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To  my  wonderful  wife,  Christine,  who  is  hoping  that 
we  still  get  along  once  she  begins  to  see  me  more  often, 
and  to  the  friends  and  partners  of  all  the  123  authors,  similarly  oppressed. 


Preface  to  the  Third  Edition 


Once  the  second  edition  was  safely  off  to  the  printer,  the  110 
authors  breathed  a  sigh  of  relief  and  relaxed,  secure  in  the  belief 
that  they  would  "never  have  to  do  that  again.”  That  lasted  for  10 
years.  When  we  finally  awoke,  it  seemed  that  a  lot  had  happened. 
In  particular,  people  were  trying  to  use  the  Handbook  as  a  text¬ 
book  even  though  it  lacked  the  practical  chapters  needed.  There 
had  been  tremendous  progress  in  lasers  and  fiber-optics  and  in  our 
understanding  of  the  mechanisms  underlying  photobleaching  and 
phototoxicity.  It  was  time  for  a  new  book.  I  contacted  "the  usual 
suspects'"  and  almost  all  agreed  as  long  as  the  deadline  was  still  a 
year  away. 

That  was  in  2002.  Three  years  later,  most  of  the  old  chapters 
have  been  substantially  or  totally  rewritten.  Although  12  of  the 
chapters  are  on  topics  that  have  either  been  rendered  obsolete  by 
improvements  in  instrumentation  or  changes  in  research  interest 
have  been  dropped,  some  have  been  replaced  by  chapters  on 
similar  topics.  To  make  the  Handbook  of  more  use  as  a  textbook, 
we  have  added  an  extended  appendix  about  practical  multiphoton 
imaging  and  another  describing  the  operation  of  CCD  cameras  in 
some  detail.  There  is  a  new  series  of  practical  chapters  on  confo- 
cal  microscopy  and  the  selection  of  dyes,  as  well  as  on  ion 
imaging,  and  on  methods  for  studying  brain  slices,  embryos, 
biofilms  and  plants  (two).  There  is  also  a  new  chapter  describing 
in  some  detail  how  such  components  as  interference  filters, 
acousto-optical  devices,  and  galvanometers  are  made  and  what 
parameters  limit  their  performance.  The  single  chapter  on  3D 
image  analysis  now  has  the  company  of  two  more  on  automated 
3D  image  analysis  and  a  third  on  high-content  screening  and  a 
fourth  on  database  management.  Chapters  have  been  added 
describing  techniques  that  have  only  recently  come  to  the  fore, 
such  as  patterned-illumination  fluorescence  microscopy,  fluores¬ 
cence  resonance  energy  transfer  (FRET)  and  the  generation  and 
detection  of  second-  and  third-harmonic  signals.  In  addition,  new 
imaging  techniques  such  as  stimulated  emission  depletion  (STED), 
coherent  anti-Stokes  Raman  (CARS)  imaging  and  selected  plane 
illumination  (SPIM)  now  have  their  own  chapters  and  there  are 
also  chapters  that  connect  the  world  of  3D  light  microscopy  to  the 


larger  world  of  micro-CT  and  micro-MRI  and  the  smaller  world 
revealed  by  the  scanning  and  transmission  electron  microscopes. 
To  round  out  the  story  we  even  have  a  chapter  on  what  PowerPoint 
does  to  the  results,  and  the  annotated  bibliography  has  been 
updated  and  extended. 

As  with  the  previous  editions,  the  editor  enjoyed  a  tremendous 
amount  of  good  will  and  cooperation  from  the  124  authors 
involved.  Both  I,  and  the  light  microscopy  community  in  general, 
owe  them  all  a  great  debt  of  gratitude.  On  a  more  personal  note,  I 
would  like  to  thank  Kathy  Lyons  and  her  associates  at  Springer  for 
their  unstinting  support  on  one  of  the  biggest  books  they  have  done 
in  microscopy  and  the  assistance  of  her  co-workers  at  Chernow 
Editorial  Services,  Barbara  Chernow  and  Kathy  Cleghorn.  Helen 
Noeldner  was  again  willing  to  work  long  hours  to  keep  all  the  man¬ 
uscripts  straight  in  spite  of  my  best  effort  to  confuse  them.  Thanks 
are  also  due  to  Bill  Feeny,  the  Zoology  Department  artist,  for  the 
innumerable  figures  that  he  rescued,  reconstructed  and  otherwise 
returned  to  life. 

If  the  hidden  agenda  of  the  first  edition  was  photon  efficiency, 
and  of  the  second,  spherical  aberration,  the  message  of  the  third 
edition  is  definitely  that  all  raw,  3D  data  sets  should  be  decon¬ 
volved  (or  at  least  3D-Gaussian  filtered)  before  being  viewed  or 
measured.  Not  only  is  this  required  to  meet  the  Nyquist  recon¬ 
struction  criterion,  it  also  greatly  reduces  the  apparent  effects  of 
Poisson  Noise  by  effectively  averaging  the  signal  over  the  50-100 
voxels  needed  to  make  a  Nyquist-sampled,  3D  image  of  a  single 
point  object.  This  last  factor  allows  one  to  obtain  acceptable 
images  using  much  less  excitation,  thereby  reducing  the  chance 
that  studies  of  living  cells  will  be  compromised  by  artifacts  caused 
by  phototoxicity.  As  evermore  studies  in  3D  light  microscopy  are 
carried  out  on  living  cells,  nothing  is  more  important.  Now  we 
need  dyes  that  produce  less  toxicity  because  they  do  not  cross  to 
the  triplet  state  and  photodetectors  that  operate  with  lower  noise 
and  higher  quantum  efficiency!  That  will  take  another  book. 

James  B.  Pawley 
January  2006 
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Preface  to  the  Second  Edition 


Confocal  microscopy  is  a  good  idea  that  was  invented,  forgotten 
and  then  reinvented  about  once  every  decade  in  the  years  between 
1957  and  1985.  However,  when  White  and  Amos  demonstrated  an 
instrument  that  was  sufficiently  user-friendly  to  become  the  ideal 
tool  for  the  3D  localization  of  specific,  fluorescent  labels  in  bio¬ 
logical  specimens,  the  field  finally  took  off.  Soon  after  the  publi¬ 
cation  of  their  1985  article  in  the  Journal  of  Cell  Biology,  requests 
to  fund  the  purchase  of  similar  equipment  increased  at  such  a  rate 
that,  in  the  fall  of  1988,  the  U.S.  National  Science  Foundation 
(NSF)  realized  that  it  needed  some  hard  information  about  the 
capabilities  of  this  new  technique.  They  funded  a  two-day  sym¬ 
posium  on  the  subject  as  part  of  the  August  1989  annual  meeting 
of  the  Electron  Microscope  Society  of  America  and  also  financed 
the  publication  of  18  papers  by  the  participants  as  The  Handbook 
of  Biological  Confocal  Microscopy  for  free  distribution  at  the 
meeting. 

This  first  edition  of  the  Handbook  differed  from  most  of  the 
many  other  compiled  volumes  on  the  subject  in  that,  rather  than 
each  author  concentrating  on  his  or  her  own  work,  an  outline  for 
the  entire  book  was  written  first,  and  then  authors  were  solicited 
to  cover  particular  aspects  of  the  instrumentation  or  its  use. 
Although  the  necessity  of  having  a  volume  ready  for  distribution 
by  August  1989  imposed  stringent  deadlines  on  the  authors  and 
required  the  typography  and  printing  to  be  done  locally,  every 
effort  was  made  to  try  to  edit  the  chapters  so  that  they  fit  together 
to  form  a  cohesive  whole.  The  success  of  the  project  was  due 
almost  entirely  to  the  enthusiasm  the  authors  had  for  sharing  their 
knowledge  of  this  fascinating  subject  with  a  wider  audience.  Man¬ 
uscripts  originally  expected  to  be  10  pages  in  length  ended  up 
being  more  than  twice  this  length,  and  several  were  more  that  50 
pages  long. 

The  resulting  volume  included  chapters  that  described  and 
compared  each  of  the  component  parts  of  the  microscope  itself 
(laser  and  conventional  light  sources,  intermediate  optics, 
alternative  scanning  systems,  objectives,  pinholes,  detectors,  and 
antecedent  and  related  optical  techniques),  chapters  that  discussed 
the  digital  aspects  of  data  acquisition  (pixelation,  digitization,  and 
display  and  measurement  of  3D  data  sets)  and  chapters  that 
reviewed  the  properties  of  fluorescent  dyes,  the  techniques  of  3D 
specimen  preparation,  and  the  fundamental  limitations  and  practi¬ 
cal  complexities  of  quantitative  confocal  fluorescence  imaging.  An 
annotated  bibliography  of  the  field  was  also  included. 

If  this  first  book  had  any  underlying  theme,  it  was  probably 
the  importance  of  photon  efficiency.  This  came  about  because,  as 
the  chapters  came  together,  it  became  clear  that  technical  limita¬ 
tions  of  the  early  instruments,  in  combination  with  suboptimal 
operating  techniques,  often  had  an  effect  such  that  the  signal  actu¬ 
ally  recorded  was  only  about  1%  of  the  expected  signal.  The  Hand¬ 
book  included  several  concrete  suggestions  for  increasing  this 
fraction,  and  it  is  a  pleasure  to  report  that  instruments  incorporat¬ 
ing  many  of  these  improvements  now  demonstrate  an  efficiency 
figure  that  is  closer  to  10-20%. 

Because  of  the  widespread  acceptance  of  the  NSF-sponsored 
volume  by  users  of  the  confocal  microscope,  a  revised  edition  (the 
“red  book”)  was  published  by  Plenum  in  1990.  Although  this  hard¬ 


cover  version  included  over  40  new  figures,  updated  tabular  infor¬ 
mation  and  over  1,400  typographical  improvements,  it  was  other¬ 
wise  generally  very  similar  to  the  initial  offering. 

However,  the  past  five  years  has  seen  a  virtual  explosion  in  the 
field  of  biological  confocal  microscopy.  As  it  became  more  and 
more  evident  that  the  original  Handbook  could  no  longer  claim  to 
cover  the  entire  held,  I  contacted  the  original  set  of  authors  about 
producing  an  updated  edition.  Remembering  the  frantic  urgency 
that  had  typified  the  production  of  the  first  edition,  I  did  this  with 
some  trepidation;  but  I  need  not  have  worried.  The  response  was 
uniformly  enthusiastic,  and  several  authors  were  not  only  willing 
to  completely  revise  their  original  chapters  but  also  volunteered 
to  write  additional  chapters  describing  several  new  areas.  The 
response  from  the  17  new  authors  was  similarly  enthusiastic. 

The  final  product  includes  37  chapters  (15  updated  from  the 
first  edition,  21  new  ones,  and  an  annotated  bibliography)  and  is 
almost  three  times  as  long  as  the  original.  Chapters  covering 
confocal  operation  in  the  UV,  in  the  transmission  mode,  and 
when  scanning  at  video  rates  using  a  variety  of  either  point¬ 
scanning  or  line-scanning  techniques  have  been  added.  The  use 
of  pulsed  laser  sources  for  both  two-photon  excitation  and 
fluorescence-lifetime  imaging  is  covered  in  depth,  and  there  is  an 
entire  chapter  on  the  functional  principles  of  modern  fiberoptic 
components  and  the  manifold  ways  that  these  can  be  applied  to 
confocal  microscopy.  In  addition,  chapters  on  the  joys  and  perils 
of  observing  living  specimens  in  the  confocal  microscope  and  on 
the  detection  of  gold-conjugated  labels  now  complement  a  revised 
version  of  the  earlier  chapter  describing  the  preparation  of  dead 
specimens. 

No  less  than  3  of  the  new  chapters  address  the  comparative 
advantages  of  the  confocal  and  wideheld/deconvolution  methods 
of  obtaining  3D  data  sets  from  biological  specimens  with  the 
minimum  possible  damage.  Although  each  of  these  chapters  pro¬ 
ceeds  from  a  very  different  perspective  (algebraic  optics,  actual 
measurements,  and  minimum-entrope  image  processing),  I  believe 
that  together  they  give  a  balanced  view  of  this  complex  and  impor¬ 
tant  subject  and  make  it  clear  that  the  confocal  microscope  could 
still  be  improved  if  the  present  photodetector  were  replaced  with 
one  having  a  higher  quantum  efficiency.  The  longest  chapter  in  the 
book  describes  the  inner  workings  of  the  17  currently  available 
systems  applicable  to  the  analysis  and  display  of  3D  digital  image 
data,  and  there  is  now  also  a  chapter  describing  the  features  of  all 
of  the  current  hardware  systems  for  the  storage,  display,  and  hard¬ 
copy  output  of  3D  and  4D  image  data  sets. 

The  subtext  of  this  second  edition  is  probably  an  increased 
recognition  of  the  extent  to  which  the  resolution  and  signal 
strength  of  confocal  images  can  be  degraded  by  spherical  aberra¬ 
tion  introduced  whenever  there  is  a  refractive-index  mismatch, 
such  as  that  occurring  when  an  oil-immersion  objective  is  used 
with  an  aqueous  specimen.  Not  only  is  an  entirely  new  chapter 
devoted  to  the  subject,  but  many  other  authors  emphasize  the  same 
point  in  their  chapters.  Again,  the  manufacturers  have  responded 
with  the  introduction  of  a  number  of  superb  new  water-immersion 
objectives  to  simplify  confocal  observations  of  living  specimens; 
these  are  also  described. 
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On  the  subject  of  optics  there  are  also  two  chapters  on  real¬ 
time  3D  imaging.  In  one,  the  approach  is  to  combine  a  high  speed 
slit  scanner  with  rapid  motion  of  the  focus  plane,  while  the  other 
demonstrates  the  truth  of  the  almost  paradoxical  premise  that  it  can 
be  useful  to  actually  increase  the  chromatic  aberration  of  an  objec¬ 
tive  if  it  is  to  be  used  to  examine  surface  height  in  the  back- 
scattered  light  mode  with  “white”  light.  Of  more  interest  to  those 
wishing  to  improve  axial  resolution  in  the  fluorescent  mode  is  the 
chapter  describing  new,  high-resolution  techniques  that  combine 
either  two  or  even  three  confocal  objectives  with  two-photon  exci¬ 
tation  to  improve  resolution  to  a  level  heretofore  believed  to  be 
impossible. 

Finally,  there  is  a  tutorial  chapter  intended  for  the  novice  user, 
as  well  as  two  appendixes.  The  first  appendix  describes  the  rela¬ 
tionship  between  real-space  and  optical  coordinates,  while  the 
second  provides  a  compilation  of  the  optical  path  layouts  of  the 
major  commercial  confocal  instruments. 

The  topics  in  this  book  cover  a  very  wide  range  of  disciplines. 
While  this  is  good  in  that  it  shows  the  integrating  nature  of  the 
field,  it  can  lead  to  problems  with  notation  when  optical  physicists, 
experts  on  information  theory,  microscope  designers,  and  just  plain 
biologists  have  to  try  to  agree  on  a  common  system  of  notation. 
In  the  first  edition,  we  did  not  even  try  to  overcome  this  problem. 
Although  this  led  to  some  confusion,  I  must  confess  that  my  efforts 
to  remedy  the  problem  in  the  present  volume  have  not  been  totally 
successful.  Index  of  refraction  has  been  rendered  as  T|,  so  that  n 
can  be  reserved  for  the  number  of  quantum  events;  where  t  has 
been  used  for  thickness,  we  have  tried  to  use  italics,  so  that  t  could 
be  used  for  time  as  a  variable  and  T  for  temperature,  while  spe¬ 
cific  times  (lifetimes,  pulse  times)  are  shown  as  T  or  x;  wherever 
x,  y  and  z  are  used  as  directions,  we  have  italicized  them,  while 
we  have  tried  to  keep  r  as  actual  dimensions  in  the  x—y  plane  (rp  = 
pinhole  radius,  rs  =  slit  width,  rd  =  detector  diameter,  etc.);  and 
numerical  aperture  appears  almost  everywhere  as  NA  but  becomes 
Ana  in  some  equations.  Perhaps  most  debatable  was  my  decision 
to  try  to  save  space  by  replacing  the  word  "wavelength”  with  X  in 
the  body  of  the  text.  On  reflection,  this  change  probably  did  not 
repay,  in  space,  the  interruption  of  the  reader  that  it  produces,  but, 
unfortunately,  by  the  time  this  became  evident,  it  was  too  late  to 
change  it.  In  spite  of  our  best  efforts,  problems  arose  because, 
while  authors  wanted  to  fit  in  with  the  book  as  whole,  they  also, 
understandably,  wished  to  remain  consistent  with  their  previous 
publications.  I  would  like  to  thank  them  all  for  their  cooperation 
on  this  complex  issue,  and  I  hope  that  our  efforts  at  consistency 
have  not  introduced  any  errors  into  the  text. 

This  brings  us  to  the  Index.  There  was  not  enough  time  to 
prepare  an  index  for  the  NSF  version.  One  was  put  together  for 
the  “red  book,”  but  it  was  somewhat  less  extensive  than  one  might 
have  wished  for  a  handbook.  This  time,  when  faced  with  the  need 
to  do  it  all  again,  and  also  having  all  of  the  text  in  electronic  form, 
I  was  mindful  of  the  two  opposing  indexing  concepts  currently  per¬ 
vasive  in  the  popular  culture.  What  one  might  call  the  minimalist 
view  of  indexing  comes  from  the  Douglas  Adams  book  The  Hitch¬ 
hiker’s  Guide  to  the  Galaxy,  where  the  original  entry  for  Earth  is 
“Harmless;”  this  is  only  slightly  improved  later  by  being  updated 
to  “Mostly  Harmless.”  The  opposing  view  was  crystallized  by 


Barry  Commoner  as:  “Everything  is  connected  to  everything  else” 
—  a  concept  amply  demonstrated  within  the  field  of  confocal 
microscopy.  Trying  to  steer  a  middle  course  between  these  two 
extremes,  I  have  concocted  a  new  Index  that  is  over  twelve  times 
the  size  of  the  previous  one  (now  with  nearly  7,000  topics  and 
about  twice  that  many  page  listings),  while  the  book  itself  has 
almost  tripled.  This  Index  contains  entries  for  almost  every 
diagram,  plot,  image,  and  table  in  the  book.  It  also  lists  under 
“Summaries”  the  pages  of  the  summary  sections  that  conclude 
most  chapters  and  contain  their  “take-home  lessons.”  The  listing 
“chapter”  refers  to  an  entire  chapter  starting  on  the  page  noted  and 
dealing  predominantly  with  the  listed  topic.  Although  subjects  in 
the  text  are  extensively  cross-indexed,  literally  “connecting  every¬ 
thing  to  everything  else”  would  have  required  another  book.  I 
settled  for  making  sure  that  each  text  topic  appeared  at  least  once 
under  all  of  the  Index  topics  that  seemed  appropriate,  but  I  did  not 
attempt  to  list  all  the  pages  in  each  chapter  on  which  a  term  was 
mentioned.  As  a  result,  the  reader  would  probably  be  well-advised 
to  look  for  additional  information  on  the  pages  adjacent  to  (usually 
following)  those  pages  listed  in  the  Index.  I  beg  indulgence  for  all 
of  the  “inevitable  omissions.” 

Confocal  microscopy  is  not  the  only  technology  to  have  devel¬ 
oped  over  the  last  five  years.  Constant  improvements  in  the  inter¬ 
national  digital  communication  network  have  brought  e-mail  and 
electronic  file  transfer  into  the  normal  working  lives  of  most  of  the 
authors,  and  this  made  the  editing  of  the  present  edition  much  more 
of  a  two-way  process.  Chapters  could  be  modified  to  fit  better  with 
their  neighbors,  returned,  checked,  and  resubmitted  all  in  a  matter 
of  days,  even  when  the  authors  concerned  were  in  Australia, 
Taiwan,  and  Europe.  Although  this  process  added  a  welcome  level 
of  flexibility  not  present  for  the  earlier  book,  it  also  imposed  an 
additional  strain  on  the  authors,  who  often  were  just  congratulat¬ 
ing  themselves  on  finally  getting  their  chapter  “out  the  door”  only 
to  have  them  reappear  with  a  lot  of  suggested  changes  and  requests 
for  expansion  to  cover  additional  areas.  Again,  the  authors 
responded  to  this  challenge  in  the  most  positive  manner  possible, 
and  this  seems  the  most  appropriate  place  to  record  my  sincere 
thanks  to  them  for  the  cooperative  spirit  that  they  invariably  dis¬ 
played.  Thanks  are  also  due  to  Helen  Noeldner,  who  provided  the 
order  and  secretarial  assistance  without  which  we  could  not  have 
succeeded;  to  Mary  Born,  my  editor  at  Plenum,  whose  kind  voice 
prevented  me  from  jumping  out  of  my  twelfth-floor  window  on 
several  occasions;  to  those  manufacturers  who  provided  support 
for  publishing  some  of  the  color  figures  and  to  their  representa¬ 
tives  for  providing  the  diagrams  and  other  information  included  in 
Appendix  2;  to  NSF,  which  provided  me  with  grant  DIR-90- 17534, 
to  my  wife;  Christine,  who  toiled  many  late  nights  on  the  Index; 
and  to  my  family  (and  doubtless  the  families  of  the  authors),  who 
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Foundations  of  Confocal  Scanned  Imaging  in 
Light  Microscopy 

Shinya  Inoue 


Seldom  has  the  introduction  of  a  new  instrument  generated  as 
instant  an  excitement  among  biologists  as  the  laser-scanning  con- 
focal  microscope.  With  the  new  microscope,  one  can  slice  incred¬ 
ibly  clean,  thin  optical  sections  out  of  thick  fluorescent  specimens; 
view  specimens  in  planes  tilted  to,  and  even  running  parallel  to, 
the  line  of  sight;  penetrate  deep  into  light-scattering  tissues;  gain 
impressive  three-dimensional  (3D)  views  at  very  high  resolution; 
obtain  differential  interference  or  phase-contrast  images  in  exact 
register  with  confocal  fluorescence  images;  and  improve  the  pre¬ 
cision  of  microphotometry. 

While  the  instrument  that  engendered  such  excitement  became 
commercially  available  first  in  1987,  the  optical  and  electronic 
theory  and  the  technology  that  led  to  this  sudden  emergence  had 
been  brewing  for  several  decades.  The  development  of  this  micro¬ 
scope  stems  from  several  roots,  including  light  microscopy,  con¬ 
focal  imaging,  video  and  scanning  microscopy,  and  coherent  or 
laser-illuminated  optics  (see  historic  overview  in  Table  1.1).  In  this 
chapter,  I  will  first  discuss  some  basic  principles  relating  to  lateral 
and  axial  resolution  as  well  as  depth  of  field  in  light  microscopy, 
highlight  some  history  that  lays  a  foundation  to  the  development 
of  laser-scanning  confocal  microscopy,  and  end  with  some  general 
remarks  regarding  the  new  microscopes,  including  a  disk-scanning 
confocal  system. 


LIGHT  MICROSCOPY 
Lateral  Resolution1 

The  foundations  of  modem  light  microscopy  were  established  a 
century  ago  by  Ernst  Abbe  (1873,  1884).  He  demonstrated  how  the 
diffraction  of  light  by  the  specimen,  and  by  the  objective  lens, 
determined  image  resolution;  defined  the  conditions  needed  to 
design  a  lens  whose  resolution  was  diffraction  limited  (rather  than 
limited  by  chromatic  and  spherical  aberrations);  and  established 
the  role  of  the  objective  and  condenser  numerical  apertures  (NA) 
on  image  resolution  (Eq.  1).  Thus, 


1  For  extensive  discussions  on  modern  microscope  lens  design  and  aberrations 
and  a  more  rigorous  treatment  of  the  optical  principles  and  applications  of 
light  microscopy  than  is  appropriate  for  this  revised  chapter,  refer  to  com¬ 
plementary  chapters  in  Handbook  of  Optics  (e.g.,  Inoue  and  Oldenbourg, 
1994)  and  in  Video  Microscopy ,  2"J  edition  (Inoue  and  Spring.  1997). 


d  1-22X0 

^  NAobj  +  NAc, 


(1) 


where  dmm  is  the  minimum  spacing  in  a  periodic  grating  that  can 
just  be  resolved.  dmin  is  expressed  as  lateral  distance  in  the  speci¬ 
men  space;  A,0  is  the  wavelength  of  light  in  vacuum;  and  AA„b|  and 
NAQon d  are  the  numerical  apertures  of  the  objective  and  condenser 
lenses,  respectively.  The  NA  is  the  product  of  the  sine  of  the  half¬ 
angle  (a)  of  the  cone  of  light  either  acceptable  by  the  objective 
lens  or  emerging  from  the  condenser  lens  and  the  refractive 
indexes  (T|)  of  the  imbibing  medium  between  the  specimen  and  the 
objective  or  condenser  lens,  respectively. 

Equation  1  demonstrates  that,  in  addition  to  the  wavelength 
and  the  NA  of  the  objective  lens,  the  condenser  NA  also  affects 
image  resolution  in  the  microscope.  For  objects  that  are  illumi¬ 
nated  fully  coherently  (a  condition  that  pertains  when  NAconi 
approaches  0,  namely  when  the  condenser  iris  is  closed  down  to  a 
pinhole),  the  minimum  resolvable  lateral  spacing  increases  (i.e., 
the  resolution  decreases)  by  a  factor  of  2  compared  to  the  case 
when  the  condenser  iris  is  opened  so  that  AAcond  =  AAobj.  As  the 
condenser  iris  is  opened  and  NAamd  becomes  larger,  the  illumina¬ 
tion  becomes  progressively  less  coherent  and  resolution  increases. 
[Note,  however,  that  laser  beams  tend  to  illuminate  objects  coher¬ 
ently  even  when  the  condenser  iris  is  not  closed  down  (see  Chapter 
5,  this  volume).] 

Equation  1  describes  the  relation  between  NA  and  resolution 
for  line-grating  objects.  A  complementary  method  of  defining  the 
limit  of  resolution  uses  point  objects  instead  of  line  gratings.  The 
image,  in  focus,  of  an  infinitely  small  luminous  object  point  is 
itself  not  infinitely  small,  but  is  a  circular  Airy  diffraction  image 
with  a  central  bright  disk  and  progressively  weaker  concentric  dark 
and  bright  rings.  The  radius  rAky  of  the  first  dark  ring  around  the 
central  disk  of  the  Airy  diffraction  image  depends  on  X  and  the  NA 
of  the  objective: 


'Airy 


=  0.61- 


NA 


obj 


(2) 


where  rAily  is  expressed  as  distance  in  the  specimen  plane. 

When  there  exist  two  points  of  light  separated  by  a  small  dis¬ 
tance  d  in  the  specimen  plane,  their  diffraction  images  lie  side  by 
side  in  the  image  plane.  The  images  of  two  equally  bright  spots 
are  said  to  be  resolved  if  d  is  larger  or  equal  to  the  radius  of 
the  Airy  disk.  This  is  the  Rayleigh  criterion,  and  it  relies  on  the 
assumption  that  the  two  point  sources  radiate  incoherently.  If  the 
two  point  sources  emit  light  coherently,  their  amplitude  rather  than 
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Confocal  Microscopy 


Minsky  Patent  (1957) 
Insight"’*’*’** 

Stage  scanning^ 


Petran  et  al.  (1968) 

Tandem  scanning^ 
Davidovits  and  Egger  (1971) 
Laser  illumination 
Lens  scanning** 


Sheppard  and  Choudhury  (1977) 

Theory*'*’** 

Sheppard  et  al.  (1978) 

Stage  scanning*’**’*^ 

Cremer  and  Cremer  (1978) 
Auto-focus  stage  scanning** 
“4-7t-point  illumination”*'’*’’** 
Brakenhoff  et  al.  (1979) 
Specimen  scan*'’*’’**’* 

Koester  (1980) 

Scanning  mirror** 


Cox  and  Sheppard  (1983) 

Digital  recording**’* 

Aslund  et  al.  (1983) 

2-mirror  laser  scanning** 

Hamilton  et  al.  (1984) 

Differential  phase*  ** 

Wilson  and  Sheppard  (1984) 

Extended  depth  of  field*’**’*^ 

Boyde  (1985a) 

Nipkow  type*** 

Carlsson  et  al.  (1985) 

Laser  scan,  Stacks  of  confocal  images'** 
Wijnaendts  van  Resandt  et  al.  (1985) 
xz-view**’* 


TABLE  1.1.  Historic  Overview3 

Microscopy  Video  (Microscopy) 

Abbe  (1873,  1884)"’*  Nipkow  (1884) 

Berek  (1927)** 

Zworykin  (1934) 

Zemicke  (1935)"’* 

Gabor  (1948)" 

Hopkins  (1951)"  Llory  (1951) 

Young  and  Roberts  (1951) 
Plying  spot* 

Linfoot  and  Wolf  (1953) 

3D  diffraction  by  annul,  apert."’** 

Tolardo  di  Prancia  (1955) 

Limited  field*' 

Nomarski  (1955)" 

Linfoot  and  Wolf  (1956)  Montgomery  et  al.  (1956) 

3D  diffraction  pattern"**  Plying  spot  UV* 

Ingelstam  (1956) 

Resolution  and  info,  theory" 


Kubota  and  Inoue  (1959)"’* 

Smith  and  Osterberg  (1961)" 

Freed  and  Engle  (1962) 
Flying  spot* 

Harris  (1964)"’*’ 

Ellis  (1966) 

Holomicrography" 


Hellwarth  and  Christensen  (1974) 
Second  harmonic  generation*** 
Hoffman  and  Gross  (1975) 
Modulation  contrast* 


Ellis  (1978) 

Single  sideband  edge  enhancement  microscopy*’** 


Castleman  (1979) 

Digital  image  processing"’*’* 

Quate  (1980) 

Acoustic  microscopy"’* 

Inoue  (1981)a,c 
Allen  et  al.  (1981a,b)"’* 

Fuchs  et  al.  (1982/ 

Agard  and  Sedat  (1983)"’*’** 


Ellis  (1985) 

Light  scrambler** 


Sher  and  Barry  (1985/ 
Fay  et  al.  (1985)"’*’** 
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TABLE  1.1.  ( Continued ) 


Confocal  Microscopy 

Microscopy 

Video  (Microscopy) 

Suzuki-Horikawa  (1986) 

Cox  and  Sheppard  (1986) 

Inoue  (1986) 

Video-rate  laser  scang 

Acousto-optical  modulator 

No  exit  pinhole 

Overview,  How  to"  ""^ 

Xiao  and  Kino  (1987) 

Nipkow  type'7 

Amos  et  al.  (1987)"’47’" 

Castleman  (1987)" 

McCarthy  and  Walker  (1988) 

Ellis  (1988) 

Nipkow  type"7 

Denk  et  al.  (1990) 

Two  photon"’",‘7/ 

Hell  and  Wichmann  (1994) 

PSF  reduction  by  stimulated  emission  depletion"’7’’'7 

Ichihara  et  al.  (1996) 

High-throughput  spinning  disk'/,/,g 

Scanned  aperture  phase  contrast"’"’'7 

Gustaffson  et  al.  (2000) 

Structured  illumination"’7’’"’'7 

Oldenbourg  and  Mei  (1995) 
LC-pol  system"’'7 

Conchello  et  al.  (1997) 
Aperture  scanning"’7’’"’'7^ 

Volkmer  et  al.  (2001) 

Inoue  et  al.  (200 1  a, b) 

Coherent  anti-stokes  raman" 

Inoue  et  al.  (2002) 

Fluorescence  pol" 

Centrifuge  pol  scope"’g 

"Diffraction  theory. 
b  Superresolution. 

"Contrast  modes. 

^Optical  sectioning/depth  of  field. 
e  Stereo. 

^3D  in  objective  space. 
sHigh  speed. 


their  intensity  distribution  in  the  image  must  be  considered,  and 
resolution  generally  decreases.  The  impact  of  the  quality  and  NA 
of  the  condenser  on  the  lateral  resolution  of  point  objects  was  con¬ 
sidered  by  Hopkins  and  Barham  (1950).  Their  results  are  similar 
to,  but  not  strictly  identical  with,  the  case  of  line-grating  objects 
(see  Born  and  Wolf,  1980). 

It  is  important  to  realize  that  these  resolution  criteria  apply 
only  to  objective  lenses  used  under  conditions  in  which  the  image 
is  free  from  significant  aberrations  (see  Chapters  6,  7,  8,  9,  11,  and 
22,  this  volume ;  Inoue  and  Oldenbourg,  1994;  Chapters  2  and  3  in 
Inoue  and  Spring,  1997).  This  implies  several  things: 

•  A  well-corrected  clean  objective  lens  is  used  within  the  wave¬ 
lengths  of  light  and  diameter  of  field  for  which  the  lens  was 
designed  (commonly  in  conjunction  with  specific  oculars 
and/or  tube  lenses). 

•  The  refractive  index,  dispersion,  and  thickness  of  the  coverslip 
and  immersion  media  are  those  specified  for  the  particular 
objective  lens. 

•  The  correct  tube  length  and  ancillary  optics  are  used  and  the 
optics  are  axially  aligned. 

•  The  full  complement  of  image-forming  rays  and  light  waves 
leaving  all  points  of  the  objective-lens  aperture  converge  to  the 
image  plane  without  obstruction. 

•  The  condenser  aperture  is  homogeneously  and  fully 
illuminated. 

•  The  condenser  is  properly  focused  to  produce  Kohler 
illumination. 


These  considerations  for  resolution  assume  that  the  specimen  is 
viewed  in  conventional  widefield  (WF)  microscopy.  When  the 
(instantaneous)  field  of  view  becomes  extremely  small,  as  in  con- 
focal  microscopy,  the  resolution  can  in  fact  be  greater  than  when 
the  field  of  view  is  not  so  limited.  We  shall  return  to  this  point 
later.2 

Axial  Resolution 

We  now  turn  to  the  axial  (z-axis)  resolution,  measured  along  the 
optical  axis  of  the  microscope,  that  is,  perpendicular  to  the  plane 
of  focus  in  which  the  lateral  resolution  was  considered. 

To  define  axial  resolution,  it  is  customary  to  use  the  3D  dif¬ 
fraction  image  of  a  point  source  that  is  formed  near  the  focal  plane. 
In  the  case  of  lateral  resolution,  that  is,  the  resolution  in  the  plane 
of  focus,  the  Rayleigh  criterion  makes  use  of  the  infocus  diffrac¬ 
tion  images  (the  central  cross-section  of  the  3D  diffraction  pattern) 
of  two  point  sources  and  the  minimum  distance  that  they  can 
approach  each  other  laterally,  yet  still  be  distinguished  as  two. 
Similarly,  axial  resolution  can  be  defined  by  the  minimum  distance 


2  Note  also  that  one’s  ability  to  determine  the  location  of  an  object  is  not  deter¬ 
mined  by  the  resolution  limit  of  the  system.  In  fact,  the  location  of  an  object 
(diffraction  pattern)  can  be  determined  under  a  microscope  with  precisions 
that  are  many  times,  or  even  orders  of  magnitude,  greater  than  the  resolution 
limit  (e.g.,  Denk  and  Webb,  1987;  also  see  Inoue,  1989). 
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that  the  diffraction  images  of  two  points  can  approach  each  other 
along  the  axis  of  the  microscope,  yet  still  be  seen  as  two.  To  define 
this  minimum  distance,  we  use  again  the  diffraction  image  of  an 
infinitely  small  point  object  and  ask  for  the  location  of  the  first 
minimum  along  the  axis  of  the  microscope. 

The  precise  distribution  of  energy  in  the  image-forming  light 
above  and  below  focus,  especially  for  high  NA  objective  lenses, 
cannot  be  deduced  by  geometric  ray  tracing  but  must  be  derived 
from  wave  optics.  The  wave  optical  studies  of  Linfoot  and  Wolf 
(1956)  show  that  the  image  of  a  point  source  produced  by  a  dif¬ 
fraction-limited  optical  system  (e.g.,  a  well-designed  and  properly 
used  light  microscope)  is  not  only  periodic  around  the  point  of 
focus  in  the  focal  plane,  but  is  also  periodic  above  and  below  the 
focal  plane  along  the  axis  of  the  microscope.  [Such  3D  diffraction 
images  (including  those  produced  in  the  presence  of  lens  aberra¬ 
tions)  are  presented  photographically  by  Cagnet  et  al.  (1962;  also 
see  Chapter  7,  this  volume.  Fig.  7.4).  The  intensity  distribution  cal¬ 
culated  by  Linfoot  and  Wolf  for  an  aberration-free  system  is  repro¬ 
duced  in  Born  and  Wolf  (1980)  and  also  in  Inoue  and  Spring  (1997, 
Fig.  2-30).  The  3D  pattern  of  a  point  source  formed  by  a  lens  pos¬ 
sessing  an  annular  aperture  was  calculated  by  Linfoot  and  Wolf 
(1953).] 

The  distance  from  the  center  of  the  3D  diffraction  pattern  to 
the  first  axial  minimum  (in  object  space  dimensions)  is  given  by 

2Xnr\ 

Zmin=(^J  (3) 

where  T|  is  the  refractive  index  of  the  object  medium.  -mln  corre¬ 
sponds  to  the  distance  by  which  we  have  to  raise  the  microscope 
objective  in  order  to  focus  the  first  intensity  minimum  observed 
along  the  axis  of  the  3D  diffraction  pattern  instead  of  the  central 
maximum.3 

As  with  the  lateral  resolution  limit,  we  can  use  zmin  as  a 
measure  of  the  limit  of  axial  resolution  of  the  microscope  optics. 
Note,  however,  that  zm in  shrinks  inversely  proportionally  with  the 
square  of  the  NAobi,  in  contrast  to  the  lateral  resolution  limit  which 
shrinks  with  the  first  power  of  the  M40bj.  Thus,  the  ratio  of  axial- 
to-lateral  resolution  (ZmiJrAhy  =  3.28t|/AAobj)  is  substantially  larger 
than  A,  and  is  inversely  proportional  to  the  NA  of  the  objective  lens. 

Depth  of  Field 

The  depth  of  field  of  a  microscope  is  the  depth  of  the  image  (mea¬ 
sured  along  the  microscope  axis  translated  into  distances  in  the 
specimen  space)  that  appears  to  be  sharply  in  focus  at  one  setting 
of  the  fine-focus  adjustment.  In  brightfield  microscopy,  this  depth 
should  be  approximately  equal  to  the  axial  resolution,  at  least  in 
theory.  The  actual  depth  of  field  has  been  determined  experimen¬ 
tally,  and  the  contribution  of  various  factors  that  affect  the  mea¬ 
surement  have  been  explored  by  Berek  (1927). 

According  to  Berek,  the  depth  of  field  is  affected  by  (1)  the 
geometric  and  diffraction-limited  spreading,  above  and  below  the 
plane  of  focus,  of  the  light  beam  that  arose  from  a  single  point  in 
the  specimen;  (2)  the  accommodation  of  the  observer’s  eye; 
and  (3)  the  final  magnification  of  the  image.  The  second  factor 
becomes  irrelevant  when  the  image  is  not  viewed  directly  through 
the  ocular  but  is  instead  focused  onto  a  thin  detector  (as  in  video 


3  As  discussed  later,  the  distance  can  be  reduced  significantly  below  the 
classical  limit  given  by  Eq.  1.3,  for  example,  by  reducing  the  effective  point- 
spread  function  by  special  use  of  two-photon  confocal  imaging. 


microscopy  or  confocal  microscopy  with  a  minute  exit  pinhole). 
The  third  factor  should  also  disappear  once  the  total  magnification 
is  raised  sufficiently,  so  that  the  unit  diffraction  image  becomes 
significantly  larger  than  the  resolution  element  of  the  detector  (e.g., 
Flansen  in  Inoue,  1986;  Castleman,  1987,  1993;  Schotten,  1993; 
Inoue  and  Spring,  1997,  Section  12.2). 

When  the  detector  can  be  considered  to  be  infinitely  thin  and 
made  up  of  resolution  elements  spaced  sufficiently  (at  least  2-fold) 
finer  than  the  Airy  disk  radius,  then  one  need  only  to  consider  the 
diffraction-limited  depth  of  field.  In  that  case,  the  depth  of  field  is 
taken  to  be 

5  =  —  (z  .  +  —z  .  -)  W 

that  is,  one  quarter  of  the  distance  between  the  first  axial  minima 
above  (tw)  and  below  (Zmin-)  the  central  maximum  in  the  3D  Airy 
pattern  converted  to  distances  in  specimen  space  (see  Eq.  3;  zmin* 
and  zmin-  correspond  to  Z1  and  -Z1  in  Chapter  7,  Fig.  7.4,  this 
volume). 

In  conventional  fluorescence  and  darkfield  microscopy,  the 
light  arising  from  each  image  point  produces  significant  intensity 
within  a  solid  cone  that  reaches  a  considerable  distance  above  and 
below  focus  (as  seen  in  the  point-spread  functions  for  these  modes 
of  microscopy;  e.g.,  Streibl,  1985;  also  Chapters  11  and  23,  this 
volume).  Therefore,  fluorescent  (or  light-scattering)  objects  that 
are  out  of  focus  produce  unwanted  light  that  is  collected  by  the 
objective  and  reduces  the  contrast  of  the  signal  from  the  region  in 
focus. 

For  these  reasons,  the  depth  of  field  may  be  difficult  to  measure 
or  even  to  define  precisely  in  conventional  fluorescence  and  dark- 
field  microscopy.  Put  another  way,  one  could  say  that  when  objects 
that  are  not  infinitely  thin  are  observed  in  conventional  fluores¬ 
cence  or  darkfield  microscopy,  the  apparent  depth  of  field  is  very 
much  greater  than  the  axial  resolution. 

The  unwanted  light  that  expands  the  apparent  depth  of  field  is 
exactly  what  confocal  imaging  eliminates.  Thus,  we  can  view  only 
those  fluorescent  and  light- scattering  objects  that  lie  within  the 
depth  that  is  given  by  the  axial  resolution  of  the  microscope  and 
attain  the  desired  shallow  depth  of  field. 

As  mentioned  earlier,  the  lateral  resolution  of  a  microscope  is 
also  a  function  of  the  size  of  the  field  observed  at  any  one  instant. 
Tolardo  di  Francia  (1955)  suggested,  and  Ingelstam  (1956)  argued 
on  the  basis  of  information  theory,  that  one  gains  lateral  resolution 
by  a  factor  of  V2  as  the  field  of  view  becomes  vanishingly  small. 
These  theoretical  considerations  set  the  stage  for  the  development 
of  confocal  imaging. 


CONFOCAL  IMAGING 

As  a  young  postdoctoral  fellow  at  Harvard  University  in  1957, 
Marvin  Minsky  applied  for  a  patent  for  a  microscope  that  used  a 
stage-scanning  confocal  optical  system.  Not  only  was  the  concep¬ 
tion  farsighted,  but  his  insight  into  the  potential  application  and 
significance  of  confocal  microscopy  was  nothing  short  of  remark¬ 
able.  [See  the  delightful  article  by  Minsky  (1988)  that  shows  even 
greater  insight  into  the  significance  of  confocal  imaging  than  do 
the  following  extracts  culled  from  his  patent  application.] 

In  Minsky’s  embodiment  of  the  confocal  microscope,  the  con¬ 
ventional  microscope  condenser  is  replaced  by  a  lens  identical  to 
the  objective  lens.  The  field  of  illumination  is  limited  by  a  pinhole, 
positioned  on  the  microscope  axis.  A  reduced  image  of  this  pinhole 
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is  projected  onto  the  specimen  by  the  “condenser.”  The  field  of 
view  is  also  restricted  by  a  second  (or  exit)  pinhole  in  the  image 
plane  placed  confocally  to  the  illuminated  spot  in  the  specimen  and 
to  the  first  pinhole  (Fig.  1.1).  Instead  of  trans-illuminating  the  spec¬ 
imen  with  a  separate  “condenser”  and  objective  lens,  the  confocal 
microscope  could  also  be  used  in  the  epi-illuminating  mode, 
making  a  single  objective  lens  serve  as  both  the  condenser  and  the 
objective  lens  (Fig.  1.2). 

Using  either  transmitted  or  epi-illumination,  the  specimen  is 
scanned  with  a  point  of  light  by  moving  the  specimen  over  short 
distances  in  a  raster  pattern.  (The  specimen  stage  was  supported 
on  two  orthogonally  vibrating  tuning  forks  driven  by  electromag¬ 
nets  at  60  and  6000  Hz.)  The  variation  in  the  amount  of  light,  mod¬ 
ulated  by  the  specimen  and  passing  the  second  pinhole,  is  captured 
by  a  photoelectric  cell.  The  photoelectric  current  is  amplified  and 
modulates  the  beam  intensity  of  a  long-persistence  cathode-ray 
tube  (CRT)  scanned  in  synchrony  with  the  tuning  forks.  As  a  result, 
the  image  of  the  specimen  is  displayed  on  the  CRT.  The  ratio  of 
scanning  distances  between  the  electron  beam  and  the  specimen 
provides  image  magnification,  which  is  variable  and  can  be  very 
large. 

With  this  stage-scanning  confocal  microscope,  Minsky  says, 
light  scattered  from  parts  other  than  the  illuminated  point  on  the 
specimen  is  rejected  from  the  optical  system  (by  the  exit  pinhole) 
to  an  extent  never  before  realized.  As  pointed  out  in  the  patent 
application,  there  are  several  advantages  to  such  an  optical  system: 

•  Reduced  blurring  of  the  image  from  light  scattering 

•  Increased  effective  resolution 

•  Improved  signal-to-noise  ratio 

•  Permits  unusually  clear  examination  of  thick,  light-scattering 
objects 

•  xy-scan  possible  over  wide  areas  of  the  specimen 

•  Inclusion  of  a  z-scan  is  possible 

•  Electronic  adjustment  of  magnification 


FIGURE  1 .1 .  Optical  path  in  simple  confocal  microscope.  The  condenser  lens, 
C,  forms  an  image  of  the  first  pinhole.  A,  onto  a  confocal  spot,  D.  in  the  spec¬ 
imen,  S.  The  objective  lens,  O,  forms  an  image  of  D  onto  the  second  (exit) 
pinhole,  B,  which  is  confocal  with  D  and  A.  Another  point,  such  as  E  in  the 
specimen,  would  not  be  in  focus  with  A,  so  that  the  illumination  would  be  less. 
In  addition,  most  of  the  light,  g-h,  scattered  from  E  would  not  pass  the  second 
pinhole,  B.  The  light  reaching  the  phototube,  P,  from  E  is  thus  greatly  attenu¬ 
ated  compared  to  that  from  the  confocal  point,  D.  In  addition,  the  exit  pinhole 
can  be  made  small  enough  to  exclude  the  diffraction  rings  in  the  image  of  D, 
so  that  the  resolving  power  of  the  microscope  is  improved.  As  the  specimen  is 
scanned,  the  phototube  provides  a  signal  of  the  light  passing  through  sequen¬ 
tial  specimen  points  Di,  D2,  D3,  etc.  (not  shown).  D,,  D2,  D3,  etc.,  can  lie  in  the 
focal  plane  as  in  conventional  microscopy  or  perpendicular  to  it,  or  at  any  angle 
defined  by  the  scanning  pattern,  so  that  optical  sections  can  be  made  in  or  at 
angles  tilted  from  the  conventional  image  plane.  Because,  in  the  stage¬ 
scanning  system,  the  small  scanning  spot,  D,  lies  exactly  on  the  axis  of  the 
microscope,  the  lenses  C  and  O  can  be  considerably  less  sophisticated  than 
conventional  microscope  lenses,  which  must  form  images  from  points  some 
distance  away  from  the  lens  axis.  (After  Minsky,  1957.) 


FIGURE  1.2.  Optical  path  in  epi-illuminated  confocal  microscope.  The 
entrance  pinhole,  A,  point  D  in  the  specimen,  S,  and  exit  pinhole,  B,  are  con¬ 
focal  points  as  in  Figure  1.1.  A  partial,  or  dichromatic,  mirror,  M,.  transmits 
the  illuminating  beam  a-b-c  and  reflects  the  beam  d-e  which  passed  D  and 
was  reflected  by  the  mirror,  M2,  on  which  the  specimen  is  lying.  Only  the 
reflected  beam  that  passes  point  D  focuses  onto  the  detector  pinhole  and  reaches 
the  photocell,  P.  A  single  lens,  O,  replaces  the  condenser  and  objective  lenses 
in  Figure  1.1.  (After  Minsky,  1957.) 


•  Especially  well  suited  for  making  quantitative  studies  of  the 
optical  properties  of  the  specimen 

•  An  infinite  number  of  aperture  planes  in  the  microscope  are 
potentially  available  for  modulating  the  aperture  with  darkfield 
stops,  annuli,  phase  plates,  etc. 

•  Complex  contrast  effects  can  be  provided  with  comparatively 
simple  equipment 

•  Permits  use  of  less  complex  objective  lenses,  including  those 
for  long  working  distance,  ultraviolet  (UV),  or  infrared 
imaging,  as  they  need  to  be  corrected  only  for  a  single  axial 
point. 

The  high-resolution  acoustic  microscope  developed  by  Quate 
and  co-workers  (Quate,  1980)  and  the  laser  disk,  video,  and  audio 
recorder/players  are  object-scanning-type  confocal  microscopes. 
The  designers  of  these  instruments  take  advantage  of  the  fact  that 
only  a  single  axial  point  is  focused  or  scanned  (see,  e.g.,  Inoue  and 
Spring,  1997,  Sect.  11.10). 

IMPACT  OF  VIDEO 
Nipkow  Disk 

lust  about  the  same  time  that  Abbe  in  Jena  laid  the  foundation  for 
modern  light  microscopy,  a  young  student  in  Berlin,  Paul  Nipkow 
(1884),  figured  out  how  to  convert  a  two-dimensional  (2D)  optical 
image  into  an  electrical  signal  that  could  be  transmitted  as  a  one¬ 
dimensional  (ID),  or  serial,  time-dependent  signal,  over  a  single 
cable  (as  in  a  Morse  code).  Prior  to  Nipkow,  most  attempts  at  the 
electrical  transmission  of  optical  images  involved  the  use  of  mul¬ 
tiple  detectors  and  as  many  cables. 

Nipkow  dissected  the  image  by  scanning  over  it  in  a  raster 
pattern,  using  a  spinning  opaque  wheel  perforated  by  a  series  of 
rectangular  holes.  The  successive  holes,  placed  a  constant  angle 
apart  around  the  center  of  the  disk  but  on  constantly  decreasing 
radii  (i.e.,  arranged  as  an  Archimedes  spiral),  generated  the  raster¬ 
scanning  pattern  (Fig.  1.3).  The  brightness  of  each  image  element, 
thus  scanned  by  the  raster,  was  picked  up  by  a  photocell.  The 
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FIGURE  1.3.  Nipkow  disk.  The  perforations  in  the  opaque  disk.  A,  which  is 
rotating  at  a  constant  velocity,  scan  the  image  in  a  raster  pattern  as  shown  in 
B.  (After  Zworykin  and  Morton,  1954.) 


output  of  the  photocell  reflected  the  brightness  of  the  sequentially 
scanned  image  elements  and  drove  a  neon  bulb  that,  viewed 
through  another  (part  of  the)  Nipkow  disk,  reproduced  the  desired 
picture. 

A  similar  type  of  scanner  disk,  but  with  multiple,  centrosym- 
metric  sets  of  spirally  placed  holes,  was  used  by  Mojmir  Petran 
and  co-workers  at  Prague  and  New  Haven  to  develop  their  epi- 
illuminated  tandem-scanning  confocal  microscope  (TSM)  (Egger 
and  Petran,  1967;  Petran  et  al.,  1968).  In  Petran’s  microscope, 
holes  on  a  portion  of  the  spinning  disk  placed  in  front  of  the  light- 
source  collector  lens  are  imaged  onto  the  specimen  by  the  objec¬ 
tive  lens.  Each  point  of  light  reflected  or  scattered  by  the  specimen 
is  focused  by  the  same  objective  lens  back  onto  the  centrosym- 
metric  portion  of  the  Nipkow  disk.  The  pinholes  at  this  region 
exclude  the  light  originating  from  points  in  the  specimen  not  illu¬ 
minated  by  the  first  set  of  pinholes,  giving  rise  to  confocal  opera¬ 
tion  (Chapter  10,  this  volume). 

As  with  Nipkow's  initial  attempt  at  television,  this  TSM  tends 
to  suffer  from  the  low  fraction  (l%-2%)  of  light  that  is  transmitted 
through  the  source  pinholes.  Also,  very  high  mechanical  precision 
is  required  for  fabricating  the  symmetrical  Nipkow  disk  and  for 
spinning  it  exactly  on  axis.  In  addition,  some  of  the  advantages 
pointed  out  by  Minsky  for  the  stage-scanning  type  confocal  optics 
are  lost  because  the  objective  lenses  are  no  longer  focusing  a  single 
axial  point  of  light. 

However,  for  biological  applications,  the  tandem-scanning 
system  provides  the  decided  advantage  that  the  specimen  remains 
stationary.  As  a  result,  the  speed  of  the  raster  scan  is  not  limited 
by  the  mass  of  the  specimen  support  as  it  is  in  stage  scanning,  and 
the  scanning  system  is  unlikely  to  introduce  any  geometrical  dis¬ 
tortion.  Thus,  with  a  TSM,  one  can  observe  objects  that  reflect  or 
scatter  light  moderately  strongly,  in  real  time,  either  by  using  a 
television  or  photographic  camera  or  by  observing  the  image 
directly  through  the  eyepiece. 

In  addition  to  Petran  and  co-workers,  Alan  Boyde  (1985a)  in 
London  took  advantage  of  the  good  axial  discrimination  and  light- 
penetrating  capability  of  the  tandem  scanning  confocal  microscope 
and  pioneered  its  use  for  viewing  biological  objects.  In  particular, 
he  used  it  for  imaging  below  the  surface  of  hard  tissue  such  as 
bone  and  teeth  to  visualize  the  cells  and  lacunae  found  there  (see 


Lewin,  1985).  Boyde  also  provides  striking  stereoscopic  images 
obtained  with  the  tandem-scanning  confocal  microscope  (Boyde, 
1985b,  1987). 

Gordon  Kino  and  co-workers  at  Stanford  University  have 
designed  a  confocal  microscope  using  a  Nipkow  disk  in  a  manner 
that  differs  somewhat  from  the  Petran  type  (Xiao  and  Kino,  1987; 
also  see  Chapter  10,  this  volume).  In  the  Kino  type,  the  rays  illu¬ 
minating  the  specimen  and  those  scattered  by  the  specimen  tra¬ 
verse  the  same  set  of  pinholes  on  the  spinning  Nipkow  disk  rather 
than  those  that  are  centrosymmetrical.  By  using  a  special  low- 
reflection  Nipkow  disk,  tilted  somewhat  to  the  optic  axis  of  the 
microscope,  and  by  employing  crossed  polarizers  and  a  quarter- 
wave  birefringent  plate  to  further  reduce  the  spurious  reflections 
from  the  disk,  they  are  able  to  use  only  one  side  of  it,  thus  allevi¬ 
ating  some  of  the  alignment  difficulties  of  the  Petran  type. 

More  recently,  spinning-disk  confocal  units  have  been  vastly 
improved  by  adding  microlenses  to  the  pinholes.  As  described  later 
(see  “Yokogawa  Disk-Scanning  Confocal  System”  below),  the 
microlens-equipped  disk-scanning  systems  effectively  provide 
video-rate  and  faster  confocal  full-frame  imaging  and  in  real  color. 
While  in  part  depending  on  older  technology,  the  confocal- 
scanning  unit  (CSU)  systems  turn  out  to  have  certain  advantages 
not  achievable  with  point-scanning  confocal  systems. 

Electron-Beam-Scanning  Television 

While  Nipkow’s  invention  laid  the  conceptual  groundwork  for 
television,  raster  scanning  based  on  a  mechanical  device  was 
simply  too  inefficient  for  practical  television.  Thus,  it  was  not  until 
five  decades  after  Nipkow,  following  the  advent  of  vacuum  tube 
and  electronic  technology,  that  Zworykin  (1934)  and  his  col¬ 
leagues  at  RCA  were  able  to  devise  a  practical  television  system. 
These  workers  developed  the  image  iconoscope,  an  image-storage- 
type  electron-beam-scanning  image  pickup  tube.  The  image  icono¬ 
scope,  coupled  with  a  CRT  for  picture  display,  permitted  very 
rapid,  ‘‘inertialess”  switching  and  scanning  of  the  image  and 
picture  elements.  With  these  major  breakthroughs,  television  not 
only  became  practical  for  broadcasting  but  emerged  as  a  tool  that 
could  be  applied  to  microscopy  (see  Inoue  and  Spring,  1997, 
Sects.  1.1  and  1.2). 

An  early  application  of  video  (the  picture  portion  of  television) 
was  the  flying  spot  UV  microscope  of  Young  and  Roberts  (1951). 
With  this  microscope,  the  specimen  remains  stationary  and  single 
object  points  are  scanned  serially  in  a  raster  pattern  by  a  moving 
spot  of  UV  light  emitted  by  the  face  of  a  special  high-intensity  UV- 
CRT.  The  optical  elements  (condenser)  of  the  microscope  demag- 
nify  this  moving  spot  onto  the  specimen,  which  modulates  its 
brightness.  The  modulated  UV  light  is  then  picked  up  by  a  photo¬ 
tube  and  amplified  electronically  before  being  displayed  on  a 
visible-light  CRT  scanned  in  synchrony  with  the  UV-CRT. 

Young  and  Roberts  point  out  that  by  illuminating  only  a  single 
specimen  point  at  a  time  with  a  flying-spot  microscope,  flare  is 
reduced  and  the  image  becomes  a  closer  rendition  of  the  speci¬ 
men’s  optical  properties  than  that  obtained  with  a  non-scanning 
microscope.  They  also  point  out  that  for  these  same  reasons  —  and 
because  a  photoelectric  detector  can  provide  a  sequential,  linear 
output  of  the  brightness  of  each  specimen  point  —  quantitative 
analysis  becomes  possible  with  a  flying-spot  microscope.  In  addi¬ 
tion,  they  note  that  the  electronic  photodetector  raises  the  sensi¬ 
tivity  of  image  capture  by  perhaps  two  orders  of  magnitude 
compared  to  photography. 

It  should  be  noted  that  the  flare  which  would  otherwise  arise 
from  the  unilluminated  parts  of  the  specimen  is  significantly 
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reduced  with  a  flying-spot  microscope  (see  Sheppard  and 
Choudhury,  1977),  even  though  the  exit  pinhole  used  in  a  confocal 
microscope  is  not  present.  Thus,  for  example,  Wilke  et  al.  (1983) 
and  Suzuki  and  Hirokawa  (1986)  developed  laser-scanning  flying- 
spot  microscopes  (coupled  with  digital  image  processors)  to  raise 
image  contrast  (at  video  rate)  in  fluorescence,  differential- 
interference-contrast  (DIC),  and  brightfield  microscopy.  Naturally, 
the  exit  pinhole  in  a  confocal  system  is  very  much  more  effective 
at  excluding  unwanted  light  arising  from  different  layers  or  por¬ 
tions  of  the  specimen  not  currently  illuminated  by  the  source 
“pinhole,”  but  it  does  so  at  the  cost  of  reduced  image  brightness, 
lower  scanning  speed,  and  increased  instrumental  complexity  and 
price. 

While  the  flying-spot,  or  beam-scanning,  microscope  was 
developed  and  applied  in  UV  microscopy  for  about  a  decade  after 
its  introduction,  its  further  development  as  an  imaging  device  was 
eclipsed  for  some  time  by  the  need  and  the  opportunity  to  develop 
automated  microscopy  for  rapid  cell  sorting  and  diagnosis.  Here, 
the  aim  was  not  the  imaging  of  cell  structures  as  such  but  rather 
the  rapid  and  efficient  classification  of  cells  based  on  their  bio¬ 
chemical  characteristics,  taking  advantage  of  the  emerging  power 
of  high-speed  digital  computers.  The  size,  shape,  absorbance,  light 
scattering,  or  light  emission  of  cells  (labeled  with  specific  fluores¬ 
cent  markers)  was  used  either  to  classify  the  cells  by  scanning  the 
slide  under  a  microscope  or  to  sort  the  cells  at  very  high  rates  as 
the  cells  traversed  a  monitoring  laser  beam  in  a  flow  cell  or  a 
Coulter-type  cell  separator. 

Impact  of  Modern  Video 

Meanwhile,  starting  in  the  late  1970s,  the  introduction  of  new  solid 
state  devices,  especially  large-scale  integrated  circuits  and  related 
technology,  led  to  dramatic  improvements  in  the  performance 
and  availability,  and  reduction  in  price,  of  industrial-grade  video 
cameras,  video  tape  recorders,  and  display  devices.  Concurrently, 
ever  more  compact  and  powerful  digital  computers  and  image- 
processing  systems  appeared  in  rapid  succession.  These  advances 
led  to  the  birth  of  modern  video  microscopy,  which  in  turn  brought 
about  a  revitalized  interest  in  the  power  and  use  of  the  light  micro¬ 
scope  (for  reviews  see  Allen  et  al. ,  1981a, b;  Inoue,  1981,  1989; 
Allen,  1985;  Inoue  and  Spring,  1997). 

In  brief,  dynamic  structures  in  living  cells  could  now  be  visu¬ 
alized  with  a  clarity,  speed,  and  resolution  never  achieved  before 
in  DIC,  fluorescence,  polarized-light,  darkfield,  and  other  modes 
of  microscopy;  the  growth  and  shortening  of  individual  molecular 
filaments  of  tubulin  and  f-actin,  and  their  gliding  motion  and  inter¬ 
action  with  motor  molecules,  could  be  followed  in  real  time 
directly  on  the  monitor  screen;  and  the  changing  concentration  and 
distribution  of  ions  and  specific  protein  molecules  tagged  with  flu¬ 
orescent  reporter  molecules  could  be  followed,  moment  by 
moment,  in  physiologically  active  cells  (Chapters  19,  29,  and  42, 
this  volume). 

In  addition  to  its  immediate  impact  on  cellular  and  molecular 
biology,  video  microscopy  and  digital  image  processing  also  stim¬ 
ulated  the  exploration  of  other  new  approaches  in  light  microscopy 
along  several  fronts.  These  include  the  development  of  ratio 
imaging  and  new  reporter  dyes  for  quantitative  measurement  of 
local  intracellular  pH,  calcium  ion  concentration,  etc.  (Tanasugarn 
et  al.,  1984;  Bright  et  al.,  1989;  Tsien,  1989;  Chapters  16,  19,  and 
29,  this  volume );  the  computational  extraction  of  pure  optical  sec¬ 
tions  from  whole-mount  specimens  in  fluorescence  microscopy 
(based  on  deconvolution  of  multi-layered  images  utilizing  knowl¬ 
edge  of  the  microscope’s  point-spread  function;  Agard  and  Sedat, 


1983;  Agard  et  al.,  1989;  see  also  Chapters  23,  24,  and  25,  this 
volume)',  3D  imaging  including  stereoscopy  (Brakenhoff  et  al., 
1986,  1989;  Inoue  and  Inoue,  1986;  Aslund  et  al.,  1987;  Stevens 
et  al.,  1994;  Inoue  and  Spring,  1997,  Sect.  12.7.7);  and,  finally,  the 
further  development  of  laser-scanning  microscopy  and  confocal 
microscopy. 


LASERS  AND  MICROSCOPY 
Holography 

In  1960,  Maiman  announced  the  development  of  the  first  operat¬ 
ing  laser.  However,  “his  initial  paper,  which  would  have  made  his 
findings  known  in  a  more  traditional  fashion,  was  rejected  for 
publication  by  the  editors  of  Physical  Review  Letters  —  this  to 
their  everlasting  chagrin.”  (For  historic  accounts  including  this 
quotation  and  a  comprehensive  discussion  of  the  principles 
and  application  of  lasers  and  holography,  see  Sects.  14.2  and 
14.3  in  Hecht,  1987;  see  also  Chapter  5,  this  volume.)  Shortly 
thereafter,  two  types  of  applications  of  lasers  were  sought  in 
microscopy.  One  took  advantage  of  the  high  degree  of  monochro¬ 
maticity  and  the  attendant  long  coherence  length.  Coherence 
length  is  the  distance  over  which  the  laser  waves  could  be  shifted 
in  path  and  still  remain  coherent  enough  to  display  clear  inter¬ 
ference  phenomena  (note  that,  in  fact,  this  reflects  a  very  high 
degree  of  temporal  coherence).  These  characteristics  made  the 
laser  an  ideal  source  for  holography  (Leith  and  Upatnieks,  1963, 
1964). 

To  explore  the  use  of  holography  with  the  microscope,  Ellis 
(1966)  introduced  a  conventional  light  microscope  into  one  of  two 
beams  split  from  a  laser.  When  this  beam  was  combined  with  the 
other  beam  passing  outside  of  the  microscope,  the  two  beams  could 
be  made  to  interfere  in  a  plane  above  the  ocular.  The  closely  spaced 
interference  fringes  were  recorded  on  very  fine-grained  photo¬ 
graphic  film  to  produce  the  hologram. 

What  Ellis  found  was  that  the  coherence  length  of  the  laser 
beam  was  so  long  that  the  hologram  constructed  as  described 
above  could  be  viewed  not  only  to  reconstruct  an  image  of  the 
specimen  being  magnified  by  the  microscope,  but  also  to  recon¬ 
struct  images  of  the  inside  of  the  microscope.  Indeed,  in  the  holo¬ 
gram  one  could  see  the  whole  optical  train  and  interior  of  the 
microscope,  starting  with  the  substage  condenser  assembly,  the 
specimen,  the  objective  lens  and  its  back  aperture,  the  interior  of 
the  body  tube  up  to  the  ocular,  and  even  the  light  shield  placed 
above  it!  This  made  it  possible  for  Ellis  to  view  the  hologram 
through  appropriately  positioned  stops,  phase  plates,  etc.,  and  to 
generate  contrast  from  the  specimen  in  imaging  modes  such  as 
darkfield  or  oblique  illumination,  phase  contrast,  etc.,  after  the 
hologram  itself  had  been  recorded.  In  other  words,  the  state  of  the 
specimen  at  a  given  point  of  time  could  be  reconstructed  and 
viewed  after  the  fact  in  contrast  modes  different  from  the  one 
present  when  the  hologram  was  recorded. 

In  principle,  holomicrography  presents  many  intriguing  possi¬ 
bilities  including  3D  imaging.  But  the  very  virtue  of  the  long 
coherence  length  of  the  laser  beam  means  that  the  hologram  also 
registers  all  the  defects  and  dirt  in  the  microscope.  Without  laser 
illumination,  the  optical  noise  produced  by  these  defects  would  be 
far  out  of  focus.  With  a  laser  illuminating  the  whole  field  of  view 
of  the  microscope,  the  interference  fringes  from  these  out-of-focus 
defects  intrude  into  the  holographic  image  of  the  specimen  where 
they  are  prominently  superimposed.  Because  of  this  problem, 
holomicrography  has  so  far  not  been  widely  used.  [However,  see 
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Shamoff  et  al.  (1986),  who  have  figured  out  how  to  obtain  holomi- 
crograms  that  display  only  the  changes  taking  place  in  the  speci¬ 
men  (contracting  muscle  striations)  over  an  interval  of  time  and 
thus  eliminate  the  fixed-patterned  optical  noise.] 

Laser  Illumination 

Another  practical  application  of  lasers  in  microscopy  is  its  use  as 
an  intense,  monochromatic  light  source.  Lasers  can  produce  light 
beams  with  a  very  high  degree  of  monochromaticity  and  polariza¬ 
tion,  implying  a  high  degree  of  coherence.  Some  lasers  also  gen¬ 
erate  beams  with  very  high  intensity.  Thus,  an  appropriate  laser 
could  serve  as  a  valuable  light  source  in  those  modes  of 
microscopy  where  monochromaticity,  high  intensity,  and  a  high 
degree  of  coherence  and  polarization  are  important. 

To  use  the  laser  as  an  effective  light  source  for  microscopy, 
three  conditions  must  be  satisfied: 

•  Both  the  microscope’s  field  of  view  and  the  condenser  aper¬ 
ture  must  appropriately  be  filled. 

•  The  coherence  length  of  the  laser  beam  (i.e.,  the  temporal 
coherence)  must  be  reduced  to  eliminate  interference  from  out- 
of-focus  defects. 

•  The  coherence  at  the  image  plane  must  be  reduced  to  elimi¬ 
nate  laser  “speckle”  and  to  maximize  image  resolution. 

In  fact,  these  three  conditions  are  not  totally  independent,  but  they 
do  specify  the  conditions  that  must  be  met. 

One  of  the  following  five  approaches  can  be  used  to  fulfill 
these  conditions  (see  also  Chapter  6,  this  volume). 

Spinning-Disk  Scrambler 

The  laser  beam,  expanded  to  fill  the  desired  field,  is  passed  through 
a  spinning  ground-glass  diffuser  placed  in  front  of  the  beam 
expander  lens  (Hard  et  al.,  1977).  The  ground  glass  diffuses  the 
light  so  that  the  condenser  aperture  is  automatically  filled. 
However,  if  the  ground  glass  were  not  moving,  small  regions  of 
its  irregular  surface  would  act  as  coherent  scatterers  and  the  image 
field  would  still  be  filled  with  laser  speckle.  Spinning  or  vibrating 
the  ground  glass  reduces  the  temporal  coherence  of  each  of  the 
coherent  scattering  points  to  a  period  shorter  than  the  integration 
time  of  the  image  sensor.  Thus,  when  averaged  over  the  period  of 
the  motion,  the  field  also  becomes  uniformly  illuminated.  This 
approach,  while  simple  to  understand,  can  result  in  considerable 
light  loss  at  the  diffuser.  Also,  inhomogeneity  of  the  diffuser’s 
texture  can  give  rise  to  concentric  rings  of  varying  brightnesses 
which  traverse  the  field. 

Oscillating-Fiber  Scrambler 

The  laser  beam  is  focused  onto  the  entrance  end  of  a  single- 
stranded  multi-mode  optical  fiber  whose  output  end  lies  at  the  focal 
point  of  a  beam-expanding  lens.  This  lens  projects  an  enlarged 
image  of  the  fiber  tip  to  fill  the  condenser  aperture.  The  fiber,  which 
is  fixed  at  both  ends,  is  vibrated  at  some  point  along  its  length.  The 
field  and  aperture  are  then  uniformly  filled  with  incoherent  light 
with  little  loss  of  intensity  (Ellis,  1979).  If  the  fiber  were  not 
vibrated,  the  simple  fact  that  the  light  beam  is  transmitted  through 
the  fiber  could  make  the  laser  beam  highly  multi-modal.  That 
would  reduce  the  lateral  coherence  of  the  beam  at  the  aperture 
plane,  but  the  image  would  still  be  filled  with  speckle.  Vibration 
that  reduces  the  temporal  coherence  of  the  beam  below  the  inte¬ 
gration  time  of  the  image  sensor  integrates  out  the  speckle  without 
loss  of  light  (see  also  Chapter  6,  this  volume). 


Multi-Length  Fiber  Scrambler 

None  of  the  mechanical  scramblers  mentioned  above  can  be  used 
where  speckles  have  to  be  removed  within  extremely  brief  time 
periods.  For  example,  in  a  centrifuge  polarizing  microscope,  the 
laser  output  must  be  made  spatially  incoherent  within  the  few 
nanoseconds  required  to  freeze  the  image  of  the  specimen  flying 
through  the  field  of  the  objective  lens  at  speeds  up  to  lOOm/s 
(Inoue  et  al.,  2001b).  Our  solution  for  reducing  the  coherence  of 
the  laser  pulse  was  to  introduce  a  fiber  bundle  made  up  of  up  to 
100  fibers  of  multiple  lengths  between  two  multi-mode  single-fiber 
scramblers.  The  first  multi-mode  fiber  introduced  some  phase  ran¬ 
domizing  effects,  while  the  multi-length  fibers  provided  a  fiber 
bundle  output  whose  phase  varied  depending  on  the  length  of  the 
fiber.  However,  the  intensities  of  the  fiber  output  varied  depending 
on  their  location  in  the  bundle.  The  final  multi-mode  fiber  made 
the  non-uniform  brightness  of  the  bundle  output  homogeneous,  so 
that  the  microscope  condenser  received  uniform  illumination. 
Thus,  without  using  any  mechanically  moving  parts,  the  phase  of 
the  monochromatic  laser  beam  is  randomized,  and  speckles  are 
eliminated  from  the  field  image,  while  the  whole  condenser  aper¬ 
ture  is  filled  uniformly. 

Field  Scanning 

The  field  is  scanned  by  a  minute  focused  spot  (the  diffraction 
image)  of  a  single-mode  laser  beam  that  has  been  expanded  to  fill 
the  condenser  aperture  (as  in  a  laser-scanning  confocal  micro¬ 
scope).  Thus,  the  specimen  is  scanned  point  by  point,  and  the 
signal  light  reflected,  transmitted,  or  emitted  by  the  specimen  is 
collected  and  focused  by  the  objective  lens.  This  imaging  mode 
avoids  the  generation  of  speckle  from  laser-illuminated  specimens 
because  speckle  arises  from  the  interference  between  the  coherent 
light  waves  scattered  from  different  parts  of  a  specimen.  (This 
optical  setup  is  less  effective  at  removing  speckle  when  a  smooth 
reflecting  surface  is  presented  slightly  away  from  the  plane  of 
focus.) 

This  fourth  approach  leads  to  field-scanning  microscopy.  A 
focused  spot  of  laser  light  can  be  made  to  scan  the  field  as  in  a 
flying-spot  microscope,  or  the  specimen  can  be  moved  and 
scanned  through  a  fixed  focus  point.  Alternatively,  an  exit  pinhole 
and  beam  scanners  can  be  added  to  generate  a  laser-scanning  con- 
focal  microscope. 

Aperture  Scanning 

The  minute  diffraction  image  of  a  single-mode  laser  is  focused  by 
a  beam  expander  onto  an  off-axis  point  on  the  condenser  aperture. 
The  small  spot  is  scanned  (made  to  precess)  over  the  condenser 
aperture  in  such  a  way  that  the  field  is  uniformly  illuminated.  At 
any  instant  of  time,  the  specimen  is  illuminated  by  a  tilted  colli¬ 
mated  beam  of  light  emerging  from  the  condenser  and  originating 
from  the  illuminated  aperture  point.  Selected  regions  of  the  aper¬ 
ture  are  filled  in  rapid  succession  by  scanning  the  spot,  so  that  the 
whole  field  is  illuminated  by  collimated,  coherent  beams  at  suc¬ 
cessively  changing  azimuth  angles.  The  rapid  scanning  of  the 
source  reduces  the  temporal  coherence  of  illumination  at  the  object 
plane  to  less  than  the  response  time  of  the  image  detector.  Never¬ 
theless,  the  lateral  coherence  is  maintained  for  each  instantaneous 
beam  that  illuminates  the  specimen  (Ellis,  1988). 

Ellis  has  argued  the  theoretical  advantage  provided  by  this  fifth 
approach  and  has  demonstrated  its  practical  attractiveness.  With 
aperture  scanning,  one  gains  new  degrees  of  freedom  for  optical 
image  processing  because  the  aperture  function  (which  controls 
the  image  transfer  function  of  the  microscope)  can  be  regulated 
dynamically  for  each  point  of  the  aperture.  The  image  resolution 
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and  the  shallow  depth  of  field  that  can  be  achieved  with  aperture¬ 
scanning  phase-contrast  microscopy  is  most  impressive  (see,  e.g., 
Inoue  and  Spring,  1997,  Fig.  2-47). 

Laser-Illuminated  Confocal  Microscopes 

During  the  early  1970s,  Egger  and  co-workers  at  Yale  University 
developed  a  laser-illuminated  confocal  microscope  in  which  the 
objective  lens  was  oscillated  in  order  to  scan  the  beam  over  the 
specimen.  Davidovits  and  Egger  obtained  a  U.S.  patent  on  this 
microscope  (1972;  see  review  by  Egger,  1989). 

A  few  years  later,  Sheppard  and  Choudhury  (1977)  provided 
a  thorough  theoretical  analysis  on  various  modes  of  confocal  and 
laser-scanning  microscopy.  The  following  year,  Sheppard  et  al. 
(1978)  and  Wilson  et  al.  (1980)  described  an  epi-illuminating  con¬ 
focal  microscope  of  the  stage-scanning  type,  equipped  with  a  laser 
source  and  a  photomultiplier  tube  (PMT)  as  the  detector,  using  a 
novel  specimen  holder.  The  specimen  holder,  supported  on  four 
taut  steel  wires  running  parallel  to  the  optical  axis,  allowed  precise 
z-axis  positioning  as  well  as  fairly  rapid  voice-coil-actuated 
scanning  of  the  specimen  in  the  xy-plane.  Using  this  instrument, 
Sheppard  et  al.  demonstrated  the  value  of  the  confocal  system 
particularly  for  examining  integrated  circuit  chips.  With  stage¬ 
scanning  confocal  imaging,  optical  sections  and  profile  images 
could  be  displayed  on  a  slow-scan  monitor  over  areas  very  much 
larger  than  can  be  contained  within  the  field  of  view  of  any  given 
objective  lens  by  conventional  microscopy. 

These  authors  capitalized  on  the  fact  that  the  confocal  signal 
falls  off  extremely  sharply  with  depth,  and  the  image  is  therefore 
completely  dark  for  regions  of  the  specimen  that  are  not  near  the 
confocal  focus  plane.  For  example,  with  a  tilted  integrated  circuit 
chip,  only  the  portion  of  the  surface  within  the  shallow  depth  of 
field  (at  any  selected  z-value)  could  be  displayed,  as  a  strip-shaped 
region  elongated  parallel  to  the  chip’s  axis  of  tilt.  Other  areas  of 
the  image  were  dark  and  devoid  of  structure.  Conversely,  by  com¬ 
bining  all  the  xy-scan  images  made  during  a  slow  z-scan,  they 
could  produce  a  final  “extended  focus”  image  of  the  whole  tilted 
surface,  which  demonstrated  maximum  spatial  resolution  on  all 
features  throughout  the  focus  range  (Wilson  and  Sheppard,  1984; 
Wilson,  1985)  (Chapter  22,  this  volume). 

This  could  be  done  even  when  the  specimen  surface  was  not 
a  single  tilted  plane  but  was  wavy  or  consisted  of  complex  sur¬ 
faces.  In  their  monograph  Scanning  Optical  Microscopy,  Wilson 
and  Sheppard  (1984)  show  shallow  optical  sections  of  insect  anten¬ 
nae  shining  on  a  dark  background.  They  also  show  stereo-pair 
images  of  the  same  object  consisting  of  two  “extended  focus” 
images  made  by  focusing  along  two  focal  axes  that  were  tilted  by 
several  degrees  relative  to  the  optical  axis.  Extended-focus  images 
demonstrate  that  the  confocal  system  can  either  decrease  or 
increase  the  effective  depth  of  field  without  loss  of  resolution. 

As  described  in  the  final  section  of  this  article,  the  lateral  res¬ 
olution  that  is  practically  attainable  can  be  improved  by  using  con¬ 
focal  optics.  In  addition,  the  removal  of  the  extraneous  light 
contributed  by  out-of-focus  objects  dramatically  improves  the  con¬ 
trast  and  gives  rise  to  a  brilliantly  sharp  image. 

Sheppard  et  al.  also  managed  to  display  different  regions  on 
the  surface  of  an  integrated  circuit  chip  with  varying  intensity  or 
pseudocolor  corresponding  to  the  height  of  the  region.  This  is  pos¬ 
sible  because  the  amount  of  light  reflected  by  an  (untilted)  step  on 
the  surface  of  the  chip  and  passing  the  second  pinhole  varies  with 
the  distance  of  the  reflecting  surface  from  the  focal  plane.  The 
authors  also  showed  that,  by  processing  the  photoelectric  signal 
electronically,  the  edges  of  the  steps  alone  could  be  outlined  or  the 


gradient  of  the  steps  could  be  displayed  in  a  DIC-like  image 
(Hamilton  and  Wilson,  1984).  [For  the  basics  of  digital  image  pro¬ 
cessing,  see  Castleman  (1979),  Baxes  (1984),  Gonzales  and  Wintz 
(1987),  Chapter  12  in  Inoue  and  Spring  (1997),  and  Chapter  14, 
this  volume .] 

The  integrated  circuit  chip  could  also  be  displayed  with  con¬ 
trast  reflecting  the  status  of  the  local  circuit  elements,  for  example, 
reflecting  its  temperature  or  the  amount  of  photo-induced  current 
flowing  through  the  circuit,  superimposed  on  the  confocal  image 
of  the  chip  made  with  reflected  light  (Wilson  and  Sheppard,  1984). 

In  addition  to  the  Oxford  group,  the  brothers  Cremer  and 
Cremer  (1978)  of  Heidelberg  designed  a  specimen-scanning  laser- 
illuminated  confocal  microscope.  This  epi-fluorescence  system 
was  equipped  with  (1)  a  circular  exit  pinhole,  in  front  of  the  first 
PMT,  whose  diameter  was  equal  to  the  principal  maximum  of 
the  diffraction  pattern;  and  (2)  an  annular  aperture,  in  front  of  a 
second  PMT,  whose  opening  corresponded  to  the  first  subsidiary 
maximum  of  the  diffraction  pattern.  The  output  of  the  two  PMTs 
was  used  to  provide  autofocus  as  well  as  displays  of  surface 
contour  and  fluorescent  intensity  distribution. 

In  the  1978  article,  the  Cremers  also  discussed  the  possibility 
of  laser  spot  illumination  using  a  “4jt-point  hologram”  that  could, 
at  least  in  principle,  provide  long  working  distance  relative  to  the 
small  spot  size  that  could  be  produced. 

CONFOCAL  LASER-SCANNING  MICROSCOPE 

In  addition  to  those  already  mentioned,  the  pioneering  work  of  the 
Oxford  electrical  engineering  group  was  followed  in  several  Euro¬ 
pean  laboratories  by  Brakenhoff  et  al.  (1979,  1985),  Wijnaendts 
van  Resandt  et  al.  (1985),  and  Carlsson  et  al.  (1985).  These  inves¬ 
tigators  respectively  developed  the  stage-scanning  confocal 
microscope  further,  verified  the  theory  of  confocal  imaging,  and 
expanded  its  application  into  cell  biology.  I  shall  defer  further  dis¬ 
cussions  on  these  important  contributions  to  authors  of  other  chap¬ 
ters  in  this  volume.  In  the  meantime,  video  microscopy  and  digital 
image  processing  were  also  advancing  at  a  rapid  rate. 

These  circumstances  culminated  in  the  development  of  the 
confocal  laser-scanning  microscope  (CLSM,  Figs.  1.4,  1.5;  Aslund 
et  al.,  1983,  1987)  and  publication  of  its  biological  application  by 
Carlsson  et  al.  (1985),  Amos  et  al.  (1987),  and  White  et  al.  (1987). 
The  publications  were  followed  shortly  by  introduction  of  laser¬ 
scanning  confocal  microscopes  to  the  market  by  Sarastro,  Bio- 
Rad,  Olympus,  Zeiss,  and  Leitz.  It  was  White,  Amos,  and  Fordham 
of  the  Cambridge  group  that  first  enraptured  the  world’s  biologi¬ 
cal  community  with  their  exquisite  and  convincing  illustrations  of 
the  power  of  the  CLSM.  Here  at  last  was  a  microscope  that  could 
generate  clear,  thin  optical  sectioned  images,  totally  free  of  out-of- 
focus  fluorescence,  from  whole  embryos  or  cells  and  at  NAs  as 
high  as  1.4.  Not  only  could  one  obtain  such  remarkable  optical- 
sectioned  fluorescence  images  in  a  matter  of  seconds,  but  x-z  sec¬ 
tions  (providing  views  at  right  angles  to  the  normal  direction  of 
observation)  could  also  be  captured  and  rapidly  displayed  on  the 
monitor.  A  series  of  optical  sections  (stored  in  the  memory  of  the 
built-in  or  add-on  digital  image  processor)  could  be  converted  into 
3D  images  or  displayed  as  stereo  pairs.  The  confocal  fluorescent 
optical  sections  could  also  be  displayed  side  by  side  with  non- 
confocal  brightfield  or  phase-contrast  images,  acquired  concur¬ 
rently  using  the  transmitted  portion  of  the  scanning  laser  beam. 
These  images  could  also  be  displayed  superimposed  on  top  of  each 
other,  for  example,  with  each  image  coded  in  different  pseudo¬ 
color,  but  unlike  similar  image  pairs  produced  by  conventional 
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FIGURE  1 .4.  Schematic  of  laser-scanning  confocal  micro¬ 
scope.  (From  Aslund  el  al.,  1987.) 


microscopes,  the  two  images  were  in  exact  register  and  showed  no 
parallax  as  each  was  generated  by  the  same  scanning  spot. 

Most  of  the  laser-scanning  systems  discussed  in  this  section 
employed  epi-illunrination  using  some  form  of  mechanical  scan¬ 
ning  devices.  They  could  not  readily  be  applied  to  confocal 
imaging  of  transmitted  light,  for  example,  for  high-extinction 
polarization  or  DIC  microscopy.  Nevertheless,  Goldstein  et  al. 
(1990)  developed  a  system  using  an  Image  Dissector  Tube  which, 
in  principle,  should  be  able  to  provide  confocal  imaging  in  the 
trans-illumination  mode.  Such  an  approach  may  eventually  lead  to 
workable  transmission  laser-scanning  confocal  microscopes  with 
multiple  contrast  modes. 

TWO-  AND  MULTI-PHOTON  MICROSCOPY 

As  noted,  conventional  point-scanning  confocal  microscopes  dra¬ 
matically  reduce  the  contribution  of  fluorescence  from  out-of¬ 
focus  regions  of  the  specimen.  Nevertheless,  regions  of  the 
specimen  above  and  below  the  focal  plane  are  exposed  to  the 


Laser 


FIGURE  1.5.  Depth  discrimination  in  a  laser-scanning  confocal  fluorescent 
microscope.  Compare  with  Figure  1.2.  (Courtesy  of  Dr.  H.  Kapitza,  Carl  Zeiss, 
Oberkochen.) 


full  cones  of  intense  excitation  light,  converging  and  diverging 
from  the  illuminated  spot.  Thus,  with  conventional  confocal 
microscopy,  biological  specimens  tend  to  suffer  from  photon- 
induced  damage  and  rapid  bleaching  of  fluorescence,  while  the 
fraction  of  the  short  wavelength  excitation  beam  that  reaches  the 
focal  plane  is  reduced  by  absorption  in  the  intervening  material. 
Many  of  these  shortcomings  are  circumvented  by  two-  and  multi¬ 
photon  microscopy. 

By  focusing  a  pulse  of  very  intense  laser  beam  with  twice  the 
wavelength  (half  the  frequency)  of  the  standard  short  wavelength 
excitation  beam,  and  within  a  period  shorter  than  the  fluorescence 
decay  time  of  the  fluorophore,  the  coherently  interfering  photons 
can  excite  molecules  at  half  the  wavelength  of  the  long  wavelength 
laser,  and  do  so  selectively  in  the  focused  spot.  In  other  words,  the 
output  of  an  intense  near-infrared  (IR)  laser  induces  fluorescence 
in  a  blue  or  UV  excitable  fluorophore  at,  and  only  at,  the  focused 
spot  where  the  coherent  electromagnetic  field  strength  is  so  high 
(within  the  required  brief  period)  that  it  acts  nonlinearly  to  excite 
the  chromophores  at  twice  the  frequency  of  the  IR  field.  The  flu- 
orophores  in  the  cone  of  the  illuminating  light  above  and  below 
focus  do  not  experience  the  two-photon  effect  and,  therefore,  are 
not  excited  or  damaged.  Additionally,  in  contrast  to  conventional 
confocal  microscopy,  two-photon  laser  scanning  systems  do  not 
require  an  exit  pinhole  or  an  image-forming  objective  lens.  This  is 
because  the  minute  two-photon  excited  fluorescent  spot  is  totally 
isolated  in  space  and  is  free  of  “parasitic”  fluorescence  in  the  xy- 
plane  as  well  as  along  the  z-axis.  Therefore,  the  fluorescence  emis¬ 
sion  needs  only  to  be  collected  by  an  efficient  photodetector  as  the 
excitation  spot  is  being  scanned  (see  Chapter  28,  this  volume). 

Thus,  compared  to  conventional  confocal  microscopy,  two- 
photon  microscopy  permits  confocal  imaging  of  planes  much 
deeper  in  the  tissue,  and  with  considerably  higher  light-gathering 
efficiency,  as  well  as  with  less  fluorescence  bleaching  and  speci¬ 
men  damage  outside  of  the  focal  plane.  Denk  et  al.  (1990)  and 
Squirrel  et  al.  (1999)  have  made  extended  time-lapse  recordings 
of  dividing  tissue-cultured  cells  and  mammalian  embryos  in  two- 
photon  microscopy. 

In  another  interesting  and  ingenious  application  of  two-photon 
microscopy,  the  fluorescence  excitation  volume  (point-spread 
function)  has  been  reduced  considerably  below  that  defined  by 
wave  optics  by  use  of  two  partially  overlapping  excitation 
volumes.  The  first  excites  fluorescence  in  the  standard  two-photon 
volume,  while  the  second  volume,  concurrently  generated  by  a 
somewhat  longer  wavelength,  quenches  the  fluorescence  (by  stim- 
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ulated  emission  depletion)  in  the  zone  where  the  two  volumes 
overlap  each  other.  A  phase  plate  in  the  depletion  beam  path  has 
the  effect  that  this  beam  is  almost  the  inverse  of  the  Airy  disk  and 
has  a  null  at  the  focus.  Thus,  the  volume  of  region  actually  fluo¬ 
rescing  is  carved  smaller  than  the  standard  two-photon  excitation 
volume,  and,  in  fact,  Hell  and  Wichmann  (1994)  report  having 
reduced  the  height  of  the  point-spread  function  (PSF)  by  as  much 
as  a  factor  of  five  (see  also  Klar  et  al.,  2000,  Chapter  31,  this 
volume). 

IS  LASER-SCANNING  CONFOCAL 
MICROSCOPY  A  CURE-ALL? 

With  the  impressively  thin  and  clean  optical  sections  that  are 
obtainable,  and  the  x-z  sections  and  stereoscopic  images  that  can 
neatly  be  displayed  or  reconstructed,  one  can  be  tempted  to  treat 
the  CLSM  as  a  cure-all.  One  may  even  think  of  the  instrument  as 
the  single  microscope  that  should  be  used  for  all  modern  cell 
biology  or  embryology.  How  valid  is  such  a  statement  and  what, 
in  fact,  are  the  limitations  of  the  current  instruments  beyond  their 
high  costs? 

The  fundamental  limits  of  confocal  imaging  will  be  covered 
in  the  next  chapter.  Here  I  will  comment  on  three  topics:  the  speed 
of  image  or  data  acquisition,  comparison  with  the  depth  of  field 
in  phase-dependent  imaging,  and  some  optical  and  mechanical 
factors  affecting  confocal  microscopy. 

Speed  of  Image  or  Data  Acquisition 

Several  factors  affect  the  time  needed  to  acquire  a  usable  image 
with  a  confocal  microscope.  These  include  (1)  the  type  of  confo¬ 
cal  system  used;  (2)  the  optical  magnification  and  numerical  aper¬ 
ture  of  the  system;  (3)  the  desired  area  covered;  (4)  required  quality 
of  the  image  (e.g.,  lateral  and  axial  resolution,  levels  of  image  gray 
scale,  degree  of  freedom  from  graininess);  and  (5)  the  amount  of 
light  reaching  the  sensor.  Here  we  will  survey  a  few  general  points 
relating  to  the  choice  of  instruments,  specifically  as  applied  to 
biology. 

Among  the  different  confocal  systems,  the  stage-scanning  type 
requires  the  longest  time  (~10s)  to  acquire  a  single  image  because 
the  specimen  support  has  to  be  translated  (vibrated)  very  precisely. 
Biological  specimens  are  often  bathed  in  a  liquid  medium,  and  for 
these,  any  movement  presents  a  problem.  Even  if  the  specimen 
chamber  is  completely  sealed  and  the  gas  phase  excluded  to  min¬ 
imize  the  inertial  effects  of  stage  scanning,  specimen  motion  still 
can  occur  during  stage  scanning.  The  alternative  lens-scanning 
system  can  encounter  worse  problems  when  oil-immersion  lenses 
are  used.  Very  often  structures  in  biological  specimens  are  moving 
or  changing  dynamically  at  rates  incompatible  with  very  slow  scan 
rates.  Thus,  despite  the  many  virtues  of  the  stage-scanning  system 
recognized  by  Minsky  (1957)  and  by  Wilson  and  Sheppard  (1984), 
there  is  little  chance  that  the  stage-scanning  microscope  will  be 
widely  used  in  biology.  An  exception  might  be  for  large-area  3D 
scanning  of  fixed  and  permanently  mounted  specimens.  Such  spec¬ 
imens  require,  or  can  take  advantage  of,  those  virtues  of  the  stage¬ 
scanning  system  that  cannot  be  duplicated  by  other  confocal 
designs. 

In  the  Petran-type  TSM  or  the  Kino-type  confocal  microscope, 
the  disk  can  be  spun  rapidly  enough  to  provide  images  at  video 
rate  (30  frames/s).  When  speed  of  image  acquisition  is  of  para¬ 
mount  importance,  as  in  the  study  of  moving  cells,  living  cells  at 
high  magnification,  or  microtubules  growing  in  vitro ,  the  type  of 


speed  provided  by  the  Nipkow  disk  system  may  be  indispensable. 
For  example,  at  the  ~10,000x  magnification  needed  for  clear  visu¬ 
alization,  the  Brownian  motion  of  microtubules  (even  those  many 
micrometers  long)  is  so  great  that  an  image  acquisition  time  of 
>0.1s  blurs  the  image  beyond  use. 

As  discussed  earlier,  the  downside  of  the  classical  Nipkow 
disk-type  system  is  that  the  efficiency  of  light  transmission  is  low, 
light  reflected  by  the  spinning  disk  reduces  image  contrast,  and  the 
image  may  suffer  from  intrusive  scan  lines.  Also,  observation  is 
usually  by  direct  viewing  through  the  ocular,  or  via  some  photo¬ 
graphic  or  video  imaging  device,  rather  than  using  a  PMT.  While 
video  imaging  does  have  its  own  advantages,  video  sensors  other 
than  cooled  charge-coupled  devices  (CCDs)  and  special  return- 
beam-type  pickup  tubes  operate  over  a  limited  dynamic  range. 
Conventional  video  pickup  tubes  seldom  respond  linearly  over  a 
range  of  >100: 1  (more  commonly  somewhat  less;  see  Inoue  and 
Spring,  1997),  and  they  have  relatively  high  measurement  noise. 
By  contrast,  a  PMT  can  have  a  dynamic  range  of  >106.  When 
exceedingly  weak  signals  need  to  be  detected  from  among  strong 
signals,  or  when  image  photometry  demands  dynamic  range  and 
precision  beyond  those  attainable  with  standard  video  cameras,  an 
imaging  system  using  a  cooled  CCD  or  a  PMT  detector  may  be 
required.  Modem  stage-scanning-  and  laser-scanning-type  con¬ 
focal  microscopes  use  such  detectors  (Chapter  12,  this  volume). 
Nevertheless,  for  some  applications  improved  versions  of  the 
Nipkow-disk-type  confocal  instruments  may  provide  optical  sec¬ 
tions  with  better  signal  and  image  quality  than  with  CLSMs  as  dis¬ 
cussed  below  under  “Yokogawa  Disk-Scanning  Confocal  System.” 

The  frame-scanning  rate  of  the  CLSM  falls  somewhere 
between  that  of  the  stage-  and  tandem- scanning  types,  normally 
about  1  to  2  s/frames.  This  rate  is  the  minimum  time  required  by 
the  mirror  galvanometers  (that  are  used  to  scan  the  illuminating 
and  return  beams)  to  produce  an  image  of,  say,  512  x  768  picture 
elements.  This  limitation  in  scanning  speed  relates  to  the  absolute 
time  required  to  scan  along  the  fastest  axis  (usually  the  x,  or  hor¬ 
izontal,  scan).  The  scanning  speed  cannot  be  increased  without 
affecting  image  resolution  or  confocal  discrimination  (Chapters  3, 
21,  and  25,  this  volume). 

The  .r-scanning  speed  can  be  increased  by  using  a  resonance 
galvanometer,  a  spinning  mirror,  or  an  acousto-optical  modulator 
instead  of  the  mirror  galvanometers  (Chapters  3,  9,  and  29,  this 
volume).  However,  doing  so  may  reduce  both  scan  flexibility  (i.e., 
no  optical  “zoom”  magnification)  and  inefficient  use  of  the  duty 
cycle.  Furthermore,  in  a  scanning  confocal  system  used  for  fluo¬ 
rescence  microscopy,  one  cannot  use  the  same  acousto-optical 
device  (or  other  diffraction-based  electro-optical  modulator)  to 
both  scan  the  exciting  beam  and  de-scan  the  emitted  beam  because 
the  modulator  would  deviate  the  two  beams  by  different  amounts 
based  on  their  X. 

Of  even  greater  importance,  the  image  captured  by  a  CLSM 
in  a  single,  1-  to  2-s  scan  time  is  commonly  too  noisy  because  the 
image-forming  signal  is  simply  not  made  up  of  enough  photons. 
The  image  generally  must  be  integrated  electronically  over  several 
frame  times  to  reduce  the  noise,  just  as  when  one  is  using  a  high- 
sensitivity  video  camera.  Thus,  with  a  CLSM,  it  often  requires 
several,  or  many,  seconds  to  acquire  a  well-resolved,  high-quality 
fluorescence  image. 

If,  in  an  attempt  to  reduce  the  number  of  frames  that  must  be 
integrated,  one  tries  to  increase  the  signal  reaching  the  PMT  by 
raising  the  source  brightness,  by  opening  up  the  exit  pinhole,  or 
by  increasing  the  concentration  of  fluorochrome,  each  alteration 
introduces  new  problems  of  its  own.  In  fact,  in  CLSMs  used  for 
fluorescence  imaging,  if  anything,  one  wants  to  reduce  the  light 
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reaching  the  specimen  in  order  to  avoid  saturation  of  the  fluo- 
rophores,  significant  bleaching,  and  other  excitation-induced 
damage.  There  is  almost  an  indeterminacy  principle  operating 
here:  One  simply  cannot  simultaneously  achieve  high  temporal 
resolution,  high  spatial  resolution,  large  pixel  numbers,  and  a  wide 
gray  scale  simultaneously.  This  speed  limitation  must  be  seen  as  a 
disadvantage  of  the  CLSM. 

As  already  discussed,  two-photon  confocal  fluorescence  micro¬ 
scopy  (Chapter  28,  this  volume)  is  a  promising  new  approach 
that  may  reduce  the  effect  of  some  of  these  limitations  in  addition 
to  providing  excellent  lateral  and  axial  resolution.  However, 
because  the  time  between  pulses  is  long  ( 1 0—  12  s)  compared  to 
the  fluorescence  time  of  organic  dyes,  it  only  produces  signal  10% 
to  20%  of  the  time.  This  low-duty  cycle  exacerbates  the  data  rate 
limit. 

While  the  sampling  rate  for  obtaining  whole  images  with  the 
CLSM  is  limited,  this  does  not  imply  that  the  temporal  resolution 
of  the  detector  system  is  inherently  low.  For  example,  one  can 
measure  relatively  high-speed  events  with  the  CLSM,  if  one 
decides  to  sacrifice  pixel  numbers  by  reducing  the  size  of  the 
scanned  area  or  even  by  using  a  single,  or  a  few,  line  scan(s).  In 
addition  to  the  high  temporal  resolution,  the  bleaching  of  diffusible 
fluorochromes  and  photodynamic  damage  to  the  cell  are  reported 
to  be  significantly  reduced  when  the  scan  is  restricted  to  a  single 
line  (Chapter  19,  this  volume). 

Another  alternative  for  gaining  speed  is  to  use  a  slit  instead  of 
a  pinhole  for  confocal  scanning.  This  approach,  although  some¬ 
what  less  effective  than  confocal  imaging  with  small  round  pin¬ 
holes,  is  surprisingly  effective  in  suppressing  the  contribution 
of  out-of-focus  features.  Several  manufacturers  have  produced 
laser-illuminated,  slit-scanning  confocal  microscopes  that  provide 
video-rate  or  direct-view  imaging  systems  that  are  quite  easy  to 
operate,  at  a  fraction  of  the  price  of  the  normal  CLSM.  However, 
the  rapid  bleaching  of  fluorescent  dyes  encountered  with  the  slit¬ 
scanning  system  has  been  a  disappointment  for  those  hoping  to 
gain  confocal  scanning  speed  for  studies  on  living  cells. 

Yokogawa  Disk-Scanning  Confocal  System 

A  new  confocal  disk-scanning  unit  (CSU-10  and  CSU-21) 
designed  by  Yokogawa  Electric  Corporation  provides  video-rate 
and  faster  confocal  imaging  with  several  advantages  while  over¬ 
coming  the  two  major  factors  that  had  limited  earlier  TSM 
systems.  The  new  system  uses  two  Nipkow-type  disks  located  one 
above  the  other  with  precisely  aligned  perforations.  In  place  of  pin¬ 
holes,  the  first  disk  contains  some  10,000  microlenses,  each  of 
which  focuses  the  collimated  laser  beam  onto  a  corresponding 
pinhole  on  the  second  disk.  The  microlenses  increase  the  through¬ 
put  of  excitation  laser  from  a  scant  1%  to  2%  of  conventional 
Nipkow  disks  to  nearly  50%.  At  the  same  time,  a  dichromatic  filter 
cube  is  placed  between  the  two  disks,  so  that  light  reflected  or  scat¬ 
tered  from  the  initial  disk  no  longer  contributes  unwanted  back¬ 
ground  to  the  fluorescence  signal  received  by  the  detector  (Fig. 
1.6).  These  confocal  scanning  units  can  be  attached  to  any  upright 
or  inverted  research-grade  light  microscope.  The  1000  or  so  pin¬ 
holes  that  scan  the  specimen  in  parallel  at  any  instant  of  time  are 
arranged  in  a  unique  geometrical  pattern.  The  unique  pattern 
reduces  image  streaking  (found  with  conventional  Nipkow  disks) 
and  provides  uniform  illumination  of  the  whole  field  of  view 
(Inoue  and  Inoue,  2002). 

With  any  multiple-pinhole-  (or  slit-)  scanning  system,  some 
light  originating  from  outside  the  focal  plane  is  transmitted  through 
“neighboring”  pinholes,  so  that  focal  discrimination  is  not  as  effec- 
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FIGURE  1.6.  Schematic  of  optics  in  the  CSU-10.  The  expanded  and  colli¬ 
mated  laser  beam  illuminates  the  active  portion  of  the  upper  Nipkow  disk  con¬ 
taining  some  20,000  microlenses.  Each  microlens  focuses  the  laser  beam 
(through  the  dichromatic  filter  cube)  onto  its  corresponding  pinhole,  thus  sig¬ 
nificantly  raising  the  fraction  of  the  illuminating  beam  that  is  transmitted  by 
the  main  Nipkow  disk  containing  the  pinhole  array.  From  the  pinholes,  the 
beams  progress  down  to  fill  the  aperture  of  the  objective  lens.  The  objective 
lens  generates  a  reduced  image  of  the  pinholes  in  the  focal  plane  of  the 
specimen. 

Fluorescence  given  off  by  the  illuminated  points  in  the  specimen  is  cap¬ 
tured  by  the  objective  lens  and  focused  back  onto  the  Nipkow  disk  containing 
the  pinhole  array.  Each  pinhole  now  acts  as  its  own  confocal  exit  pinhole  and 
eliminates  fluorescence  from  out-of-focus  regions,  thus  selectively  transmitting 
fluorescence  that  originated  from  the  specimen  points  illuminated  by  that  par¬ 
ticular  pinhole.  (However,  for  specimens  with  fluorescence  distributed  over 
large  depths,  some  out-of-focus  fluorescence  can  leak  through  adjacent  pin¬ 
holes  in  multiple  pinhole  systems  such  as  the  CSU-10;  see  text.)  The  rays  trans¬ 
mitted  by  the  exit  pinholes  are  deflected  by  the  dichromatic  beam  splitter, 
located  between  the  two  Nipkow  disks,  and  proceed  to  the  image  plane.  (Figure 
courtesy  of  Yokogawa  Electric  Corporation.) 


tive  as,  or  the  confocal  stringency  does  not  match  that  of,  a  point¬ 
scanning  confocal  system.  Nevertheless,  the  Yokogawa  system 
provides  very  effective  focal  discrimination  and  capability  for  pro¬ 
viding  striking  optical  sections  in  real  time,  either  for  direct  obser¬ 
vation  through  the  eyepiece  or  captured  on  a  video  or  photographic 
camera  or  a  CCD.  The  residual  out-of-focus  contribution  can  be 
rapidly  and  effectively  reduced  by  “unsharp  masking”  or  "neigh¬ 
borhood  deconvolution”  digital  processing.  (Several  examples  of 
the  dynamic  cellular  changes  captured  with  the  CSU-10,  as  well 
as  the  effectiveness  of  postprocessing,  are  illustrated  in  Inoue  and 
Inoue,  2002.  See  also  Chapter  10,  this  volume.) 

In  addition  to  the  effective,  real-time  and  faster-than- video-rate 
confocal  fluorescence  imaging  in  real  color  (which  can  be  viewed 
superimposed  with  the  brightfield  or  DIC  image  of  the  specimen), 
several  observers  have  been  impressed  by  the  significantly  slower 
fluorescence  bleaching  rate  and  much  longer  survival  time  for 
living  cells  observed  with  the  CSU-10  (coupled  to  low-noise  CCD 
or  video  cameras)  compared  to  imaging  of  the  same  objects  with 
point-scanning  confocal  systems.  For  example,  CSU-10  imaging 
was  found  indispensable  for  capturing  the  dynamic  growth, 
motion,  and  gliding  of  GFP-expressing  microtubules  in  yeast  cells 
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as  well  as  speckle  images  of  tubulin  flux  in  pTk-1  tissue  cells  and 
in  the  thick  spindles  undergoing  mitosis  in  Xenopus  egg  extracts 
(Waterman-Storer  and  Salmon,  1997;  Grego  et  al.,  2001;  Tran  et 
al.,  2001;  Maddox  et  a /.,  2002  ).  The  reasons  for  low-fluorescence 
bleaching  and  extended  cell  survival  are  discussed  in  Inoue  and 
Inoue  (2002  and  in  Chapter  38,  this  volume.) 

The  advantage  of  the  real-time,  direct-view  confocal  system 
extends  beyond  capturing  sharp  images  of  moving  or  dynamically 
changing  objects,  whose  images  would  be  blurred  or  distorted  by 
the  slow  frame-capture  rate  of  conventional  CLSMs.  For  example, 
with  a  CLSM  it  is  difficult  to  visualize,  or  even  to  find,  minute  flu¬ 
orescent  objects  that  are  sparsely  distributed  in  three  dimensions. 
With  an  image  intensifier  CCD  camera  coupled  to  a  direct-view 
system,  the  signal  from  such  sparsely  distributed  objects  is  readily 
found  in  real  time  as  one  focuses  through  the  specimen. 

Depth  of  Field  in  Phase-Dependent  Imaging 

The  z-axis  resolution  measured  in  epi-fluorescence  imaging  with 
a  confocal  laser  scanning  microscope  is  reported  to  be  1.5  |im  with 
an  NA  0.75  objective  lens  (Cox  and  Sheppard,  1993)  and  0.48  pm 
with  an  NA  1.3  objective  lens  (Hell  et  al.,  1993)  at  a  wavelength 
of  5 14  nm.  Kino  reports  a  depth  of  field  of  0.35  pm  for  NA  1 .4  con¬ 
focal  optics,  when  imaging  point-like  reflecting  objects.  These 
numbers  are  in  good  agreement  with  Eqs.  2  and  3,  and  the  height 
of  the  3D  diffraction  pattern  of  a  point  object  discussed  earlier. 
In  addition,  Stephan  Hell,  as  described  above,  has  achieved  even 
shallower  field  depths  by  superimposing  two  3D  diffraction  spots 
of  differing  wavelengths  in  stimulated  depletion  point¬ 
scanning  confocal  microscopy. 

How  do  these  shallow  depth  of  fields  attainable  with  a  confo¬ 
cal  microscope  compare  with  those  obtainable  in  the  absence  of 
confocal  imaging?  While  I  could  come  up  with  no  hard  numbers 
for  fluorescence  microscopy  without  confocal  imaging  (except 
where  3D  deconvolution  is  employed,  see  Chapters  23,  24,  and  25, 
this  volume),  it  is  well  known  that  the  fluorescence  from  out-of¬ 
focus  objects  substantially  blurs  the  in-focus  image.  On  the  other 
hand,  for  contrast  generated  by  phase-dependent  methods  such  as 
phase-contrast,  DIC,  and  polarized-light  microscopy,  Gordon  Ellis 
and  I  have  obtained  data  that  show  remarkably  thin  optical  sec¬ 
tions  in  the  absence  of  confocal  imaging. 

Thus,  using  a  lOOx  NA  1.4  Nikon  PlanApo  objective  lens, 
combined  with  an  NA  1.35  rectified  condenser  whose  full  aperture 
was  uniformly  illuminated  through  a  light  scrambler  with  546-nm 
light  from  a  100-W  high-pressure  Hg  arc  source  (as  described  in 
Ellis,  1985,  and  in  Inoue,  1986,  Appendix  3),  I  obtained  depth  of 
fields  of  ca.  0.2,  0.25,  and  0.15  pm,  respectively,  for  phase- 
contrast,  rectified  DIC,  and  rectified  polarized-light  microscopy. 

These  values  were  obtained  by  examining  video  images  of 
surface  ridges  on  a  tilted  portion  of  a  human  buccal  epithelial  cell. 
The  video  signal  was  contrast  enhanced  digitally  but  without 
spatial  filtration.  The  change  in  image  detail  that  appeared  with 
each  0.2-pm  shift  of  focus  (brought  about  by  incrementing  a  cal¬ 
ibrated  stepper  motor)  was  inspected  in  the  image  and  enlarged 
to  ~10,000x  on  a  high-resolution  video  monitor.  As  shown  in 
Figure  1.7,  the  fine  ridges  on  the  cell  surface  are  not  contiguous 
in  the  succeeding  images  stepped  0.2  pm  apart  in  the  polarized- 
light  and  phase-contrast  images,  but  they  are  just  contiguous  in  the 
DIC  images.  From  these  observations,  the  depth  of  field  in  the  rec¬ 
tified  polarized-light  image  is  estimated  to  be  somewhat  below, 
and  the  DIC  image  just  above,  the  0.2-pm  step  height.  [The  phase- 
contrast  images  here  should  not  be  compared  literally  with  the 
images  in  the  two  other  contrast  modes  because  the  diameter  of 


the  commercially  available  phase  annulus  was  rather  small,  and 
out-of-focus  regions  intruded  obtrusively  into  the  image.  With 
Ellis’  aperture-scanning  phase-contrast  microscope,  the  illuminat¬ 
ing  rays,  and  the  correspondingly  minute  phase  absorber  spot,  scan 
the  outermost  rim  of  the  objective  lens  aperture  in  synchrony. 
Therefore,  essentially  the  full  NA  of  the  objective  lens  is  available 
to  transmit  the  waves  diffracted  by  the  specimen.  Under  these  con¬ 
ditions,  the  z-axis  resolution  of  the  optical  section  in  phase- 
contrast  appears  to  be  even  higher  than  that  of  the  two  other 
contrast  modes  shown  here  (Ellis,  1988;  Inoue,  1994).] 

For  polarization  microscopy  of  specimens  with  low  retar- 
dances,  the  LC-Pol  scope  system  devised  by  Oldenbourg  and  Mei 
(1995;  see  also  Oldenbourg,  1996)  also  provides  effective  optical 
sectioning.  The  LC-Pol  scope  generates  an  image  (retardance  map) 
whose  pixel  brightness  is  proportional  to  the  retardance  of  the 
specimen  at  each  pixel,  independent  of  the  specimen’s  azimuth  ori¬ 
entation,  while  the  algorithm  used  to  compute  the  retardance  map 
also  reduces  the  polarization  aberrations  introduced  by  the  optics 
(that  otherwise  degrade  the  image;  Shribak  et  al.,  2002).  Thus, 
with  the  LC-Pol  scope  system,  objective  lenses  with  NA  as  high 
as  1 .4  can  be  used  at  their  full  aperture  to  detect  retardances  as  low 
as  0.03  nm.  The  use  of  the  high  NA  lenses  at  full  aperture  then  pro¬ 
vides  the  shallow  depth  of  field  (of  less  than  1-pm  thickness)  as 
illustrated  in  Figure  1.8.  In  fact,  the  LC-Pol  scope  can  individu¬ 
ally  resolve  two  flagellar  axonemes  that  cross  each  other  and 
are  separated  by  no  more  than  their  diameter  of  about  0.2  |im 
(Oldenbourg  et  al.,  1998). 

We  do  not  yet  quite  understand  why  the  depth  of  field  of  the 
non-confocal  phase-dependent  images  should  be  so  thin.  It  may 
well  be  that  contrast  generation  in  phase-dependent  imaging 
involves  partial  coherence  even  at  very  high  NAs,  and  that  an 
effect  similar  to  the  one  proposed  elsewhere  for  half-wave  masks 
(Inoue,  1989)  is  giving  us  increased  lateral  as  well  as  axial  reso¬ 
lution.  Whatever  the  theoretical  explanation  turns  out  to  be,  our 
observations  show  that  for  phase-dependent  imaging  of  relatively 
transparent  objects,  even  in  the  absence  of  confocal  optics,  optical 
sections  can  be  obtained  (at  video  rate)  that  appear  to  be  some¬ 
what  thinner  than  for  fluorescence  imaging  in  the  presence  of  con¬ 
focal  optics.  Moreover,  they  perform  this  function  without 
requiring  that  energy  be  deposited  in  the  specimen,  i.e.,  without 
producing  photodamage. 

OTHER  OPTICAL  AND  MECHANICAL  FACTORS 
AFFECTING  CONFOCAL  MICROSCOPY 

Lens  Aberration 

With  stage-  or  object-scanning  confocal  microscopes,  we  saw 
earlier  that  high  NA  lenses  with  simplified  design  and  long 
working  distances  could  be  used  because  the  confocal  image  points 
(source  pinhole,  illuminated  specimen  point,  and  detector  pinhole) 
all  lie  exactly  on  the  optical  axis  of  the  microscope.  This  same 
principle  is  now  used  widely  in  the  design  of  optical  disk 
recorder/players. 

In  contrast,  with  TSM  and  CLSM  sharp  images  of  the  source 
“pinhole(s)”  must  be  focused  over  a  relatively  large  area  away  from 
the  lens  axis.  In  addition,  the  objective  lens  and  the  scanner  must 
bring  images  of  the  illuminated  spot(s)  and  the  source  pinhole(s) 
into  exact  register  with  the  exit  pinhole(s),  and  for  fluorescence 
microscopy,  do  so  at  different  wavelengths.  Thus,  for  these  systems 
to  function  efficiently,  the  microscope  objective  lens  has  to  be 
exceptionally  well  corrected.  The  field  must  be  flat  over  an 
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FIGURE  1.7.  Optical  sections  of  surface  ridges  on  an  oral  epithelial  cell.  These  ultrathin  optical  sections  were  obtained  without  confocal  imaging  in  phase- 
contrast  (left),  rectified  DIC  (middle),  and  rectified  polarized-light  microscopy  (right).  The  focus  planes  for  the  successive  frames  in  each  contrast  mode  were 
incremented  0.2  pm.  Scale  bar  10  pm.  (See  text  and  original  article  for  details.  From  Inoue,  1988.) 


appreciable  area,  axial  and  off-axis  aberrations  must  be  corrected 
over  the  field  used,  and  lateral  and  longitudinal  chromatic  aberra¬ 
tions  must  be  well  corrected  for  both  the  emission  and  illuminating 
wavelengths.  As  far  as  is  possible,  the  aberrations  should  be  cor¬ 
rected  within  the  objective  lens  without  the  need  to  use  a  compli¬ 
mentary  ocular.  [For  details  of  these  subjects  and  design  of  modern 
lenses  to  overcome  the  aberrations,  see  Inoue  and  Oldenbourg 
(1994),  Shimizu  and  Takenaka  (1994),  and  Chapter  7,  this  volume.] 
Finally,  the  lens  and  other  optical  components  must  have  good 
transmission  over  the  needed  wavelength  range. 

These  combined  conditions  place  a  strenuous  requirement  on 
the  design  of  the  objective  lens.  Fortunately,  with  the  availability 
of  modern  glass  stocks  and  high-speed  computer-optimized 


design,  a  series  of  excellent-quality,  high-NA,  PlanApo,  and  high- 
UV-transmitting  lenses  have  appeared  from  all  four  major  micro¬ 
scope  manufacturers  (Leitz,  Nikon,  Olympus,  Zeiss)  during  the 
past  decade. 

Even  with  excellent  lenses,  however,  the  image  loses  its  sharp¬ 
ness  when  one  focuses  into  a  transparent,  live,  or  wet  specimen  by 
more  than  a  few  micrometers  from  the  inside  surface  of  the 
coverslip.  The  problem  here  is  that  oil-immersion  microscope 
objectives  are  designed  to  be  used  under  rather  stringent  optical 
conditions,  namely  homogeneous  immersion  of  everything, 
including  the  specimen  itself,  in  a  medium  of  Tj  =  1.52.  When  such 
a  lens  is  used  on  live  or  wet  specimens  immersed  in  water  or  phys¬ 
iological  saline  solution,  even  with  the  coverslip  properly  oil  con- 
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FIGURE  1 .8.  Optical  sections  of  meiosis-I  metaphase  spindle  in  live  spermatocyte  of  a  crane  fly  Nephrotoma  sturalis  observed  with  the  LC  Pol  scope.  In  focus 
are:  (A)  the  upper  kinetochores  (Ks)  and  their  two  K-fibers  for  the  left  bivalent  chromosome;  (B)  upper  Ks  and  fibers  for  the  middle  and  right  bivalents;  (C) 
lower  Ks  and  fibers  for  the  middle  and  right  bivalents.  The  birefringence  retardation  of  the  K-fibers,  made  up  of  a  dense  bundle  of  microtubules,  is  ca.  1  nm 
greater  than  that  due  to  the  background  array  of  spindle  microtubules.  The  effect  of  optical  sectioning  is  more  obvious  in  the  mitochondrial  threads  (which  are 
much  thinner  than  the  K-fibers  and  free  of  background  birefringence)  that  surround  the  spindle.  Imaged  with  546-nm  illumination  in  LC-Pol  scope  with  Nikon 
60x/1.4  NA  “DIC”  PlanApo  objective  lens  combined  with  condenser  NA  at  1.0.  The  fine  focus  control  was  shifted  1.2  pm  between  (A)  and  (B),  and  0.9  pm 
between  (B)  and  (C).  White  =  3.0nm  retardance.  Scale  bar  =  5pm. 


tacted  to  the  objective  lens,  aberrations  are  no  longer  properly  cor¬ 
rected  once  the  image-forming  rays  traverse  a  significant  distance 
in  the  Tj  =  1.33  aqueous  medium.  Doing  so  distorts  the  unit  dif¬ 
fraction  image  and  alters  the  shape  (and  even  the  location  of)  the 
point-spread  function,  and  does  so  to  varying  degrees  as  one 
focuses  to  different  depths  into  the  aqueous  medium.4  This  impor¬ 
tant  topic  is  discussed  in  detail  in  Chapters  7  and  20  (this  volume) 
and  in  Inoue  and  Spring  (1997,  Sect.  2.5). 

The  same  holds  true  also  for  high-NA  dry  objectives  because 
they  are  designed  under  the  assumption  that  (unless  embedded  in  a 
Tj  =  1 .52  medium)  the  specimen  lies  in  an  infinitely  thin  layer  placed 
directly  against  a  coverslip  whose  thickness  (generally  0.17  mm), 
refractive  index,  and  dispersion  conform  to  specification. 

One  approach  to  overcoming  these  problems  is  to  switch  to  a 
water-immersion  objective  lens.  Then  the  cumulative  depth  of  the 
water  layer  between  the  objective  lens  and  the  focused  portion  of 
the  specimen  should  remain  unchanged  with  focus.  Whether  the 
objective  lens  is  designed  for  homogeneous  water  immersion  or 
for  use  in  the  presence  of  a  coverslip  does  not  matter,  so  long  as, 
in  the  latter  case,  a  coverslip  with  the  proper  specifications  is  used. 
In  fact,  however,  even  the  small  difference  in  Tj  between  physio¬ 
logical  solutions,  seawater,  tissue,  and  pure  water  must  be  taken 
into  account.  While  this  approach  does  overcome  some  of  the  aber¬ 
ration  problems,  water-immersion  lenses  cannot  be  made  with 
NAs  of  much  above  1.25  (because  of  the  1.33  refractive  index  of 
water).  Several  manufacturers  now  produce  high-NA  water- 
immersion  objectives  with  excellent  correction,  some  with  high 
transmissions  for  UV  down  to  wavelengths  of  340  nm.  In  our  ex¬ 
perience,  the  Nikon  60x  1.2  NA  Plan  Apochromatic,  correction- 
collar-equipped  water-immersion  objective  gave  an  impressive 
DIC  image  of  diatom  frustule  through  a  220-|im-thick  layer  of 


4  Note  also  that  the  refractive  index  and  dispersion  of  the  immersion  media 
could  also  be  significantly  affected  by  temperature.  Some  high  NA  immer¬ 
sion  lenses  are  thus  equipped  with  correction  collars  to  compensate  for  these 
variations  in  addition  to  the  thickness  of  the  coverslip. 


water  between  specimen  and  coverslip.  Adjustment  of  the  collar, 
as  in  dry-  or  variable-immersion  objective  lenses,  compensates  for 
the  relative  thickness  of  layers  having  higher  or  lower  Tj.  With  the 
increasing  use  of  electronic  and  electro-mechanical  controls  in 
confocal  and  conventional  microscopes,  it  is  now  possible  to 
design  a  superior  high-NA  lens  with  an  auto-compensating  cor¬ 
rection  device  (possibly  built  outside  of  the  objective  lens)  that  is 
electronically  linked  to  the  fine  focus  control. 

A  (motor-driven)  optical-correcting  unit,  such  as  the  In-Focus 
system  (Infinity  Photo-Optical  Company,  Boulder,  CO),  placed  in 
the  parallel-beam  region  of  a  microscope  can  be  used  to  change 
the  focal  level  and/or  correct  for  residual  spherical  aberration 
without  displacing  the  objective  lens.  In  fact  we  find  that  the  unit 
can  often  improve  the  point-spread  function,  so  that  the  z-axis  dis¬ 
tribution  of  the  3D  diffraction  pattern  becomes  more  symmetric, 
even  for  “highly  corrected”  Plan  Apochromatic  objective  lenses 
used  following  the  manufacturer's  exact  specifications  (see 
Chapters  7  and  9,  this  volume).  Conversely,  by  appropriately 
linking  the  In-Focus  drive  with  the  objective  (and  condenser)  lens 
motor  drive(s),  one  can  now  substantially  improve  the  point-spread 
function  even  when  focusing  deeper,  for  example,  into  a  specimen 
residing  in  an  aqueous  medium. 

Unintentional  Beam  Deviation 

The  intensity  of  each  point  in  the  final  image  from  a  confocal 
microscope  is  designed  to  measure  the  amount  of  light  trans¬ 
mitted  by  the  detector  pinhole  for  the  corresponding  point  in  the 
specimen  as  it  is  being  scanned.  However,  if  the  amount  of  light 
transmitted  by  the  detector  pinhole  is  modulated  by  factors  not 
related  to  the  interaction  of  the  illuminating  point  of  light  and  the 
specimen  at  that  raster  point,  or  if  the  confocality  between  the 
entrance  pinhole,  the  illuminated  specimen  point,  and  the  detector 
pinhole  were  to  be  transiently  lost  for  any  reason,  one  would  obtain 
a  false  reading  of  the  brightness  at  that  point. 

One  such  error  could  be  introduced  if  a  localized,  lens-  or 
prism-shaped  region  having  a  Tj  different  from  that  of  the  sur- 
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roundings  were  present  in  the  path  of  the  scanning  or  imaging 
beam.  The  scanning  or  imaging  beam  would  then  be  refracted  or 
deviated  and  the  intensity  of  light  reaching  the  detector  falsely 
modified.  Such  a  false  signal  could  be  difficult  to  distinguish  from 
variations  in  a  genuine  signal  arising  from  a  specimen  point  in  the 
focus  plane.  The  plane  of  focus  may  also  be  distorted  by  the  pres¬ 
ence  of  such  refracting  regions  (Pawley,  2002),  so  that  one  may 
no  longer  be  scanning  a  flat  optical  section  through  the  specimen 
(Chapter  17,  this  volume).  Moreover,  as  discussed  in  the  previous 
section,  the  diffraction  pattern  of  each  image  point  formed  by  an 
oil-immersion  lens  can  be  distorted  and  displaced  along  the  z-axis 
even  when  a  layer  of  optically  homogeneous  aqueous  medium  is 
present  between  the  specimen  and  the  coverslip  [see  Fig.  2^-0 
(after  Gibson  and  Lanni,  1991)  in  Inoue  and  Spring,  1997]. 
Attempts  to  correct  some  beam  deviations  by  interferometric 
measurements  of  the  specimen  have  been  published  (Kam  et  al., 
2001). 

Clearly,  vibration  of  the  microscope,  and  even  minor  distor¬ 
tions  of  the  mechanical  components  that  support  the  optics  or  the 
specimen,  may  introduce  misalignment  between  the  two  pinholes 
and  what  was  supposed  to  be  the  confocal  point  in  the  specimen. 
This  could  lead  to  short-term  periodic  errors  or  longer-term  drift. 
Antivibration  tables  that  isolate  the  instrument  from  building  and 
floor  vibrations  are  commonly  used  to  support  confocal  micro¬ 
scopes.  While  such  a  support  is  useful  and  may  be  essential  in 
some  building  locations,  it  does  not  eliminate  the  influence  of  air¬ 
borne  vibration,  which  can  in  fact  raise  major  havoc  in  microscopy 
(G.W.  Ellis,  personal  communication,  1966).  Nor  does  it  eliminate 
the  influence  of  thermal  drift  or  vibration  arising  from  the  opera¬ 
tion  of  the  instrument. 

Given  the  need  to  precisely  maintain  the  confocal  alignment 
and  to  use  some  form  of  mechano-optical  scanning  within  the 
instrument,  a  confocal  microscope  is  especially  susceptible  to 
vibration  and  problems  of  mechanical  distortion.  Indeed,  once 
the  complex  optical,  electro-optical,  mechanical,  and  electronic 
systems  have  been  appropriately  designed,  the  success  of  one  con¬ 
focal  instrument  over  another  may  well  depend  on  its  immunity 
to  vibration,  in  addition  to  the  friendliness  of  its  user  interface. 
Yokogawa’s  disk-scanning  system,  while  with  somewhat  reduced 
confocal  stringency,  turns  out  to  be  remarkably  immune  to 
vibration  problems. 

CONTRAST  TRANSFER  AND  RESOLUTION  IN 
CONFOCAL  VERSUS  NON-CONFOCAL 
MICROSCOPY 

In  addition  to  designing  and  successfully  demonstrating  the  power 
of  stage-scanning  confocal  microscopy,  Wilson  and  colleagues 
have  extensively  analyzed  the  theoretical  foundations  of  confocal 
microscope  imaging  (see  Wilson  and  Sheppard,  1984;  Wilson, 
1990;  Chapters  11,  22,  and  23,  this  volume).  Their  mathematical 
treatment  leads  to  the  somewhat  surprising  conclusion  that  the  ulti¬ 
mate  limit  of  resolution  (i.e.,  the  cut-off  spatial  frequency  or 
spacing  at  which  image  contrast  of  periodic  objects  drops  to  zero) 
obtainable  with  coherent  confocal  microscopy  is  identical  with  that 
for  incoherent  non-confocal  microscopy.  However,  compared  to 
incoherent  non-confocal  optics,  image  contrast  should  rise  much 
more  sharply  with  coherent  confocal  optics  as  the  spatial  period  is 
increased.  Therefore,  the  practical  resolution  attained  at  threshold 
contrast  (i.e.,  the  minimum  contrast  required  for  the  spacing  to  be 
detected)  was  expected  to  be  significantly  greater  with  confocal 
optics  than  with  conventional  non-confocal  optics. 


FIGURE  1 .9.  Experimental  contrast  transfer  values  measured  as  a  function  of 
the  spatial  period  of  line  gratings  using  a  laser  beam-scanning  microscope  in 
the  confocal  reflection  mode  (solid  points)  and  in  the  non-confocal  transmission 
mode  (circles).  In  both  imaging  modes,  PlanApo  objective  lenses  (Nikon  Inc., 
Melville,  NY )  with  numerical  apertures  (NAs)  ranging  from  0.45  to  1.4  and 
laser  wavelengths  (X)  of  514.5nm  or  488  nm  were  used.  Spatial  periods  are 
expressed  in  units  of  the  limiting  wavelength.  A72(NA),  to  normalize  the  data 
taken  with  different  laser  X  and  lenses  of  different  NA.  We  call  data  presented 
in  this  fashion  the  contrast  transfer  characteristic  (CTC). 

Continuous  lines  are  theoretical  curves  displaying  calculated  CTCs  for  the 
coherent  confocal  and  the  incoherent  non-confocal  imaging  modes.  Compari¬ 
son  of  the  two  CTC  curves  shows  that,  while  the  limiting  resolutions  are  iden¬ 
tical  for  both  imaging  modes,  the  contrast  due  to  line  detail  in  the  specimen  is 
maintained  much  better  with  confocal  optics.  (The  microscope  was  a  prototype 
built  by  Hamamatsu  Photonics  Kabushiki  Kaisha,  Japan.  It  was  used  with  the 
detector  pinhole  diameter  reduced  to  a  small  fraction  of  the  Airy  disk 
diameter.  From  Oldenbourg  et  al..  1993.) 

We  have  confirmed  these  predictions  by  direct  measurement 
on  test  gratings  that  we  fabricated  by  electron  lithography  with 
spacings  down  to  0. 1  pm.  Indeed,  with  confocal  optics  equipped 
with  a  small  exit  pinhole,  the  contrast  transfer  efficiency  rose  to 
80%  at  twice  (and  reached  100%  at  three  times)  the  cut-off 
spacing.  With  incoherent  non-confocal  imaging,  80%  contrast 
transfer  was  not  attained  until  four  times  and  did  not  even  reach 
the  100%  transfer  rate  at  eight  times  the  cut-off  spacing  (Fig.  1.9)! 

SUMMARY 

•  The  limiting  resolution  of  all  microscopes  depends  on  the  A,  of 
the  light  used  and  the  NA  of  the  objective  and  condenser 
lenses.  Dirty  or  misaligned  optics  or  vibration,  or  both,  can 
reduce  the  achieved  resolution.  Test  resolution  regularly,  and 
especially  pay  attention  to  the  iris  setting  and  full  illumination 
of  the  condenser  aperture,  to  assure  optimal  performance. 

•  A  small  detector  pinhole  in  the  confocal  microscope  is  essen¬ 
tial  if  the  maximum  optical  sectioning  capability  and  resolu¬ 
tion  of  the  instrument  are  to  be  realized  concurrently.  Correct 
alignment  and  use  of  this  control  is  very  important.  However, 
a  larger  pinhole  may  be  required  to  improve  the  signal  when 
there  is  limited  light  level,  motion  in  the  specimen,  or  fading 
of  fluorescence. 

•  By  opening  the  confocal  exit  pinhole  (to  not  much  greater  than 
the  Airy  disk  diameter),  one  loses  the  advantage  of  higher  X-Y 
resolution  in  fluorescence  microscopy  but  retains  much  of  the 
capability  for  rejecting  out-of-focus  information  while  also 
gaining  in  the  fluorescence  signal. 
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•  The  ultimate  resolution  limit  of  a  CLSM  in  the  reflection 
mode  is  the  same  as  that  of  a  conventional  light  microscope, 
but  the  contrast  that  it  produces  from  features  is  higher. 
For  fluorescence  imaging,  the  resolution  in  a  confocal  micro¬ 
scope  can  be  ~V2  greater  than  with  conventional  microscopy, 
but  only  if  the  confocal  detector  pinhole  is  appreciably  smaller 
than  the  Airy  disk  produced  by  a  point  fluorescent  object. 

•  CLSM  is  not  a  cure-all  for  all  biological  studies.  Its  sampling 
speed  is  limited,  and  it  does  not  lend  itself  to  using  either  inter¬ 
ference  effects  to  produce  contrast,  such  as  phase  or  DIC, 
which  have  been  found  to  be  relatively  innocuous  to  living 
cells,  or  polarization  contrast  that  can  reveal  fine  structural 
dynamics  noninvasively. 

•  Video  microscopy  can  take  advantage  of  the  various  types  of 
interference  and  polarizing  contrast  not  easily  implemented  in 
the  CLSM.  Therefore,  it  is  ideal  for  dynamic,  high-resolution 
observations  of  living  specimens  and  for  tracking  the  behav¬ 
ior  of  macromolecular  assemblies. 

•  Holographic  microscopy  is  a  field  with  intriguing  promise  that 
is  so  far  beset  by  practical  difficulties. 

•  Two-photon  microscopy  and  CLSMs  provide  high-confocal 
stringency  and,  in  general,  are  methods  of  choice  for  obtain¬ 
ing  clear,  high-resolution  optical  sections  of  the  fluorescence 
distribution  in  3D  fluorescent  specimens.  However,  for 
capturing  well-resolved  optical  sections  of  highly  dynamic 
specimens,  or  for  detection  of  sparsely  distributed  minute 
fluorescent  objects,  the  newer  disk-scanning  confocal  systems 
employing  microlenses  on  the  pinholes  may  well  be  the  system 
of  choice. 
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Fundamental  Limits  in  Confocal  Microscopy 

James  B.  Pawley 


INTRODUCTION 

The  previous  chapter  described  how  the  confocal  approach  devel¬ 
oped  from  conventional  light  microscopy  and  outlined  the  basic 
advantages  gained  by  the  use  of  confocal  sampling  techniques,  pri¬ 
marily  that  the  exclusion  of  light  by  the  confocal  pinhole  makes  it 
possible  to  record  data  from  optical  sections.  This  chapter  will 
discuss  the  fundamental  considerations  that  limit  the  performance 
of  all  confocal  microscopes.  Though  at  present  no  commercially 
available  equipment  approaches  these  limits,  some  simple  tests 
will  be  described  to  help  the  user  assess  how  well  a  given  instru¬ 
ment  performs.  Additional  information  to  help  the  user  to  operate 
the  confocal  microscope  in  an  optimal  manner  can  be  found  in 
Chapter  35,  “A Tutorial  on  Confocal  Microscopy,”  and  Chapter  36, 
“Practical  Confocal  Microscopy.”  These  also  include  methods  for 
measuring  resolution  and  other  useful  parameters. 

What  Limits? 

The  task  of  the  confocal  light  microscope  is  to  measure  optical  or 
fluorescent  properties  within  a  number  of  small,  contiguous  sub¬ 
volumes  of  the  specimen  (Fig.  2.1;  Pawley  and  Centonze,  1994). 
The  fundamental  limits  on  this  process,  therefore,  are  related  to  the 
quantitative  accuracy  with  which  these  measurements  can  be 
made,  a  factor  that  depends  on  the  number  of  photons  that  pass 
into  n i  and  out  of,  n2,  the  sub-volume;  its  size  (5x,  8y,  5z);  and  its 
position  (x,  y,  z).  Additional  limitations  are  imposed  on  the  rate  at 
which  these  measurements  can  be  made  by  the  effects  of  photo¬ 
damage  to  the  specimen,  finite  source  brightness,  and  fluorescence 
saturation.  Finally,  limitations  are  imposed  by  the  fact  that  the  con¬ 
tinuous  specimen  must  be  measured  in  terms  of  discrete  volume 
elements  called  voxels  [a  voxel  is  the  three-dimensional  (3D) 
equivalent  of  a  pixel,  which  is  the  smallest  element  of  a  two- 
dimensional  (2D)  digital  image].  This  chapter  will  try  to  define  the 
factors  that  ultimately  limit  the  accuracy  with  which  these  mea¬ 
surements  can  be  made.  As  such,  it  will  serve  as  an  introduction 
to  many  of  the  chapters  that  follow  in  which  the  practical  and  the¬ 
oretical  aspects  of  these  problems  are  discussed  in  greater  detail. 
The  discussion  should  be  applicable  to  the  consideration  of  any 
type  of  confocal  microscope,  though  here,  as  elsewhere  in  this 
volume,  microscopes  in  which  scanning  is  accomplished  by 
moving  the  light  beam(s)  rather  than  the  specimen  will  be  empha¬ 
sized  because  they  are  more  easily  applied  to  living  specimens. 
Most  of  the  discussion  will  focus  on  the  performance  of  the 
confocal  laser  scanning  microscope  (CLSM),  but  in  some  cases, 
differences  between  the  mirror- scanning  and  disk-scanning  instru¬ 
ments  (including  both  tandem  and  single-sided  disks)  will  dictate 
a  separate  consideration. 


The  data  recorded  from  a  confocal  microscope  will,  in  the  sim¬ 
plest  case,  be  a  set  of  intensity  values  (usually  representing  the 
concentration  of  fluorophore)  for  every  voxel  throughout  a  3D 
volume  within  the  specimen.  Though  these  data  may  often  be  dis¬ 
played  as  an  image,  it  should  always  be  remembered  that  the 
single-beam  confocal  microscope  is  intrinsically  a  serial  or  sam¬ 
pling  instrument,  not  a  parallel-imaging  instrument.  While  it  is  true 
that  one  may  choose  to  sample  a  plane  by  sequentially  scanning 
the  illumination  over  a  large  number  of  overlapping  sub- volumes 
that  cover  the  focus  plane  and  that  by  doing  so,  one  may  produce 
a  2D  image,  given  sufficiently  flexible  equipment,  one  could  also 
use  the  same  total  sampling  (imaging)  time  to  measure  a  single 
point  at  a  great  many  different  times,  a  smaller  volume  within  the 
sample  or,  indeed,  any  other  small  collection  of  points  within  the 
specimen  a  great  many  times. 

The  distinction  between  sampling  and  imaging  is,  of  course, 
not  absolute;  after  all,  most  of  us  will  view  the  final  result  as  some 
sort  of  image.  However,  the  distinction  is  still  useful  because  it 
requires  one  to  explicitly  confront  many  problems  that  are  not 
always  so  obvious  when  microscope  images  are  viewed  directly 
by  eye  or  after  photographic  recording. 

The  sampling  approach,  which  is  covered  in  more  detail  in 
Chapter  4,  allows  an  image  to  be  built  up  from  a  number  of  indi¬ 
vidual  measurements,  each  of  which  reflect  properties  within  a 
specific  region  of  the  sample.  If  the  measured  properties  are 
optical,  the  measurements  involve  counting  photons,  and  this 
process  itself  implies  limits  on  both  the  data  rate  and  the  statisti¬ 
cal  accuracy  that  are  often  ignored  in  normal  widefield  (WF) 
microscopy.  These  limitations  are  associated  with  factors  such  as 
counting  statistics,  source  brightness,  and  specimen  response,  and 
are  discussed  next. 

Although  the  points  raised  so  far  constitute  a  fairly  complete 
list  of  the  physical  limits  on  the  accuracy  and  completeness  of  the 
data  stored  in  the  image  memory  of  a  confocal  microscope,  they 
say  nothing  about  factors  affecting  the  response  of  either  the  dye 
molecules  in  the  specimen  or  the  response  of  these  molecules  to 
the  excitation.  A  more  complete  understanding  of  these  and  other 
important  variables  can  be  found  outlined  in  Pawley  (2000)  and  in 
greater  detail  throughout  the  other  chapters  of  this  book. 

Counting  Statistics:  The  Importance  of  n 

The  accuracy  of  any  particular  measurement  involving  fundamen¬ 
tal,  quantum  interactions  (such  as  counting  photons)  is  limited  by 
Poisson  statistics.  Without  going  into  the  details,  this  means  that 
if  the  same  measurement  is  made  repeatedly,  and  the  average  result 
of  these  measurements  is  n  photons/measurement,  the  chance  that 
any  specific  measurement  is  in  the  range  between  n  +  Vn  and  n 
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From  Source 


FIGURE  2.1.  Parameters  related  to  the  fundamental  limitations  of  confocal 
microscopy. 

-  V«  is  only  63%.  For  example,  if  n  is  100  photons,  63%  of  the 
measurements  will  be  in  the  range  from  100  -  VTOO  to  100  +  VI  Of) 
or  between  90  and  110.  Such  a  measurement  is  said  to  have  10% 
statistics.  It  can  be  seen  that  increasing  the  precision  to  1%  requires 
that  n  be  increased  to  ( 100)2  or  10,000  photons.  While  similar  con¬ 
siderations  limit  the  performance  of  all  types  of  image  measure¬ 
ments,  they  are  more  explicit  in  their  effect  on  confocal 
microscopy  where  photons  are  routinely  counted  individually.  It 
might  help  to  have  a  better  idea  about  the  numbers  of  photons 
involved.  Figure  2.2  shows  how  a  1  mW  laser  beam  composed 
of  1015  photon/s  becomes  a  detected  signal  of  only  10-100 
photons/pixel. 

The  uncertainty  associated  with  counting  quantum-mechanical 
events  is  often  spoken  of  in  terms  of  it  being  the  source  of  intrin¬ 
sic  or  statistical  noise,  and  this  usage  is  the  basis  of  the  common 
belief  that,  while  a  single-scan  image  is  “noisy,”  the  “noise”  can 
be  “reduced”  by  summing  or  Kalman-averaging  the  data  from 
many  frames.  This  usage  is  accurate  to  the  extent  that  because  the 
summed  image  contains  more  data,  it  is  better  statistically  deter¬ 
mined  and  appears  less  noisy.  However,  it  is  important  to  keep 
intrinsic  noise  separate  in  one’s  mind  from  extrinsic  noise  such  as 
that  introduced  by  detector  dark-current  or  electronic  noise,  or  that 
produced  by  stray  or  out-of-focus  light1  because,  unlike  intrinsic 
noise,  these  factors  are  susceptible  to  being  reduced  by  careful 
technique  and  technological  improvements.  Furthermore,  in  the 
case  of  fixed  pattern  noise,  the  effect  may  not  be  reduced  by  aver¬ 
aging  many  frames. 


1  A  simple  test  for  stray  is  to  collect  a  frame  with  the  room  lights  on  but  the 
laser  turned  off.  If  the  average  brightness  of  this  frame  or  one  with  the  laser 
on  but  no  specimen  in  place,  is  higher  than  that  with  the  laser  and  room  lights 
off,  you  may  have  a  stray  light  problem. 


While  it  is  misleading  to  think  that  the  only  way  to  reduce 
“noise”  is  to  average  more  data,  it  is  also  true  that  in  a  well- 
designed  CLSM,  the  major  noise  source  is  intrinsic  noise.  This  fact 
highlights  the  importance  of  making  sure  that  as  many  as  possible 
of  the  available  photons  are  recorded  as  part  of  the  signal.  Photon 
efficiency  is  discussed  later. 

Source  Brightness 

A  fundamental  law  of  optics  states  that  the  brightness  of  light  in 
the  image  (measured  in  watts/cm2/steradian)  can  never  be  greater 
than  it  was  in  the  object  (or  source).  In  the  case  of  laser-scanning 
microscopes,  the  intrinsic  brightness  of  the  laser  source  is  so  high 
that  this  law  does  not  present  a  practical  limitation  on  performance 
(though  photodamage  of  the  specimen  may  do  so).  However,  it 
does  pose  a  limitation  on  the  performance  of  disk-scanning  con¬ 
focal  instruments,  many  of  which  currently  use  mercury  arc 
sources  that  lose  90%  to  98%  of  their  intensity  in  passing  through 
the  disk  (Chapters  6,  10,  this  volume ).  In  the  latter  case,  the  source 
brightness  and  the  optical  design  of  the  illuminating  optics  are 
crucial  to  being  able  to  detect  enough  photons  to  produce  a  statis¬ 
tically  well-defined  fluorescence  image  in  a  reasonable  amount  of 
time.  At  present,  the  total  power  of  the  narrow-band  illumination 
needed  to  excite  fluorescence  that  emerges  from  the  best  disk¬ 
scanning  instruments  is  at  least  an  order  of  magnitude  less  than 
that  commonly  used  with  the  laser  instruments.  However,  im¬ 
proved  non-laser  sources  are  constantly  being  developed  (Chapter 
6,  this  volume)  and,  in  addition,  the  laser-powered  disk-scanning 
microscopes  are  beginning  to  take  over  some  of  the  fast-scanning 
market  previously  dominated  by  the  arc-illuminated  disk  scanners 
(Chapter  10,  this  volume). 

Specimen  Response:  Dye  Saturation 

In  normal,  widefield  (WF)  microscopy,  it  is  safe  to  assume  that 
photons  interact  with  the  specimen  in  a  manner  that  is  independent 
of  the  illumination  intensity.  However,  this  linear  response  is  not 
characteristic  of  laser-based  confocal  microscopes  operated  at 
effective  laser  power  levels  of  >lmW.  Conceivable  departures 
include  the  possibility  that  absorption  in  the  specimen  may  cause 
sufficient  warming  to  produce  damage  or  that  the  electric  field 
strength  may  become  sufficient  to  produce  a  nonlinear  response 
(such  as  2-photon  excitation,  Chapters  28  and  37,  this  volume). 
However,  the  most  obvious  complication  is  the  phenomenon  of 
singlet-state  fluorescence  saturation. 

This  phenomenon  occurs  when  the  flux  of  exciting  illumina¬ 
tion  is  so  intense  that,  at  any  instant,  a  significant  fraction  of  the 
fluorescent  molecules  in  the  illuminated  area  are  already  in  the 
singlet-excited  state.  As  excited  molecules  no  longer  absorb  light 
at  the  usual  wavelength  (X),  this  has  the  effect  of  lowering  the 
effective  dye  concentration.  This  saturation  threshold  can  easily  be 
reached  at  flux  levels  around  106W/m2  such  as  are  found  in 
CLSMs  (see  Chapters  16,  17,  and  39,  this  volume).  The  problem 
is  more  severe  when  using  dye  molecules  with  high  absorption 
crossections,  or  long  fluorescent  decay  times  especially  when  the 
dye  must  be  used  at  low  concentration. 

Effects  of  Saturation 

Because  fluorescence  saturation  is  a  property  of  the  beam,  it  affects 
all  dye  molecules  equally,  so  those  in  “bright”  pixels  are  affected 
the  same  as  those  in  “dim”  pixels.  As  a  result,  the  contrast  of  the 
final  image  seems  unaffected:  bright  areas  are  bright,  dark  areas, 
dark.  In  other  words,  an  image  recorded  with  a  beam  intense 
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Specimen:  produces  signal  and  contrast. 
Fluorescence  signal  efficiency  -10-6 


FIGURE  2.2.  Schematic  diagram  showing  photon  numbers 
at  various  stages  of  fluorescence  microscopy. 


FIGURE  2.3.  Confocal  XZ,  backscattered  light  image  of 
cheek  cells  in  a  chamber  made  of  a  coverslip  held  away 
from  the  microscope  slide  by  dots  of  dried  nail  polish. 
The  image  of  the  upper  glass/water  interface  is  straight 
and  smooth.  The  lower  one  is  not.  This  is  an  indication 
that  the  optical  properties  of  the  cheek  cells  have  distorted 
the  optical  section.  Bio-Rad  MRC-600  confocal  mounted 
on  a  Nikon  upright  microscope  with  a  60x  NA  1.4  Plan- 
Apo  oil-immersion  objective. 


enough  to  cause  saturation  does  not  look  “clipped”  in  intensity  in 
the  same  way  that  an  overexposed  photograph  does.  However,  the 
danger  of  fluorescence  saturation  is  that  in  its  presence,  the  signal 
from  any  particular  pixel  becomes  a  function  of  variables  other 
than  dye  concentration.  These  variables  include  the  local  fluores¬ 
cent  decay  time  constant,  which  can  depend  strongly  on  the  mol¬ 
ecular  environment,  and  local  defocus  effects  that  may  affect  the 
peak  intensity  in  the  focused  spot  and  hence  the  degree  of  satura¬ 
tion.  It  follows  from  this  last  point  that,  as  saturation  occurs  mainly 
at  the  center  of  the  focused  spot,  relatively  more  signal  will  be  pro¬ 
duced  from  adjacent  planes  when  one  operates  near  saturation,  an 
effect  that  marginally  reduces  z-resolution. 

Finally,  although  the  mechanisms  of  photodegradation  are  as 
yet  imperfectly  understood  (see  Chapters  16  and  39,  this  volume ), 
it  now  seems  likely  that  the  absorption  of  a  second  photon  by  a 
molecule  still  in  the  singlet-excited  state  may  turn  out  to  be  a 
common  bleaching  mechanism.  If  high  levels  of  light  flux  increase 
the  rate  at  which  fluorescent  dye  is  bleached  (per  illuminating 
photon),  operation  near  saturation  may  increase  the  bleach  rate. 

This  fundamental  limitation  on  the  rate  of  fluorescent  data 
acquisition  can  be  side-stepped  if  the  light  illuminates  more  than 
one  focused  spot  on  the  specimen.  However,  to  preserve  confocal 
conditions,  a  separate  excitation  and  detection  pinhole  must  be 
used  for  each  spot,  a  condition  that  is  only  present  in  the  disk¬ 
scanning  and  line-scanning  instruments  at  this  time. 


The  Effect  of  Refractile  Cellular  Structures 

As  is  diagramed  in  countless  textbooks,  the  pinhole  of  the  confo¬ 
cal  optical  system  excludes  light  emitted  or  scattered  from  features 
that  are  above  or  below  the  focus  plane,  creating  the  “optical 
section.”  However,  all  these  diagrams  assume  that  the  specimen  is 
optically  uniform.  While  this  assumption  may  be  met  for  fixed 
specimens  that  are  embedded  in  a  mounting  media  having  a  high 
refractive  index  (RI),  it  is  less  accurate  in  describing  the  optics  of 
living  cells.  The  fact  that  living  cells  produce  high-contrast  images 
when  viewed  using  phase-contrast  or  differential-interference  con¬ 
trast  optics,  is  a  clear  indication  that  the  RI  of  many  cellular  struc¬ 
tures  is  substantially  different  from  that  of  water.  Figure  2.3  is  an 
xt-image  of  a  clump  of  cheek  cells  mounted  between  the  cover- 
slip  (top)  and  the  slide  (bottom)  made  using  backscattered  light 
(BSL).2  As  it  should  be,  the  flat  surface  of  the  coverslip  is  imaged 
as  a  thin  horizontal  white  line.  However,  because  of  the  presence 
of  the  cheek  cells,  the  equally  flat  surface  of  the  glass  slide  is  ren¬ 
dered  as  a  fuzzy  line  that  undulates  up  and  down  by  as  much  as 


2  Backscattered  light  is  a  more  general  term  for  what  is  often  referred  to  as 
“reflected  light.”  It  is  recommended  that  the  term  “reflected”  be  reserved  to 
describe  light  scattered  by  features  sufficiently  large  and  smooth  so  that  “the 
angle  of  incidence  equals  the  angle  of  reflection.”  Most  cells  produce  much 
more  backscattered  light  than  reflected  light. 
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FIGURE  2.4.  (A)  Confocal  XZ,  backscattered  light  image  of  cheek  cells  in  a  chamber  made  of  a  coverslip  held  away  from  the  microscope  slide  by  dots  of  dried 
nail  polish.  (B)  Acridine  orange  fluorescence  image  of  the  same  specimen  as  in  (A).  It  is  easy  to  see  how  the  presence  of  the  nucleus  (the  large  blob  in  the 
fluorescence  image)  has  displaced  the  BSL  image  of  the  top  surface  of  the  slide.  Optical  system  the  same  as  in  Figure  2.3.  Field  width,  50pm;  field  height, 
-60  pm. 


6|im.  Figures  2.4  and  2.5  make  clear  the  role  that  the  nuclei  in 
some  cells  can  play  in  distorting  the  “plane  of  focus.”  In  this  case, 
the  “surface  of  best  focus”  is  clearly  not  a  plane  (Pawley,  2002). 

The  idea  that  the  RI  anomalies  caused  by  common  cellular 
structures,  such  as  nuclei,  are  sufficiently  severe  to  displace  the 
location  of  any  optical  section  on  their  far  side  by  distances  on  the 
order  of  micrometers  is  deeply  disturbing.  As  living  cells  neces¬ 
sarily  have  nuclei  and  other  refractive  organelles,  it  is  hard  to  see 
how,  or  even  if,  we  can  compensate  for  the  sequellae  of  their 
optical  properties.  Although  one  might  imagine  computer  calcula¬ 
tions  that  would  model  these  optical  effects  and  thereby  improve 
matters,  such  calculations  will  be  immensely  complex  and  there¬ 
fore  take  time:  time  seldom  available  when  viewing  living  cells. 
In  general,  it  seems  that  we  must  accustom  ourselves  to  more 
modest  performance  when  viewing  living  specimens  in  a  3D  light 
microscope. 

Given  the  distortion  noted  above,  it  seems  inevitable  that  these 
RI  anomalies  must  also  deform  the  point-spread-function  (PSF)  of 
the  microscope  in  nearby  regions.  This  raises  questions  about  the 
confidence  one  can  have  in  images  produced  by  widefield/image- 
deconvolution  of  living  specimens.  It  is  not  that  deconvolution  will 
not  produce  a  sharper  image.  The  problem  is,  does  this  sharper 
image  accurately  represent  the  3D  structure  of  the  specimen? 

In  some  tissues,  cellular  structures  are  naturally  less  optically 
disruptive:  cornea  and  lens  come  to  mind.  For  other  living  tissues, 
it  seems  that  one  must  be  mindful  that  the  visibility  of  peri-nuclear 
structures  may  depend  not  only  on  their  presence  or  absence  but 
also  on  whether  these  structures  are  located  on  the  near  or  far  side 
of  the  nucleus.  When  they  are  located  on  the  far  side,  mounting 
the  specimen  between  two  coverslips  so  that  it  can  be  flipped  over 
may  be  worthwhile  (see  Chapter  15,  this  volume). 

When  looking  far  below  the  surface  of  a  tissue,  it  may  be  worth 
the  effort  to  try  to  orient  the  specimen  so  that  the  foreground  con¬ 
tains  as  few  nuclei  as  possible. 

A  first  approximation  of  how  serious  this  problem  is  on  any 
particular  specimen  can  be  obtained  by  viewing  the  specimen 
using  phase-contrast  or  darkfield  optics.  Contrast  in  both  of  these 
imaging  modes  is  proportional  to  changes  in  local  RI. 


FIGURE  2.5.  Stereo  view  from  part  of  az-stack  of  12S,  512x512  BSL  images 
of  a  cheek  cell  specimen.  The  projection  orientation  was  chosen  to  emphasize 
the  spatial  relationship  between  the  nucleus  (central  blob)  and  the  distorted  area 
of  the  image  of  the  water/slide  interface  originally  located  below  it.  The  four¬ 
way  symmetry  of  the  "distortion”  image  is  caused  by  the  interaction  of  the 
nucleus  and  the  polarized  laser  beam  and  modified  by  the  fact  that  high-NA 
light  is  more  likely  to  reflect  from  the  slide  interface.  Bio-Rad  Radiance  con¬ 
focal  mounted  on  a  Zeiss  Axiophot  upright  microscope  with  a  40x  NA  1 .2  C- 
PlanApo  water-immersion  objective.  To  obtain  stereo  effect,  view  through 
red/blue  or  red/green  glasses. 
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A  Typical  Problem 

The  many  limits  on  CLSM  performance  do  not  act  alone  but  can 
combine  in  complex  and  subtle  ways.  To  highlight  the  interactions 
between  counting  statistics  and  more  widely  recognized  limits 
such  as  spatial  resolution,  let  us  define  a  characteristic  microscop¬ 
ical  problem  shown  schematically  in  Figure  2.6.  The  specimen  is 
a  cell  in  which  some  of  the  protein  sub-units  making  up  the 
cytoskeletal  fibers  have  been  replaced  with  fluorescent  analogs. 
These  fibers  are  very  small,  about  10  times  smaller  than  the  x, ir¬ 
resolution  limit  of  light  optics,  and  excitation  of  the  fluorescent 
dye  causes  it  to  bleach,  a  process  that  is  probably  toxic  to  the  cell. 
Because  the  object  is  to  observe  the  formation  and  movement  of 
the  linear  cytoskeletal  elements  within  the  living  cell,  one  cannot 
take  advantage  of  the  higher  spatial  resolution  of  the  electron 
microscope  because  this  instrument  cannot  be  used  on  living  cells. 

This  example,  though  perhaps  overly  specific,  is  not  unchar¬ 
acteristic,  and  it  has  the  advantage  of  highlighting  interactions 
between  many  of  the  fundamental  limitations.  The  important  fea¬ 
tures  of  this  example  are  as  follows: 

•  The  observer  would  benefit  from  high  spatial  resolution  in  all 
three  dimensions. 

•  Accuracy  in  measuring  intensity-of-stain/unit-length  will  be 
important,  because  doing  so  may  permit  determination  of  the 
number  of  sub-resolution,  linear  polymers  bundled  together  to 
make  up  each  visible  fiber. 

•  A  number  of  images  must  be  recorded  at  different  times  in 
order  to  show  change/motion. 

•  Each  measurement  will  cause  bleaching  and  cellular  toxicity. 

Clearly,  these  conditions  contain  an  inherent  contradiction:  to 
obtain  more  quantitative  temporal  or  spatial  accuracy,  more  light 
must  pass  through  the  sample  but  this  will  produce  more  fading 
and  cytotoxicity.  As  a  result,  the  improved  images  may  be  of  a 
dying  cell  rather  than  a  living  one,  and,  therefore,  the  biological 
reliability  of  the  measurement  may  be  inversely  proportional  to 
the  physical  accuracy  set  by  counting  statistics. 

Other  interactions  between  these  parameters  are  also  possible. 
High  spatial  resolution  implies  that  the  micro- volumes  (pixels) 
excited  by  the  beam  and  sampled  by  the  detector  must  be  very 
close  together  (actually  the  spacing  between  lines  is  less  than  one 


Light  beam 


FIGURE  2.6.  Diagram  of  a  notional  specimen:  a  living  cell  process  contain¬ 
ing  a  filamentous  structure  smaller  than  the  resolution  limit  and  sparsely  stained 
with  fluorescent  analog  molecules. 


quarter  the  diameter  of  the  Airy  disk).  The  measurement  at  each 
pixel  is  really  the  detection  of  the  small  fraction  (<1%)  of  the  flu¬ 
orescent  excitations  excited  by  the  beam  that  actually  produces 
signal  in  the  detector  while  this  pixel  is  illuminated. 

Higher  spatial  resolution  implies  smaller  pixels  and,  hence,  the 
need  to  count  more  photons  from  any  given  volume  of  the  speci¬ 
men.  The  increases  in  detected  signal  implied  by  improved  reso¬ 
lution  are  not  insignificant:  maintaining  the  statistical  accuracy  of 
the  measurements  when  the  resolution  increases  by  a  factor  of  2 
requires  four  times  more  signal  in  order  to  image  a  single  plane 
and  eight  times  more  if  a  3D  volume  is  to  be  sampled.  Fortunately, 
the  only  way  to  actually  increase  the  resolution  by  2x  is  to  increase 
the  numerical  aperture  (NA)  by  2x,  and  doing  so  increases  the  frac¬ 
tion  of  the  photons  emitted  from  the  specimen  that  are  captured  by 
the  lens  by  a  factor  of  (ANA)2  or  4x.  As  a  result,  some  of  the 
improved  image  quality  can  sometimes  be  retained  by  collecting 
a  larger  fraction  of  the  emitted  light  rather  than  simply  by  exposing 
the  specimen  to  increased  illumination.3 

This  interaction  emphasizes  the  importance  of  keeping  the 
pixel  size  appropriate  to  the  operating  resolution.  Those  who 
usually  use  WF  microscopes  to  record  images  on  film  may  be  less 
familiar  with  the  idea  that  pixel  size  is  an  explicit  experimental 
variable.4  It  is  important  to  understand  that  the  “zoom”  magnifi¬ 
cation  factor  used  to  change  the  size  of  the  area  scanned  on  the 
specimen  on  most  commercial  CLSMs  usually  does  so  by  chang¬ 
ing  the  pixel  size  (referred  to  the  specimen).  Therefore,  although 
the  ability  to  vary  the  display  magnification  by  a  factor  of  about 
10:1  may  seem  to  be  great  convenience,  with  a  given  A,  and  NA, 
only  one  zoom  setting  provides  optimal  Nyquist  sampling  of 
the  optical  image  data.  All  other  zoom  settings  must  necessarily 
either  over-  or  under-sample  the  data  and  either  produce  more 
beam  damage  or  record  less  resolution  than  they  should.  (See 
Chapter  4,  this  volume ,  for  how  to  choose  the  optimal  zoom 
setting.) 

Given  the  interrelated  constraints  highlighted  by  these  exam¬ 
ples,  the  two  features  needed  for  a  confocal  microscope  to 
approach  its  ultimate  performance  are: 

•  Photon  Efficiency:  The  system  must  count  as  many  as  possi¬ 
ble  of  the  photons  transmitted  or  emitted  by  the  sample  at  the 
plane  of  focus. 

•  Spatial  and  Temporal  Resolution:  Generally  one  should  focus 
the  laser  into  a  spot  that  is  as  small  as  possible  though,  on  occa¬ 
sion,  it  may  be  advantageous  to  make  this  volume  somewhat 
larger  to  avoid  saturation  or  photodamage  effects. 

These  two  topics  will  now  be  discussed  in  more  detail. 

PRACTICAL  PHOTON  EFFICIENCY 

Photon  efficiency  (y)  is  a  measure  of  the  fraction  of  the  signal 
photons  generated  by  the  action  of  the  laser  beam  on  the  specimen 
that  are  actually  represented  in  the  final  image  data  set  stored  in 
the  computer.  Although  photons  can  be  lost  both  between  the  light 
source  and  the  specimen  and  between  the  specimen  and  the  detec¬ 
tor,  those  lost  before  reaching  the  sample  can  usually  be  “replaced” 


3  Although  this  reciprocity  works  in  the  vy-plane,  it  doesn’t  compensate  in  the 
--direction  because  ^-resolution  is  proportional  to  1/(NA)2. 

4  These  problems  are  strictly  analogous  if  we  relate  pixel  size  to  the  size  of  the 
film  grain.  To  record  faint  fluorescence,  we  choose  a  “fast”  grainy  film  with 
a  high  ISO  rating  only  to  discover  that  it  does  not  record  details  as  small  as 
those  visible  when  using  a  film  with  a  lower  ISO  rating. 
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FIGURE  2.7.  A  system  of  two  filter  blocks  for  use  with  the  Bio- 
Rad  MRC-500-1024  which  permits  optimal  removal  of  specu¬ 
lar  reflection  artifacts  and  the  simultaneous  collection  of  BSL 
and  fluorescence  signals.  Block  I  contains  a  beam-splitter  con¬ 
sisting  of  a  6-mm-thick  Pyrex  optical  flat,  anti-reflection  coated 
on  its  rear  (upper)  side.  A  1/2-wave  plate  after  the  laser  is  used 
to  rotate  the  laser  polarization  so  that  0.8%  to  5%  of  the  laser 
light  is  reflected  down  the  optic  axis  and  through  a  1/4-wave 
plate  between  the  ocular  and  the  objective.  Almost  all  of  the 
returning  BSL  and  fluorescent  light  passes  through  the  beam¬ 
splitter  to  Block  II,  where  the  two  signals  are  separated  with  a 
dichroic  mirror.  The  long  X  signal  passes  to  PMT  1  via  an  emis¬ 
sion  filter.  Signal  at  the  excitation  wavelength  passes  through  a 
second  1/4-wave  plate  and  analyzer  (both  rotatable)  and  a  laser 
line  filter  to  PMT  2.  The  orientation  of  the  1/2-wave  plate  is 
adjusted  to  give  maximum  light  from  the  objective,  then  the 
rotatable  elements  in  Block  II  are  adjusted  to  eliminate  spurious 
reflections.  Finally,  the  1/4-wave  plate  in  the  microscope  body 
is  rotated  to  produce  the  maximum  BSL  signal  from  a  living 
cell. 


with  relative  ease  by  an  increase  in  laser  power.5  However,  photons 
lost  after  leaving  the  sample  represent  a  more  fundamental  loss: 
they  carry  information  obtained  at  the  expense  of  radiation  damage 
to  the  specimen. 

Current  instruments  waste  photons  in  a  number  of  straightfor¬ 
ward  ways,  some  of  which  are  discussed  later  in  this  section.  Two 
somewhat  less  obvious  cases  of  photon  inefficiency  that  can  be 
grouped  under  the  heading  “wasted  light”  are  worthy  of  mention 
here. 

Whenever  the  laser  beam  passes  from  a  region  having  one  RI 
to  another  region  having  a  different  RI,  some  light  is  scattered  back 
towards  the  source.  As  a  result,  almost  any  biological  specimen, 
particularly  a  living  one,  produces  BSL.  In  most  current  instru¬ 
ments  used  in  the  fluorescent  mode,  the  BSL  is  reflected  back  to 
the  source  by  the  dichroic  beam-splitter  and  thereby  wasted.  As  is 
also  pointed  out  in  Chapter  9,  it  is  possible  to  design  the  confocal 
microscope  in  such  a  way  that  both  the  BSL  and  the  fluorescent 
signal  can  be  detected  separately  and  in  a  totally  non-interfering 
manner  to  produce  a  BSL  image  that  is  perfectly  aligned  with  the 
fluorescent  image  (Pawley  et  al.,  1993a,  Pawley,  2002).  This  image 
provides  information  on  the  optical  inhomogeneities  in  the  cell  at 
no  additional  cost  in  terms  of  radiation  damage  to  the  specimen 
(Fig.  2.7). 

The  second  example  of  wasted  light  is  that  which  is  elicited 
by  the  laser  during  line  retrace  in  raster-scanning  instruments.  As 
photons  excited  from  the  specimen  by  this  light  are  not  detected, 
it  represents  a  significant  (30%  for  Is  scans,  10%  for  4s  scans) 
and  unnecessary  assault  on  the  specimen.  Fortunately,  it  can  be 
eliminated  by  gating  the  light  source  during  retrace  using  a  Pockels 
Cell  or  the  acousto-optical  deflector  (AOD),  now  available  on  most 


5  As  noted  above,  such  an  increase  is  more  difficult  when  using  the  arc  sources 
commonly  employed  in  the  disk-scanning  approach. 


commercial  instruments.  Early  video-rate  scanners  using  resonant 
galvanometer  scanning  employed  a  system  of  blades  to  mask  the 
sides  of  the  raster  so  that  light  did  not  reach  the  specimen  as 
the  beam  slowed  down  before  turning  around  at  either  end  of  the 
sinusoidal  motion. 

Light  generated  inside  the  specimen  can  be  lost  through 
several  mechanisms: 

1.  Absorption  or  scattering  in  either  the  objective  lens  or  the 
medium  that  couples  it  to  the  specimen,  or  by  the  fixed  and/or 
moving  mirrors  and  transfer  optics  needed  to  “de-scan”  the 
beam  (see  also  Chapters  7  and  9,  this  volume). 

2.  Incorrect  alignment  of  the  optical  system  resulting  in  the 
improper  placement  or  orientation  of  the  pinhole  or  slit  (see 
also  Chapters  3,  8,  11,  22,  35,  and  36,  this  volume). 

3.  Low  quantum  efficiency  (QE)  of  the  photon  detector  (see  also 
Chapter  12,  this  volume). 

4.  Imprecise  digitization  of  the  output  of  the  photon  detector  (see 
also  Chapter  12,  this  volume). 

While  these  subjects  will  be  covered  in  much  more  detail  in 
the  chapters  noted  above,  a  few  points  will  be  made  here  as  a  back¬ 
ground  to  descriptions  of  the  practical  tests  of  performance  to  be 
described  later. 

Losses  in  the  Optical  System 
Objectives 

To  a  first  approximation,  fluorescent  light  proceeds  from  the  site 
of  its  generation  in  all  directions  equally.  As  a  result,  only  a  small 
fraction  even  strikes  the  objective  lens:  ~30%  of  the  total  for  NA 
1.4  and  this  fraction  decreases  with  1/(NA)2.  The  fraction  of  this 
light  that  emerges  from  the  other  side  of  the  objective  depends  on 
its  transmittance.  Although  measuring  the  absolute  transmittance 
of  an  objective  lens  at  full  NA  is  far  from  easy  (see  Chapter  7,  this 
volume,  for  details),  useful  measurements  comparing  the  effective 
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transmission  of  individual  objectives  can  be  made  on  epi- 
illuminated  instruments  with  significantly  less  trouble.  All  that  is 
needed  is  a  photodiode  light  sensor  (one  without  an  air  space 
between  it  and  any  clear  glass  “window”)  linked  to  a  sensitive 
current  meter  (or  better,  a  basic  photometer)  and  a  front-surfaced 
mirror.  After  adjusting  the  instrument  to  measure  BSL  and  fitting 
objective  lens  A,  align  the  instrument  if  necessary,  then  switch  to 
a  high  zoom  setting  and  measure  the  light  emerging  from  the 
objective  with  the  sensor  (be  sure  to  exclude  stray  light  from  the 
room  and  couple  the  lens  to  the  sensor  with  immersion  oil  if  it  is 
an  oil-immersion  lens).  Call  this  light  reading  7a.  Now  place  a 
front-surfaced  mirror,  slightly  tilted,  on  the  specimen  stage  and  set 
up  the  microscope  to  image  the  reflecting  surface  using  the  BSL 
signal.  Be  careful  to  keep  the  illumination  level  very  low  so  as  not 
to  damage  the  photomultiplier  tube  (PMT)  in  the  microscope  (and 
to  realign  the  instrument  if  reducing  the  illumination  requires  the 
addition  of  ND  filters  in  front  of  the  laser,  see  Chapter  35,  this 
volume).  Focus  on  some  dust  on  the  mirror.  Because  of  the  slight 
tilt,  the  surface  will  only  be  near  focus  (bright)  along  a  broad  band. 
Use  the  computer  controls  to  measure  the  peak  brightness  (Ba)  of 
this  band  and  that  of  some  dark  area  (7>a)  well  away  from  the  bright 
band  (or  better  yet,  with  the  laser  obscured).  This  second,  back¬ 
ground  reading  is  to  compensate  for  non-specific  reflections  in  the 
optical  system,  poor  DC  balance  in  the  PMT  head  amplifier,  and 
the  Brightness  or  Black-level  offset  setting,  etc.  The  reading 
should  be  slightly  positive  to  ensure  that  “zero  signal”  is  within 
the  linear  range  of  the  digitizer. 

Some  microscopes  produce  severe  specular  reflection  artifacts 
when  used  in  the  BSL  mode:  make  measurements  in  a  part  of  the 
image  field  unaffected  by  these  reflections.  Also  be  sure  to  adjust 
the  PMT  gain  so  that  at  a  single  setting,  the  signal  remains  well 
within  the  linear  region  of  the  digitizing  system  (about  “three- 
quarters-full”  or  192  counts/pixel  in  an  8-bit  system)  throughout 
the  entire  procedure. 

Now  change  lenses,  realign,  and  make  a  second  set  of  mea¬ 
surements  7b,  Bb,  and  bb  without  changing  the  PMT  gain. 

To  a  reasonable  approximation,  the  comparative  transmission 
(7c0mp)  of  the  first  lens  as  a  fraction  of  the  second  will  be  the  fol¬ 
lowing: 
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(Transmission  specifications  for  a  number  of  common  objec¬ 
tives  can  be  found  in  tables  in  Chapter  7,  this  volume .) 


Mirrors 

On  some  scanners,  a  similar  setup  can  be  used  to  test  the  perfor¬ 
mance  of  the  internal  mirrors.  Again,  the  light  intensity  leaving  the 
objective,  7a,  is  measured.  Then  the  beam  is  stopped  (or  made  to 
scan  a  very  small  raster)  at  the  part  of  the  scan  when  it  is  focused 
on  the  mirror  surface  (brightest).  After  turning  off  the  PMT(!),  a 
second  reading  is  made  (P.J  with  the  same  photodiode  placed  just 
in  front  of  the  pinhole,  making  sure  that  all  of  the  light  strikes  the 
sensitive  part  of  the  sensor. 

The  Pa/7a  ratio  will  usually  be  depressingly  small  (~10%-20%), 
but  the  difference  covers  reflection  losses  at  the  various  mirror  and 
filter  surfaces,  including  the  beam-splitter  (50%-60%  loss  on  each 
pass)  as  well  as  those  at  the  scan  lens,  the  various  transfer  lenses 
and  the  objective.  (A  method  for  making  a  similar  measurement 
that  does  not  require  placing  the  photodiode  in  front  of  the  pinhole 
is  given  in  the  section  “Measuring  Photon  Efficiency.”) 


Though  indirect  and  somewhat  cumbersome,  such  measure¬ 
ments  can  be  useful  for  two  reasons:  (1)  as  a  rough  method  of  com¬ 
paring  the  performance  of  different  instruments  (or  different  types 
of  mirrors  fitted  to  the  same  type  of  instrument)  and  (2)  to  monitor 
performance  of  a  specific  instrument  over  time.  Performance 
degrades  as  dust  and/or  hydrocarbon  or  other  atmospheric 
vapors  condense  onto  the  surfaces  of  mirrors  and  other  optical 
components.  In  instruments  having  up  to  11  reflective  surfaces  in 
the  detector  chain,  a  change  in  reflectance  of  even  1%  can  have 
marked  effects  on  photon  efficiency  (see  Chapter  7,  this  volume). 


Pinhole 

Is  the  Confocal  Pinhole  a  "Good  Thing"? 

It  is  argued  by  some  that  the  pinhole  is  the  Achilles’  heel  of  the 
confocal  approach  because  it  excludes  photons  that,  while  not  orig¬ 
inating  from  the  plane  of  focus,  do  still  carry  relevant  information 
regarding  other  planes  because  of  the  non-point  nature  of  the 
focused  optical  probe.  The  process  of  “deconvolution”  will  “put 
the  photons  back  where  they  came  from”  and  all  will  be  well  (see 
Chapters  23,  24,  and  25,  this  volume).  Elaborate  and  authoritative 
mathematical  analyses  seem  to  support  an  argument  that  can  be 
summarized,  not  too  unfairly,  as  suggesting  that,  “as  all  measured 
photons  are  good,  excluding  photons  must  be  bad.”  The  problem 
is  that  the  mathematics  that  describe  so  well  the  actions  of  light 
waves,  generally  has  more  trouble  accounting  for  the  behavior  of 
photons,  or  more  particularly  of  the  errors  in  counting  them  caused 
by  Poisson  noise. 

Leaving  aside  possible  modifications  to  the  confocal  detector 
(noted  in  the  next  segment)  that  would  allow  photons  from  nearby 
planes-of-focus  to  be  separately  collected  in  a  confocal  instrument, 
the  contention  that  light  from  out-of-focus  planes  contains  in¬ 
formation  deserves  some  consideration.  In  the  discussion  that 
follows,  all  non-confocal  (WF)  images  will  be  composed  of  light 
from  both  in-focus  and  out-of-focus  planes  while  confocal  images 
consist  of  light  from  only  in-focus  planes.  The  argument  turns  on 
the  relative  quantum  efficiency  (QE)  of  the  detectors  normally 
used  with  each  technique  [cooled  charged-coupled  device  (CCD) 
vs.  PMT]  and  on  when  and  to  what  extent  the  out-of-focus  light 
can  yield  information  regarding  a  3D  structure  beyond  that  con¬ 
tained  in  the  in-focus  light.  This  matter  is  covered  in  more  detail 
in  the  appendix  to  Chapter  13,  and  also  in  Chapters  22,  23,  and  24 
and  in  Pawley  (1994). 

Let  us  take  some  examples.  All  will  agree  that  an  out-of-focus 
WF  image  of  an  isolated  point  object  will  be  a  larger  and  less 
intense  image  than  an  in-focus  one  (Fig.  2.8)  and,  furthermore, 
that  a  knowledge  of  the  optical-transfer-function  of  the  system, 
together  with  information  from  other  adjacent  planes  as  to  whether 
the  current  plane  of  focus  is  above  or  below  the  plane  containing 
the  point  object,  will  allow  us  to  use  computer  image-processing 
techniques  to  gain  some  information  from  this  out-of-focus  data. 
Supposing  then  that  only  out-of-focus  images  were  available,  one 
would  be  justified  in  saying  that,  from  an  analysis  of  out-of-focus 
data,  information  had  been  gained  as  to  the  actual  location  of  the 
object  (i.e.,  in-focus  information).  But  beyond  that,  would  one  be 
justified  in  saying  that  this  would  be  a  significant  addition  to  the 
information  present  in  an  image  of  the  plane  containing  the  object 
(the  in-focus  image)?  In  other  words,  is  the  out-of-focus  informa¬ 
tion  “useful”  only  because  in-focus  information  is  lacking? 

Another  aspect  of  a  measurement  in  which  WF  techniques  will 
give  a  more  accurate  estimate  is,  for  instance,  a  measurement  of 
the  total  fluorescence  from  an  entire  specimen  (say,  all  of  the  chro¬ 
mosomes  in  a  nucleus).  For  a  given  dose  to  the  specimen,  the  WF 
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FIGURE  2.8.  Image  geometry.  A  schematic  diagram  showing  how  the  cone  of  light  leaving  a  point  object  at  angles  that  will  be  accepted  by  the  objective, 
looking  at  different  focus  planes. 


technique  can  be  more  accurate  simply  because  it  will  detect  more 
total  photons. 

On  the  other  hand,  there  must  be  some  limit  to  the  extent  that 
light  from  sources  remote  from  the  in-focus  plane  can  provide 
more  information  about  that  plane  than  it  obscures  (Chapter  10, 
this  volume.  Appendix).  For  instance,  suppose  the  specimen  con¬ 
sists  of  a  small,  0.2-pm  feature  in  the  plane  of  focus  and  a  larger 
and  lOx  brighter  feature  1pm  above  it.  Let  us  also  suppose  that 
the  point-spread  function  (PSF)  of  the  optical  system  is  such  that, 
in  the  WF  image  focused  on  the  plane  containing  the  small  object, 
75%  of  the  light  reaching  the  pixels  centered  on  this  feature  orig¬ 
inates  from  the  larger,  brighter  out-of-focus  feature.  As  this  extra 
signal  will  double  the  statistical  noise  associated  with  intensities 
measured  in  these  pixels,  it  is  hard  to  see  how  the  WF  measure¬ 
ment  could  be  better  than  a  confocal  measurement  that  senses  the 
in-focus  feature  but  excludes  most  of  the  out-of-focus  light  before 
it  reaches  the  detector. 

As  a  more  complex  example,  consider  a  BSL  image  of  a  tooth 
in  which  virtually  every  voxel  scatters  light.  In  such  a  thick 
“dense”  specimen,  the  fraction  of  the  WF  signal  originating  from 
the  in-focus  plane  is  so  small  and  so  uniform  that  even  detecting 
it  using  WF  imaging  and  signal  processing  would  be  extremely 
difficult,  and  the  suggestion  that  the  out-of-focus  signal  actually 
contributes  information  beyond  that  available  in  the  in-focus  or 
confocal  image  seems  difficult  to  support. 

Somewhere  in  between  these  extremes  might  be  the  more  real¬ 
istic  situations  defined  by  the  two,  vertical  “sight  lines”  labeled  A 
and  B,  in  Figure  2.9. 


The  question  then  becomes:  Up  to  what  level  of  3D  staining 
density  (or  sparsity)  can  information  about  the  specimen  (as  dis¬ 
tinct  from  signal  defined  as  “output  for  the  detector”)  be  gained 
from  out-of-focus  light?  The  answer  will  not  be  a  simple  one,  as 
it  depends  on  a  number  of  practical  factors  apart  from  the  sparsity 
of  the  staining,  such  as  (1)  how  well  one  knows  the  optical 
response  function  of  the  microscope  and  how  it  changes  with  z  or, 
(2)  the  relative  amounts  of  specific  and  non-specific  staining  and 
the  exact  effects  of  applying  nonlinear  image  processing 
algorithms  such  as  non-negativity  or  “maximum  entropy”  (see 
Chapters  23  and  24,  particularly  the  Appendix,  this  volume ). 

Excellent  results  from  the  WF/image-processing  approach  are 
also  shown  in  these  chapters  and  elsewhere,  but,  because  of  the 
difficulty  of  producing  fluorescent  specimens  that  have  an  inter¬ 
esting  3D  geometry  but  which  do  not  bleach  when  they  are  looked 
at  using  first  one  method  and  then  the  other,  no  meaningful  side- 
by-side  comparison  has  yet  been  performed  although  there  have 
been  some  computer  simulations.  More  sophisticated  comparisons 
are  described  in  (Chapters  13  and  24,  this  volume ),  but  it  becomes 
clear  that,  to  be  both  realistic  and  useful,  any  comparisons 
are  likely  to  be  complex  as  they  must  include  differences  in  the 
QEs  of  the  respective  detectors  (3-1  Ox  higher  for  WF)  as  well 
as  the  details  of  the  optics,  the  geometry  of  the  stained  features  in 
the  specimen,  and  differences  in  the  response  of  the  dye  to  the 
much  more  intense  light  that  must  be  used  in  any  scanning 
technique. 

The  important  parameters  characterizing  the  three  most 
popular  methods  of  obtaining  3D  images  from  fluorescent  bio- 
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FIGURE  2.9.  Schematic  diagram  showing  areas  of  an  imagi¬ 
nary  fluorescent  structure  to  illustrate  how  local  variations  in 
stain  density  change  the  levels  of  out-of-focus  light  detected. 


logical  specimens  are  summarized  and  compared  in  Chapter  35, 
Table  35.1  and  35.2. 

Proper  Use  of  the  Confocal  Pinhole 

In  a  confocal  microscope,  the  pinhole  is  present  to  prevent  light 
originating  from  anywhere  but  the  plane  of  focus  from  reaching 
the  detector.  It  is  mounted  in  an  image  plane,  and,  if  it  is  misaligned 
or  if  its  size  is  reduced  beyond  that  corresponding  to  a  diffraction- 
limited  spot  on  the  sample  (i.e.,  a  pinhole  diameter  of  0.7-1. 5  Airy 
units),  then  it  will  severely  reduce  the  number  of  photons  reach¬ 
ing  the  detector  while  producing  only  a  marginal  improvement  in 
xy-  or  ^-resolution.  Making  the  pinhole  larger  than  the  diameter  of 
the  diffraction-limited  spot  allows  more  photons  to  be  detected  but, 
as  -80%  of  those  originating  from  the  plane  of  focus  were  already 
being  collected,  most  of  the  additional  signal  comes  from  adja¬ 
cent  planes,6  reducing  the  "-resolution.  Choice  of  the  proper  size 
and  shape  for  the  pinhole  is  a  sensitive  function  of  X,  and  the  objec¬ 
tive  lens  NA  and  magnification.  However,  even  an  “optimum”  1 
Airy  aperture  will  exclude  at  least  some  photons  (the  -20%  present 
in  the  outer  rings  of  the  Airy  disk),  and  this  represents  a  funda¬ 
mental  cost  of  using  the  confocal  pinhole. 

Given  these  uncertainties,  it  seems  unlikely  that  little,  if  any, 
additional,  useful  information  will  be  gained  from  the  out-of-focus 
light  as  long  as: 

•  The  “bright”  voxels  are  stained  at  least  lOx  more  intensely 
than  background  voxels  and  are  each  capable  of  producing  at 
least  20  detectable  photons/pixel  when  present  in  the  focus 
plane  of  a  confocal  instrument. 

•  The  confocal  detector  pinhole  is  kept  at  1  to  2  Airy  units,  the 
size  needed  to  demonstrate  a  lateral  resolution  equal  to  that  of 
a  WF  instrument  with  the  same  NA,  X,  etc. 

•  Both  instruments  are  designed  and  adjusted  in  accordance  with 
the  sampling  and  photon-efficiency  criteria  proposed  in  this 
book. 

•  The  advantage  of  the  confocal  approach  will  increase  as  the 
total  number  of  planes  sampled  (or  imaged)  is  reduced. 


6  Assuming  that  fluorescent  dye  is  located  there. 


Detection  and  Measurement  Losses 
The  Detector 

The  detector  characteristics  of  importance  to  3D  microscopy  have 
been  reviewed  by  several  authors  (Pawley,  1994;  Sheppard  et  at, 
1992;  Sandison  et  at,  1994).  The  characteristics  of  most  impor¬ 
tance  to  photon  efficiency  are: 

•  Effective  quantum  efficiency  (QEeS):  The  proportion  of  the 
photons  arriving  at  the  detector  that  actually  contribute  to  an 
output  signal  that  is  linearly  proportional  to  the  photon  input. 
(QE  is  often  a  strong  function  of  the  X  of  the  detected  photons, 
Fig.  2.10.) 

•  Noise  level:  This  includes  both  additive  noise,  in  the  form  of 
PMT  or  CCD  dark  current  noise  or  electronic  amplifier  noise, 
and  multiplicative  noise  in  the  form  of  random  variations  in 
size  of  the  actual  PMT  output  pulses  derived  from  identical 
input  photons. 

The  Photomultiplier  Tube 

Although  the  PMT  is  the  most  common  detector  used  in  the 
CLSM,  it  is  not  necessarily  the  ideal  detector.  While  the  raw  QE 
of  the  most  modem  PMTs  may  be  as  high  as  30%,  this  still  means 
that  70%  of  the  photons  produce  no  signal.  At  565  nm,  only  about 
15%  photons  striking  the  outer  surface  of  a  selected,  end- window 
PMT  with  an  S-20  multi-alkali  photocathode  are  absorbed  in  the 
thin  photocathode  (PC)  layer  evaporated  onto  the  inner  surface  of 
the  window  (Fig.  2.10).  The  remaining  85%  are  either  transmitted 
or  reflected.  As  only  those  photons  that  are  absorbed  can  produce 
photoelectrons  (PE)  for  subsequent  amplification  in  the  multiplier 
section  of  the  PMT,  it  is  clear  that  any  mechanism  that  reduces 
transmission  or  reflection  losses  may  substantially  improve  PMT 
QE.  This  can  be  done  by  introducing  the  light  into  the  end  window 
of  the  PMT  at  an  angle  such  that  it  is  totally  internally  reflected 
by  the  PC  (Gunter  et  at,  1970).  The  light  not  absorbed  in  the  PC 
then  totally  reflects  off  the  outer  surface  of  the  window  and  strikes 
the  PC  again.  This  process  continues  until  the  light  is  either 
absorbed  or  it  reaches  the  far  side  of  the  PC.  Using  such  an  optical 
enhancer,  we  have  reported  an  increase  in  QE  of  60%  at  520  nm, 
180%  at  690  nm,  and  220%  at  820  nm  (Pawley  et  al.,  1993). 

Transmission  losses  can  also  be  decreased  by  employing  a  new 
GaAs  and  GaAsP  photocathode  or  using  an  opaque  photocathode. 
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as  is  found  in  side-window  PMTs  (Fig.  2.10).  These  have  markedly 
higher  QE,  especially  in  the  red.  On  the  other  hand,  it  is  important 
to  remember  that  published  QE  curves  tend  to  represent  the  very 
best  performance  measured  and,  especially  on  end-window  tubes, 
there  is  lot  of  tube-to-tube  variation.  In  addition,  published  QE 
curves  only  show  the  fraction  of  photons  producing  photoelec¬ 
trons,  not  the  fraction  that  actually  reach  the  first  dynode  and  start 
a  cascade. 

Though  a  significant  achievement,  improved  PC  performance 
in  the  red  is  usually  accompanied  by  significantly  higher  dark- 
count  rates.  Although  the  dark  current  of  a  PMT  is  usually  low  (0. 1 
count/pixel),  it  is  a  strong  function  of  the  temperature  (Fig.  2.11). 
In  the  heated  confines  of  some  commercial  instruments,  it  may  not 
always  be  small  compared  to  the  signal  level  of  a  weakly  fluores¬ 
cent  sample,  and  this  is  especially  true  if  excitation  levels  have 
been  reduced  to  permit  photon-counting  without  pulse-pileup.  The 
fact  that  dark  count  rate  is  proportional  to  PC  size  explains  the 
trend  to  PMTs  with  10-mm  PC  diameters  rather  than  the  25-mm 
tubes  previously  common.  Dark  current  can  also  be  reduced  by 
cooling  the  PC. 

Of  more  concern  is  the  fact  that  PMT  output  pulses  produced 
by  single  PEs  vary  in  size  by  an  order  of  magnitude  because  of 
statistical  variations  in  the  small  number  of  particles  present  in  the 
early  stages  of  the  electron  multiplier  [Fig.  2.12(A)].  The  distri¬ 
bution  in  the  height  of  these  single  PE  pulses  is  shown  in  Figure 
2.12(B)  for  several  representative  types  of  PMTs.  Clearly,  if  one 
merely  sums  the  output  pulses,  some  electrons  will  count  more 
than  others.  This  produces  multiplicative  noise  and  the  value 
sensed  by  integrating  the  pulses  and  recording  the  sum  in  the  com¬ 
puter  will  have  more  uncertainty  than  had  the  photon  pulses  been 
counted  directly. 

As  most  of  the  scatter  in  the  pulse-height  distribution  reflects 
Poisson  noise  applied  to  the  small  number  of  secondary  electrons 
generated  at  the  first  dynode,  it  is  important  to  keep  this  number 
as  high  as  possible.  Figure  2.13  shows  that  first-dynode  gain  can 
be  as  high  as  20  on  average  but  only  if  a  relatively  high  voltage 
(600  V)  is  placed  on  the  first  stage  of  the  voltage  divider.  Current 
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FIGURE  2.1 0.  Variation  of  quantum  efficiency  with  wavelength  of  some  rep¬ 
resentative  PMT  photocathode  materials  and  different  types  of  CCDs.  Unless 
they  are  thinned  and  rear-illuminated,  the  performance  of  the  Si  sensors  in  a 
CCD  is  substantially  (50%)  less  in  the  green  and  blue  because  of  absorption 
and  scattering  in  the  overlying  poly  silicon  charge-transfer  electrodes.  Curves 
for  the  bialkyli,  multialkyli,  GaAs,  and  GaAsP  photocathodes  are  “raw”  figures 
of  the  photolelectrons  leaving  the  photocathode,  and  do  not  take  into  account 
the  ~30%  of  these  that  fail  to  multiply  at  the  first  dynode. 


FIGURE  2.1 1 .  Variation  of  PMT  dark  current  with  photocathode  temperature 
for  a  number  of  photocathode  materials.  The  SI  curve  represents  performance 
of  the  only  classical  photocathode  material  with  significant  response  in  the 
near-IR.  As  the  modern  replacement  of  SI,  GaAs,  also  has  high  dark  current 
and  is  often  used  with  a  cooled  photocathode.  Please  note  that  the  vertical  axis 
is  logarithmic. 


commercial  confocals  are  more  likely  to  use  less  than  half  this 
voltage,  leading  to  a  likely  gain  of  10.  Clearly,  this  will  introduce 
an  additional  uncertainty  in  the  process  of  trying  to  determine  how 
many  photons  were  detected  in  each  pixel.  Thirty  percent  multi¬ 
plicative  noise  can  only  be  overcome  by  collecting  about  60% 
more  signal.7  In  this  way  multiplicative  noise  reduces  the  effective 
QE  well  below  that  of  the  published  graphs.  Indeed,  when  one 
includes  the  “optimism”  that  often  characterizes  photocathode  QE 
curves,  the  signal  lost  because  some  photoelectrons  fail  to  propa¬ 
gate  at  the  first  dynode,  and  the  effect  of  multiplicative  noise,  the 
effective  QE  of  the  entire  PMT  is  usually  less  than  50%  of  what 
one  would  have  predicted  from  the  published  curves  alone. 

There  are  some  reasons  for  hope.  Hybrid  PMTs  amplify  pho¬ 
toelectrons  by  accelerating  them  directly  onto  a  silicon  photodiode 
[Fig.  2.14(A)].  By  using  an  acceleration  of  tens  of  kilovolts,  the 
electrons  arrive  at  the  diode  with  enough  energy  to  make  thou¬ 
sands  of  electron-hole  (e-h)  pairs.  This  extremely  high  “first-stage 
gain”  substantially  reduces  multiplicative  noise,  as  can  be  seen  in 
Figure  2.14(B),  which  shows  the  pulse-height  distribution  from  a 
hybrid  PMT  exposed  to  about  6  ±  2.5  photoelectrons.  Because 
every  photoelectron  produces  a  nearly  identical  number  of  e-h 
pairs  in  the  silicon,  pixels  in  which,  for  instance,  eight  arrive, 
produce  integrated  output  pulses  that  are  clearly  different  in  size 
from  those  made  by  either  seven  or  nine  photoelectrons.  This  pro- 


7  Or  by  photon  counting,  an  ability  lost  to  the  confocal  world  when  Bio-Rad 
ceased  making  confocal  microscopes. 
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FIGURE  2.12.  (A)  Interactions  of  photons  and  photoelectrons  in  the  photomulti¬ 
plier  tube  (PMT).  Most  photons  do  not  interact  with  the  photocathode.  Those  that 
do  create  photoelectrons,  some  of  which  strike  the  first  dynode  with  sufficient  energy 
to  create  a  number  of  secondary  electrons.  By  sequential  collisions  with  10  to  14 
subsequent  dynodes,  the  charge  packet  initiated  by  a  single  photoelectron  comes  to 
contain  more  than  106  electrons,  a  number  large  enough  to  be  well  above  the  noise 
floor  of  the  electronics  that  follow  the  PMT.  (B)  Spectrum  of  single-PE  pulse  heights 
for  three  representative  PMTs.  The  spread  in  the  major  peak  is  due  to  statistical  vari¬ 
ations  in  the  gain  of  the  early  stages  of  the  dynode  multiplier.  The  peak  of  very  small 
pulses  on  the  left  represents  PE  produced  thermally  from  the  later  dynodes.  In  pulse¬ 
counting,  these  smaller  pulses  are  eliminated  by  setting  the  discriminator  threshold 
near  the  valley  in  the  distribution.  [This  figure  redrawn  from  one  published  by 
Thorn/EMI  (Ruislip,  UK)]. 
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vides  the  advantages  of  photon  counting  without  the  danger  of  satu¬ 
ration  due  to  pulse  pileup.  The  problem  at  present  is  high  cost,  rel¬ 
atively  short  lifetime  and  sensitivity  to  damage  from  overexposure. 

Finally,  as  the  PMT  is  a  single-channel  device,  it  can  provide 
a  measure  of  only  those  photons  that  pass  through  the  pinhole 
mask.  This  makes  the  proper  setting  of  this  parameter  very  impor¬ 
tant.  One  cannot  decide  later  that  it  should  have  been  bigger  or 
smaller. 

The  Cooled  Charge-Coupled  Device 

The  only  practical  alternative  to  the  PMT  is  the  solid-state  detec¬ 
tor  of  which  the  cooled  charge-coupled  device  (CCD)  is  the 
optimal  example.  This  detector  has  both  different  capabilities  and 
different  problems  (see  Chapter  12  and  Appendix  3,  this  volume). 
The  QE  can  be  very  high  (70%-80%)  and  extends  well  into  the 
infrared  (Fig.  2.10).  Furthermore,  as  each  photon  is  recorded  as  an 
identical  amount  of  current,  there  is  no  multiplicative  noise.  Unfor¬ 
tunately,  to  keep  the  noise  level  acceptably  low  (+5  photoelec¬ 
tron/measurement),  this  detector  must  be  cooled  to  -40  to  80°C 
and  read  out  at  the  relatively  low  rate  of  100,000  pixels/s  (vs. 
400,000  pixels/s  for  a  normal  CLSM).  This  noise  level  is  clearly 
too  high  if  the  peak  signal  level  is  only  10  photons/pixel,  as  it  can 
be  on  many  CLSMs.8  It  is  less  serious  when  the  signal  from  the 


8  Many  users  may  find  it  hard  to  accept  that  the  brightest  pixel  in  their  con- 
focal  images  represents  a  signal  of  only  8-12  photons.  Indeed,  they  may  point 
to  a  histogram  of  such  an  image  and  note  that  there  are  some  pixels  present 
with  every  possible  value  from  0-255.  Surely,  if  there  are  256  possible  values, 
the  brightest  must  correspond  to  255  photons.  In  fact,  the  relationship  between 
photon  signal  level  and  digital  value  stored  in  the  image  memory  is  arbitrar¬ 
ily  determined  by  the  particular  PMT  gain  and  black-level  setting  chosen.  The 
reason  that  a  signal  representing  at  most  8  photons  is  not  displayed  as  a  pos- 
terized  image  containing  only  8  possible  gray  levels  is  that  multiplicative 
noise  in  the  PMT  blends  these  8  levels  into  each  other  so  that  the  transitions 
are  not  apparent.  Quite  simply,  some  photoelectrons  are  counted  as  "more 
equal  than  others.” 


darkest  pixel  is  >25  photoelectrons  because  then  statistical  varia¬ 
tions  in  this  number  (i.e.,  V25  =  5)  are  similar  in  size  to  the  mea¬ 
surement  noise.  These  features  make  the  cooled  CCD  detector 
more  suitable  for  slowly  scanned  images  (10-100s/frame)  pro¬ 
ducing  relatively  high  signal  levels.  (See  discussion  of  this  subject 
in  Chapters  10,  12,  and  23,  this  volume.) 

In  the  disk-scanning  and  line-scanning  confocal  microscopes, 
the  image  data  emerges  as  a  real  image  rather  than  as  a  time 
sequence  of  intensity  values  from  a  single  detector.  Although  this 
real  image  can  be  detected  photographically  or  by  eye,  both  of 
these  sensors  have  fairly  low  QE.  However,  these  confocal  micro¬ 
scopes  can  approach  the  photon  efficiency  of  the  CLSM  if  they 
incorporate  a  cooled  CCD  sensor  having  detection  performance 
similar  to  that  described  in  the  last  paragraph.  This  combination  is 
now  implemented  in  several  commercial  instruments. 

Of  even  more  importance  is  the  recent  introduction  of  the  elec¬ 
tron  multiplier  CCD  (EM-CCD).  In  the  EM-CCD,  a  gain  register 
is  introduced  between  the  end  of  the  horizontal  register  and  the 
read  amplifier.  The  gain  register  consists  of  -500  charge-coupled 
elements  in  which  one  of  the  three  charge  transfer  voltages  is  much 
higher  than  normal.  When  exposed  to  this  higher  voltage,  each 
electron  in  the  charge  pocket  has  about  a  1%  chance9  of  multiply¬ 
ing  by  collision  with  covalent  electrons  from  the  silicon  lattice. 
This  process  produces  two  electrons  and  when  repeated  -500 
times,  the  average  gain  can  be  over  lOOOx,  which  is  sufficient  to 
lift  the  signal  from  even  one  signal  electron  well  above  the  read 
noise  of  the  output  field  effect  transistor  (FET)  amplifier. 

This  produces  an  image  detector  with  the  QE  of  silicon  and 
the  noise  characteristics  of  a  PMT.  There  is  a  snag,  of  course:  the 
random  nature  of  the  electron  multiplication  process  produces 


9  The  useful  range  is  0.5%  to  1.5%  and  depends  on  how  high  the  voltage  on 
the  transfer  electrode  is. 
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FIGURE  2.13.  Variation  of  the  gain  from  collisions  that  occur  at  the  first 
dynode  with  cathode/first-dynode  voltage. 


even  more  multiplicative  noise  than  is  present  in  a  good  PMT.  As 
a  result,  the  output  signal  has  about  1.4x  more  noise  than  can  be 
attributed  to  Poisson  noise  alone.  As  this  can  only  be  reduced  by 
counting  2x  more  photons,  the  easiest  way  to  think  of  the  EM- 
CCD  is  that  it  has  no  read  noise,  but  the  QEeSt  is  only  50%  as  great 
as  it  would  have  been  in  the  absence  of  the  multiplicative  noise 
(i.e.,  a  back-illuminated  CCD  detector  having  a  QE  of  80%  would 
have  an  effective  QE  of  only  40%  if  it  were  read  out  by  an  EM- 
CCD  amplifier).  This  means  that  the  EM-CCD  is  only  optimal  in 
the  0  to  20  PE/pixel  signal  range  — just  where  the  disk-scanning 
confocals  operate10  (see  Chapters  10  and  43,  this  volume). 

In  addition  to  high  QE,  Si  photon  detectors  have  a  variety  of 
other  practical  advantages.  As  the  sensitive  element  in  such  a 
detector  is  typically  very  small  (5-30  pm  on  a  side),  selective  use 
of  only  a  few  elements  in  a  small,  planar,  2D  array  could  permit 
it  to  operate  in  the  CLSM  as  a  combination  pinhole  and  detector. 
Figure  2.15  is  a  sketch  of  what  such  a  detector  might  look  like. 
After  each  pixel  interval  of  the  microscope,  the  charge  pattern  in 
the  5x5  sensor  array  at  the  top  would  be  transferred  to  the  read 
register  and  then  the  signal  in  all  25  pixels  would  be  read  out 
sequentially  at  about  35  MHz.  These  25  values  could  then  be 
“decoded”  in  a  number  of  possible  ways,  the  most  straightforward 
of  which  would  be  to  provide  three  separate  signals  correspond- 


10On  first  hearing,  20  c/pixel  may  sound  like  a  very  low  level  of  signal.  In  fact, 
however,  considerable  experience  shows  that  much  fluorescence  confocal 
microscopy  is  performed  at  much  lower  signal  levels.  The  next  section  gives 
methods  for  those  with  instruments  capable  of  fast-photon  counting  to  cali¬ 
brate  their  PMT  gain  controls.  Those  who  follow  this  procedure  may  be 
surprised  to  find  that  when  they  are  using  “normal”  PMT  gain  (8.00  and 
above  on  recent  Bio-Rads),  256  stored  in  the  memory  corresponds  to  —10 
photons  or  less. 
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FIGURE  2.14.  Hybrid  PMTs.  (A)  Diagram  of  hybrid  PMT  layout.  Photoelec¬ 
trons  leaving  the  photocathode  are  accelerated  onto  a  silicon  photodiode  (or  an 
array  of  such  diodes)  with  sufficient  energy  to  produce  thousands  of  electron- 
hole  pairs  in  the  silicon.  The  very  high  gain  of  the  first  (and  only)  stage  greatly 
reduces  the  effect  of  multiplicative  noise.  (B)  A  plot  of  single-pixel  signal  levels 
obtained  from  such  a  hybrid  PMT  under  conditions  where  the  photocathode 
was  emitting  on  average,  about  6  photoelectrons/pixel.  The  clearly  defined 
peaks  represent  signal  levels  of  from  1  to  1 1  photoelectrons/pixel. 


ing  to  the  summed  signals  from  the  brown,  orange,  and  red  areas 
of  the  sensor  array.  In  this  way,  it  would  be  possible  to  collect 
signal  simultaneously  at  three  different  pinhole  sizes. 

With  such  a  detector,  pinhole  alignment  could  be  done  elec¬ 
tronically  simply  by  searching  for  the  detector  element  producing 
the  most  signal  from  a  planar  specimen  and  misalignment  could 
be  detected  on  the  fly  by  comparing,  for  example,  summed  output 
from  the  5  pixels  on  the  left  with  the  5  on  the  right  (Pawley,  1996). 

Digitization 

In  the  simplest  CLSM  system,  the  output  of  the  PMT  head  ampli¬ 
fier  is  passed  directly  to  the  analog-to-digital  converter  (ADC), 
which  samples  the  voltage  for  a  few  nanoseconds  during  the  time 
of  each  pixel  (tp)  and  turns  the  sensed  voltage  into  a  digital  number. 
As  tp  is  usually  a  few  microseconds,  it  is  important  to  ensure  that 
the  voltage  present  during  the  much  shorter  sampling  time  is  a 
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FIGURE  2.15.  Schematic  diagram  showing  a  proposed  solid-state  detector  that  could  be  used  to  replace  the  PMT  used  in  confocal  microscopes.  The  detector 
uses  an  electron-multiplier  register  to  reduce  the  read  noise  below  0.01  electron/pixel.  By  reading  out  all  25  pixels  in  the  sensor  array  at  each  pixel  of  the  con- 
focal  raster  and  then  apportioning  the  signal  to  the  three  concentric  areas  of  the  sensor,  this  device  would  allow  one  to  simultaneously  collect  signal  at  three  dif¬ 
ferent  effective  pixel  sizes. 


good  measure  of  the  average  signal  level  during  tp.  This  is  usually 
accomplished  by  limiting  the  bandwidth  of  the  amplifier  immedi¬ 
ately  preceding  the  ADC  by  giving  it  a  time-constant  of  tJA.  This 
approach  is  simple  and  effectively  expands  the  sampling  time  from 
~tp/1000  to  tp/4  without  excessively  blending  the  signal  from  each 
pixel  with  that  of  its  neighbors.  However,  this  still  means  that  the 
system  is  only  “counting”  about  25%  of  the  time. 

The  situation  can  be  improved  if,  as  mentioned  above,  the 
system  adjusts  the  electronic  bandwidth  to  coincide  with  the  optical 
bandwidth  of  the  microscope  and  changes  the  pre-amp  time  con¬ 
stant  to  compensate  for  changes  in  optical  resolution,  scan  speed 
(in  pm/s  as  the  focus  plane)  and  fp  (see  also  Chapter  4,  this  volume). 

Alternatively,  it  can  also  be  improved  if  the  CLSM  uses  a  dig¬ 
itizer  employing  full  integration.  Such  a  system  can  be  imple¬ 
mented  in  two  ways.  As  first  used  by  Sarastro/Molecular  Dynamics 
and  later  by  Bio-Rad,  the  output  of  the  PMT  is  integrated  by 
feeding  a  proportional  current  into  a  capacitor,  then  reading  the 
capacitor  voltage  out  to  an  ADC  and  finally  resetting  the  capaci¬ 
tor  voltage  back  to  0  (Fig.  2.16,  lower  left).  The  Bio-Rad  MRC- 
600  and  later  instruments  incorporate  three  circuits  of  this  type  in 
each  digitizing  channel:  one  accumulating,  the  second  being  read 
out,  and  the  third  being  reset. 

The  second  method  of  implementing  full  integration  is  to  feed 
the  output  from  a  high-bandwidth  (fp/20)  head  amplifier  into  a  high 
speed  ADC  running  at,  say,  lOx  the  pixel  rate  and  then  utilizing 
fast  digital  circuitry  to  average  the  10  successive  digital  readings 
needed  to  produce  the  single  value  actually  stored  for  each 
pixel. 


Compared  to  fp/4  bandwidth  limiting,  either  method  of  full 
integration  effectively  provides  4x  more  useful  signal  without  any 
more  light  being  detected. 

Photon  Counting 

Obtaining  a  digital  representation  of  optical  data  is  ultimately  a 
question  of  counting  photons.  Ideally,  this  means  not  only  using  a 
high-QE  detector  but  also  using  a  signal  to  which  the  contribution 
of  each  photon  is  equal.  In  the  case  of  the  solid-state  sensors,  the 
uniformity  condition  is  automatically  met  by  the  sensing  process 
(1  photon  =  1  PE)  but  this  condition  can  also  be  met  by  the  PMT 
if  it  is  operated  in  a  pulse-counting  mode. 

In  pulse-counting,  the  goal  is  not  to  measure  the  average  level 
of  the  output  current  during  the  Tp  but  rather  attempts  to  eliminate 
the  effects  of  multiplicative  noise  by  discriminating,  and  then 
counting,  the  individual  output  pulses  resulting  from  the  emission 
of  individual  PEs  from  the  PC  of  the  PMT.  To  reduce  the  effect  of 
small  noise  pulses  generated  from  the  dynodes  of  the  PMT,  photon 
pulses  are  passed  through  a  discriminator  set  to  trigger  on  pulses 
larger  than  those  at  the  bottom  of  the  “valley”  seen  near  the  left 
edge  of  the  pulse-height  plot  in  Figure  2.12(B  ).  Each  time  the  PMT 
output  goes  above  this  preset  threshold,  one  pulse  is  counted 
(Fig.  2.16). 

Unfortunately,  problems  arise  when  large  numbers  of  photons 
must  be  counted  in  a  short  time  because  the  output  of  the  PMT 
preamp  does  not  immediately  return  to  zero  after  each  pulse.  If  a 
second  pulse  arrives  before  the  first  one  is  over,  the  second,  or 
piled-up,  pulse  will  be  missed.  Beyond  the  problem  of  pile-up  is 
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FIGURE  2.16.  Two  alternative  approaches  to  counting  single-photon  pulses. 
In  both,  the  signal  from  a  fast  head  amplifier  is  passed  to  a  discriminator. 
Uniform  pulses  from  the  discriminator  can  either  be  counted  with  digital  cir¬ 
cuitry  or  integrated  in  a  capacitor  and  then  read  out  through  an  analog-to-digital 
converter  (ADC). 


the  difficulty  of  actually  counting  the  output  from  the  discrimina¬ 
tor  at  very  high  asynchronous  count  rates. 

Suppose  a  laser-scanning  instrument  scans  a512x512  raster 
in  1  s  and  25%  of  this  time  is  used  for  retrace.  That  leaves  Tp  = 
~3|is/pixel.  If  each  photon  pulse  occupies  30  ns  ( tp ),  the  maximum 
number  of  pulses  that  one  could  possibly  count  in  each  pixel  would 
be  100  but,  because  the  photons  arrive  at  random  times,  even  at 
one  tenth  of  this  rate  (i.e.,  10  counts/pixel),  10%  of  the  photons 
will  still  arrive  when  the  circuit  is  already  busy. 

In  general,  the  rate  that  pulses  are  recorded  RKC  is  given  by 

Rrec='rpAp(1-e'i?taP”fp/7’P)  (2) 

The  Bio-Rad  MRC-1024  had  a  maximum  count  rate  of  58 
counts/pixel  and  its  response  was  fairly  linear  up  to  about  15 
counts/pixel.  This  instrument  (and  its  successors)  was  unusual  for 
allowing  the  user  to  switch  to  the  photon-counting  mode  without 
any  complicated  adjustments.  This  ability  made  it  fairly  easy  to 
occasionally  switch  modes,  just  to  determine  the  number  of 
photons/pixel  that  were  actually  being  detected.  On  first  hearing, 
10  counts/pixel  may  sound  like  a  very  low  level  of  signal.  In  fact, 
however,  considerable  experience  with  just  this  sort  of  mode 
switching  shows  that  much  fluorescence  confocal  microscopy  is 
performed  at  much  lower  peak  signal  levels.  Although  one  should 
be  concerned  about  the  signal  saturation  that  occurs  when  one 
exceeds  the  linear  range  of  the  fast-photon  counting  circuitry,  it  is 
also  important  to  remember  that,  to  some  degree,  one  can  correct 
for  piled-up  losses  with  a  simple  look-up  table.11  Furthermore,  such 
losses  need  not  be  reduced  to  zero  but  merely  made  small  com¬ 
pared  to  the  intrinsic  statistical  noise.  Pile-up  losses  will  become 
less  important  if  more  manufacturers  switch  to  pulse-counting 
PMTs  and  use  faster  head  amplifiers  and  counters. 

With  regard  to  the  latter,  digital  counters  are  not  strictly 
mandatory.  All  that  must  be  done  in  order  to  remove  PMT  multi¬ 
plicative  noise  is  to  clip  all  the  single-PE  pulses  to  a  uniform  size 
and  feed  them  to  a  fully  integrating  ADC  (Fig.  2.16).  In  fact,  in 


11  If  10  were  counted  and  this  is  known  to  cause  10%  dead  time,  record  11. 


some  commercial  instruments,  much  of  the  beneficial  effect  of 
photon-counting  is  incorporated  into  the  analog  digitization 
system  by  the  simple  expedient  of  arranging  the  electronic  and 
PMT  gain  so  that  the  single-PE  pulses  saturate  a  fast,  high-gain 
amplifier  installed  between  the  PMT  and  the  slower  amplifier  stage 
that  leads  to  the  ADC.  Because  this  saturable  amplifier  is  fast,  each 
pulse  is  separately  clipped  to  a  uniform  height,  thereby  meeting 
the  criterion  that  each  photon  contribute  equally  to  the  final 
number  recorded. 


Where  Have  All  the  Photons  Gone? 

All  present  instruments  embody  design  compromises  that  prevent 
them  from  obtaining  the  ultimate  in  photon  efficiency  throughout 
the  four  processes  discussed  above.  Many  systems  employ  more 
refractive  optics  than  is  absolutely  necessary  and  most  suffer  trans¬ 
mission  losses  in  the  range  of  30%  to  65%  (transmission  figures  as 
a  function  of  X  for  a  number  of  modern  objectives  are  given  in 
Table  7.3,  this  volume).  Although  the  metal  mirrors  that  produced 
losses  of  85  %  in  early  confocal  instruments  have  now  been  replaced 
with  broadband,  dielectric-multilayer  mirrors  that  reflect  98%  of 
in-bandwidth  light,  the  loss  at  each  surface  is  multiplicative.  The 
main  culprit  is  often  the  triple  dichroic  beam  splitter.  Far  from  just 
reflecting  at  laser  lines,  this  component  often  sends  as  much  as  40% 
of  the  light  between  the  laser  lines  back  towards  the  laser.  The  use 
of  an  acousto-optical  device  as  a  beam-splitter  (AOBS)  by  Leica  has 
made  a  significant  contribution  to  alleviating  this  problem  (see  Figs. 
3.10  and  3.23,  this  volume ). 

Insufficient  attention  is  often  paid  to  the  selection  and  adjust¬ 
ment  of  the  PMT  itself.  While  many  recognize  that  any  specific 
tube  will  operate  best  over  only  a  narrow  range  of  accelerating 
voltages  and  that  those  with  bialkali  photocathodes  have  lower 
noise  and  higher  QE  in  the  green,  while  those  with  S-20,  multi¬ 
alkali,  photocathodes  are  better  in  the  red  and  near-infrared,  it  is 
usually  forgotten  that  the  performance  of  individual  tubes  often 
varies  from  the  mean  for  their  type  by  more  than  a  factor  of  2.  In 
addition,  the  new  side-window  PMTs  that  are  used  in  a  number  of 
systems  have  a  response  in  the  red  that  is  almost  3x  that  of  tradi¬ 
tional  end-window  tubes.  While  this  performance  also  implies  a 
substantial  increase  in  dark  current,  cooled  versions  are  now 
available. 

Additional  degradation  is  imposed  on  the  data  by  multiplica¬ 
tive  noise  and  poor  digitizing  circuitry.  Finally,  signal  is  often  lost 
because  of  poor  alignment,  and  the  improper  choice  of  pinhole 
diameter  may  exclude  as  much  as  90%  of  the  useful  signal  from 
the  detector  in  a  vain  attempt  to  extract  an  “imaginary”  improve¬ 
ment  in  resolution  (imaginary  because  the  low  signal  levels 
prevent  useful  information  from  being  obtained  before  the  speci¬ 
men  is  destroyed). 

Taken  together,  all  of  these  factors  can  add  up  to  a  factor  of 
100X  or  more  in  photon  efficiency  between  state-of-the-art  and 
sloppy  operation  on  poorly  chosen  equipment.  Every  action  that 
results  in  more  efficient  use  of  the  photons  generated  within  the 
specimen  should  be  thought  of  as  being  directly  responsible  for 
making  it  possible  to  collect  a  proportionally  larger  number  of 
images  (or  images  with  better  statistics)  before  the  specimen  is 
damaged. 


Measuring  Photon  Efficiency 

The  PMT  output  signal  is  the  only  source  of  readily  available  data 
with  which  to  try  to  measure  the  photon  efficiency.  Unfortunately, 
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a  large  number  of  parameters  can  have  a  major  effect  on  this  single 
measurement  (Pawley,  2002).  Those  discussed  above  include: 

•  Laser  power,  which  is  a  function  of  temperature,  cavity  gain, 
precision  of  stabilizing  circuit,  etc. 

•  The  transmission  of  ND  and  other  filters  used  at  a  par¬ 
ticular  X. 

•  Proper  alignment  of  the  optics  launching  the  light  into  the  fiber. 

•  The  NA  and  transmission  of  the  objective  lens  and  other  optics. 

•  Reflectivity  of  the  mirrors  for  the  particular  X  and  polarization 
in  use. 

•  The  fraction  of  the  laser  beam  that  is  actually  accepted  by  the 
objective  back  entrance  pupil  (varies  with  NA/magnification). 

•  Pinhole  diameter  and  alignment. 

•  PMT  voltage  and  black-level  setting. 

•  Staining  density  and  type  of  dye. 

•  Focus  level  and  RI  of  embedding  media. 

The  number  and  diversity  of  these  parameters  make  it  difficult 
to  obtain  a  measure  of  the  fraction  of  the  photons  leaving  the 
focused  spot  that  are  actually  counted  as  part  of  the  stored  image 
data. 

What  is  needed  is  a  stable  point-source  of  light  of  known  inten¬ 
sity  that  can  be  mounted  conveniently  below  the  objective.  One 
way  to  make  such  a  source  is  by  allowing  a  known  amount  of  laser 
light  to  strike  a  stable  phosphor.  One  first  measures  the  light 
emerging  from  the  objective  (as  noted  above)  and  adjusts  it  to 
some  standard  amount.  Specimens  that  maintain  a  constant  level 
of  fluorescent  efficiency  (i.e.,  ones  that  do  not  bleach  or  change 
with  time)  include  such  inorganic  phosphors  as  single  crystals  of 
CaF2-Eu  or  YAG-Ce  and  uranyl  glass.  Unfortunately,  although 
these  materials  are  very  stable  under  intense  laser  illumination, 
they  also  have  very  high  RI.  Consequently,  high  NA  objectives  are 
unable  to  form  an  aberration-free  focus  within  them,  and  because 
of  this  the  signal  that  they  generate  at  the  PMT  decreases  rapidly 
as  the  focus  plane  moves  into  the  material.  However,  they  can  be 
useful  to  those  who  normally  use  objectives  of  lower  NA  where 
spherical  aberration  effects  are  a  less  serious  factor.  Cubes  of  flu¬ 
orescent  plastic  or  uranyl  glass  also  offer  a  convenient  way  of 
demonstrating  correct  alignment  of  the  laser  with  the  objective 
back-focal  plane  (BFP;  see  Chapter  36,  this  volume).  An  alterna¬ 
tive  fluorescence  standard  can  be  fabricated  by  dissolving  dye  in 
immersion  oil  or  water  (depending  on  the  correction  of  the  objec¬ 
tive),  but  such  specimens  are  not  always  as  stable  over  time  as  one 
might  wish. 

A  more  direct  approach  to  measuring  photon  efficiency 
involves  using  the  microscope  simply  to  image  a  small  light  source 
such  as  a  light-emitting  diode  (LED)  specimen,  or  the  light  formed 
by  the  microscope’s  normal  transmission  illumination  system  set 
up  for  Kohler  illumination  (Fig.  2.17).  The  arc  or  incandescent 
transmission  source  must  be  provided  with  a  known  and  constant 
power  supply.  Once  this  has  been  set  up,  the  only  major  variables 
remaining  are  the  amount  of  metal  deposited  on  the  inside  of  the 
glass  envelope  surrounding  the  source,  the  bandpass  effects  of  any 
filters  that  remain  in  the  light  path,  the  pinhole  diameter,  and  the 
PMT  voltage.  In  many  instruments,  it  is  relatively  easy  to  remove 
all  of  the  dichroic  and  bandpass  filters  and  let  the  light  from  the 
image  plane  pass  directly  to  the  PMT.  Under  these  conditions,  one 
should  get  a  standard  reading  with  a  given  objective,  pinhole  size, 
and  lamp  power.  Although  the  microscope  is  now  a  flying  spot 
detector  and  only  collects  photons  from  one  pixel  at  a  time  with 
the  result  that  the  effective  signal  intensity  is  about  250,000x  less 
than  if  the  PMT  were  measuring  the  entire  field,  it  will  often  still 
be  necessary  to  place  ND  filters  between  the  source  and  the  con¬ 


denser  lens  to  permit  the  PMT  to  be  operated  at  a  voltage  similar 
to  that  normally  used.  Line-frequency  variations  in  the  filament 
heating  current  may  be  visible  in  the  data  as  will  any  instability 
in  the  plasma  of  arc  sources.  For  this  reason,  it  may  be  more 
convenient  to  measure  the  PMT  output  with  an  analog  DC  volt¬ 
meter  than  with  the  ADC  and  image  memory  (see  Chapter  36,  this 
volume). 

With  such  a  setup,  one  can  also  calibrate  the  actual  effective 
size  of  the  pinhole.  To  do  this,  record  the  PMT  signal  level  (7S)  as 
a  function  of  the  pinhole  setting.  Is  should  vary  as  the  square  of 
the  pinhole  diameter,  and  a  plot  of  versus  pinhole  diameter 
should  be  a  straight  line  passing  through  the  origin.  Departures 
from  linearity  may  indicate  that  dust  is  obscuring  part  of  the  aper¬ 
ture  or  that  the  pinhole  control  does  not  give  a  very  accurate  esti¬ 
mate  of  pinhole  diameter. 

By  introducing  ND  filters  below  the  stage,  it  is  possible 
to  reduce  the  intensity  of  the  light  to  the  level  at  which  photon¬ 
counting  is  appropriate.  On  those  instruments  that  have  this  ability, 
one  can  easily  make  measurements  to  determine  a  rough  ratio 
between  the  actual  number  of  photons  being  detected  (using 
photon-counting)  and  analog-intensity-values  stored  in  the 
memory  at  the  same  PMT  gain  settings.  This  is  done  by  recording 
the  same  brightfield  “image”  in  both  analog  and  photon-counting 
modes.  Such  information  should  be  used  to  reach  a  rational  deci- 


FIGURE  2.17.  Optical  setup  for  measuring  the  detection  efficiency  or  the 
effective  size  of  the  pinhole  using  the  internal  transmitted  illumination  system 
of  the  microscope  as  a  standard  light  source. 
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FIGURE  2.18.  Effect  of  photon  counting.  The  two  images  were  made  using  the  confocal  detection  system  as  a  flying-spot  scanner  to  record  the  light  from  the 
internal  transmitted-light  illumination  system  set  up  for  Kohler  illumination.  Both  images  were  recorded  with  a  PMT  setting  of  8.00  but  (A)  was  recorded  in  the 
analog  mode  while  (B)  was  recorded  using  the  fast-photon-counting  mode  (and  the  contrast  was  subsequently  stretched  as  the  brightest  pixel  had  only  12  counts 
recorded  in  it). 


sion  about  when  one  can  use  photon-counting  without  fear  of  pulse 
pile-up  (i.e.,  that  if  the  PMT  gain  is  above  XX,  the  signal  level 
registering  as  255  in  the  image  memory  is  only  “x”  photons/pixel 
and  so  one  can  switch  to  photon-counting  for  lower  multiplicative 
noise  without  any  worry  of  pile-up  losses).  Figure  2.18  shows  two 
such  images  with  their  accompanying  histograms.  The  circle  is 
formed  by  the  field  diaphragm  that  has  been  partially  closed 
to  provide  a  “zero-signal”  reference.  Different  output  look-up 
tables  had  to  be  used  to  display  the  two  images  because  the  photon¬ 
counting  image  had  only  12  counts  in  the  brightest  pixel. 

Finally,  if  a  calibrated  radiometer  is  available,  it  is  possible  to 
use  this  setup  to  measure  the  actual  detection  efficiency  of  the 
instrument,  Deff.  From  an  image  such  as  that  in  Figure  2.19(B),  it 
is  easy  to  measure  the  actual  radius  (rf)  of  the  field  diaphragm  in 
the  focus  plane  in  micrometers.  If  one  can  then  measure  the  total 
light  power  (P)  passing  through  this  plane  by  placing  the  sensor 
of  a  photometer  calibrated  to  measure  milliwatts  and  one  can 
specify  the  X  of  these  photons  by  placing  a  narrow-band  filter  in 
the  illumination  light  path,  one  can  then  calculate  the  flux  of 
photons  passing  through  this  plane  in  photons/pm2,  ( F ). 

The  energy  of  a  single  photon  is  hv  =  hc/X,  where  c  is  the  speed 
of  light  and  h  is  Planck’s  constant,  so  the  total  number  of  photons 
having  wavelength  X  coming  through  the  condenser  and  into  the 
sensor  of  the  photometer  is  n  =  PX/hc  (photons/s).  The  fraction  of 
those  that  should  then  reach  the  PMT  is  simply  the  ratio  of  the 
effective  area  of  the  pinhole  (nrl)  to  the  effective  area  of  the  field 
diaphragm.  The  effective  radius  of  the  pinhole  is  its  physical  size 
( rp )  divided  by  the  total  magnification  between  it  and  the  focus 
plane  ( ).  In  the  Bio-Rad,  rp=  8mm  with  the  pinhole  all  the  way 
open  and  Meff=  SSfM^eXMobj).  With  a  lOx  lens  and  a  Mtube=  1.5 
(assuming  1.25x  for  the  fluorescence  attachment  and  1.25x  for  the 
differential  interference  contrast  (DIC)  attachment.  For  modern, 
infinity-conjugate  microscopes,  Mtube  =  1),  rQ=  5  pm.  (Values  of 
MtS  for  a  number  of  confocal  instruments  are  tabulated  in 
Appendix  2.) 


If  the  radius  of  the  field  diaphragm  is  set  to  rf=  500  pm  in  the 
focus  plane,  then  the  fraction  of  the  light  from  this  plane  entering 
the  detector  pinhole  is  ( rjrf )2  =  (5/500)2  =  0.01%,  and  the  total 
number  of  photons  striking  the  PMT,  nPMT ,  in  sampling  time  ts( s) 
will  be 


FIGURE  2.19.  Contrast  transfer  function  (CTF)  of  a  microscope  optical 
system.  The  black  line  shows  how  the  contrast  in  the  image  of  a  high-contrast 
bar  pattern  varies  with  the  spatial  frequency  of  the  pattern  (spacings/|im).  The 
gray  line  represents  the  response  measured  if  the  dark  bars  in  the  test  target 
reflect  (or  transmit)  70%  of  the  incident  light  rather  than  0%.  The  dashed  line 
represents  the  imaginary  situation  of  a  CTF  in  which  all  spatial  frequencies  up 
to  the  “resolution  limit”  really  have  equal  contrast.  The  diagram  makes  clear 
why  high  frequency  features  of  a  test  object  are  more  likely  to  remain  visible 
above  any  arbitrary  noise  level  (5%  of  26.5%  shown)  if  they  have  100%  con¬ 
trast  rather  than  30%. 
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where  DcS  is  the  transmission  efficiency  of  the  optical  system  and 
K  is  a  constant  that  relates  the  power  measured  coming  out  of  the 
condenser  P,  to  Deff  and  the  number  of  photons  counted  in  the 
PMT.  For  a  Bio-Rad  setup  as  noted  above  and  with  P  measured 
in  nanowatts  and  X  =  546  nm,  K  =  0.04  photons/pixel/nW.  The 
comparison  between  photons/pixel  measured  at  the  image  plane 
with  those  detected  by  photon-counting  is  a  worthwhile,  though 
sobering,  exercise. 

Nothing  increases  the  probability  that  normal  operating  pro¬ 
cedures  are  optimal  as  much  as  practicing  these  techniques  under 
test  conditions  because,  in  this  case,  one  knows  if  one  is  getting 
the  correct  answer  (see  Chapters  35  and  36,  this  volume,  for  addi¬ 
tional  performance  tests). 


RESOLUTION:  HOW  MUCH  IS  ENOUGH? 

A  peculiar  feature  of  any  properly  aligned  confocal  microscope  is 
that  it  is  always  “in  focus,”  in  that  the  exciting  and  detecting  light 
paths  are  always  focused  into  the  same  diffraction-limited  spot. 
Details  of  the  size  and  shape  of  the  volume  sampled  in  this  way 
are  dependent  almost  entirely  on  the  design  of  the  optical  system 
(especially  the  NA  of  the  objective  lens  and  the  RI  of  the  medium 
surrounding  the  specimen  as  formulated  in  the  Abbe  equation).  In 
broad  terms,  the  effective  xy-resolution  in  biological  confocal  flu¬ 
orescence  microscopy  is  essentially  the  same  as  it  is  in  widefield,12 
while  the  z-resolution  is  about  3x  worse  at  NA  1 .4  and  4x  worse 
at  NA  1.2.  A  more  detailed  analysis  can  be  found  in  Chapters  1,  7, 
and  8.  In  addition,  there  is  the  important  trade-off  between  spatial 
resolution,  statistical  accuracy,  and  radiation  damage,  as  outlined 
above  and  related  to  the  setting  of  the  pinhole  diameter. 

Rather  than  repeat  the  straightforward  aspects  of  theoretical 
resolution  again  here,  this  section  will  discuss  three  other  aspects 
that  are  perhaps  less  obvious  but  which  are  both  characteristic  of 
confocal  microscopy  and  that  fundamentally  limit  the  ability  of  the 
instrument  to  solve  biological  problems.  These  topics  are: 

•  Circumstances  under  which  it  may  be  desirable  to  reduce 
resolution. 

•  The  effect  of  image  digitization  on  spatial  and  temporal 
resolution. 

•  Practical  considerations  that  may  degrade  resolution  or 
produce  distortion. 


Can  Resolution  Be  Too  High? 

Normally,  the  microscopist  makes  every  effort  to  obtain  the  highest 
possible  spatial  resolution,  and,  to  the  extent  that  the  specimen 
is  not  degraded  by  being  so  observed,  this  is  entirely  proper. 
Although,  in  the  case  of  biological  samples,  the  condition  of  non¬ 
degradation  is  never  entirely  met,  such  samples  can  be  rendered 
very  robust  by  treatment  with  anti-bleaching  agents  and  fade- 


12  Although  other  chapters  in  this  volume  note  that  the  ultimate  xy-resolution 
in  confocal  is  1.4x  smaller  than  that  in  widefield,  this  assumes  the  use  of  a 
very  small  pinhole.  Given  the  low  signal  levels  that  characterize  stained  bio¬ 
logical  specimens  one  virtually  never  uses  such  a  small  pinhole.  The  state¬ 
ment  above  assumes  a  pinhole  diameter  of  2  Airy  units. 


resistant  dyes  (Chapters  16,  18,  and  39,  this  volume).  However, 
such  fixed  and  chemically  protected  structures  cannot  be  consid¬ 
ered  the  general  case.  Recently,  significant  improvements  in  the 
photon  efficiency  of  confocal  instruments  has  led  to  them  being 
used  increasingly  for  viewing  living  specimens  (Chapter  19,  this 
volume).  Indeed,  experiments  in  which  thousands  of  images  have 
been  recorded  from  particularly  hardy  cell  lines  are  no  longer 
unusual. 

To  determine  when  it  may  be  advisable  to  intentionally  reduce 
spatial  resolution,  it  will  be  helpful  to  consider  a  second  specimen 
type:  a  living  cell  that  has  been  micro-injected  with  a  fluorescent 
substance  that  changes  its  absorption  spectrum  in  response  to 
the  local  concentration  of  certain  ionic  species,  a  concentration 
expected  to  change  on  the  scale  of  a  few  micrometers.  The  object 
of  the  experiment  is  to  monitor  changes  in  the  concentration  of  this 
ion  as  a  function  of  time  and  experimental  conditions.  The  major 
complication  is  that  the  dye  is  cytotoxic,  and  this  is  especially  true 
when  it  is  excited  by  light.  Furthermore,  the  fluorescence  must  be 
measured  accurately  at  two  different  exciting  wavelengths  to  deter¬ 
mine  the  ionic  concentration  by  ratioing  the  results.  Because  of 
diffusion,  significant  changes  in  ion  concentration  are  not  expected 
to  occur  on  the  scales  smaller  than  1  pm. 

How  does  one  optimize  an  instrument  to  perform  this  experi¬ 
ment?  Clearly  the  cytotoxicity  problem  implies: 

•  Using  a  water-immersion  lens  of  large  NA  to  collect  as  much 
light  as  possible. 

•  Using  the  lowest  dye  concentration  consistent  with  the  statis¬ 
tical  accuracy  required. 

•  Using  a  magnification  such  that,  when  referred  to  the  speci¬ 
men,  a  pixel  is  on  the  order  of  1  pm2. 

The  amount  of  illuminating  light  that  should  be  used  must  be 
determined  on  the  basis  of  the  number  and  accuracy  of  the  mea¬ 
surements  that  are  to  be  made.  As  it  is  important  to  measure 
changes  in  ionic  concentration,  it  can  be  assumed  that  the  time 
available  for  each  measurement  is  a  constant,  probably  set  by  the 
maximum  scan  speed  of  the  galvanometers,  although  the  intensity 
of  the  illumination  used  to  make  the  images  need  not  be.  Both  sta¬ 
tistical  accuracy  and  toxicity  will  increase  with  the  intensity  of  the 
exciting  beam.  As  “biological  accuracy”  cannot  be  sacrificed  to 
obtain  better  statistical  accuracy,  conditions  must  be  adjusted  to 
maximize  the  number  of  measurements  that  can  be  made  before 
the  cell  is  damaged.  Considering  this  problem  from  the  point  of 
view  of  a  single-beam  confocal  will  highlight  the  effect  of  fluo¬ 
rescence  saturation. 

The  need  for  ratio  imaging  implies  a  fairly  short  scan-time  to 
avoid  changes  in  the  specimen  between  the  two  component 
images.  The  need  for  statistical  accuracy  implies  an  intensely  illu¬ 
minated  spot  to  produce  many  countable  photons  but,  in  single¬ 
beam  confocal  systems,  saturation  puts  a  limit  on  the  maximum 
useful  flux  that  can  be  used.  Fortunately,  the  low  spatial  resolution 
required  provides  some  flexibility.  The  flux  of  exciting  illumina¬ 
tion  is  highest  at  the  neck  of  the  cone  of  illumination  formed  by 
the  objective  lens,  and  the  diameter  of  this  cone  will  be  smaller 
(and  the  flux  higher)  when  the  lens  used  to  form  it  is  operated  at 
full  NA.  In  the  example  described,  however,  the  large  NA  was 
chosen  to  provide  high  collection  efficiency,  not  high  resolution. 
As  the  pixel  size  is  to  be  1  pm,  there  is  no  need  to  focus  the  entire 
beam  into  a  0.2-pm  spot  because  this  will  produce  a  maximum 
flux  density  25  times  greater  than  that  present  in  a  1-pm  spot  and 
incur  a  proportionally  greater  risk  of  dye  saturation.  (Note:  To 
insure  that  the  signal  is  collected  from  the  entire  area  of  the  larger 
spot,  a  5x  larger  pinhole  diameter  must  be  used.) 
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When  using  coherent  laser  illumination,  the  only  way  to  make 
the  spot  larger  is  to  under-fill  the  BFP  of  the  objective  lens,  thereby 
reducing  its  effective  NA.  However,  if  one  is  using  a  more  diffuse 
source  such  as  an  Hg  arc,  one  can  use  a  larger  source  “pinhole”  in 
the  intermediate  image  plane.  Although  under-filling  the  objec¬ 
tive  reduces  the  ^-resolution,  considerable  depth  discrimination 
remains,  and  the  confocal  microscope  retains  all  of  the  other  desir¬ 
able  features,  such  as  quantitative  measurement  accuracy,  and  the 
ability  to  scan  a  plane  within  a  larger  specimen  often  makes  it  ideal 
for  studies  of  this  kind. 

The  lesson  here,  then,  is  that  because  of  the  twin  considera¬ 
tions  of  dye  diffusion  and  saturation,  it  is  often  important  to  be 
able  to  adjust  the  image-forming  properties  of  the  instrument  to 
produce  a  spot  size  appropriate  to  the  experiment  at  hand.  The 
ability  to  intentionally  under-fill  a  high-NA  objective  is  poorly 
implemented  on  current  commercial  instruments. 

Limitations  Imposed  by  Spatial  and 
Temporal  Quantization 

Although  the  image  viewed  in  a  disk-scanning  or  slit-scanning 
confocal  microscope  is,  in  principle,  as  continuous  as  that  from  a 
WF  microscope,  this  distinction  is  lost  when  the  image  is  finally 
sensed  using  a  cooled  CCD  or  a  digital-video  image  sensor.  The 
fact  that  all  digital  confocal  images  must  be  recorded  and  treated 
in  terms  of  measurements  made  within  discrete  pixels  can  limit  the 
effective  resolution  of  the  instrument  in  ways  that  may  not  be 
familiar  to  some  who  approach  digital  microscopy  for  the  first 
time.  In  a  sense,  these  limits  are  more  practical  than  fundamental 
because,  if  the  microscope  operation  follows  the  rules  of  Nyquist 
sampling  theory  as  discussed  below  (and  in  greater  detail  in 
Chapter  4,  this  volume ),  these  limits  should  present  no  obstacle  to 
recording  good  images.  However,  because  incautious  use  of  the 
“zoom”  magnification  control  found  on  all  commercial  CLSMs 
makes  it  relatively  easy  to  operate  these  instruments  outside  the 
Nyquist  conditions,  a  brief  discussion  of  sampling  theory  is  also 
included  here.  It  is  mentioned  under  the  heading  “Resolution” 
because  it  involves  the  ability  to  record  "the  separation  of  two 
closely  spaced  objects.” 

Spatial  Frequencies  and  the  Contrast 
Transfer  Function 

Sampling  theory,  like  resolution  itself,  is  often  easier  to  think  about 
accurately  in  the  “spatial  frequency  domain”  where  one  consid¬ 
ers  not  the  size  of  objects  themselves  but  the  inverse  of  their  size: 
“How  many  objects  are  visible  per  millimeter?”  In  the  spatial  fre¬ 
quency  domain,  an  image  is  seen  as  being  composed  of  the  spac- 
ings  between  features  rather  than  the  features  themselves.  The 
reason  for  using  this  seemingly  obscure  mental  construct  is  that 
the  ability  of  an  optical  system  to  transmit  information  depends 
entirely  on  the  spatial  frequency  of  this  information.  More  specif¬ 
ically,  all  optical  systems  reduce  the  contrast  of  the  high  spatial 
frequencies  (representing  smaller  spacings)  more  than  they  do 
lower  spatial  frequencies  (representing  larger  features).  This  fact 
is  made  evident  when  one  plots  the  contrast  transfer  function 
(CTF)  of  an  optical  system  by  measuring  the  contrast  present  in 
an  image  of  a  test  object  made  up  of  regular  arrays  of  black-and- 
white  bars  each  having  a  specific  spacing  or  frequency 
(Oldenbourg  et  al.,  1993;  Chapter  1  and  Figures  35.6-35.11  in 
Chapter  35,  this  volume ).  Such  a  CTF  is  shown  by  the  black  line 
in  Figure  2.19  and  the  gray  line  below  it  represents  the  image 


contrast  produced  by  a  test  target  that  is  100%  white  and  70%  gray 
(i.e.,  one  with  only  30%  contrast). 

Contrast  Transfer  Function  and  Resolution 

Although  in  common  parlance  the  word  “resolution”  is  often  used 
as  though  it  were  an  independent  parameter.  Figure  2.19  makes  it 
clear  that  the  number  chosen  to  represent  the  “resolution”  is  really 
somewhat  arbitrary.  It  refers  to  the  highest  spatial  frequency  at 
which  the  contrast  is  above  some  given  value.  For  instance,  the 
Rayleigh/Abbe  criterion  for  bright,  point-objects  on  a  black 
background  (i.e.,  stars  on  a  clear  night)  really  assumes  that  the 
minimum  visible  contrast  is  25%.  The  problem  with  this  simplis¬ 
tic  attitude  to  resolution  is  that  it  can  give  one  the  idea  that,  no 
matter  what  their  original  contrast  (i.e.,  the  staining  specificity)  in 
the  object,  all  spatial  frequencies  up  to  “the  resolution”  are  equally 
visible  as  is  implied  by  the  upper  dashed  line  in  Figure  2.19.  In 
fact,  the  most  important  message  from  Figure  2.19  is  that  the 
contrast-in-the-image  is  proportional  to  contrast-in-the-object  as 
degraded  by  the  CTF  of  the  imaging  system,  and  in  particular,  the 
contrast  of  small  features  just  within  the  “resolution  limit”  is 
much  lower  than  that  of  larger  features. 

Visibility,  Resolution,  and  the  Rose  Criterion 

The  reciprocal  relationship  between  contrast  and  resolution  is 
important  because  usually  what  one  is  actually  interested  in  is  not 
resolution  per  se  but  visibility:  the  ability  of  an  observer  to  rec¬ 
ognize  two  closely  spaced  features  as  being  separate.  Visibility  is 
a  much  more  slippery  concept  because  it  depends  not  only  on  the 
calculable  aspects  of  diffraction  theory  but  on  the  higher  functions 
that  our  visual  system  uses  to  determine  how  a  particular  pattern 
of  light  intensity  should  be  interpreted.  As  a  result,  there  are  few 
standard  methods  for  measuring  visibility. 

On  the  other  hand,  visibility  has  the  advantage  of  requiring  us 
to  consider  another  important  parameter  of  the  image  data:  its 
signal-to-noise  ratio  (S/N).  The  Rose  Criterion  states  that,  to  be 
visible,  a  dark  feature  that  is  a  single  pixel  in  size  must  have  an 
intensity  that  differs  from  that  of  a  white  background  by  at  least  5 
times  the  noise  level  of  the  background13  (Rose,  1948).  Although 
the  factor  5  is  somewhat  lower  for  lines  and  other  geometrical  fea¬ 
tures,  the  point  is  that  visibility  depends  on  more  than  geometri¬ 
cal  optics  and  the  CTF.  Assuming  that  the  contrast  is  measured  in 
units  that  are  proportional  to  the  statistical  precision  with  which 
each  signal  level  is  known,  visibility  also  depends  on  both  the 
absolute  and  the  relative  difference  between  the  signal  levels  of 
the  feature  and  the  background.  In  the  case  of  being  able  to  resolve 
two  point  features,  the  Rose  Criterion  requires  that  the  noise  be 
low  enough  and  the  contrast  be  high  enough  that  one  can  dis¬ 
criminate  the  lower  intensity  of  the  pixel  between  the  two  features 
as  being  different  from  that  of  the  peaks.  A  number  of  authors  have 
pointed  out  that,  in  fluorescence  confocal  microscopy,  one  seldom 
counts  enough  photons  for  the  25%  contrast  of  the  Abbe  resolu¬ 
tion  to  be  visible  with  any  degree  of  confidence. 

As  with  the  CTF  curve,  where  resolution  is  defined  in  terms 
of  contrast  in  the  final  image,  visibility  requires  that,  as  smaller 
objects  have  lower  contrast,  more  photons  will  have  to  be  counted 
to  reduce  the  Poisson  noise  of  the  measurement  well  below  the  low 
contrast  of  the  features. 

The  reader  can  see  that  visibility  brings  together  the  contrast 
of  the  object  (as  embodied  in  its  staining  characteristics),  the  focus- 


13  The  choice  of  a  dark  feature  on  a  white  ground  is  not  arbitrary.  A  white  signal 
has  significant  but  measurable  Poisson  noise.  A  truly  black  background  does 
not. 
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ing  properties  of  the  optical  system  as  defined  by  the  CTF  and  the 
statistical  requirements  of  the  Rose  Criterion.  However,  there  is 
still  one  other  process  that  can  limit  whether  or  not  a  feature  is 
visible  in  the  final  image  and  that  is  how  the  signal  is  digitized. 

Digitization  and  the  Nyquist  Criterion 

To  convert  any  continuous,  analog  signal  into  a  digital  represen¬ 
tation,  it  is  sampled  by  measuring  its  intensity  at  regular  intervals 
in  time  (or  space).  Suppose  that  one  is  recording  the  air  tempera¬ 
ture  at  a  particular  location  over  time.  How  often  should  one  make 
such  measurements?  Clearly  the  answer  depends  on  the  rate  of 
change  of  the  “features”  of  the  temperature  record  that  are  of  inter¬ 
est.  These  might  be  the  effects  of  breezes  or  thermals  that  may 
change  over  seconds  or  less,  or  they  might  be  climatic  changes  that 
might  only  be  noticeable  over  centuries  or  millennia.  Nyquist’s 
crucial  insight  was  that  there  is  a  fixed  relationship  between  the 
highest  temporal  (or  spatial)  frequency  of  interest  in  the  data  set 
and  the  minimum  rate  at  which  samples  must  be  taken  if  they  are 
to  record  all  the  possible  significant  variations  in  that  signal  accu¬ 
rately.  Specifically,  for  non-periodic  data,  the  sampling  frequency 
should  be  at  least  2.4x  higher  than  the  highest  frequency  in  the 
data.  Consequently,  to  preserve  all  of  the  information  that  could 
be  recorded  using  a  CTF  such  as  that  shown  in  Figure  2.19,  it  will 
be  necessary  to  have  the  pixels  smaller  than  1/2.4  Fc  where  Fc  is 
the  cut-off  frequency.14  This  means  that  the  Airy  figure  image  of  a 
point  object  should  be  at  least  4  to  5  pixels  across  the  diameter  of 
its  first  dark  ring. 

The  practical  problem  that  arises  in  CLSM  is  that,  although 
changing  the  “zoom”  magnification  changes  the  area  scanned  on 
the  specimen,  it  does  not  normally  do  so  by  varying  the  number 
of  pixels,  and  consequently,  pixel  size  is  inversely  proportional  to 
the  zoom  factor.  However,  this  does  not  change  the  fact  that,  for  a 
given  optical  system,  only  one  pixel  size  (and  one  zoom  factor!) 
matches  the  Nyquist  criterion.  At  zoom  settings  that  provide 
smaller  pixels,  the  data  will  be  over-sampled  with  the  result  that  it 
will  be  bleached  more  than  necessary  and  only  a  smaller  field  of 
view  can  be  scanned  in  a  given  period.  It  is  becoming  more 
common  for  manufacturers  to  display  the  pixel  size  as  part  of  the 
setup  display,  but  if  this  is  not  done,  it  is  not  difficult  to  calculate 
if  one  has  a  knowledge  of  the  length  of  some  feature  in  the  image 
in  both  pixels  and  micrometers. 

On  the  other  hand,  at  lower  zoom  settings  where  the  pixels  are 
larger  than  those  prescribed  by  Nyquist,  not  only  may  some 
smaller  features  be  missed  entirely  but  features  not  present  in  the 
object  may  “appear"  in  the  data  because  of  a  phenomenon  called 
aliasing. 

One  can  understand  aliasing  by  returning  to  the  temperature 
analogy.  Assume  that  the  thermometer  is  capable  of  responding  to 
changes  in  temperature  on  the  order  of  minutes  but  that  only  annual 
changes  are  of  interest.  Nyquist  would  seem  to  say  that  one  can 
find  out  the  average  yearly  temperature  by  making  on  average  2.4 
measurements/year.  However,  there  could  clearly  be  problems  if 


14  The  original  Nyquist  theory  related  to  1 -dimensional  data.  As  2D  images  may 
contain  spatial  frequencies  at  any  angle  to  the  raster  axes,  it  might  seem 
logical  to  increase  the  sampling  frequency  by  1.4  to  account  for  features 
running  at  45°  to  these  axes.  However,  as  is  shown  in  Fig.  4.10,  adjacent 
rows  of  pixels  fill  in  this  data.  In  addition,  because  the  low  S/N  of  most  con- 
focal  data  limits  the  effective  resolution  of  final  data  more  severely  than  does 
the  Abbe  criterion,  the  rule  that  pixels  should  be  2x  smaller  than  the  Abbe 
resolution  limit  works  well. 


one  sample  is  made  at  6  am  in  December  and  the  next  at  1 1  am  in 
May !  Large  day-to-day  and  time-of-day  variations  will  swamp  out 
small  changes  in  the  average  annual  temperature.  Nyquist  can  only 
work  if  the  response  time  of  the  sensor  is  reduced  until  it  is  about 
2.4x  our  sampling  interval.  This  could  be  accomplished  by  burying 
the  thermometer  a  few  meters  in  the  earth  where  the  temperature 
changes  much  more  slowly  (Pollock  and  Chapman,  1993). 

An  analogous  situation  occurs  in  a  CLSM  operated  at  a  low 
zoom  setting.  When  images  are  recorded  on  a  “continuous” 
medium,  there  is  no  possibility  that  small  but  bright  objects  will 
be  entirely  missed.  However,  this  is  not  true  of  sampling  systems. 
When  pixels  are  large  compared  with  the  optical  resolution 
(i.e..  low  zoom  setting),  it  is  possible  that  small  features  will  be 
missed  altogether  (so-called  “blind  spots”).  Blind  spots  only  occur 
because  Nyquist  sampling  conditions  are  not  met  and  small 
features  lying  between  scan  lines  are  not  sampled.  However, 
on  instruments  that  employ  Tv/4  integration  rather  than  full- 
integration  in  their  digitizing  circuitry,  small  features  that  the 
scanning  beam  passes  during  the  dead  time  of  the  digitizer  may  be 
recorded  with  lower  contrast  or  not  at  all. 

Aliasing  may  cause  features  to  appear  larger,  smaller,  or  in  dif¬ 
ferent  locations  from  where  they  should  be.  Like  aliasing,  blind 
spots  are  only  a  problem  when  the  pixel  size  is  much  larger  than 
the  spot  size  and  where  the  object  contains  a  lot  of  small  features 
(see  Chapters  4  and  35,  this  volume ,  for  a  more  complete  analysis). 

There  are  two  strategies  for  reducing  these  problems  (both 
discussed  in  the  previous  sections):  increasing  the  size  of  the 
focal  spot  and  using  a  fully  integrating  digitization  system. 
However,  the  latter  approach  only  avoids  blind  spots  in  the 
horizontal  direction. 

The  Nyquist  sampling  rule  is  often  flaunted  by  practicing 
microscopists  but  the  errors  produced  by  this  practice  usually  pass 
unnoticed  for  two  reasons: 

•  Working  images  that  are  somewhat  over-sampled  tend  to  be 
more  pleasant  to  view  on  the  display  screen.  As  the  size  of  the 
individual  pixels  (referred  to  the  sample)  is  considerably 
smaller  than  the  Nyquist  limit,  the  data  looks  “as  it  should” 
but  the  radiation  dose  needed  to  record  it  was  higher  than 
necessary. 

•  The  pixel  intensities  making  up  the  images  involve  so  few 
quanta  that  statistical  variations  in  the  signal  pose  a  greater 
limitation  to  defining  the  position  of  the  edge  of  a  feature  than 
does  the  quantization  of  the  image  in  space.  This  is  less  of  a 
limitation  if  one  uses  more  photons,  which  happens  when  one 
uses  more  pixels. 

Nevertheless,  as  applications  of  confocal  technology  more 
closely  approach  the  absolute  limits  (e.g.,  by  more  closely  match¬ 
ing  the  pixel  size  to  that  of  the  data  of  interest  and  improving  the 
photon  efficiency  so  that  the  statistical  noise  is  reduced)  the  costs 
of  incorrect  spatial  (and  temporal)  quantizing  will  become  more 
apparent. 

Fortunately,  a  technique  is  available  to  help  solve  the  noise 
problem.  Because  Poisson  noise  affects  the  measurement  of  inten¬ 
sity  in  each  pixel  independently,  positive  noise  excursions  can 
appear  to  be  small  bright  features  that  are  only  one  pixel  wide. 
These  can  be  eliminated  by  adhering  to  the  second  aspect  of 
Nyquist  sampling,  the  part  related  to  reconstructing  an  analog 
image  from  the  digital  data.  Nyquist  mandates  that  the  recon¬ 
structed  signal  be  passed  though  an  “amplifier”  having  the  same 
bandwidth  as  that  from  which  the  original  signal  was  obtained.  In 
the  case  of  microscopy,  the  input  “bandwidth”  in  question  is 
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defined  by  the  CTF  of  the  optics.  As  single-pixel  noise  includes 
spatial  frequencies  that  are  at  least  4x  higher  than  the  Abbe  limit, 
no  information  is  lost  by  passing  the  recorded  data  through  an 
“amplifier”  having  the  same  3D  bandwidth  as  the  microscope.  The 
simplest  way  of  doing  this  is  to  deconvolve  it. 

This  point  bears  repeating.  Although  deconvolution  and  con- 
focal  imaging  are  often  seen  as  competing  methods  aimed  at  the 
same  goal,  namely  producing  images  of  3D  stain  distributions, 
in  fact,  they  are  not  exclusive,  and  there  is  much  to  be  said  for 
combining  them.  Not  only  does  deconvolution  suppress  the 
“single-pixel”  features  created  by  Poisson  noise,  it  also  effectively 
averages  the  signal  over  16  to  25  pixels  in  2D  and  64  to  125  voxels 
in  3D  because  this  is  the  number  of  pixel/voxels  that  carry  appre¬ 
ciable  signal  in  the  Nyquist-sampled  image  of  a  point  object  (Fig. 
2.20).  In  other  words,  it  has  the  same  effect  of  reducing  the 
uncertainty  in  the  estimate  of  the  brightness  in  individual 
voxels  as  Kalman  averaging  for  64  to  125  frames.15  This  point 
is  so  important  that  the  present  edition  of  this  volume  devotes  an 
entire  chapter  to  it:  Chapter  25. 

Figure  2.21  shows  what  can  be  accomplished.  This  example 
was  kindly  provided  by  Flans  vander  Voort  of  SVI,  Delft,  NE.  The 
top  row  shows  a  through-focus  series  of  an  actual  Nyquist-sampled 
PSF.  The  next  two  rows  show  two  simulated  through-focus  series 
in  which  Poisson  noise  has  been  added  as  though  the  brightest 
pixel  recorded  only  25  photons,  and  the  remainder  proportionally 
less.  Although  the  noise  is  different  in  each  column,  when  they  are 
deconvolved,  the  results  (seen  in  the  two  bottom  rows),  are  almost 
identical. 


FIGURE  2.20.  Relationship  between  the  Rayleigh-criterion  resolution,  the 
point-spread  function  (PSF),  and  Nyquist  sampling.  If  the  PSF  is  sampled  at 
the  Nyquist  level,  each  pixel  will  be  less  than  1/4  of  the  distance  from  one 
side  of  it  to  the  other  and  -16  pixels  will  have  significant  signal  in  them  in 
each  .ry-plane. 


15  Although  such  filtering  will  reduce  image  contrast,  the  contrast  can  be 
increased  arbitrarily  by  changing  the  display  look-up  table,  a  process  that 
does  not  decrease  the  S/N. 


Because  the  confocal  PSF  is  so  much  more  confined  than  that 
of  WF  data,  it  takes  much  less  time  to  deconvolve  confocal  data. 
Indeed  it  is  quite  reasonable  to  approximate  the  process  by  a  3D 
Gaussian  blur  (see  Fig.  19.2,  this  volume). 

There  are  two  other  matters  about  sampling  that  should  be 
considered: 

•  Temporal  aliasing  effects'.  The  accuracy  with  which  temporal 
changes  can  be  imaged  is  limited  by  the  frame-scan  rate. 
Failure  to  sample  sufficiently  often  can  create  artifacts  such  as 
the  “back-rotating  wagon-wheel  effect”  familiar  from  the 
movies.  In  the  CLSM,  temporal  aliasing  not  only  limits  the 
ability  to  follow  motion,  it  also  has  a  resolution  component  in 
terms  of  the  imprecision  of  the  measurement  of  the  location 
(or  motion)  of  objects.  Specifically,  because  the  pixel  time  is 
short  while  the  frame  time  is  long,  motion  of  the  specimen  pro¬ 
duces  distortion  rather  than  the  directional  blurring  that  would 
occur  if  the  whole  field  was  recorded  continuously  over  the 
same  period  of  time. 

•  Mismatch  of  probe  and  pixel  shape:  There  is  a  mismatch  in 
shape  between  the  circular  shape  of  the  moving  Airy  disk  and 
the  “hard-edged”  square  shape  of  a  pixel  on  the  liquid-crystal 
display  (LCD),  a  difference  made  more  pronounced  by  the 
innate  tendency  of  the  human  visual  system  to  emphasize 
edges. 

In  the  case  of  a  signal  recorded  by  a  cooled  CCD  detector  on 
a  disk-scanning  confocal  system,  the  CCD  detector  spatially  quan¬ 
tizes  the  signal  in  a  manner  entirely  determined  by  its  sensor  geom¬ 
etry  and  the  total  optical  magnification.  However,  because  the 
scanning  mirrors  in  the  CLSM  follow  ballistic  rather  than  stepped 
trajectories,  the  output  from  the  PMT  of  these  instruments  is,  in 
principle,  continuous  in  the  horizontal  direction.  As  a  result,  aver¬ 
aged  over  the  digitizing  time,  the  Airy  disk  is  not  round  but  is 
slightly  more  blurred  in  the  horizontal  direction. 

Practical  Considerations  Relating 
Resolution  to  Distortion 

To  obtain  the  theoretical  spatial  resolution  of  a  confocal  micro¬ 
scope,  it  is,  of  course,  necessary  to  have  a  diffraction-limited 
optical  system,  but  this  is  not  sufficient.  Leaving  the  practical 
aspects  of  alignment  and  optical  quality  for  Chapters  7  through  11, 
22,  and  35,  this  section  discusses  the  limitations  imposed  by 
mechanical  stability  and  the  repeatability  of  the  scanning  system, 
a  topic  that  highlights  one  of  the  most  important  differences 
between  disk-scanning  and  laser-scanning  instruments. 

In  disk-scanning  instruments,  the  image  is  real  and,  therefore, 
cannot  be  distorted  by  the  scanning  system.16  If  a  CCD  sensor  is 
used  to  detect  it,  geometrical  distortion  in  the  digitized  image  is 
extremely  low,  and  because  of  the  inherent  mechanical  stability  of 
the  sensor,  any  distortion  that  remains  from  the  optics  is  stable  and 
can  be  corrected  by  minor  digital  image  processing.  The  mechan¬ 
ical  problems  of  this  instrument  are,  therefore,  confined  to  the 
effects  of  vibration  and  the  relative  motion  of  the  stage  and  lens. 

In  all  confocal  instruments,  it  is  vital  that  relative  motion 
between  the  objective  and  the  specimen  be  kept  to  less  than  10% 
of  the  resolution  limit  in  x,  y,  and  z.  In  tandem  disk-scanning 


16  Although  it  can  be  distorted  by  any  barrel  or  pincushion  distortion  present 
in  the  optics. 
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FIGURE  2.21.  Deconvolution  of  simulated  confocal  data  improves  S/N  and  resolution.  (A)  shows  a  through-focus  series  of  an  actual  Nyquist-sampled  PSF. 
(B)  and  (C)  show  simulated  images  in  which  Poisson  noise  has  been  added  as  though  the  brightest  pixel  recorded  only  25  photons.  Although  the  noise  is  dif¬ 
ferent  in  each  column,  when  they  are  deconvolved,  the  results  (D)  and  (E)  are  almost  identical.  (This  example  was  kindly  provided  by  Hans  vander  Voort  of 
SVI,  Delft,  NE.) 


instruments,  both  the  rotating  disk  and  the  cooling  system  of  the 
often-large  illumination  sources  represent  potential  sources  of 
vibration  in  addition  to  those  normally  present  in  the  environment. 
All  vibrations  must  be  reduced  below  the  tolerance  levels  or 
resolution  will  be  degraded  in  a  fairly  straightforward  manner.  In 
this  regard,  it  is  not  sufficient  to  mechanically  isolate  the  micro¬ 
scope  on  some  sort  of  isolation  table  because  laser-cooling  fans, 
spinning  disks,  scanning  galvanometers,  etc.  can  all  introduce 
vibration  into  the  isolated  system. 

Not  so  straightforward  is  the  effect  of  vibration,  and  possibly 
stray  electrical  and  magnetic  fields  acting  either  directly  on  the  gal¬ 
vanometers  or  introducing  spurious  signals  into  the  current  ampli¬ 
fiers  that  control  them.  In  these  instruments,  accurate  imaging 
depends  on  the  mirrors  causing  the  beam  to  scan  over  the  sample 
in  a  precise  pattern  duplicating  the  mathematically  perfect  raster 
represented  by  the  data  locations  in  the  image  memory.  Failure  to 
duplicate  this  raster  has  the  result  that  data  will  be  displayed  in  the 
wrong  pixel,  producing  distortion.  On  a  system  with  a  1000-line 
raster  and  a  10 : 1  zooming  ratio,  keeping  beam  placement  to  within 
0.1  of  a  pixel  requires  accuracy  of  one  part  in  105.  The  electro¬ 
mechanical  properties  of  the  galvanometers  (mass,  spring  constant, 
frequency  response,  overshoot,  resonant  frequency,  bearing  toler¬ 
ance,  rigidity,  etc.)  produce  additional  errors  (Chapter  3,  this 
volume).  Image  distortions  produced  by  these  errors  are  often 
masked  by  the  paucity  of  test  samples  having  an  accurately  defined 
geometry  (Pawley  et  ah,  1993)  and  by  the  fact  that,  at  high  zoom. 


current  instruments  greatly  over-sample  the  image  so  the  smallest, 
visible  structural  features  are  many  pixels  wide. 

This  problem  merits  mention  here  because  it  is  possible  to 
measure  the  x,  v,  z  position  of  the  centroid  of  an  object  in  a  digital 
image  to  an  accuracy  that  is  much  smaller  than  the  spatial  res¬ 
olution  limit.  Indeed,  WF  light  microscopy  techniques  have  been 
used  to  measure  motion  on  the  order  of  1  nm  (Gelles  et  al. ,  1988). 
However,  due  to  the  random  imprecision  in  the  systems  used  to 
position  the  mirrors,  it  is  unlikely  that  measurements  of  similar 
reliability  could  be  made  on  any  present  CLSM.  In  this  context, 
then,  the  accuracy  and  precision  with  which  mirror  position  can 
be  controlled  is  a  fundamental  limitation  on  the  ability  of  CLSM 
to  determine  position. 

The  presence  of  scan  instability  can  be  detected  either  by  visu¬ 
ally  comparing  sequential  single-scan  images  of  a  diagonal  knife- 
edge  viewed  at  the  highest  possible  magnification,  contrast  and 
signal  level  or,  alternatively,  by  computing  the  apparent  motion  of 
the  centroids  of  two  fixed  objects,  each  covering  100  to  200  pixels 
as  they  are  recorded  on  a  number  of  sequential  scans.  The  varia¬ 
tions  measured  by  the  second  method  should  decrease  rapidly  with 
increasing  illumination  intensity  because  of  improved  statistical 
accuracy,  although  they  may  then  begin  to  measure  stage  drift. 

However,  before  all  position  errors  are  blamed  on  the  gal¬ 
vanometers,  it  is  important  that  living  cells  are  far  from  the  innocu¬ 
ous  transparent  objects  often  assumed  in  articles  on  microscope 
optics.  As  noted  above  the  pronounced  variations  in  RI  that  allow 
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TABLE  2.1. 

Fundamental  Limits  in 

Confocal  Microscopy 

Parameter 

Theoretical 

Practical 

Resolution 

Spatial:  Y  L,  k2,  oq,  a2 

Alignment 

Off-axis  aberrations 

X  Aj,  A2,  OCj,  CX2 

Alignment 

Off-axis  aberrations 

Bandwidth 

Ballistic  scan 

Z  Aq,  A2,  OCj,  OC2 

Alignment 

Pinhole 

Off-axis  aberrations 

diam. 

A  tube  length 

Temporal:  Scan  speed 
Signal  decay 
times 

Quantization  of  t 

Position 

Objective  lens  distortion 

Mirror  accuracy 

Pixellation  limitations 
Sampling  time 

Vibration 

Quantitative 

Poisson  statistics 

Mirror  and  digitizing  losses 

measurement 

Detector  QE  and  noise 

Bleaching/photodamage 

Saturation 

Source  brightness 

us  to  image  cellular  organelles  in  phase-contrast  or  differential- 
interference  contrast  microscopy  can  also  profoundly  distort  the 
focus  plane  in  the  ^-direction. 

CONCLUSION 

I  have  attempted  to  highlight  some  aspects  of  confocal  instrumen¬ 
tation  that  must  be  addressed  in  order  to  attain  performance  limited 
only  by  fundamental  considerations  (Table  2.1).  Although  the  con¬ 
stituent  aspects  of  both  photon  efficiency  and  resolution  have  been 
addressed  separately,  I  have  tried  to  emphasize  that  the  effects  of 
both  of  these  factors  overlap  and  interact  in  a  fairly  complex 
manner.  To  summarize: 

•  Although  resolution  in  the  confocal  microscope  is  primarily  a 
function  of  the  NA  of  the  optical  system  and  the  X  of  the  light, 
it  can  also  be  further  limited  if  the  signal  level  represents  so 
few  quanta  that  the  detected  signal  lacks  the  statistical  preci¬ 
sion  to  produce  a  “visible”  feature  or  if  the  data  are  not  cor¬ 
rectly  sampled  because  the  pixels  are  too  large. 

•  To  improve  the  statistical  precision  of  the  signal,  every  effort 
should  be  made  to  count  as  many  of  the  photons  emerging 
from  the  specimen  as  possible: 

—  Reduce  optical  losses. 

—  Select  the  best  type  of  PMT  for  the  signal  to  be  detected. 

New  photodetectors  may  soon  become  available. 

—  Use  photon-counting  when  appropriate. 

—  Check  alignment  often. 

—  Routinely  check  performance  by  making  and  analyzing 
images  of  test  objects. 

•  To  remove  “single-pixel  noise  features”  and  provide  the  advan¬ 
tages  of  multi-voxel  averaging,  all  Nyquist-sampled,  3D  con¬ 
focal  results  should  be  deconvolved  (or  at  least  3D  Gaussian 
filtered)  before  being  viewed.  Likewise,  Nyquist-sampled  2D 
data  should  be  filtered. 

•  The  effects  of  image  quantization  should  not  be  ignored.  Only 
one  pixel  size  (zoom  setting?)  is  really  optimal  for  each  X,  NA, 
and  lens  magnification. 


•  Care  should  be  taken  to  keep  the  pinhole  diameter  larger  than 
that  of  the  Airy  disk  at  its  half-power  points  (0.5  Airy  unit). 
This  setting  will  change  with  X,  NA,  and  lens  magnification. 

•  Special  circumstances  may  dictate  breaking  sampling  and 
pinhole  “rules,”  but  they  should  still  be  recognized  and 
acknowledged. 

•  It  should  be  possible  to  operate  the  system  both  at  its 
diffraction-limited  resolution  and  at  larger  spot  sizes. 

•  In  laser-scanned  instruments,  imperfect  scanning  precision  can 
introduce  distortion. 

•  Fluorescence  saturation  places  unexpected  limits  on  the  speed 
with  which  experiments  can  be  performed  with  laser-scanning 
microscopes. 
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INTRODUCTION 

The  amazing  array  of  microscopic  instrumentation  for  generating 
three-dimensional  images  of  biological  specimens  that  are 
described  elsewhere  in  this  volume  rely  on  a  relatively  small 
number  of  optical  and  mechanical  components.  Although  most 
biologists  have  a  general  idea  of  their  function  and  how  to  operate 
them,  some  mysteries  remain. 

It  is  the  purpose  of  this  chapter  to  provide  the  reader  with  a 
description  of  how  components  are  specified,  which  components 
are  used  in  commercially  available  equipment,  and  how  they 
operate,  along  with  some  insight  in  terms  of  their  specific  appli¬ 
cations  to  confocal  or  multi-photon  microscopy.  We  will  try  to 
emphasize  those  specifications  and  limitations  that  impinge 
directly  on  the  performance  of  the  entire  instrument  so  that  the 
reader  can  understand  the  various  trade-offs  and  raise  sensible 
questions  during  the  purchasing  process.  Those  wishing  to  make 
their  own  confocal  from  scratch  will  need  to  consult  more  detailed 
sources. 

The  comments  provide  insights  developed  over  many  years 
during  which  we  designed,  built,  and  used  a  large  number  of 
different  confocal  microscopes.  Our  comments  are  meant  to 
improve  the  understanding  of  the  fine  points  of  microscope 
operation  among  users,  and  should  not  be  construed  as  recom¬ 
mendations  either  in  favor  of  or  against  any  particular  brand.  If 
some  of  our  remarks  seem  vague  or  inconclusive  this  may  reflect 
the  fact  that  some  elements  make  sense  in  this  but  not  in  that 
design.  Parts  that  might  have  been  important  in  the  past  often 
become  less  important  due  to  technical  improvements  in  other 
areas. 

The  best  approach  to  optical  design  is:  “Efficiency  through 
simplicity.”  Use  as  few  optical  elements  as  are  absolutely  neces¬ 
sary  to  achieve  a  certain  goal  and  place  them  in  the  optimal 
location  along  the  optical  path.  In  general,  flat  surfaces  are  more 
easily  manufactured  and  easier  and  less  expensive  to  coat  than 
curved  surfaces.  Lenses  with  long  focal  lengths  usually  have  fewer 
aberrations  and  are  less  expensive  than  those  with  short  focal 
lengths.  Additionally,  the  actual  performance  of  an  optical 
instrument  always  seems  to  reflect  engineering  and  financial 
compromises. 

This  chapter  describes  the  operation  of  some  of  the  impor¬ 
tant  components  used  in  confocal  microscopes:  filters,  scanners, 
acousto-optical  devices,  electro-optical  modulators,  and  polarizers. 
It  complements  in  particular  Chapter  9,  which  discusses  the  inter¬ 
mediate  optics  of  confocal  microscopes. 


REGULATING  THE  INTENSITY 

In  modern  laser  confocal  microscopes,  the  intensity  of  the  light  is 
typically  regulated  using  acousto-optical  tunable  filters  (AOTFs; 
see  below),  combinations  of  several  AOTFs,  or  combinations  of 
AOTFs  and  neutral-density  (ND)  filters.  A  polarizer  can  also  serve 
as  a  continuously  variable  attenuator  as  most  lasers  emit  polarized 
light  (Callamaras  and  Parker,  1999).  However,  if  the  polarization 
axis  needs  to  be  preserved  [e.g.,  to  maintain  proper  differential 
interference  contrast  (DIC)  operation]  a  second  polarizer  must  be 
included  and  adjusted  to  pass  the  original  beam.  Neutral-density 
filters  are  either  absorptive  gray  glass  filters  (for  low-power  appli¬ 
cations)  or  reflective  metallic  filters  (for  high-power  applications) 
that  attenuate  the  intensity  of  the  transmitted  light  independent 
of  the  wavelength  (A,).  Circular,  neutral-density  filters,  which  have 
a  band  of  optical  density  around  their  perimeter  that  increase 
linearly  or  logarithmically  with  the  angle,  also  allow  one  to  control 
the  intensity  continuously,  but  the  intensity  profile  of  a  large  beam 
passing  such  a  filter  becomes  slightly  inhomogeneous. 

Because  most  lasers  used  in  confocal  microscopy  produce  only 
milliwatts  of  power,  there  is  little  danger  of  overheating  the  ND 
filter.  However,  this  does  not  apply  to  filters  used  with  arc  sources 
or  lasers  for  multiphoton  excitation,  where  the  beam  power  may 
reach  several  watts.  ND  filters  come  in  two  general  types,  those 
made  of  darkened  glass  that  absorbs  light  and  those  coated  with  a 
metal  film  that  reflects  light.  The  former  are  more  likely  to  over¬ 
heat  and  the  latter  more  likely  to  produce  stray  light  or  to  deflect 
potentially  damaging  power  elsewhere. 

WAVELENGTH  SELECTIVE  FILTERING  DEVICES 

The  following  sections  discuss  the  different  filter  types  available 
for  use  as  excitation  or  emission  filters  or  as  dichroic  mirrors.  In 
fluorescence  microscopy,  filtering  devices  (e.g.,  Melles  Griot 
Catalog,  Chapter  13)  are  used  to  separate  light  beams  on  the  basis 
of  their  wavelengths.  Four  different  types  of  filters  are  used  to 
selectively  transmit  or  block  a  desired  range  of  wavelengths  (Fig. 
3.1):  (a)  short-pass  filters  (blue  line)  that  cut-off  wavelengths 
longer  than  a  certain  wavelength  (A,o)  —  heat  filters  are  used  to 
exclude  infrared  light  to  reduce  specimen  heating  by  the  illumina¬ 
tion;  (b)  long-pass  filters  (red  line)  that  only  transmit  light  longer 
than  a  certain  wavelength  —  fluorescence  emission  filters  are  often 
long-pass  filters  used  to  prevent  the  excitation  light  from  reaching 
the  detector;  (c)  bandpass  filters  (light  green  line)  that  only  trans- 
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shorter  Wavelength  longer 


FIGURE  3.1 .  Types  of  filters.  Filters  are  flat  optical  components  designed  only 
to  transmit  light  of  certain  wavelengths.  There  are  four  general  types:  short- 
pass  (dark  blue),  long-pass  dichroic  (dark  green),  bandpass  (yellow-green),  and 
long-pass  (red). 


mit  light  between  a  cut-on  and  a  cut-off  wavelength  —  these  may 
be  used  to  select  either  excitation  or  emission  wavelengths  espe¬ 
cially  when  one  is  trying  to  image  signals  from  more  than  one 
fluorophore  simultaneously;  and  (d)  dichroic  mirrors  (dark  green 
line)  that  separate  the  emitted  light  from  the  excitation  light.  In 
fluorescence  microscopy,  dichroic  mirrors  are  long-pass  filters 
designed  to  operate  when  oriented  at  45°  to  the  optical  axis. 

Note  that  active  optical  devices  such  as  acousto-optical  (AO) 
elements  can  also  fulfill  these  functions  (see  description  of  AODs 
and  AOBSs  below)  and  some  confocal  systems  have  now  replaced 
all  filter  components  by  AO  devices  and  dispersive  elements  (e.g., 
Leica  AOBS  SP2  and  RS). 

Selecting  the  Wavelength  of  the  Illumination 
and  the  Detected  Light 

While  filament-based  lamps  emit  a  continuous  spectrum  of  light 
that  is  very  similar  to  black-body  radiation,  high-pressure,  plasma- 
based  light  sources  emit  a  large  number  of  distinct  lines  on  top  of 
a  black-body  radiation  background  (see  Chapter  6,  this  volume). 
Lasers  come  in  various  types  and  may  emit  one  or  a  number  of 
lines,  each  with  an  extremely  narrow  bandwidth  (see  Chapter  5, 
this  volume).  Finally,  femtosecond  pulsed  lasers  emit  bands  that 
can  be  up  to  several  tens  of  nanometers  wide. 

The  spectral  problems  posed  by  epi-fluorescence  microscopy 
can  be  understood  by  looking  at  Figure  3.2,  which  shows  the 
important  variables  schematically.  The  excitation  spectrum  of  a 
fluorophore  (blue)  is  skewed  so  that  the  peak  is  towards  its  long- 
X  side  while  the  emission  spectrum  (green)  is  peaked  near  its  short- 
X  side.  The  difference  in  nanometers  between  the  peaks  of  these 
spectra  is  referred  to  as  the  Stokes  shift.  A  large  Stokes  shift  has 
the  advantage  that  it  is  relatively  easy  to  separate  the  excitation 
light  from  the  much  fainter  signal  light,  but  it  also  “uses  up”  more 
of  the  spectrum,  making  it  more  difficult  to  excite  and  record  signal 
from  two  or  more  dyes  at  the  same  time. 

The  object  is  to  excite  the  dye  as  efficiently  as  possible  while 
collecting  as  much  of  the  emitted  light  as  possible  without  trans¬ 
mitting  any  excitation  light  into  the  detection  channel,  a  task  com¬ 
plicated  by  the  fact  that  the  excitation  light  is  usually  about  105to 
106  times  brighter  than  the  fluorescence  emission. 

Ideally,  one  should  illuminate  with  a  narrow  wavelength  band 
that  matches  the  peak  excitation  wavelength  of  a  fluorophore 
selected.  This  requires  both  that  the  excitation  filter  pass  only  this 


band  and  that  the  dichroic  beam-splitter  reflect  it.  Although  this 
can  work  well  if  one  has  a  laser  line  that  coincides  with  the  peak 
of  the  excitation  spectrum,  more  commonly,  the  excitation  is  from 
a  Hg  or  Xe  arc.  As  such  sources  are  far  less  bright  than  lasers,  in 
terms  of  being  able  to  concentrate  a  lot  of  photons  onto  a  single 
point  in  the  specimen  (see  also  Chapter  6,  this  volume),  one 
normally  chooses  to  excite  the  dye  over  a  band  of  wavelengths 
that  matches  the  “half-power”  points  of  the  excitation  spectrum. 
Although  this  allows  more  photons  to  strike  the  dye,  it  also  means 
that  the  window  of  wavelengths  available  for  detecting  the  fluo¬ 
rescent  light  must  now  start  at  a  slightly  longer  wavelength  if  it  is 
to  exclude  all  the  excitation  light. 

On  the  other  hand,  no  matter  what  the  bandwidth  of  the  exci¬ 
tation,  fluorophores  emit  light  over  a  spectral  band  and  the  wider 
the  detection  bandwidth,  the  more  signal  will  be  recorded.1  An 
emission  filter  (sometimes  called  a  barrier  or  rejection  filter),  is 
necessary  to  block  any  excitation  light  from  reaching  the  detector. 
This  filter  is  at  least  a  long-pass  filter,  but  may  be  a  bandpass  filter 
if  this  is  needed  to  block  undesired  autofluorescence  or  when  the 
emission  from  two  or  more  fluorophores  must  be  discriminated. 

Generally  speaking,  each  fluorophore  requires  different 
illumination  and  detection  filters,  and  yet  other  filter  packages 
when  two  or  more  fluorophores  are  present  in  the  same  specimen. 
Although  total  signal  is  maximized  when  large  filter  bandwidths 
are  used,  spectral  discrimination  is  enhanced  if  the  filters  are 
chosen  to  provide  a  good  separation  of  the  bands.  In  addition,  nar¬ 
rowing  the  illumination  spectrum  to  match  the  excitation  peak  will 
increase  the  excitation  efficiency  while  reducing  the  bleaching  and 
excitation  of  non-specific  fluorescence,  by  avoiding  wavelengths 
that  excite  the  fluorophore  less  efficiently  (Kramer,  1999). 

Separating  the  Light  Paths 

In  fluorescence  microscopes,  part  of  the  light  path  is  used  by  both 
the  excitation  and  the  fluorescent  light  and  these  two  components 
must  be  separated,  based  on  their  spectral  properties  and  their 


Wavelength 


FIGURE  3.2.  Fluorescence  parameters.  To  a  first  approximation,  the  excita¬ 
tion  and  emission  spectra  are  mirror  images  of  each  other.  The  wavelength  dif¬ 
ference  between  their  peaks  is  called  the  Stokes  shift  and  varies  with  the  dye 
and  its  environment.  The  bandwidth  (light  blue  and  light  green  rectangles)  is 
a  measure  of  the  width  of  each  spectrum,  defined  as  the  wavelength  difference 
between  the  half-power  points  of  each  spectrum. 


1  The  fact  that  all  common  photodetectors  cease  to  respond  once  the  energy  of 
each  photon  drops  below  a  fixed  level,  places  a  practical  limit  on  the  long-A, 
end  of  the  detection  bandwidth. 
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direction  of  propagation.  This  separation  is  produced  either  by 
dichroic  mirrors,  which  transmit  one  part  of  the  spectrum  and 
reflect  the  other  part  when  placed  at  45°  in  the  light  path,  or  by 
acousto-optical  beam-splitters  (see  AOBS  below).  For  single  flu- 
orophores,  a  long-pass  dichroic  that  reflects  the  short-A,  illumina¬ 
tion  light  and  transmits  the  fluorescent  light  is  generally  chosen. 
Note  that,  in  a  two-photon  microscope,  the  fluorescent  light  has  a 
shorter  A,  than  the  excitation  light.  Therefore,  if  a  single-photon 
setup  is  also  used  for  two-photon  excitation,  the  long-pass  dichroic 
must  be  replaced  by  a  short-pass  dichroic. 

Common  fluorophores  usually  have  a  Stokes  shift  of  20  to 
lOOnm,  that  is,  the  peak  of  the  emission  spectrum  occurs  at  a  A, 
that  is  20  to  lOOnm  longer  than  that  of  the  absorption  peak  (e.g.. 
Molecular  Probes,  2002,  pp.  903-905;  Chapter  16,  this  volume). 
Therefore,  in  order  to  transmit  most  of  the  short-/,  part  of  the 
fluorescence  to  the  detector,  dichroic  mirrors  must  have  a  very 
sharp  edge  between  their  reflection  and  transmission  bands.  While 
a  loss  of  illumination  intensity  due  to  reduced  reflection  by  the 
dichroic  mirror  can  often  be  compensated  for  by  increasing  the 
intensity  of  the  light  source,  fluorescence  mistakenly  reflected 
towards  the  source  rather  than  proceeding  to  the  detector  repre¬ 
sents  a  loss  that  cannot  be  remedied  without  subjecting  the  speci¬ 
men  to  more  light. 

Modern  confocal  microscopes  may  have  several  detectors 
to  allow  the  simultaneous  detection  of  signals  in  different 
wavebands. 


Conventional  Filters 

Conventional  filters  generally  consist  of  colored  glass,  metallic 
films,  or  polymers  that  absorb  unwanted  wavelengths.  Colored 
glass  filters  are  relatively  inexpensive  and  easy  to  handle,  but  have 
some  disadvantages  (Reichman,  2000,  p.  11):  the  cut-off  between 
the  transmission  and  absorptions  bands  is  rather  gradual  and  the 
peak  transmittance  is  usually  low.  Additionally,  because  they 
absorb  light,  they  may  overheat  and  crack  if  subjected  to  high  light 
intensities  (i.e.,  in  the  illumination  path).  In  addition,  many  of  the 
glasses  involved  are  autofluorescent.  In  recent  years,  advances 
in  the  manufacture  of  interference  filters  have  made  them  the  X- 
selective  components  of  choice  for  all  filter-based  microscopes. 


Interference  Filters 

It  is  not  an  overstatement  to  say  that  the  development  of  epi- 
fluorescence  microscopy  owed  more  to  the  perfection  of  interfer¬ 
ence  filters,  and  more  particularly  of  dichroic  beam-splitters,  than 
to  that  of  any  other  optical  component. 

When  light  passes  from  a  smooth  medium  having  a  refractive 
index  (RI)  of  tii  to  a  second  medium  with  an  RI  =  n2,  an  amount 
of  the  light  proportional  to  («!  -  n2)2,  is  reflected  [Fig.  3.3(A)]. 
When  light  strikes  a  thin  layer  of  high-RI  material  covering  a  sub¬ 
strate  of  low-RI,  optical-grade  glass  or  fused  silica/quartz,  some 
light  reflects  back  from  each  interface.  If  the  thickness  of  the  layer 
is  such  that  the  light  reflected  by  the  top  surface  is  out-of-phase 
with  the  light  reflected  from  the  next  surface  down  (i.e.,  the  layer 
is  X/4  (or  3A74,  5A./4  .  .  .)  thick,2  these  two  beams  will  be  out  of 


2  One  must  remember  that,  when  measuring  film  thickness  in  terms  of  wave¬ 
length,  we  are  referring  to  optical  thickness  (i.e.,  geometric  thickness  X  RI) 
but  that,  as  RI  changes  with  wavelength,  some  subtlety  is  needed  in  deter¬ 
mining  the  correct  actual  thickness.  In  practice  this  is  accomplished  by  actu¬ 
ally  monitoring  the  transmission  of  the  filter  as  each  layer  is  deposited. 
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FIGURE  3.3.  Interference  filtering.  (A)  When  light  strikes  a  transparent 
surface  covered  by  a  thin  transparent  film  having  a  different  refractive  index, 
some  light  is  reflected  by  each  interface.  (B)  If  the  thickness  of  the  film  is  such 
that  these  two  wave-trains  are  out-of-phase  as  they  leave  the  surface,  they  will 
tend  to  cancel  out,  reducing  the  overall  reflectivity  (and  increasing  trans¬ 
mission).  (C,  D)  On  the  other  hand,  for  light  with  a  wavelength  such  that  the 
waves  leaving  the  surface  are  in  phase,  the  total  reflectivity  will  be  increased. 
(These  diagrams  modified  from  originals  kindly  provided  by  Turan  Erdogan, 
Semrock,  Rochester,  NY.) 


phase  and  will  tend  to  cancel  each  other  out,  with  the  result  that 
less  total  light  will  be  reflected  [Fig.  3.3(B)].  Alternatively,  if  either 
X  or  the  thickness  of  the  layer  is  changed  so  that  the  two  reflected 
waves  are  now  in  phase  (i.e.,  the  layer  is  X2  thick),  then  the  total 
reflectivity  at  this  wavelength  is  increased,  although  still  quite  low 
[about  10%  from  only  two  surfaces.  Fig.  3.3(C,D)].  The  total 
reflectivity  can  be  increased  almost  arbitrarily  by  adding  more 
layers  of  alternating  high-  and  low-RI  materials  (Fig.  3.4). 

Traditionally,  the  materials  used  for  the  high-index  layers  were 
zinc  sulfide  (RI  =  2.35),  zinc  selenide  (RI  =  2.67),  and  sodium  alu¬ 
minum  fluoride,  or  cryolite,  was  used  for  the  low-index  material. 
Because  these  materials  are  both  soft  and  hygroscopic,  they  must 
be  protected,  usually  by  sticking  the  coated  sides  of  two  glass  sub¬ 
strates  together  with  epoxy  [Fig.  3.5(A)].  This  has  the  disadvan¬ 
tage  of  making  it  very  difficult  to  make  all  the  flat  surfaces  parallel 
with  the  result  that  most  such  filters  suffer  from  “wedge-error”  and 
light  passing  through  them  is  deflected  at  a  slight  angle. 

More  recently,  these  coating  materials  have  been  replaced  by 
semi-transparent  layers  of  metal  oxides.  Although  more  complex 
and  time-consuming  to  deposit,3  these  so-called  hard  coatings,  are 
unaffected  by  water,  temperature  changes,  or  normal,  careful  han¬ 
dling,  and  because  they  deposit  as  denser,  more  uniform  layers,  the 
interfaces  are  more  uniform  and  hence  scatter  less  light.  As  a  result, 
more  light  is  transmitted.  Best  of  all,  because  they  are  deposited 
on  a  single  piece  of  glass  [Fig.  3.5(B)],  both  the  wedge  error  and 
wavefront  error  (“flatness”)  can  be  reduced  substantially  (respec¬ 
tively  to  <0.1  minute  of  arc  and  <A,/inch  of  surface),  and  this 
reduces  pixel  shift  when  the  filters  (or  dichroics)  must  be  changed 
for  sequentially  recording  several  dyes  in  a  single  specimen. 


3  The  process  involves  ion-assisted  deposition,  a  recent  modification  of  evap¬ 
oration  methods  that  utilizes  either  direct  thermal  or  electron  beam  to  vapor¬ 
ize  source  materials  and  an  energetic  ion  source  that  ionizes  the  evaporated 
materials  and  then  bombards  the  substrate  with  them.  This  process  increases 
the  packing  density  in  the  film  layers,  which  increases  the  RI  and  improves 
the  mechanical  characteristics  of  the  coat.  Fewer  voids  reduces  water  absorp¬ 
tion,  a  common  cause  of  mechanical  failure  and  unstable  optical  properties. 
Depending  on  the  films  being  deposited,  various  ion  species  may  be 
employed,  including  oxygen  or  inert  gases  such  as  argon. 


46  Chapter  3  •  J.  Rietdorf  and  E.H.K.  Stelzer 


FIGURE  3.4.  Interference  mirrors.  (A)  Light  impinging  on  a  stack  of  transparent  layers  of  alternating  high  and  low  refractive  index,  will  be  highly  reflected  if 
each  layer  is  A/4  of  a  wavelength  in  thickness.  (B)  This  also  applies  to  surfaces  struck  at  an  angle  of  45°,  although  in  this  case,  shorter  wavelengths  will  be 
reflected  for  a  given  layer  thickness. 


Traditional  dielectric  coatings  are  often  stacked  into  units 
called  cavities,  which  are  constructed  of  groups  of  alternating 
layers  separated  by  a  wider  layer  of  one  of  the  materials  called  a 
spacer  (see  Fig.  3.6).  The  spacers  are  produced  with  a  thickness 
that  corresponds  to  even  multiples  of  a  half  wavelength  in  order 
to  reflect  or  transmit  light  in  registration  with  the  dielectric  layers. 
Increasing  the  number  of  cavities  utilized  to  build  an  interference 
filter  produces  a  proportional  increase  in  the  slope  of  the  cut-on 
and  cut-off  wavelength  transmission  boundaries.  Conventional 
interference  filters  featuring  up  to  15  stacked  cavities  can  have  a 
total  of  over  75  individual  dielectric  layers  and  provide  a  band¬ 
width  only  a  few  nanometers  wide.  However,  because  the  surfaces 
of  soft,  evaporated  films  are  not  perfectly  smooth,  they  produce 
incoherent  scattering  that  reduces  peak  transmission  substantially 
when  many  layers  are  used.  Because  the  surfaces  formed  by  the 
hard  coating  materials  are  smoother  [Fig.  3.6(C)]  and  can  be 
deposited  more  precisely,  it  is  possible  to  deposit  more  than  200 
of  them  without  reducing  peak  transmission  substantially  and 
while  restricting  cumulative  thickness  errors  so  that  the  order  of 
the  array  is  maintained. 


New  approach 


I  I  epoxy  I  I  soft  coating  hard  coating  I  I  glass 

FIGURE  3.5.  Fabrication  methods.  (A)  As  the  materials  conventionally  used 
for  the  thin  films  that  make  up  interference  filters  are  both  soft  and  hygroscopic, 
they  must  be  protected  from  the  environment  by  placing  them  on  the  inside  of 
a  filter  assembly  composed  of  at  least  two  pieces  of  glass  held  together  with 
epoxy.  (B)  As  the  coatings  used  on  more  modern  filters  are  hard  enough  to  be 
applied  to  the  outside  of  a  single  glass  blank,  the  assembly  can  both  be  thinner 
and  have  surfaces  that  are  more  parallel.  (These  diagrams  modified  from  orig¬ 
inals  kindly  provided  by  Turan  Erdogan,  Semrock,  Rochester,  NY.) 


Types  of  Interference  Filters 

Virtually  any  type  of  filter  can  be  designed  and  constructed  with 
thin-film  interference  coating  technology,  including  bandpass, 
short-pass,  long-pass,  dichroic  beam-splitters,  neutral  density,  and 
a  variety  of  mirrors,  including  those  used  to  confine  laser  cavities 
(Fig.  3.1).  We  will  start  by  considering  a  long-pass  cut-off  beam¬ 
splitter  (Fig.  3.7)4  that  reflects  all  light  <X„.  Clearly,  this  could  be 
made  by  depositing  onto  the  substrate  a  stack  of  cavities,  each  with 
A./2-thick  layers  tuned  to  different  wavebands  longer  than  A,0.  Those 
cavities  reflecting  light  near  to  A,0  must  have  steeper  cut-offs  (i.e., 
more  layers)  than  those  centered  farther  away  but  the  result  is  that 
only  wavelengths  shorter  than  A»0  will  be  reflected.  It  is  impor¬ 
tant  to  remember  that,  as  any  light  that  is  not  transmitted  by  an 
interference  filter  is  reflected  rather  than  absorbed  or  scattered, 
provision  must  be  made  to  absorb  it  lest  it  become  a  source  of  stray 
light. 

A  short-pass  filter  is  made  using  the  reverse  logic  (reflect  A, 
longer  than  A«),  while  a  bandpass  filter  could  be  made  by  essen¬ 
tially  coupling  a  short-pass  and  a  long-pass  filter.  Alternatively,  one 
can  deposit  A./4  layers  for  the  wavelengths  that  one  does  want  to 
transmit  with  the  assumption  that  other  wavelengths  will  be 
reflected.  As  the  latter  approach  involves  thinner  layers  that  take 
less  time  to  deposit,  it  is  therefore  more  commonly  used.  Figure 
3.8  shows  the  performance  of  both  “hard”  and  “soft”  bandpass 
filters  as  might  be  used  as  the  excitation  and  emission  filters  for 
a  fluoroscein  cube.  When  the  “hard”  curves  are  combined  with 
the  filter  shown  in  Figure  3.7  (now  used  as  a  beam-splitter),  the 
performance  of  the  resulting  set  can  be  seen  in  Figure  3.9. 

Although  in  fact  most  filters  are  made  using  film-thickness 
patterns  considerably  more  complex  than  those  just  described, 
state-of-the-art  methods  now  allow  one  to  fabricate  a  component 
as  complex  as  the  triple-dichroic  beam-splitter  designed  to  reflect 
364,  488,  and  633  nm  light  so  accurately  that  the  measured  per¬ 
formance  closely  approximates  the  theory  (Fig.  3.10).  One  can  see 


4  Because  they  have  been  a  leader  in  introducing  this  new  technology  to 
microscopy,  this  chapter  uses  many  illustrations  provided  by  Semrock,  Inc. 
(Rochester,  NY).  However,  it  seems  likely  that,  by  the  time  you  read  this  text, 
other  filter  companies  will  also  be  offering  similar  products  fabricated  using 
the  new  hard-coating  techniques. 
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FIGURE  3.6.  Layers,  cavities,  filters  and  surfaces.  (A)  Most  interference  filters  use  tens  or  even  hundreds  of  layers.  These  layers  are  often  grouped  in  sets  of 
5,  with  a  thicker  spacer  layer  between  them.  Larger  sets  are  referred  to  as  cavities.  (B)  Scanning  electron  microscope  image  of  the  edge-view  of  a  part  of  a  stack 
of  layers.  The  lighter  layers  are  thinner  and  have  a  higher  refractive  index.  (C)  Atomic  force  micrographs  of  the  surface  of  a  variety  of  optical  surfaces:  (Cl) 
“super-polished”  substrate  with  a  surface  quality  of  0.5A  rms;  (C2)  same  substrate  with  a  25-layer,  mirror  coating  deposited  by  electron-beam  (EB)  evapora¬ 
tion,  10  A  surface  quality;  (C3)  same  substrate  with  a  25-layer,  mirror  coating  deposited  by  continuous,  plasma-enhanced  or  ion-assisted  EB  coating,  4  A  surface 
quality;  (C4)  same  substrate  with  a  50-layer,  ion-beam-sputtered  coating,  0.5  A  surface  quality.  The  improvement  in  surface  roughness  is  clear  and  this  results 
in  lower  inter-layer  and  overall  scattering.  (The  diagram  modified  from  an  original  kindly  provided  by  Melles  Griot,  Rochester,  NY.  The  micrograph  is  from 
Turan  Erdogan,  Semrock,  Rochester,  NY.) 


FIGURE  3.7.  Performance  of  a  45°  beam-splitter.  The  black  line  represents 
the  transmission  characteristics  of  a  traditional,  high-quality  dichroic  beam¬ 
splitter.  The  green  line  represents  the  performance  of  a  newer,  hard-coated  filter. 
The  superior  optical  properties  of  the  hard  coatings  allow  one  to  use  more 
layers  to  produce  steeper  cut-offs  while  still  maintaining  very  high  peak  trans¬ 
mission.  (This  diagram  modified  from  an  original  kindly  provided  by  Turan 
Erdogan,  Semrock,  Rochester,  NY.) 
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FIGURE  3.8.  Excitation  and  emission  bandpass  filters.  The  difference  in 
performance  between  the  older  “soft”  coatings  and  the  newer  "hal'd”  ones  is 
evident.  (This  diagram  modified  from  an  original  kindly  provided  by  Turan 
Erdogan,  Semrock,  Rochester,  NY.) 


FIGURE  3.10.  Theory  and  practice  in  filter  design.  The  hard  optical  coating 
performance  is  so  predictable  that  theoretical  and  actual  performance  can  be 
very  similar,  even  on  a  component  as  complex  as  a  triple-bandpass  dichroic. 
(This  diagram  modified  from  an  original  kindly  provided  by  Turan  Erdogan, 
Semrock,  Rochester,  NY.) 


the  close  agreement  between  designed  and  measured  performance 
that  is  possible  when  using  the  new  hard  coatings. 

Although  the  average  biological  microscopist  is  unlikely  to  try 
to  manufacture  an  interference  filter,  one  factor  worth  remember¬ 
ing  is  that  “reflections  add.”  In  other  words,  cavities  shape  the 
spectral  properties  of  the  incident  light  by  subtraction:  by  reflect¬ 
ing  unwanted  wavelengths.  Once  these  wavelengths  have  been 
removed  from  the  beam,  they  cannot  be  “restored”  merely  by 
depositing  a  subsequent  layer  that  would  have  transmitted  them. 
In  practice,  this  means  that  it  is  easier  to  make  a  filter  that  trans¬ 
mits  only  one  specific  wavelength. 


The  number  of  layers  and  cavities  deposited  is  the  variable 
used  to  control  the  nominal  wavelength,  bandwidth,  and  blocking 
level  of  the  filter  with  very  high  precision.  Figure  3.11  shows  the 
performance  of  a  pair  of  “hard”  filters  made  for  use  in  Raman 
spectroscopy.  In  this  technique,  photons  scattered  by  individual 
molecules  in  a  transparent  specimen  (or  even  in  the  optics!),  are 
found  to  have  lost  very  small  amounts  of  energy,  and  the  amount 
of  this  loss  is  related  to  the  electronic  structure  of  the  scattering 
material.  As  a  result,  the  scattered  light  contains  spectral  data  that 
allows  one  to  identify  the  scattering  molecules.  Because  Raman 
scattering  is  produced  by  every  element  in  the  optical  system  and 
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FIGURE  3.9.  Filter  cube  for  FITC.  Combined  performance  of  the  filters  dia¬ 
gramed  in  Figures  3.7  and  3.8.  (This  diagram  modified  from  an  original  kindly 
provided  by  Turan  Erdogan,  Semrock,  Rochester,  NY.) 
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FIGURE  3.11.  Laser-line  mirrors.  Modern  hard-coating  mirrors  can  be  made 
with  enough  layers  to  produce  very  narrow  reflection  bands  (red  line),  and  long- 
pass  filters  with  extremely  steep  cut-offs  (blue  line).  (This  diagram  modified 
from  an  original  kindly  provided  by  Semrock,  Rochester,  NY.) 


Special  Optical  Elements  •  Chapter  3  49 


c 

o 

'(/) 


E 

if) 

C 

CO 

l— 

H 


610  615  620  625  630  635  640  645  650  655  660 


Wavelength  (nm) 


Wavelength  (nm) 


FIGURE  3.12.  Linear  versus  logarithmic  plots  of  transmission.  Although  transmission  graphs  (left)  are  easy  to  understand,  they  don't  give  the  whole  story 
because  one  can  only  read  them  to  about  1%  transmission.  As  the  emitted  light  is  about  one  million  times  dimmer  than  the  excitation,  great  effort  is  required  to 
produce  dichroic  and  barrier  filters  with  sufficient  "blocking”  to  prevent  any  excitation  light  from  reaching  the  detector.  As  a  result,  one  can  only  usefully  esti¬ 
mate  filter  suitability  if  one  views  a  transmission  plot  displayed  in  logarithmic,  “optical  density  units”  (OD,  right).  Here  one  can  see  that  a  laser-line  “notch” 
filter  with  a  transmission  bandwidth  of  ±12  nm  has  a  transmission  ratio  of  >106,  and  that  the  cut-off  of  the  long-pass  barrier  is  similarly  steep.  (These  diagrams 
modified  from  originals  kindly  provided  by  Turan  Erdogan,  Semrock,  Rochester,  NY.) 


because  the  specimen  itself  produces  very  little  Raman-scattered 
light,  filters  such  as  those  shown  in  Figure  3.11  are  needed  to 
ensure  that  the  light  striking  the  specimen  is  uncontaminated  with 
Raman  wavelengths  picked  up  “in-transit”  (red  line),  using  a 
“laser-line”  filter  composed  of  a  very  large  number  of  A./4  layers 
that  effectively  transmits  only  the  laser  light.  A  steep-cut-off,  long- 
pass  filter  is  used  so  that  only  light  with  A,  >  A,laser  can  reach  the 
spectrometer  (blue  line). 

If  the  optical  configuration  requires  that  the  filter  pass  the  laser 
line  rather  than  block  it,  a  notch  filter  is  used.  Figure  3.12  shows 
the  performance  of  this  combination  in  two  different  ways:  (a)  as 
a  linear  plot  (as  is  the  case  in  all  the  other  figures  so  far)  and  (b) 
with  the  vertical  scale  in  optical  density  (OD)  units.  As  OD  units 
are  logarithmic,  the  figure  shows  that  the  blocking  level  of  this 
filter  at  its  center  wavelength  is  >106x.  Such  a  filter  (or  at  least  one 
centered  at  a  longer  wavelength!)  could  be  used  to  block  near-IR 
from  a  pulsed  laser  from  reaching  a  transmitted  light  detector  set 
up  to  measure  the  second-  or  third-harmonic  signal  (see  Chapter 
40,  this  volume).  More  generally,  because  the  fluorescence  signal 
is  commonly  106x  less  intense  than  the  excitation,  it  is  often  even 
more  important  that  emission  filters  have  good  blocking  than  that 
they  have  a  sharp  cut-off,  because  without  this  feature,  a  lot  of 
stray,  reflected  excitation  light  may  reach  the  detector,  effectively 
reducing  image  contrast. 

Because  light  striking  a  thin  film  at  an  angle  will  need  to  have 
a  slightly  shorter  wavelength  in  order  for  the  next  interface  to  be 
encountered  in  exactly  A/2  or  A./4,  the  spectral  properties  of  inter¬ 
ference  filters  and  beam-splitters  vary  with  incidence  angle.  Figure 
3.13(A)  shows  both  how  A,cut.off  of  a  short-pass  filter  changes  by 
about  10%  as  the  incidence  angle  increases  to  45°  and  also  how 
this  effect  depends  on  the  polarization  direction  of  the  incident 
light.  At  angles  greater  than  normal  incidence,  light  waves  undu¬ 
lating  parallel  to  the  plane  containing  the  incident  and  reflected 
rays  (p-polarization)  exhibit  a  different  transmission  profile  from 
waves  undulating  perpendicular  to  the  plane  of  incidence  and 


reflection  (.v-polarization).  Except  when  performing  anisotropy 
measurements,  it  is  usually  desirable  to  reduce  the  polarization- 
dependent  behavior  of  filters  in  the  detection  path  because  light 
emanating  from  the  sample  is  typically  less  polarized.  However, 
in  the  excitation  light  path  this  pol-dependence  can  be  exploited 
to  achieve  a  higher  reflectance  for  polarized  excitation  laser  light. 

Figure  3.13(A)  makes  clear  how  the  intensity  of  the  beam 
striking  the  specimen  in  a  single-beam,  confocal  microscope  can 
vary  substantially  if  the  light  emerging  from  the  (supposedly) 
polarization-preserving  fiber  connecting  the  scan  head  to  the  laser 
skips  between  modes.  Changing  the  pol-direction  of  the  fiber 
output  changes  the  amount  of  this  light  that  is  reflected  by  the 
45°  dichroic5  and  proceeds  to  the  objective  (see  Chapter  26,  this 
volume). 

Figure  3.13(B)  shows  how  this  angle  dependence  can  be  uti¬ 
lized  intentionally  to  tune  the  bandpass  of  a  narrow,  notch  filter, 
or  other  high  cut-off  component,  by  tilting  it  up  to  14°  from  normal 
incidence. 

It  is  widely  understood  that  the  prime  advantage  of  using  infin¬ 
ity-conjugate  objectives  is  that,  in  the  space  between  the  objective 
and  the  tube  lens,  light  rays  originating  at  the  front  focus  plane 
will  be  parallel  to  the  optical  axis  and  therefore,  that  such  rays  will 
strike  any  filter  or  dichroic  introduced  into  this  region  at  a  fixed 
angle.  However,  as  light  rays  originating  from  above  or  below  the 
plane-of-focus  will  not  be  parallel  to  the  optical  axis  in  this  space, 
the  effective  cut-off  wavelength  of  these  components  may  shift 
slightly  when  applied  to  such  light.  Although  such  light  is  excluded 
by  the  pinhole  in  a  confocal  microscope,  it  can  reach  the  detector 
in  the  widefield  microscope. 


5  A  dichroic  beam-splitter  is  very  similar  to  an  interference  filter  except  that 
the  reflective,  interference  layers  are  deposited  only  on  a  single  outside 
surface  and  the  substrate  is  often  thinner.  The  “back”  side  of  the  substrate  is 
often  coated  with  a  broadband,  anti-reflection  coating. 


50  Chapter  3  •  J.  Rietdorf  and  E.H.K.  Stelzer 


RazorEdge  Design  Spectra  vs.  AOI 

100 
90 
80 


c  60 
o 

m  50 

E 

«  40 
cc 

H  30 
20 
10 
0 

0.88  0.90  0.92  0.94  0.96  0.98  1.00  1.02  1.04  1.06  1.08 
Relative  Wavelength  (W iq) 


w 

V 

J\j 

V 

V 

sw 

W 

-  o°° 

—  1 0°°  s 

—  1 0°°  avg 

—  10°°  p 

—  30°°  s 

—  30°°  avg 

—  30°°  p 

45°°  s 

A/V 

w 

v> 

- 45°°  p 

-t— » 
'(/) 
C 
CD 

Q 


03 

O 


o 


Wavelength  (nm) 


FIGURE  3.1 3.  Reflectivity  and  incidence  angle.  As  the  angle  at  which  the  incident  light  strikes  the  filter  becomes  less  than  90°,  the  cut-off  wavelength  becomes 
shorter  (left).  The  change  in  wavelength  is  about  10%  at  45°  although  the  exact  amount  depends  strongly  on  the  polarization  of  the  incident  light  (6%  vs.  10%). 
This  angular  dependence  can  be  used  to  ‘tune’  the  cut-off  wavelengths  of  notch  and  edge  filters  (right).  (These  diagrams  modified  from  originals  kindly 
provided  by  Turan  Erdogan,  Semrock,  Rochester,  NY.) 


Dichroic  and  Polarizing  Beam-Splitters 

Dichroic  mirrors  or  beam-splitters  are  special  interference  filters 
that  reflect  light  in  defined  bands  while  transmitting  light  in  other 
bands  when  placed  at  an  angle  in  a  light  path  (in  distinction  to 
dichroic  filters,  dichroic  mirrors  are  placed  at  an  angle  to  the  inci¬ 
dent  light  and  their  reflective  properties  are  just  as  important  as 
their  transmission).  To  insure  that  the  image  made  with  one 
dichroic  is  aligned  with  that  made  with  others,  the  glass  blanks 
(substrates)  used  to  produce  them  must  be  polished  to  very  fine 
tolerances  for  thickness  and  parallelism  (see  above).  This  is  nec¬ 
essary  because,  in  epi-fluorescence  microscopy,  the  imaging  rays 
pass  through  a  beam-splitter  oriented  at  45°,  and  the  position  of 
the  transmitted  light  bundle  is  displaced  by  an  amount  proportional 
to  the  thickness  and  RI  of  the  substrate  on  which  the  interference 
layers  are  deposited.  If  the  thickness  is  reduced  much  below  3  mm, 
the  flatness  of  the  substrate  may  be  compromised  and,  of  course, 
the  refractive  index  (RI)  of  glass  is  a  function  of  the  wavelength: 
for  BK7  glass  RI  =  1.507  @  1060nm  but  RI  =  1.536  @  365  nm. 
This  difference  is  so  large  that  the  variation  in  offset  will  be  very 
noticeable  in  any  design  that  covers  a  range  from  the  ultraviolet  to 
the  near-infrared.  Instead  of  ignoring  these  effects,  it  is  probably 
wise  to  work  with  them  and  for  example,  to  use  a  thicker  piece  of 
glass  instead  of  a  thin  glass  plate  and  create  shifts  that  are  so  large 
that  secondary  or  ghost  images  are  easily  removable  with  a  regular 
aperture.6 

To  circumvent  the  spectral  problems  of  interference-based 
beam-splitters,  it  is  possible  to  use  a  polarizing  beam-splitter  that 
reflects  half  of  the  light  polarized  in  one  direction  and  transmits 
all  the  light  polarized  in  the  other  direction  (Cox,  2002).  About 


6  Indeed,  a  very  useful  achromatic  beam-splitter  can  be  made  by  using  such  a 
piece  of  glass  with  no  coating  on  the  near  surface.  Such  a  surface  will  reflect 
about  5%  of  all  light  impinging  at  45°.  Although,  this  “wastes”  laser  power, 
there  is  usually  more  than  one  needs  of  this  in  any  case.  Care  must  be  taken 
that  any  excitation  light  not  reflected  is  absorbed  before  it  can  become  stray 
light. 


80%  of  the  fluorescent  light  is  transmitted  for  all  visible  wave¬ 
lengths.  There  is  no  spectral  distortion  because  the  transmittance 
is  independent  of  the  wavelength,  and  the  beam-splitter  does  not 
need  to  be  changed,  simplifying  alignment. 

The  beam-splitter  in  the  LSM  5-Live  line-scanning  confocal 
microscope  (Carl  Zeiss,  Jena,  DE)  uses  a  linear  mirror  in  the  center 
of  a  glass  blank  located  in  an  optical  plane  corresponding  to  the 
back-focal  plane  and  placed  to  only  reflect  a  narrow  “line”  of  laser 
illumination  (Fig.  3.14).  As  only  a  small  fraction  of  the  blank  is 
coated  as  a  mirror,  more  than  95%  of  the  entire  bundle  of  fluores- 


Detector  /  Projector  Pupil  plane  Scan  lens  Image 


FIGURE  3.1 4.  Operation  of  the  "Acrogate.”  The  beam-splitter  in  the  Zeiss  5- 
Live  takes  advantage  of  the  fact  that,  in  a  line-scanner,  only  a  line  of  illumi¬ 
nation  exists  in  the  back-focal  plane.  Consequently,  a  piece  of  glass  with  a 
high-reflectivity  linear  mirror  across  the  center  will  reflect  all  of  a  line  of  coher¬ 
ent  laser  light  down  to  the  specimen  but  obstruct  only  about  5%  of  the  return¬ 
ing  fluorescent  light,  and  this  performance  is  not  affected  by  wavelength. 
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cent  light  returning  from  the  objective  can  be  transmitted  (see  also 
Chapter  10,  this  volume). 

Filters  and  Dispersive  Elements  for 
Multi-Channel  Detection 

As  light  emitted  by  fluorophores  in  the  sample  is  mainly  unpolar¬ 
ized,  polarization-sensitive  beam-splitters  can  only  separate  half  of 
it  from  the  polarized  excitation  light.  Instead,  the  emitted  light  is 
either  sequentially  split  by  a  combination  of  short-pass  and  long- 
pass  dichroic  mirrors  before  being  passed  through  a  bandpass  or 
long-pass  emission  filter  to  the  detector  (see  Fig.  3.9),  or  it  strikes 
a  dispersive  element  (e.g.,  a  grating  or  a  prism).  Light  split  by  a 
dispersive  element  can  either  be  projected  directly  onto  a  fixed 
array  of  mini-photomultiplier  tubes  (PMT)  from  which  a  relatively 
continuous  spectrum  is  recorded  (Fig.  3.15,  as  in  the  Zeiss  META), 
or  the  light  passes  through  a  system  of  slits  and/or  mirrors  that  can 
be  adjusted  to  pre-select  the  segments  of  the  spectrum  directed  to 
each  detector  (Fig.  3.16,  as  in  the  Leica  TCS  SP  II).  The  former 
system  has  the  advantage  of  more  channels  (32  vs.  4  detectors,  8 
vs.  4  channels  digitized  simultaneously)  but  the  disadvantages  that 
the  width  of  each  spectral  channel  is  quantized  by  the  spacing  of 
the  PMTs,  that  some  signal  is  lost  in  the  dead  zone  between  pho¬ 
tocathodes  and  because  mini-PMTs  have  lower  quantum  efficiency 
and  more  multiplicative  noise  than  the  more  conventional  PMT.  In 
both  systems,  optimal  spectral  resolution  requires  that  the  light 
passed  by  the  pinhole  originate  from  a  very  small  area  of  the  spec¬ 
imen  (i.e.,  it  must  be  collimated  light:  opening  the  pinhole  reduces 
spectral  resolution),  a  factor  that  is  not  important  for  systems  that 
use  interference  filters  to  separate  the  light.  More  details  about 


FIGURE  3.16.  Schematic  diagram  of  the  spectral  detector  in  the  Leica  TCS 
SP  II.  Light  passing  through  the  pinhole  is  made  parallel  by  a  collimating  lens 
and  refracted  by  a  prism.  Light  of  different  colors  is  reflected  to  one  of  four 
photomultiplier  tubes  (PMTs)  by  a  series  of  movable  mirrors  that  can  also  be 
arranged  to  form  shutters  or  slits.  Up  to  four  signals  can  be  digitized  from  the 
four  PMTs. 


FIGURE  3.1 5.  Schematic  diagram  of  spectral  detector  in  a  Zeiss  META  con- 
focal  head.  Light  passing  through  the  pinhole  is  made  parallel  by  a  collimat¬ 
ing  lens  and  diffracted  by  a  diffraction  grating.  Light  of  different  colors  strikes 
different  segments  of  a  32-channel  mini-PMT  array.  Light  that  strikes  the  active 
area  of  one  of  the  photocathodes  produces  photoelectrons  that  are  amplified  by 
the  adjoining  electron-multiplier  section.  Up  to  eight  signals  can  be  digitized 
from  eight  sets  of  adjacent  PMT  channels.  Metal  pins  are  placed  in  front  of  the 
PMT  array  to  prevent  light  at  laser  wavelength  from  reaching  the  detector. 


these  detectors  and  tests  that  can  be  applied  to  validate  their  per¬ 
formance  can  be  found  in  Chapter  36  ( this  volume ;  Lerner  and 
Zucker,  2004). 


MECHANICAL  SCANNERS 

In  most  of  the  commercial  confocal  microscopes  used  in  biology, 
the  beam  scans,  not  the  specimen.  The  laser  light  is  scanned  over 
the  image  plane  by  changing  the  angle  at  which  the  laser  beam 
passes  through  the  back-focal  plane  (BFP)  of  the  objective.  If  the 
rotational  axis  of  a  scan  mirror  coincides  with  both  its  own  surface 
and  the  optical  axis  at  a  telecentric  plane  (i.e.,  any  plane  that  is 
conjugate  to  the  back-focal  plane  of  the  microscope  objective 
lens),  rotation  of  the  mirror  moves  the  focused  spot  across  the 
focus  plane  in  a  straight  line  (Fig.  3.17).  The  devices  used  to  tilt 
the  mirror  in  a  rapid  and  precise  manner  are  called  galvanometers. 

While  the  first  generation  of  scanning  microscopes  could  only 
scan  the  beam  over  rectangular  areas  of  the  specimen  using  lines 
running  in  a  fixed  orientation,  modern  instruments  are  equipped 
with  scanners  that  allow  one  to  rotate  the  scanned  rectangle  or  even 
scan  arbitrarily  shaped  areas.  The  scan  angle  can  be  rotated  by 
apportioning  some  of  the  x-scan  signal  to  the  y-galvo  and  vice 
versa.  Scanning  an  arbitrary  area  is  particularly  useful  for  experi¬ 
ments  involving  FRAP  or  the  release  of  caged  compounds,  where 
the  high-intensity  illumination  area  can  be  precisely  matched  to  a 
region  of  interest  defined  using  biological  criteria  applied  to  the 
recorded  image. 

Flexible  scanning,  integrated  to  a  system  to  shift  excitation  and 
emission  wavelengths  rapidly,  allows  a  much  greater  flexibility 
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FIGURE  3.1 7.  How  rotation  of  a  mirror  in  an  aperture  plane  causes  the  linear 
motion  of  the  focused  spot.  Rotation  of  the  mirror  changes  the  angle  at  which 
light  leaves  the  focal  point.  The  green  line  at  45°  represents  the  neutral  mirror 
position.  Light  blue  and  dark  blue  lines  show  the  effect  of  rotating  the  mirror 
clockwise  by  one  half  of  the  total  deflection  angle.  Because  it  is  not  possible 
to  mount  the  galvanometer  at  the  actual  back-focal  plane  of  the  objective,  a 
scan  lens  is  used  to  image  this  plane  to  a  more  convenient  location.  Because 
this  lens  has  some  magnification  (M),  the  angles  actually  scanned  by  the  gal¬ 
vanometer  mirror  are  less  than  those  implied  by  this  diagram  by  a  factor  of 
1/M.  f,  focal  length. 

regarding  the  sequential  acquisition  of  signals  from  different  labels 
(line-by-line  or  frame-by-frame).  This  section  describes  the  dif¬ 
ferent  scanner  types.  More  detailed  information  can  be  found  on 
the  Web  pages  of  Cambridge  Technology  (Cambridge,  MA)  and 
GSI-Lumonics  (formerly,  General  Scanning,  Bellerica,  ME). 

Galvanometer  Scanners 

Beam  scanning  is  generally  implemented  using  mirrors  mounted 
on  galvanometer  “motors”  (see  Chapter  9,  this  volume).  The  mirror 
surface  is  deposited  on  a  substrate  of  silicon,  fused  quartz,  or,  for 
highest  performance  and  cost,  beryllium.  Such  a  rotating  mirror 
device  can  control  the  scan  angle  with  exquisite  accuracy  and  has 
the  additional  advantage  that  the  scanning  angle  is  insensitive  to 
wavelength,  something  not  true  of  the  faster  acousto-optical 
scanners. 

Like  a  motor,  the  actuator  in  a  galvanometer  has  a  rotor  and  a 
stator  and  currents  circulating  in  coils  in  one  or  the  other  interact 
with  the  magnetic  field  of  a  permanent  magnet  in  the  other  to 
produce  the  forces  that  move  the  rotor  (Fig.  3.18).  The  respon¬ 
siveness  of  the  actuator  depends  on  the  torque/inertia  ratio.  In 
general,  moving-coil  actuators  have  a  lower  moment  of  inertia  (7), 
lower  torque,  and  better  positioning  accuracy  than  moving-magnet 
actuators,  while  the  latter  can  move  faster  because  of  their  supe¬ 
rior  stiffness.  Consequently,  they  are  the  type  most  often  used  in 
confocal  microscopes. 

Although  some  cheap  scanners  achieve  high  scan  speed  by 
rotating  continuously  (rotating  multi-mirror  scanners),  the  angular 
scanning  range  of  such  scanners  is  fixed  and  consequently,  the 
scanned  area  cannot  be  “zoomed.”  In  addition,  the  scan  motion  is 
not  linear  and  it  is  difficult  to  fabricate  and  mount  a  multi-segment 


mirror  in  which  all  the  facets  are  an  equal  distance  from  the  axis. 
To  avoid  these  limitations,  the  scanners  used  in  confocal  micro¬ 
scopes  oscillate  around  a  fixed  neutral  position.  Such  galvanome¬ 
ters  come  in  two  flavors:  resonant  scanners  that  vibrate  at  a  fixed 
frequency  and  in  which  much  of  the  force  needed  to  oscillate  the 
mirror  is  provided  by  energy  stored  in  a  torsion  spring,  and  linear 
galvanometers  that  operate  at  lower  but  variable  frequencies  and 
in  which  the  spring  is  much  weaker  or  absent.7  In  both  cases,  the 
required  scan  forces  increase  with  the  square  of  both  the  scan 
frequency  and  the  scan  angle  and  the  limit  on  both  is  usually  set 
by  the  amount  of  power  that  can  be  dissipated  by  the  actuators. 
The  important  difference  is  that,  as  a  resonant  actuator  requires  far 
less  power  than  a  linear  one  at  a  given  scan  frequency  and  scan 
angle,  it  can  attain  more  of  either  (or  both)  of  these  before 
overheating. 

Resonant  galvanometers  usually  operate  at  4  kHz  and  above, 
often  near  to  a  fractional  multiple  of  the  horizontal  scan  fre¬ 
quencies  of  the  NTSC  video  standard.  The  mirror  angle  varies  in 
a  sinusoidal  manner  at  a  frequency  determined  by  the  mechanical 
resonance  of  the  device.  This  in  turn  is  determined  by  the  spring 
constant  k  and  the  moment  of  inertia  7  of  the  rotor/mirror 
assembly:  larger  mirror  or  higher  7,  lower  frequency;  stiffer  spring, 
higher  frequency  but  smaller  scan  angle. 

The  Olympus  SIM  scanner  runs  sinusoidally  in  “tornado” 
mode.  The  Leica  RS  uses  a  resonant  scanner  but  drives  it  with  a 
sawtoothlike  waveform.  However,  the  angular  velocity  of  a  reso¬ 
nant  galvanometer  varies,  reaching  a  maximum  at  a  phase  angle 
of  0°  and  a  minimum  at  ±90°,  where  the  scan  direction  changes. 
As  the  signal  intensity  per  pixel  is  inversely  proportional  to 
scanner  speed,  in  order  to  keep  geometrical  scan  nonlinearity  to 
<4%,  recording  is  restricted  to  the  two  ±30°  intervals  of  the  total 
360°  waveform.  As  sin  30°  =  0.5,  the  actual  amplitude  of  the  mirror 
motion  has  to  be  twice  as  large  as  is  actually  required  to  scan  the 
field  being  digitized  and,  because  of  the  time  lost  in  turnaround, 
signal  can  only  be  collected  15%  to  30%  of  the  time.  To  avoid 


FIGURE  3.1 8.  Two  types  of  galvanometer  actuator.  Although  moving-magnet 
actuators  (A)  are  smaller  and  more  rigid,  moving-coil  actuators  (B)  are  more 
precise.  (These  diagrams  modified  from  originals  kindly  provided  by  R. 
Aylward,  Cambridge  Technologies,  Cambridge,  MA.) 


7  Mechanical  stops  of  some  kind  must  be  used  to  restrict  rotor  movement  to 
<1  rotation. 
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bleaching  outside  the  recorded  area  during  the  remainder  of  the 
waveform,  stops  are  inserted  at  an  image  plane  to  exclude  light 
that  would  otherwise  land  outside  the  digitized  area. 

The  effective  scanning  speed  can  be  increased  by  collecting 
data  during  both  the  forward  and  the  backward  movements  of  the 
mirror.  However,  because  the  backward  movement  also  reverses 
the  direction  in  which  pixels  are  recorded,  additional  electronics 
or  appropriate  software  is  needed  to  repackage  the  data  into  a 
standard  image  format.  This  alignment  process  is  seldom  totally 
successful,  especially  when  used  to  perform  confocal  imaging  at 
video  rate,  and  the  result  is  that  every  other  line  is  displaced 
slightly  sideways  with  respect  to  its  neighbor. 

Here  at  the  EMBL,  two  high-resolution  methods  have  been 
implemented  to  solve  this  problem:  use  of  a  position  signal  from 
the  galvanometer  to  detect  the  zero-crossing  voltage  line.  This 
event  starts  a  counter.  By  running  the  galvanometer  symmetrically 
and  using  two  independently  adjusted  counters,  the  forward  and 
backward  movements  can  be  accurately  synchronized,  although 
special  software  is  still  needed  to  rearrange  the  pixel  order. 
However,  even  this  trick  does  not  overcome  the  fact  that  sinusoidal 
bidirectional  scanning  permits  one  to  record  quasi-linear  data 
during  only  -33%  of  the  scan  time.8 

For  example,  the  galvo  inside  the  Leica  TCS  SP2  RS  runs  at 
4000  lines/s  (8000  in  bidirectional  mode)  and  this  translates  into 
150fps  @  32  X  512  pixel  @  zoom  1.  Using  a  galvanometer  from 
Cambridge  Technology.  Olympus  claims  4000  lines  or  16fps  @ 
256  X  256  for  both  their  confocal  microscopes  in  the  Fluoview 
1000. 

Resonant  scanners  accelerate  fast  enough  to  actually  cause  the 
mirror  to  bend  or  flex.  This  has  two  negative  effects:  (a)  the  wave- 
front  can  become  distorted,  reducing  the  resolution  of  the  instru¬ 
ment  and  (b)  if  a  high-reflectivity  coating  is  used,  it  may  age  more 
rapidly  than  on  a  mirror  at  rest.  The  latter  problem  can  be  avoided 
by  using  the  less-efficient  “enhanced  full-metal”  coatings. 

Note  that  flexing  is  more  of  a  problem  for  large  mirrors  so 
choosing  a  mirror  is  always  a  compromise  between  rigidity/scan 
speed  and  the  mirror  size  required  to  reflect  a  beam  large  enough 
to  fill  the  BFP  of  all  the  objective  lenses  used.  Clearly  these 
requirements  can  interact  in  that  a  mirror  big  enough  to  reflect  the 
entire  ray  bundle  may  also  flex  enough  to  negate  the  resolution 
improvement  obtained  by  so  doing.  In  this  case  it  is  important  to 
remember  that,  at  a  given  NA,  the  ray  bundle  required  will  be 
smaller  when  using  an  objective  with  a  higher  magnification. 

In  a  linear  galvanometer,  almost  all  of  the  torque  needed  to 
move  the  mirror  is  provided  by  the  magnetic  forces  produced  by 
currents  flowing  in  coils.  The  maximum  speed  is  limited  by  the 
amount  of  RMS  power  the  galvo  can  dissipate  without  the  perma¬ 
nent  magnet  being  heated  beyond  its  Curie  point  ( 100— 
110°C)  where  it  becomes  demagnetized.  Although  linear  gal¬ 
vanometers  try  to  follow  a  distorted  sawtooth  waveform,  in  this 
electro-mechanical  system,  much  more  time  is  lost  to  overscan  than 
is  common  in  the  electron-magnetic  systems  used  to  scan  the  elec¬ 
tron  beam  in  a  cathode-ray  tube  (CRT).9  Time  lost  in  retrace  is  one 
of  the  reasons  that,  assuming  that  the  microscope  is  designed  prop¬ 
erly,  one  should  get  better  results  recording  signal  from  a  single. 


8  Two  ±  30°  scans  cover  120°,  or  33%  of  the  360°  sine-wave  cycle. 

9  In  other  words,  when  you  use  a  higher  line  rate,  a  significant  fraction 
(20%-50%)  of  your  collection  time  can  be  wasted  by  retrace.  This  reduces 

the  signal-to-noise  ratio  (S/N)  of  the  data  beyond  that  which  comes  about 
merely  because  the  beam  spends  less  time  in  each  pixel  and  therefore  excites 
fewer  photons. 


3-s  scan  than  by  Kalman-averaging  three,  1-s  scans.10  If  the  mirror 
is  driven  with  a  triangular  wave  and  the  signal  collected  in  both 
directions,  it  is  possible  to  increase  both  the  “usable/unusable”  duty 
cycle  and  the  line-scan  rate,  as  long  as  provision  is  made  for  invert¬ 
ing  the  pixel  order  and  aligning  the  two  sets  of  scans  (see  above). 

In  contrast  to  resonant  galvanometers,  linear  galvanometers 
follow  the  input  signal  at  any  drive  frequency  although  with  a 
phase  delay  that  is  not  negligible  under  normal  operating  con¬ 
ditions.  Ideally,  a  sawtooth  motion  produces  a  relatively  slow, 
linear  recording  line  followed  by  a  fast  retrace  (mono-directional 
scans).  Problems  occur  because  of  the  high  angular  acceleration 
needed  during  the  transition  between  the  end  of  one  line  to  the 
beginning  of  the  next  (i.e.,  the  retrace).  The  disparity  between  the 
linear  sawtooth  or  triangular-wave  input  and  the  actual  motion  of 
the  mirror  is  best  reduced  by  using  closed-loop  negative  feedback: 
the  universal  engineering  prescription  for  “distortion”  (Aylward, 
1999,  2003).  This  requires  a  means  by  which  the  position  of  the 
rotor  can  be  measured  and  the  information  fed  back  to  the  control 
circuitry  driving  the  coils. 

Figure  3.19  diagrams  four  methods  of  detecting  rotor  position: 
two  involving  changes  in  capacitance  between  fixed  and  moving 

A 


FIGURE  3.1 9.  Four  types  of  galvanometer  position  sensor.  Rotor  position  can 
be  determined  by  ascertaining  changes  in  the  capacitance  between  structures 
attached  to  it  and  other  electrodes  mounted  on  the  stator.  The  dielectric  but¬ 
terfly  design  (B)  is  more  precise  than  that  shown  in  (A)  because  it  is  easier  to 
align.  [Note  that  in  (B),  the  electrode  spacings  have  been  exploded  for  clarity.] 
(These  diagrams  modified  from  originals  kindly  provided  by  R.  Aylward, 
Cambridge  Technologies,  Cambridge,  MA.) 


10  On  the  other  hand,  from  the  sampling  point  of  view,  if  one  is  recording  three- 
dimensional  data,  it  is  better  to  record  three  1-s  scans  at  slightly  different 
defocus  positions,  if  doing  so  is  needed  to  meet  Nyquist  conditions  in  the 
.--direction. 
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FIGURE  3.20.  Galvanometer  mirror  performance.  Measured  plots  of  mirror 
position  (black),  angular  velocity  (green),  and  drive  current  (red).  (These  dia¬ 
grams  modified  from  originals  kindly  provided  by  Steven  Sequeira,  Cambridge 
Technologies,  Cambridge,  MA.) 

electrodes  and  two  involving  the  modulation  of  light  from  a  light- 
emitting  diode  (LED)  by  the  action  of  a  shutter  attached  to  the 
rotor.  Of  the  capacitance  methods,  the  newer  “dielectric  butterfly” 
method  provides  repeatability  of  1  micro-radian  (i.e.,  1  mm  at  a 
range  of  1  km).  Although  the  LED  methods  are  less  precise,  they 
are  also  much  smaller  and  less  expensive.  In  its  efforts  to  make  the 
actual  motion  of  the  mirror  mimic  the  sawtooth  waveform,  the  feed¬ 
back  circuit  produces  a  current  acceleration/deceleration  waveform 
that  hardly  resembles  a  “sawtooth”  at  all  (Fig.  3.20  red  line). 

As  useful  signals  cannot  be  recorded  during  retrace,  the  beam 
should  be  blanked  to  avoid  unnecessary  bleaching  and  this  can  be 
implemented  using  mechanical  stops  such  as  those  mentioned 
above  or  by  using  laser-blanking  systems  utilizing  an  AOD  or  a 
Pockels  Cell  (see  below).  Not  only  do  these  devices  reduce  the 
average  light  dose  to  the  sample  by  10%  to  20%,  they  also  elimi¬ 
nate  the  high-dose  damage  done  in  the  region  to  either  side  of  the 
raster  where  the  beam  slows  down,  stops,  and  re-accelerates.  As 
this  process  may  deposit  10  to  20  times  more  photons/pm2  in  these 
localized  areas,  it  is  wise  to  ensure  that  living  cells  do  not  extend 
beyond  the  side  borders  of  the  recorded  area  if  appropriate  beam¬ 
blanking  is  not  available. 

An  advantage  of  using  linear  actuators  responding  to  sawtooth 
drive  signals  is  that  the  actual  recording  time  can  be  2  to  2.5  times 
higher  than  the  33%  mentioned  above.  The  advantages  of  one  type 
of  galvanometer  over  the  other  in  terms  of  stability  and  long-term 
performance  are  not  known. 

Olympus  drives  their  optional  SIM  scanner  in  a  special 
“tornado”  mode  in  which  a  damped,  sinusoidal  signal  is  applied  to 
both  scanners,  producing  a  spiral,  Lissajous  pattern.  This  pattern 
is  used  to  generate  a  circular  bleaching  spot  without  wasting  any 
time  turning  the  scanner  around,  as  happens  when  it  is  driven  with 
sawtooth  signals.  However,  because  the  beam  paths  overlap  in  the 
center  of  this  circle,  more  energy  is  deposited  there,  creating  a  non- 
uniform  bleach. 

General  Specifications 

Scanners  wobble  above  and  below  the  scan  line  and  jitter  as  they 
go  along  it.  The  wobble  specification  tells  how  well  a  scanner 
remains  on  the  line  and  the  jitter  how  accurately  the  scanner  returns 


to  a  specific  pixel  of  that  line  at  a  given  instant  of  the  scan  cycle. 
In  both  cases,  performance  depends  on  the  quality  of  the  bearings 
and  the  care  with  which  the  moving  mass  has  been  mass  bal¬ 
anced.11  In  a  scanning  microscope,  these  specifications  can  affect 
the  resolution  limit  of  the  microscope. 

Let  us  assume  an  optical  scan  angle  of  +  3°  in  a  beam  scanner 
(i.e.,  a  complete  mechanical  scan  of  6°  or  lOOmrad).  If  there  are 
2500  pixels/line,  the  jitter  (and  the  wobble)  must  be  lower  than 
40|irad  (100mrad/2500),  which  is  about  four  times  more  than  the 
5  to  I  Op  rad  specification  for  the  Cambridge  Technology  6210H 
[with  advanced  optical  feedback,  Figure  3.16(D),  used  in  many  con- 
focal  microscopes.  Cambridge  Technology  scanners  have  15  [trad 
drift  per  °C].  Fortunately,  common  experience  has  shown  that  the 
scanner  specifications  tend  to  be  conservative  and  their  actual  per¬ 
formance  is  good  enough  that  it  does  not  limit  microscope  perfor¬ 
mance.  The  actual  test  is  to  use  the  galvanometers  at  the  very  small 
scan  angles  appropriate  to  high-NA,  low-magnification  lenses. 
The  minimal  scan  angle  of  the  instruments  built  at  EMBL  is 
~20  [irad/line,  but  even  here,  the  stability  is  so  good  that  an  indica¬ 
tor  must  be  used  to  tell  the  user  that  the  scanner  is  operating.  More 
recent  scanners  have  even  better  specifications  and  use  even  larger 
mirrors,  a  factor  that  allows  designers  to  use  simpler  scanning 
lenses. 

Current  devices  represent  a  highly  optimized  product  that  is 
the  culmination  of  intensive  research  over  many  years.  Commer¬ 
cial  units  perform  at  or  near  the  limits  imposed  by  physics  (e.g., 
galvo  mass  vs.  planarity,  mirror  diameter,  drive  power  vs.  over¬ 
heating)  and  materials  properties  (density  vs.  stiffness).  In  spite  of 
this  long  development  history,  last  year  Cambridge  Technology 
introduced  the  6215H  galvanometer  with  scan  rates  50%  to  100% 
higher  (1.5  kHz  triangular  wave  @  ±  3°)  than  the  previous  model. 

The  situation  is  less  settled  in  the  field  of  resonant  gal¬ 
vanometers  where  the  number  of  applications  for  very  fast  scan¬ 
ners  is  increasing  rapidly  and  not  all  of  the  requirements  can  be 
fulfilled  using  electro-optic  or  acousto-optic  devices.  With  reso¬ 
nant  scanners,  one-directional  line  scanning  can  now  be  performed 
at  up  to  8000  Hz  with  present  commercial  scanners  but  research 
scanners  have  gone  faster.  However,  resonant  scanners  have  the 
disadvantage  that  their  scan  speed  is  fixed  and  their  duty  cycle 
short.  To  improve  the  signal-to-noise  ratio  of  weak  fluorescence 
signals,  one  can  only  line  average,  and  this  process  accumulates 
readout  noise.  Finally,  there  is  also  no  simple  way  to  move  the 
beam  within  a  specific  region  or  along  a  line  that  isn’t  horizontal. 
For  FRAP  experiments,  one  must  bypass  the  laser  light  around  the 
resonant  scanner  to  a  second  linear  scanner. 

ACOUSTO-OPTICAL  COMPONENTS 

The  performance  of  acousto-optical  components  is  based  on  the 
optical  effects  of  acoustic  fields  on  birefringent  crystals.  The 
acoustic  field  is  generated  and  controlled  by  a  piezoelectric 


11  Mass  balancing  is  both  crucially  important  to  reduce  vibration  and  difficult 
to  achieve  if  one  adheres  to  the  optical  requirement  that  the  axis-of-rotation 
pass  through  the  plane  of  the  mirror.  If  the  mirror  surface  is  on  the  axis,  it 
follows  that  the  material  making  up  the  mirror  assembly  can  only  be  dis¬ 
tributed  symmetrically  on  both  sides  of  the  axis  if  additional  structures  are 
added  solely  to  counterbalance  it.  Such  additions  lead  in  turn  to  complex 
mirror  geometries  that  increase  both  I  and  the  chance  that  the  mirror  will 
flex.  As  a  result,  the  scanners  used  in  confocals  are  likely  to  have  silicon 
mirrors  about  1-nun  thick  mounted  symmetrically  (i.e.,  with  the  axis  going 
up  the  middle  of  the  silicon).  Moral:  One  pays  in  many  ways  for  trying  to 
scan  fast.  Do  not  do  it  unless  you  need  to. 
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crystal,  excited  at  50  to  150  MHz,  attached  to  one  side  of  the  crystal 
and  a  vibration  absorber  attached  to  the  facing  side.  At  a  specific 
frequency  of  the  acoustic  wave,  the  periodic  pattern  of  compres¬ 
sion  and  rarefaction  associated  with  it  affects  the  local  density,  and 
therefore  the  RI,  of  the  crystal  to  form  a  three-dimensional  peri¬ 
odic  diffractor.  Light  passing  through  the  crystal  is  diffracted  at  an 
angle  (0)  that  depends  on  the  frequency  (and  hence  the  wave¬ 
length)  of  the  acoustic  wave  and  on  the  wavelength  of  the  light12 
and  is  defined  by  the  equation: 

e  =  V./v. 

where  X  is  the  optical  wavelength  in  air,  Va  is  the  acoustical  veloc¬ 
ity  of  the  material,  and/ais  the  acoustic  frequency.  In  a  Te02  crystal 
excited  at  100 MHz,  the  diffraction  angle  for  500 nm  light  is  -4.3°. 

Normally  the  crystal  is  shaped  like  a  rhombus,  with  the  undif¬ 
fracted  beam  entering  and  leaving  normal  to  the  two  sides  and  the 
piezo  generating  plane  waves  that  are  parallel  to  the  other  two  sides 
(Fig.  3.21).  The  waves  are  tilted  with  respect  to  the  laser  beam  by 
an  angle  that  is  roughly  half  of  the  expected  diffraction  angle. 

Although  fused  and  crystalline  quartz  can  be  used  as  the  active 
material,  the  preferred  AO  material  is  Te02.  Most  AO  materials 
operate  well  only  over  a  limited  wavelength  range:  generally  less 
than  400  nm  (Chang,  1995).  They  also  affect  the  polarization  of 
the  light.  Light  passing  through  a  Te02  AOD,  along  the  “slow” 
direction,  will  emerge  with  its  polarization  direction  rotated  by 
90°. 

By  sweeping  the  drive  frequency  of  the  pressure  waves,  an 
acousto-optical  deflector  (AOD)  will  deflect  a  beam  of  mono¬ 
chromatic  light  over  a  range  of  angles,  making  it  useful  as  a  scan¬ 
ning  device.  Acousto-optical  modulators  (AOM)  are  used  to  blank 
a  monochromatic  light  beam  intermittently,  to  create  a  new  beam 
with  a  lower  (time-averaged)  intensity.  In  an  acousto-optical 
tunable  filter  (AOTF),  the  pressure  wave  pattern  in  the  crystal  is 


acoustic 


FIGURE  3.21.  Schematic  diagram  of  acousto-optical  device.  Light  entering 
from  the  left  strikes  the  surface  of  the  TeO,  crystal  at  normal  incidence. 
Acoustic  waves  propagating  upwards  from  the  piezoelectric  transducer  diffract 
the  light  beam,  either  up  or  down,  depending  on  the  polarization  of  the  light 
beam.  Because  diffracted  light  would  suffer  dispersion  on  leaving  the  crystal, 
the  far  face  is  cut  at  a  steeper  angle  than  would  be  needed  if  it  were  a  rhombus. 
Therefore,  the  device  shown  would  work  optimally  only  with  light  polarized 
parallel  to  an  acoustic  beam  that  diffracts  into  the  1  st  order. 


12  In  Te02,  the  crystal  most  commonly  used,  the  speed  of  sound  is  660  m/s.  As 
a  result,  a  100  MHz  wave  will  have  a  period  of  about  6.6  pm,  about  10  to  15 
time  greater  than  the  wavelength  of  the  light  being  diffracted. 


FIGURE  3.22.  AOD  performance.  The  amount  of  deflection  produced  by  an 
AOD  varies  with  the  amount  of  change  in  the  acoustic  frequency,  the  wave¬ 
length  of  the  incoming  light  (A),  and  the  material  in  which  the  deflection  takes 
place  (B). 


used  to  spectrally  filter  the  light.  An  acousto-optical  beam-splitter 
(AOBS)  is  really  just  an  AOTF  used  to  spectrally  combine  or 
separate  the  light  at  specific  laser  wavelengths  from  all  other  light. 

As  the  frequency  of  the  acoustic  wave  can  be  changed  on  the 
microsecond  time  scale,  acousto-optical  components  can  change 
their  deflection  properties  much  faster  than  when  using  mechani¬ 
cal  parts.  For  a  given  frequency  sweep,  the  amount  of  deflection 
depends  on  the  type  of  material  used  in  the  AOD  and  the  wave¬ 
length  of  the  incident  light  (Fig.  3.22).  The  response  time  is  limited 
by  the  time  it  takes  for  a  new  acoustic  wave  field  to  propagate 
across  the  width  of  the  crystal  and,  therefore,  depends  linearly  on 
the  width  of  the  light  beam  that  has  to  be  modulated.  Typical  rates 
are  -150  ns/mm  of  beam  width.  Only  a  narrow  band  of  wave¬ 
lengths  are  diffracted  and  out-of-band  contributions  are  reduced,  a 
feature  that  is  especially  suitable  for  filtering  devices.  Also,  as 
acousto-optical  components  do  not  have  any  moving  parts,  they 
require  little  maintenance. 

At  root,  all  AO  components  are  very  similar.  They  use  the  same 
crystals,  operating  at  the  same  acoustic  frequencies  and  diffract 
over  similar  angles.  As  AOTFs  and  AOBSs  operate  on  smaller 
beams,  they  can  be  faster  and  because  they  operate  at  fixed  input 
and  output  angles,  the  crystals  are  smaller  and  less  expensive.  As 
AODs  operate  at  a  fixed  input  angle  but  a  variable  output  angle, 
they  must  be  larger,  slower,  and  more  expensive.  Otherwise  they 
are  just  AOTFs  operating  with  a  swept  acoustic  frequency.  Increas- 


56  Chapter  3  •  J.  Rietdorf  and  E.H.K.  Stelzer 


ing  the  acoustic  frequency  increases  the  deflection  angle:  increas¬ 
ing  the  acoustic  power  increases  the  fraction  of  the  light  that  is 
deflected  from  the  incident  beam.  The  preferred  crystal,  Te02, 
gives  the  most  deflection  for  a  given  input  wavelength  and  acoustic 
power  level.  It  is  only  replaced  by  one  of  the  other  materials  when 
deflecting  high-power  laser  beams13  or  when  it  cannot  be  cooled 
sufficiently  well. 

Acousto-Optical  Deflectors 

Acousto-optical  deflectors  (AOD)  can  scan  a  laser  beam  at  up 
to  100  kHz,  compared  to  about  500  to  1000  Hz  for  linear  gal¬ 
vanometer  scanners  and  about  4  to  8  kHz  for  resonant  gal¬ 
vanometers.  In  addition,  they  allow  zooming  and  varying  the  scan 
speed.  As  they  do  not  have  the  momentum  associated  with  moving 
parts,  they  require  little  retrace  time  and  are  capable  of  rapidly 
accessing  random  scan  areas. 

However,  one  cannot  simply  replace  the  faster  of  the  two  scan¬ 
ners  in  your  confocal  (as  described  in  Chapter  9,  this  volume)  with 
an  AOD  because,  as  the  deflection  is  produced  by  diffraction,  the 
scan  angle  depends  on  the  wavelength  of  the  light  beam.  As  a 
result,  the  longer  wavelength  fluorescent  light  will  be  deflected  by 
a  different  amount  on  its  way  back  through  the  crystal  and  will 
therefore  fail  to  pass  through  the  pinhole.  Although  this  dispersion 
effect  can  be  theoretically  compensated  for  by  a  chromatic  cor¬ 
rection  system,  such  systems  involve  so  many  additional  expen¬ 
sive,  lossy  elements,  they  are  not  practical  for  confocal  use.  AOD 
dispersion  is  an  even  more  serious  problem  when  working  with 
pulsed  radiation  in  a  multi-photon  microscope  (Lechleiter  et  al., 
2002). 

A  workable  AOD  scan  system  can  be  developed  for  confocal 
fluorescence  microscopy  if  the  incident  laser  beam  is  first  x- 
deflected  by  the  AOD  and  then,  after  passing  the  dichroic  mirror, 
y-deflected  by  a  slower,  galvanometer  scanner.  As  the  returning 
fluorescence  emission  is  deflected  by  the  slower  scanner  and 
reflected  by  the  dichroic  mirror  in  the  direction  of  a  linear  detec¬ 
tor,  it  does  not  reach  the  AOD.  A  lens  focuses  light  from  the 
dichroic  onto  a  detector  slit,  which  must  be  used  because  the  move¬ 
ment  along  the  fast  axis  is  not  being  descanned.  Although  a  linear 
CCD  detector  behind  the  slit  might  be  used  to  “track”  the  moving 
signal  spot,  this  solution  is  still  confocal  only  along  one  axis. 
However,  it  gives  better  sectioning  than  slit-illumination/slit- 
detection  systems,  such  as  the  Zeiss  LSM-5Live  and  in  practice 
it  seems  to  work  quite  well  (Draaijer  and  Houpt,  1988;  Tsien  and 
Bacskai,  1995). 

Compared  to  galvanometers,  AODs  provide  small  deflection 
angles  (<3.5°  vs.  -6°  needed  for  most  objectives),  a  limited 
angular  resolution,14  a  non-circular  aperture,  and  a  low  “reflection” 
efficiency  (<70%,  i.e.,  only  0%-70%  of  the  incident  beam  is 
deflected  vs.  95%-99%  for  a  mirror).  These  parameters  depend  on 
each  other.  A  higher  angular  resolution  can  be  achieved,  but  only 
at  the  expense  of  a  spatial  variation  in  the  diffraction  efficiency 
(10%  at  2000  lines).  Finally,  the  drive  signals  fed  to  an  AOD  must 
be  readjusted  for  each  excitation  wavelength.  Although  this  can  be 


13  Such  as  those  used  for  laser  light  shows. 

14  About  500-1500  lines.  This  is  set  by  time-bandwidth  product  of  the  crystal: 
faster  scans  have  less  precision  and  hence  fewer  distinct  lines  in  the  raster. 
This  occurs  because  if  the  acoustic  frequency  varies  too  rapidly,  the  acoustic 
wavelength  will  be  different  on  the  near  and  far  sides  of  the  light  beam,  dif¬ 
fracting  these  rays  differently. 


arranged  for  a  small  number  of  discrete  laser  lines,  it  cannot  be 
used  to  deflect  “white"  light  in  a  useful  manner. 

Acousto-Optical  Modulators 

Acousto-optical  modulators  (AOM)  are  used  to  modulate  the 
intensity  of  a  beam  and  can  thus  also  function  as  laser-line  selec¬ 
tors.  In  FRAP  experiments,  for  example,  they  are  used  to  switch 
the  laser  on  and  off  rapidly  (Flamion  et  al.,  1991).  It  is  common 
to  choose  the  deflected  beam  as  the  one  that  is  used  by  the  optical 
system  because  it  can  be  switched  on  and  off  with  high  extinction 
ratio  (typically  >40  dB  or  100:1)  and  its  intensity  can  be  varied 
from  zero  to  more  than  85%  of  the  incident  beam  (Fig.  3.17). 

Acousto-Optical  Tunable  Filters 

Acousto-optical  deflection  causes  light  of  different  wavelengths  to 
exit  the  crystal  at  different  angles.  In  an  optical  system  designed 
so  that  input  light  strikes  the  crystal  at  only  one  angle  and  dif¬ 
fracted  light  re-enters  the  optical  system  only  at  one  diffraction 
angle,  then  one  can  choose  the  wavelength  of  the  light  diffracted 
at  this  angle  by  varying  the  frequency  of  the  acoustic  wave.  This 
makes  it  a  “tunable  filter."  Because  it  is  not  possible  to  diffract  all 
the  light  (and  therefore  turn  off  the  input  beam),  the  acousto- 
optical  tunable  filters  (AOTF)  is  usually  configured  so  that  one 
uses  the  light  in  the  diffracted  beam. 

As  temperature  affects  both  the  RI  of  the  crystal  and  the  speed 
of  sound,  it  also  affects  the  wavelengths  of  both  the  light  and  the 
acoustic  waves.  Therefore,  it  also  affects  the  diffraction  angle.  If 
the  angle  changes  enough  so  that  the  diffracted  beam  “misses”  the 
entrance  of  the  optical  system  (i.e.,  wanders  off  the  input  pupil  of 
the  optical  fiber),  this  will  result  in  a  large  reduction  in  the  signal 
entering  the  fiber  (and  finally  striking  the  specimen). 

Because  the  diffraction  properties  of  acousto-optical  filters 
(Harris  and  Wallace,  1969)  can  be  switched  within  microseconds, 
they  are  a  tremendous  improvement  over  using  filter  wheels  to 
choose  excitation  wavelengths  (Nitschke  et  al.,  1997).  AOTFs  can 
be  used  to  select  the  illumination  waveband  when  working  with 
either  a  broadband  light  source  (Lewis  et  al.,  1992)  or  a  multi-line 
laser  (Wachman  et  al.,  1997).  They  can  also  be  used  to  select  a 
light  detection  band  (  Wachman  et  al.,  1997). 

When  one  has  signals  from  two  or  more  fluorophores,  it  is 
easier  to  separate  them  if  they  are  excited  individually  and  sequen¬ 
tially.  To  accomplish  this,  the  AOTF  can  be  used  to  change  the 
laser  lines  after  each  line  scan,  that  is,  each  line  is  scanned  several 
times,  sequentially  —  once  for  each  of  the  excitation  wavelengths 
needed.  However,  one  can  only  take  full  advantage  of  this  high¬ 
speed  switching  of  the  excitation  if  the  other  components  in  the 
light  paths  can  also  be  adapted  to  operate  with  similar  speed  and 
flexibility. 

AOTFs  are  also  used  for  switching  off  the  illumination  light 
during  retrace  (see  above). 

Acousto-Optical  Beam-Splitters 

The  acousto-optical  beam-splitter  (AOBS)  is  actually  an  AOTF 
used  in  an  imaginative  manner  by  Leica  to  replace  the  dichroic 
mirror  usually  used  to  separate  the  illumination  and  detections 
paths  in  a  confocal  fluorescence  microscope  (Engelhardt  et  al., 
1999;  Birk  et  al.,  2002;  it  is  implemented  in  Leica’s  Spectral 
Confocal  Microscopes,  TCS  SP2  AOBS).  In  practice,  the  AOBS 
is  programmed  so  that  it  leaves  most  (90% — 95%)  of  the  fluores¬ 
cent  light  undeflected,  and  only  deflects  light  at  specific  laser  lines. 
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Transmission  AOBS  4  Lines 


FIGURE  3.23.  Transmission  of  Leica  acousto-optical  beam-splitter  (AOBS). 
This  plot  shows  the  transmission  of  the  beam-splitter  to  light  originating  from 
a  point  in  the  specimen.  The  notches  in  the  transmission  at  488,  542,  593,  and 
623  nm  show  that  the  AOBS  has  been  adjusted  to  diffract  these  wavelengths  of 
laser  light  into  the  beam  and,  consequently,  remove  then  on  the  return  trip.  The 
position  of  these  bands  can  be  adjusted  to  suit  any  visible  laser  that  might  be 
used. 


Because  it  is  possible  to  excite  the  piezo  with  a  signal  that  is 
the  sum  of  several  specific  frequencies,  and  because  the  resulting 
pressure  field  then  diffracts  several  wavelengths  simultaneously, 
the  AOBS  allows  efficient,  simultaneous  deflection  of  up  to  8  laser 
lines  towards  the  sample,  while  the  average  transmission  of  the 
crystal  (away  from  the  reflection  bands)  is  about  95%  (Fig.  3.23). 
The  intensity  of  each  line  can  be  controlled  independently  over  a 
range  of  more  than  20:1. 

The  advantages  of  the  AOBS  over  systems  that  use  several, 
multi-band,  dichroic  mirrors  as  the  main  beam-splitter  are  its  high 
efficiency,  its  lack  of  moving  parts,  and  its  flexibility  in  terms  of 
being  able  to  set  up  the  system  to  use  new  laser  lines  simply  by 
changing  the  software.  On  the  other  hand,  because  the  proper  oper¬ 
ation  of  the  AOBS  depends  on  the  light  input  being  parallel  as  it 
passes  through  the  crystal,  light  originating  above  or  below  the 
plane-of-focus  may  see  slightly  different  transmission  characteris¬ 
tics.  Fortunately,  in  confocal  operations,  most  out-of-focus  light  is 
removed  at  the  pinhole. 

ELECTRO-OPTICAL  MODULATORS 

Electro-optical  modulators  (EOM)  are  crystals  utilizing  the 
Pockels  effect  to  modulate  the  polarization,  the  intensity,  the  phase, 
the  frequency,  or  the  direction  of  propagation  of  laser  beams.  They 
are  faster  (1  GHz)  than  acousto-optical  devices  but  they  not  only 
are  sensitive  to  electro-magnetic  interference  but  they  also  gener¬ 
ate  quite  a  lot  of  it  (Draaijer  et  al. ,  1996).  In  addition,  they  have  a 
lower  extinction  ratio  and  are  more  sensitive  to  temperature 
changes  (Chang,  1995).  EOMs  can  be  used  to  modify  the  polar¬ 
ization  (Hofkens  et  al.,  1998)  and  the  intensity  or  the  frequency  of 
laser  beams  (Helm  et  al.,  2001).  Although,  in  principle  they  can 
also  be  used  as  scanners,  they  are  generally  not  used  as  such  in 
microscopes.  They  are  usually  optimized  for  performance  at  a 
single  wavelength  (Maldonado,  1995). 

Piezoelectric  Scanners 

Piezoelectric  scanners  are  mostly  used  to  move  the  objective  lens 
axially  for  three-dimensional  imaging  (Callamaras  and  Parker, 


1999)  or  to  move  the  object,  either  up  and  down  or  in  all  three 
directions  (in  a  stage-scanning  microscope).*  Because  massive 
objects  are  being  scanned,  settling  times  are  in  the  millisecond 
range  and  horizontal  scan  speeds  are  limited  to  10  to  100Hz. 
Piezoelectric  scanners  utilize  the  piezoelectric  effect:  the  genera¬ 
tion  of  a  voltage  across  some  crystals  when  they  are  compressed 
or  alternatively,  the  deformation  of  a  crystal  when  a  voltage  is 
applied.  In  piezoelectric  scanners,  voltages  are  applied  to  elec¬ 
trodes  on  a  tube  piezoelectric  ceramic  and  this  elongates  and 
moves  an  object  fixed  to  the  end  of  the  tube.  Typically,  extensions 
of  about  100  to  200  |im  are  possible  with  a  resolution  in  the  order 
of  1  nm  (sub-nanometer  resolution  can  be  achieved  if  smaller  travel 
ranges  are  acceptable).  Like  galvanometer  scanners,  piezoelectric 
scanners  can  be  driven  in  a  resonant  mode  with  frequencies  in  the 
kilohertz  range. 

Piezoelectric  scanners  suffer  from  two  problems:  hysteresis,  in 
which  a  given  voltage  produces  a  different  displacement  when 
approached  from  a  lower  voltage  than  when  approached  from  a 
higher  voltage  (i.e.,  the  position  does  not  track  when  reversing  the 
scanning  direction),  and  creep,  in  which  the  position  changes 
with  time,  even  though  the  applied  voltage  remains  fixed.  As  a 
result,  piezoelectric  elements  are  best  used  in  conjunction  with  an 
element  that  measures  the  position,  such  as  a  capacitance-based 
translation/rotation  sensor.  In  addition,  hysteresis  effects  are  easily 
avoided  by  always  approaching  the  final  position  from  only  one 
direction,  rather  than  moving  the  system  back  and  forth.  In  this 
case  the  illumination  is  best  switched  off  when  driving  the  objec¬ 
tive  back  towards  the  start  position.  Position  drifts  with  time  are 
seldom  a  serious  limitation  for  most  scanning  light  microscopy 
applications. 

POLARIZING  ELEMENTS 

The  combination  of  a  polarizing  beam-splitter  and  a  quarter-wave 
plate  is  very  useful  in  confocal  reflection  microscopy  because  it 
provides  access  to  a  backscattered  (reflected)  light  signal  without 
any  loss  of  photon  efficiency  (Pawley  et  al.,  1993).  A  horizontally 
polarized  beam  is  reflected  by  a  polarizing  beam-splitter  and 
then  circularly  polarized  with  the  quarter-wave  plate  before  being 
focused  into  the  sample.  The  light  scattered  back  from  inhomo¬ 
geneities  is  rotated  by  another  45°  when  passing  the  quarter-wave 
plate  on  the  return  journey  and,  now  vertically  polarized,  deflected 
by  the  polarizing  beam-splitter.  This  beam  can  be  used  to  gener¬ 
ate  backscattered  light  contrast. 

It  should  not  be  forgotten  that  the  signal  derived  from  small 
features  in  a  biological  specimen  is  usually  incoherently  backscat¬ 
tered  and  not  coherently  reflected.  From  a  physical  point  of  view, 
backscattered  and  reflected  light  have  different  properties.  The 
detection  system  described  above  will  only  produce  a  useful  signal 
if  the  process  of  contrast  generation  preserves  the  polarization  of 
the  incident  light.  The  arrangement  does  not  reduce  the  efficiency 
of  the  fluorescence  detection  path,  and  there  are  good  reasons 
to  believe  that  a  circularly  polarized  beam  is  better  focused  by 
standard  objective  lenses  than  is  a  linearly  polarized  beam  (van  der 
Voort  and  Brakenhoff,  1989). 


*  Recently  piezo  activators  have  been  used  to  provide  small  deflections  of  the 
galvo  mirror  along  an  axis  perpendicular  to  the  galvo  axis.  This  is  used  to 
provide  rapid  XY  scanning  in  multi-beam  scanners  using  a  single  galvo. 
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REMOVING  EXCESS  LIGHT 

It  is  always  a  good  idea  to  introduce  adjustable  apertures  (e.g., 
irises)  into  the  optical  path.  The  best  locations  for  these  are,  of 
course,  at  conjugates  of  the  aperture  and  field  planes.  On  one  hand 
these  apertures  can  be  used  to  adjust  the  beam  because  they  essen¬ 
tially  define  the  optical  paths  in  the  instrument  (i.e.,  NA  and  field 
of  view).  On  the  other  hand,  once  the  instrument  is  aligned  for 
Kohler  illumination  (see  the  Appendix  of  Chapter  36,  this  volume), 
these  apertures  should  be  closed  as  far  as  possible  so  that  stray 
light  is  reduced,  preventing  any  light  from  entering  the  optical 
system  beyond  that  needed  (1)  to  fill  the  back-focal  plane  and  (2) 
to  fill  the  field-of-view. 

Some  optical  elements  require  more  effective  care  of  unwanted 
light.  Good  examples  are  AOMs  and  AOTFs.  These  components 
control  not  only  the  laser  light  intensity,  they  also  “switch”  laser 
light  sources  on  and  off  by  deflecting  the  beam.  However,  light 
deflected  so  that  it  does  not  enter  the  optical  path  is  likely  to  reflect 
to  places  where  it  is  unwanted  unless  steps  are  taken  to  absorb  it. 

Flip  mirrors  can  also  be  used  to  keep  a  laser  from  entering  an 
optical  device.  Particularly,  when  lasers  in  excess  of  a  few  tens  of 
milliwatts  are  used,  any  deflected  beam  should  be  guided  towards 
a  strong  absorber.  A  blackened  surface  on  the  inner  endplate  of  a 
tube  works  very  well  for  lower  laser  powers,  while  a  20-mm  high 
stack  of  razor  blades  whose  cutting  edges  are  oriented  towards  the 
beam  will  do  an  excellent  job  for  higher  laser  powers.  Powers 
above  1 W  are  rarely  used  in  microscopy  but  if  they  are,  acousto- 
and  electro-optical  devices  may  start  having  problems  and  beam 
dumps  will  require  extra  cooling  (improved  air  circulation  and 
even  water  cooling). 

SOME  USEFUL  LINKS 

http://www.cambridgetechnology.com/news/Choosing%20A%20 

Galvanometer.html 

http://www.brimrose.com 

http://www.gsilumonics.com/ 

http://micro.magnet.fsu.edu/primer/techniques/fluorescence/ 

interferencefilterintro.html 

http://www.neostech.com/new_content.asp?content=AO_ 

Introduction 

http://www.semrock.com 
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Points,  Pixels,  and  Gray  Levels:  Digitizing  Image  Data 

James  B.  Pawley 


CONTRAST  TRANSFER  FUNCTION,  POINTS, 
AND  PIXELS 

Microscopical  images  are  now  almost  always  recorded  digitally. 
To  accomplish  this,  the  flux  of  photons  that  forms  the  final  image 
must  be  divided  into  small  geometrical  subunits  called  pixels.  The 
light  intensity  in  each  pixel  will  be  stored  as  a  single  number. 
Changing  the  objective  magnification,  the  zoom  magnification  on 
your  confocal  control  panel,  or  choosing  another  coupling  tube 
magnification  for  your  charge-coupled  device  (CCD)  camera 
changes  the  size  of  the  area  on  the  object  that  is  represented  by 
one  pixel.  If  you  can  arrange  matters  so  that  the  smallest  feature 
recorded  in  your  image  data  is  at  least  4  to  5  pixels  wide  in  each 
direction,  then  all  is  well. 

This  process  is  diagrammed  for  a  laser-scanning  confocal  in 
Figure  4.1,  where  the  diameter  of  the  scanning  beam  is  shown  to 
be  at  least  four  times  the  interline  spacing  of  the  scanning  raster. 
This  means  that  any  individual  fluorescent  molecule  should  be 
excited  by  at  least  four  overlapping,  adjacent  scan  lines  and  that, 
along  each  scan  line,  it  will  contribute  signal  to  at  least  four 
sequential  pixels.  Finally,  it  is  important  to  remember  that  infor¬ 
mation  stored  following  these  rules  will  only  properly  resemble 
the  original  light  pattern  if  it  is  first  spatially  filtered  to  remove 
noise  signals  that  are  beyond  the  spatial  bandwidth  of  the  imaging 
system.  Image  deconvolution  is  the  most  accurate  way  of  impos¬ 
ing  this  reconstruction  condition  and  this  applies  equally  to  data 
that  have  been  collected  by  widefield  or  scanning  techniques.  If 
you  do  this  right,  your  image  should  look  like  that  in  Figure  4.2. 

If  you  are  already  convinced  of  this,  jump  to  page  71  for  the 
second  half  of  this  chapter,  on  gray  levels.  But  if  it  all  seems  to  be 
irrelevant  mumbo-jumbo,  read  on.  Incorrect  digitization  can 
destroy  data. 

Pixels,  Images,  and  the  Contrast 
Transfer  Function 

If  microscopy  is  the  science  of  making  magnified  images,  a  proper 
discussion  of  the  process  of  digitizing  these  images  must  involve 
some  consideration  of  the  images  themselves.  Unfortunately, 
microscopic  images  are  a  very  diverse  breed  and  it  is  hard  to  say 
much  about  them  that  is  both  useful  and  specific.  For  the  purposes 
of  discussion,  we  assume  that  any  microscopic  image  is  just  the 
sum  of  the  blurred  images  the  individual  “point  objects”  that  make 
up  the  object. 

But  what  is  a  point  object?  How  big  is  it?  Is  it  the  size  of  a 
cell,  an  organelle,  or  a  molecule?  Fortunately,  we  don’t  have  to 
answer  this  question  directly  because  we  aren’t  so  much  interested 
in  a  point  on  the  object  itself  as  the  image  of  such  an  object.  As 


should  be  clear  from  Chapters  1  and  2,  our  ability  to  image  small 
features  in  a  microscope  is  limited  at  the  very  least  by  the  action 
of  diffraction.1 

So  point  objects  can  be  thought  of  as  features  smaller  than 
the  smallest  details  that  can  be  transmitted  by  the  optical  system. 
The  final  image  is  merely  the  sum  of  all  the  point  images.  Although 
the  images  themselves  may  be  varied  in  the  extreme,  all  are 
composed  of  mini-images  of  points  on  the  object. 

By  accepting  this  simplification,  we  can  limit  our  discussion 
to  how  best  to  record  the  data  in  images  of  points.  Of  course,  we 
need  more  than  the  ability  to  divide  the  image  flux  into  point  mea¬ 
surements:  the  intensity  so  recorded  must  tell  us  something  about 
microscopical  structure.  In  order  for  an  image  to  be  perceivable 
by  the  human  eye  and  mind,  the  array  of  point  images  must  display 
contrast.  Something  about  the  specimen  must  produce  changes  in 
the  intensity  recorded  at  different  image  points.  At  its  simplest, 
transmission  contrast  may  be  due  to  structures  that  are  partially  or 
fully  opaque.  More  often  in  biology,  structural  features  merely 
affect  the  phase  of  the  light  passing  through  them,  or  become  self- 
luminous  under  fluorescent  excitation.  No  matter  what  the  mech¬ 
anism,  no  contrast,  no  image.  And  the  amount  of  contrast  present 
in  the  image  determines  the  accuracy  with  which  we  must  know 
the  intensity  value  at  each  pixel. 

Contrast  can  be  defined  in  many  ways  but  usually  it  involves 
a  measure  of  the  variation  of  image  signal  intensity  divided  by  its 
average  value: 


Contrast  is  just  as  essential  to  the  production  of  an  image  as 
“resolution.”  Indeed,  the  two  concepts  can  only  be  thought  of  in 
terms  of  each  other.  They  are  linked  by  a  concept  called  the  con¬ 
trast  transfer  function  (CTF),  an  example  of  which  is  shown  in 
Figure  4.3. 

The  CTF  (or  power  spectrum)  is  the  most  fundamental  and 
useful  measure  for  characterizing  the  information  transmission 
capability  of  any  optical  imaging  system.  Quite  simply,  it  is  a  graph 
that  plots  the  contrast  that  features  produce  in  the  image  as  a  func¬ 
tion  of  their  size,  or  rather  of  the  inverse  of  their  size:  their  spatial 
frequency.  Periodic  features  spaced  1  mm  apart  can  also  be  thought 
of  as  having  a  spatial  frequency  of  1000  periods/m,  or  1  period/mm 
or  1/1000  of  a  period/pm.  Although  we  don’t  often  view  peri¬ 
odic  objects  in  biological  microscopy  (diatom  frustules,  bluebird 
feathers  or  butterfly  wing  scales  might  be  exceptions),  any  image 
can  be  thought  of  not  just  as  an  array  of  points  having  different 
intensities,  but  also  as  a  collection  of  “spacings”  and  orientations. 


1  It  is  usually  limited  even  more  severely  by  the  presence  of  aberrations. 
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Scan  lines 


The  geometry  of  the  beam  scannning  the  specimen 


P 


or,  as  seen  after 
re-oversampling, 
then  smoothing 


The  pixels  as  they  appear  in  red  channel  of  the  display  CRT  B 

FIGURE  4.1.  What  Nyquist  sampling  really  means:  the  smallest  feature 
should  be  at  least  4  pixels  wide.  In  (A),  a  yellow  beam  scans  over  two  red  point 
features.  Because  the  “resolution”  of  the  micrograph  is  defined,  by  the 
Abbe/Rayleigh  criterion,  as  the  distance  from  the  center  to  the  edge  of  the  beam 
while  Nyquist  sampling  says  that  pixels  should  be  one  half  this  size,  pixels  are 
one  quarter  of  the  beam  diameter.  From  this,  it  follows  that,  at  the  end  of  each 
line,  the  beam  will  move  down  the  raster  only  25%  of  its  diameter  (i.e.,  it  will 
scan  over  each  feature  at  least  four  times).  In  (B)  we  see  how  such  a  signal 
will  be  displayed  as  a  “blocky”  blob  on  the  screen,  about  4  pixels  in  diameter. 
Because  our  eyes  are  designed  to  concentrate  on  the  edges  of  each  pixel,  the 
screen  representation  doesn’t  look  like  an  Airy  disk  (and  would  look  even 
worse  were  we  to  add  the  normal  amount  of  Poisson  noise).  We  can  get  an 
“accurate”  impression  of  the  image  of  a  point  object  only  if  we  resample  the 
4x4  array  into  a  much  larger  array  and  apportion  the  detected  signal  among 
these  smaller,  less  distinct,  pixels  to  form  an  image  that  looks  like  the  circular 
blob  on  the  lower  right. 


An  image  of  a  preparation  of  circular  nuclei  7  pm  in  diameter  has 
spacings  of  all  possible  orientations  that  are  equal  to  the  diameter 
of  the  nuclei  in  micrometers.  The  inverse  of  this  diameter,  in 
features/pm,  would  be  the  spatial  frequency  of  the  nuclei  (in  this 
case,  about  150/mm). 

The  intensity  of  the  CTF  at  zero  spatial  frequency  is  a  measure 
of  the  average  brightness  of  the  entire  image.  The  CTF  graphs  the 
image  contrast  assuming  that  the  object  itself  has  100%  contrast 
(i.e.,  that  it  is  composed  of  alternating  black  and  white  bars  having 
a  variety  of  different  periodicities;  as  few  biological  specimens 
have  contrast  this  high,  contrast  in  microscope  images  will  be  cor¬ 
respondingly  lower).  Because  of  the  limitations  imposed  by  dif¬ 
fraction,  the  contrast  of  the  widest  bars  (spatial  frequency  near 
zero)  will  be  almost  100%  while  bars  that  are  closer  together  (i.e., 
have  a  spatial  frequency  nearer  the  diffraction  limit)  will  be 
recorded  with  lower  contrast  in  the  image. 


512  x  512  pixel  image 


Digitally  enlarged  view  (8x),  showing 
individual  image  pixels. 


Individual  pixel  size 
is  approximately  0.08  pm 


FIGURE  4.2.  A  properly  sampled  2D  image.  When  your  image  is  recorded 
with  Nyquist-sized  pixels  then  the  smallest  features  will  be  4  to  5  pixels  across. 
(This  figure  kindly  provided  by  Dr.  Alan  Hibbs  of  BioCon,  Melbourne, 
Australia.) 


FIGURE  4.3.  Contrast  transfer  function  (CTF).  This  graph  relates  how  the 
contrast  of  a  feature  in  the  image  is  inversely  related  to  its  size.  Smaller  “spac¬ 
ings”  (see  boxes  below  graph)  have  higher  “spatial  frequency”  and  will  appear 
in  the  image  with  much  lower  contrast  than  they  had  in  the  object.  Although 
the  Rayleigh/ Abbe  resolution  is  conventionally  set  at  the  point  where  the  CTF 
has  dropped  to  25%,  even  features  that  are  twice  this  large  (i.e.,  have  one  half 
the  spatial  frequency,  R/2)  are  still  represented  in  the  image  with  only  about 
half  of  their  original  contrast. 
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From  Figure  4.3,  one  can  see  that  the  Rayleigh-criterion  reso¬ 
lution  is  not  really  a  hard-and-fast  resolution  limit  but  merely  the 
spatial  frequency  at  which  the  CTF  of  the  optical  system  has 
dropped  to  about  25%.  In  general,  features  twice  as  big  as  the 
Rayleigh  limit  (i.e.,  R/2,  half  the  spatial  frequency)  will  be  trans¬ 
mitted  with  a  bit  less  than  twice  this  contrast  (i.e.,  ~50%),  and  so 
on  for  progressively  larger  features  (although  the  image  contrast 
can  never  be  more  than  100%). 

One  of  the  reasons  that  the  CTF  is  such  a  useful  guide  to 
optical  performance  is  that  it  emphasizes  the  performance  for 
imaging  small  features.  If  we  assume  for  a  moment  that  we  are 
using  a  high  numerical  aperture  (NA)  objective  (NA  1.4)  pro¬ 
ducing  a  Rayleigh  resolution  (R,  in  a  microscope,  this  is  often 
called  the  Abbe  limit)  of  -0.25  pm,  then  the  part  of  the  graph  to 
the  left  of  the  R/4  marking  describes  the  way  that  the  optical 
system  will  transmit  all  the  features  larger  than  1.0pm  (or  R/4). 
All  of  the  plot  to  the  right  of  the  R/4  mark  refers  to  its  transmis¬ 
sion  of  features  smaller  than  1.0  pm.  This  is  the  part  of  the  plot 
where  problems  are  likely  to  occur.  In  addition,  it  reminds  us  that 
diffraction  affects  the  appearance  of  features  that  are  larger  than 
the  Abbe  limit.  In  the  end,  resolution  can  only  be  defined  in  terms 
of  contrast.  It  is  NOT  the  case  that  everything  works  perfectly  up 
to  the  Abbe  limit  and  then  nothing  works  at  all. 

The  reason  that  the  CTF  is  particularly  useful  in  microscopy 
is  that,  if  everything  goes  right  (i.e.,  proper  illumination,  optically 
uniform  specimen,  no  lens  aberrations),  its  shape  is  entirely  deter¬ 
mined  by  the  process  of  diffraction.  If  this  is  true,  then  the  curve 
is  directly  analogous  to  what  we  can  see  in  the  back-focal  plane 
(BFP)  of  the  objective  lens.  You  may  recall  that,  when  illuminated 
by  axial  illumination,  large  features  (which  have  low  spatial  fre¬ 
quencies)  diffract  light  near  the  axis  while  smaller  features  diffract 
light  at  larger  angles.  If  you  imagine  that  the  left  axis  of  the  CTF 
plot  (zero  spatial  frequency)  is  located  at  the  exact  center  of  the 
BFP,  then  the  sloping  part  of  the  CFT  curve  can  be  thought  of  as 
representing  a  radial  plot  of  the  light  intensity  passing  through  the 
rest  of  the  BFP.2 

Light  passing  near  the  axis  has  been  diffracted  by  large  fea¬ 
tures.  As  many  diffraction  orders  from  these  features  will  be 
accepted  by  the  NA  of  the  objective,  they  will  be  represented  in 
the  image  with  high  contrast  (Fig.  4.4). 3  Light  out  at  the  edge  of 
the  BFP  consists  of  high-order  diffraction  from  large  features  plus 
low-order  diffraction  from  smaller  features.  The  smallest  features 
visible  at  this  NA  will  diffract  light  at  an  angle  that  is  almost  equal 
to  the  NA  of  the  objective,  as  defined  by  the  outer  border  of  the 
BFP.  As  only  one  diffraction  order  from  these  features  will  be 
accepted  by  the  objective,  the  features  that  diffract  at  this  angle 
will  be  represented  in  the  image  with  low  contrast. 

As  a  result,  one  can  “see”  important  aspects  of  the  CTF,  simply 
by  viewing  the  BFP,  for  example,  using  a  phase  telescope  or 
Bertrand  lens.  For  example,  when  using  a  phase  lens  for  fluores¬ 
cent  imaging,  the  phase  ring  present  in  the  BFP  of  the  objective 
partially  obscures  (50%-90%  opacity)  and  shifts  the  phase  of  any 
rays  passing  through  it.  Therefore,  features  in  the  object  that  are 


2  It  is  uncommon  to  image  using  only  axial  illumination,  at  least  in  part  because 
filling  the  condenser  BFP  increases  the  number  of  diffraction  orders  that  can 
pass  through  the  objective,  thereby  doubling  the  resolution.  It  is  assumed  here 
only  for  illustrative  purposes. 

3  Strictly  speaking,  the  following  analysis  is  only  accurate  for  axial  illumina¬ 
tion.  However,  even  for  the  convergent  illumination  used  to  get  the  highest 
resolution  in  transmission  imaging,  the  general  point  is  correct:  light  rays  car¬ 
rying  information  about  smaller  features  are  more  likely  to  be  represented  by 
rays  that  pass  near  the  edges  of  the  back-focal  plane. 


the  correct  size  to  diffract  at  the  angles  obscured  by  the  ring  will 
be  less  well  represented  in  the  image  data  recorded. 

Finally,  the  CTF  is  useful  because  it  is  universal.  Assuming 
that  you  normalize  the  spatial  frequency  axis  of  the  CTF  plot  in 
Figure  4.3  for  the  NA  and  X  in  use  (i.e.,  the  spatial  frequency  under 
the  25%  contrast  point  on  the  curve  should  be  the  reciprocal  of  the 
Abbe  resolution),  it  is  a  reasonable  approximation  of  the  CTF  of 
any  diffraction-limited  optical  system.  As  such  it  defines  the  best 
we  can  hope  for  in  terms  of  direct  imaging  (i.e.,  without  non-linear 
image  processing  such  as  deconvolution  to  be  discussed  later,  or 
the  use  of  clever  tricks  like  STED  as  discussed  in  Chapter  31,  this 
volume). 

The  CTF  can  be  used  to  characterize  the  performance  of  every 
part  of  the  imaging  system:  not  only  the  optical  system  but  also 
the  image  detector  (film  or  video  camera),  the  image  storage 
system  (film  or  digital  storage),  the  system  used  to  display  or  make 
hardcopy  of  the  stored  result,  even  the  performance  of  your 
eyes/glasses! 

The  performance  of  the  entire  imaging  chain  is  merely  the 
product  of  the  CTF  curves  defining  all  the  individual  processes. 
Because  the  CTF  always  drops  at  higher  spatial  frequencies,  the 
CTF  of  an  image  having  passed  two  processes  will  always  be  lower 
than  that  for  either  process  by  itself  (Fig.  4.5).  In  other  words, 
small  features  that  have  low  contrast  become  even  less  apparent 
as  they  pass  through  each  successive  stage  from  structures  in  the 
object  to  an  image  on  the  retina  of  the  viewer. 

As  can  be  seen  from  Figure  4.5,  the  steps  with  the  lowest  CTF 
are  usually  the  objective  and  the  video  camera.  A  digital  CCD 
camera  (i.e.,  a  CCD  camera  in  which  each  square  pixel  reads  out 
directly  into  a  specific  memory  location)  would  produce  better 
results  than  the  video-rate  television  camera/digitizer  combination 
shown  in  Figure  4.5  because  the  latter  digitizes  the  data  twice,  a 
process  that  can  reduce  the  contrast  of  fine,  vertical  lines  that  are 
sampled  in  the  horizontal  direction  by  a  factor  of  2.  The  perfor¬ 
mance  of  all  of  the  steps  past  the  ocular  can  be  "improved”  by 


FIGURE  4.4.  Relationship  between  the  CTF  and  the  position  in  the  back-focal 
plane  of  the  objective  lens  that  axial  light  will  diffract  from  features  of  dif¬ 
ferent  spatial  frequencies. 
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FIGURE  4.5.  The  CTF  of  each  component  in  the  microscope  system  affects 
the  final  result.  Every  optical  component  and  digital  process  can  be  character¬ 
ized  by  a  CTF.  The  effects  of  a  series  of  steps  can  be  determined  by  multi¬ 
plying  together  the  CTFs  of  all  the  steps. 


working  at  higher  magnification:  if  the  pattern  of  light  presented 
to  the  camera  (or  eye)  contains  larger  features,  their  contrast  will 
be  reduced  less  by  imperfections  in  the  camera  (or  eye)  itself. 
However,  this  approach  also  has  limitations.  Working  at  higher 
magnification  requires  either  a  larger  image  sensor  or  a  smaller 
field  of  view.  Much  of  the  remainder  of  this  chapter  is  concerned 
with  making  the  most  appropriate  choice  of  “magnification,” 
although  the  discussion  is  usually  in  terms  of  “How  large  should 
a  pixel  be,  referred  to  the  object?” 

Once  the  information  is  digitally  encoded,  further  CTF  degra¬ 
dation  can  be  minimized  as  long  as  certain  rules  are  obeyed  (as 
discussed  below  and  in  Chapter  48,  this  volume). 

The  lessons  so  far  are 

•  No  matter  how  high  the  contrast  of  the  optical  process  defin¬ 
ing  a  feature  in  the  object,  smaller  features  are  always  depicted 
in  the  final  image  with  less  contrast  than  larger  features. 

•  Features  that  have  low  intrinsic  contrast  in  the  object  will  have 
even  lower  contrast  in  the  image. 

On  the  other  hand,  remember  that  Figure  4.3  shows  the  best 
for  which  we  can  hope.  It  is  not  at  all  hard  to  end  up  with  system 
performance  that  is  substantially  (-50%)  worse  than  that  described 
by  Figure  4.3.  This  means  that  while  one  can  no  longer  see  the 
smallest  features,  one  now  might  just  as  well  use  larger  pixels. 

In  this  chapter,  we  will  assume  that  Figure  4.3  really  does 
describe  optical  system  performance,  and  go  on  to  consider  the 
other  factors  important  to  ensure  that  image  data  is  digitally 
recorded  in  an  optimal  manner. 

DIGITIZATION  AND  PIXELS 

Image  digitization  refers  to  the  process  whereby  an  apparently 
continuous  analog  image  is  recorded  as  discrete  intensity  values  at 
equally  spaced  locations  on  an  xv-grid  over  the  image  field.  This 
grid  is  called  a  raster. 

Typically  the  image  area  is  divided  into  an  array  of  rows  and 
columns  in  much  the  same  way  as  a  television  image.  In  North  and 
South  America  and  Japan,  the  television  image  is  composed  of  483 
lines  covering  a  rectangular  area  having  proportions  that  are  3  units 
high  by  4  units  wide.  If  each  line  in  such  an  image  is  divided  into 
about  640  equal  picture  elements  or  pixels,  then  each  pixel  will  be 
square  if  you  discard  three  lines  and  record  a  raster  of  640  X  480 
pixels. 

Newer  computer-based  CCD  image  digitization  systems  do  not 
rely  on  any  broadcast  television  standard,  and  are  more  likely  to 


use  rasters  of  512  x  512  or  1024  x  1024  pixels,  although  other 
dimensions  are  not  uncommon.  In  scientific  imaging,  it  is  advis¬ 
able  to  avoid  digitizing  schemes  involving  pixels  that  do  not  rep¬ 
resent  square  subunits  of  the  image  plane  (for  example,  those 
produced  by  digitizing  each  line  from  a  television  image  into  only 
512  pixels  rather  than  640  pixels)  as  there  is  little  support  for  dis¬ 
playing  or  printing  such  images  directly. 

Digitization  of  Images 

The  actual  process  by  which  the  signal  from  the  image  detector  is 
converted  into  the  intensity  values  stored  in  the  computer  memory 
for  each  pixel  depends  on  the  type  of  microscope  involved. 

CCD  cameras:  Typically,  a  widefield  or  disk-scanning  con- 
focal  microscope  uses  a  camera  incorporating  a  CCD  image  sensor. 
Although  we  will  not  describe  in  detail  the  operation  of  these 
sensors  (see  Chapter  12  and  Appendix  3,  this  volume),  the  camera 
operates  by  reading  out  a  voltage  proportional  to  the  number  of 
photons  absorbed  within  a  small  square  area  of  the  sensor  surface 
during  the  exposure  time.  As  long  as  the  intensity  value  readout  is 
stored  directly  into  the  computer,  this  small  area  on  the  CCD 
defines  the  pixel  size  for  the  remainder  of  the  imaging  system.4 

As  far  as  the  user  is  concerned,  the  most  important  parameters 
involved  in  attaching  the  CCD  to  the  microscope  are  the  NA  of  the 
objective,  the  wavelength,  and  the  total  magnification  up  to  the 
surface  of  the  sensor.  Together  these  parameters  determine  both 
the  proper  size  of  a  pixel  referred  to  the  plane  imaged  in  the  spec¬ 
imen,  and  also  the  optimal  pixel  size  for  the  CCD.  For  example, 
if  a  CCD  camera  with  8x8  pm  pixels  is  coupled  to  a  microscope 
with  a  40  x  1.3  NA  objective  via  a  lx  coupling  tube,  each  sensor 
pixel  will  cover  8/40  =  0.2  pm.  The  same  camera  and  coupling  will 
produce  “0.08pm  pixels”  when  used  with  a  lOOx  objective,  but 
the  number  of  photons  striking  each  pixel  during  a  given  exposure 
time  will  now  be  2.5  x  2.5  =  6.25x  less  because  signal  intensity 
goes  down  with  the  square  of  the  magnification. 

Photomultiplier  tubes  (PMTs):  On  a  laser  confocal  micro¬ 
scope,  signal  photons  strike  the  photocathode  of  a  PMT  where 
some  small  fraction  of  them  each  produce  a  single  photoelectron 
(PE).  These  PE  are  then  amplified  about  a  million  times  by  charge 
multiplication.  The  signal  current  emerging  from  the  PMT  is  dig¬ 
itized  under  the  control  of  a  pixel  clock  which  also  controls  how 
the  scanning  mirrors  sweep  over  a  rectangular  raster  on  the  spec¬ 
imen.  This  clock  divides  the  time  taken  to  scan  one  line  into  the 
appropriate  number  of  intervals,  so  that  each  time  interval  repre¬ 
sents  a  square  area  of  the  image  (i.e.,  each  time  interval  represents 
the  same  distance  along  the  scan  line  as  the  spacing  between  adja¬ 
cent  lines).  As  the  PMT  signal  is  digitized  for  each  interval,  or 
pixel,  the  pixel  value  represents  the  signal  intensity  of  a  small 
square  area  of  the  final  image. 

Because  the  shape  of  the  raster  in  a  laser  confocal  microscope 
is  defined  by  the  size  of  the  electronic  signals  sent  to  the  scan 
mirrors  (Fig.  4.6)  rather  than  by  the  fixed  array  of  electrodes  on 
the  surface  of  the  CCD,  there  is  much  more  flexibility  in  terms  of 
the  size  and  shape  of  the  rasters  that  can  be  scanned. 


4  This  is  not  true  if  the  CCD  is  read  out  to  form  an  analog  "composite  video” 
signal  which  is  then  redigitized  into  the  computer.  Such  uncorrelated  redigi¬ 
tization  can  reduce  the  effective  horizontal  resolution  of  the  data  by  almost  a 
factor  of  2  and  should  be  avoided.  Likewise,  one  should  be  careful  when 
“resizing”  images  using  image  processing  programs  because,  unless  it  is  done 
in  integer  multiples,  this  process  also  involves  resampling,  a  process  that 
reduces  image  contrast. 
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FIGURE  4.6.  Mirror  scan  angle  and  magnification.  The  galvanometer 
mirror  scans  the  laser  beam  across  the  focus  plane  of  the  objective  by 
effectively  changing  the  angle  at  which  the  laser  beam  passes  through 
the  back-focal  point  of  the  objective  lens.  A  larger  deflection  of  the 
mirror  scans  the  light  over  a  longer  line  on  the  specimen  (B).  As  the  data 
from  this  longer  line  are  finally  displayed  on  the  same  sized  computer 
monitor,  the  effect  is  to  lower  the  overall  magnification  of  the  image.  If 
the  number  of  pixels  digitized  along  each  line  remains  constant,  a  longer 
line  on  the  specimen  implies  larger  pixels. 


In  particular,  a  combination  of  the  magnification  of  the  objec¬ 
tive  and  the  zoom  magnification  on  the  scan  control  panel  defines 
the  dimensions  of  the  raster  at  the  object  plane  in  the  specimen.  If 
more  current  is  sent  to  the  scanning  mirrors  (low  zoom  magnifi¬ 
cation),  they  will  drive  the  scanning  beam  over  a  larger  area  of  the 
specimen  and,  assuming  a  fixed  raster  size  (e.g.,  512  x  512  pixels), 
this  means  that  each  pixel  will  now  represent  a  larger  area  of  the 
specimen  (Fig.  4.7,  darkest  square).  Conversely,  higher  zoom 
magnification  will  send  smaller  currents  to  the  scan  mirrors.  This 
will  make  the  raster  scan  over  a  smaller  area  on  the  specimen,  and 
make  the  area  represented  by  a  single  pixel  proportionally  smaller 
(Fig.  4.7,  lightest  square).  As  a  result,  and  unlike  the  CCD  case, 
pixel  size  is  under  continuous  control  as  the  user  changes  raster 
shape/size  and  zoom  magnification  settings.  However,  your  control 
panel  should  constantly  display  the  current  pixel  dimensions. 


FIGURE  4.7.  Relationship  between  zoom  setting  and  area  scanned  on  the 
specimen.  A  higher  zoom  magnification  setting  scans  the  beam  over  a  smaller 
area  of  the  sample.  As  each  pixel  now  represents  a  smaller  area  on  the  speci¬ 
men,  we  say  that  the  pixels  are  “smaller.”  The  important  thing  is  to  adjust  the 
zoom  magnification  so  that  the  pixel  size  is  about  50%  of  the  Abbe  resolution 
for  the  NA  and  wavelength  in  use. 


HOW  BIG  SHOULD  A  PIXEL  BE?  SAMPLING 
AND  QUANTUM  NOISE 

Clearly,  it  is  not  possible  to  represent  features  spaced,  say,  1  pm 
apart  if  the  pixel  dimensions  are  2  X  2  pm.  Having  smaller  pixels 
will  increase  the  chance  that  small  features  of  the  specimen  are 
adequately  sampled.  However,  having  smaller  pixels  also  has  dis¬ 
advantages.  It  means  either  imaging  a  smaller  area  of  the  speci¬ 
men  or  using  a  larger  raster  size  [1024  x  1024  rather  than  512  x 
512;  Fig.  4.8(A)].  If  you  choose  a  larger  raster,  you  must  store  and 
analyze  more  data.  You  must  also  either  collect  fewer  signal 
photons  from  each  pixel  [Fig.  4.8(B)]  or  take  longer  to  scan  the 
larger  raster.  Longer  counts  require  you  to  expose  the  specimen  to 
more  light  [Fig.  4.8(C)],  a  process  that  may  be  deleterious,  espe¬ 
cially  to  living  specimens. 

Settling  for  less  signal  in  each  pixel  is  also  not  without  prob¬ 
lems.  The  signal  that  is  being  counted  is  not  continuous  but  is  com¬ 
posed  of  photons,  sometimes  quite  small  numbers  of  photons.  In 
fact,  it  is  not  uncommon  for  the  signal  from  a  single  pixel  in  the 
bright  area  of  a  fluorescent  confocal  specimen  to  represent  the 
detection  of  only  9  to  16  photons. 

As  the  detection  of  a  photon  is  a  quantum  mechanical  event, 
there  is  an  intrinsic  uncertainty  in  the  number  actually  detected  on 
any  given  trial.  This  uncertainty  is  referred  to  as  Poisson,  or  sta¬ 
tistical,  noise  and  is  equal  to  the  square  root  of  the  number  of 
events  (photons)  detected.  Therefore,  reading  16  photons  really 
means  detecting  16  +  4  events.5  Like  diffraction,  Poisson  noise 
is  a  rigid  physical  limitation.  The  only  way  to  reduce  the  rela¬ 
tive  uncertainty  that  it  causes  is  to  count  more  events. 

If  we  increase  the  zoom  magnification  by  a  factor  of  2,  there 
will  be  4x  as  many  pixels  covering  any  given  scanned  area  of  a 
two-dimensional  (2D)  specimen.  If,  at  the  same  time,  we  also 
reduce  the  laser  power  by  a  factor  of  4,  the  same  total  amount  of 
signal/pm2  will  emerge  from  the  reduced  area  now  being  scanned, 
producing  the  same  bleaching  or  other  photo  damage  but  the 
average  signal  level  in  each  bright  pixel  will  now  be  not  16 
photons,  but  only  4  +  2  photons.  The  uncertainty  of  each  mea¬ 
surement  is  now  50%.  In  other  words,  when  photons  are  scarce, 
one  seldom  wants  to  use  pixels  smaller  than  are  absolutely  neces¬ 
sary  to  record  the  information  in  the  image. 

It  is  simply  a  case  of  “winning  on  the  swings  what  you  lose 
on  the  roundabouts.”  Either  scenario  has  advantages  and  dis- 


5  That  is,  67%  of  a  series  of  measurements  of  this  intensity  would  be  in  the 
range  of  12  to  20  photons  and  33%  of  such  measurements  will  be  outside 
even  this  range.  In  other  words,  if  you  detect  10  photons  you  really  have  very 
little  idea  about  what  the  signal  intensity  really  “should  have  been.” 
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FIGURE  4.8.  Relationship  between  raster  size,  pixel  size,  and 
“light  dose.”  (A)  If  a  microscope  scanning  at  a  zoom  setting  of 
1  is  switched  from  a  512  x  512  raster  to  one  of  1024  x  1024 
pixels,  the  dimensions  of  each  pixel  on  the  specimen  will  drop 
by  50%.  (B)  If  the  same  amount  of  signal  is  split  between  more 
pixels,  the  signal  level  from  each  one  goes  down  (and  the 
Poisson  noise  goes  up),  but  if  the  beam  scans  more  slowly  or  is 
made  more  intense  so  that  the  same  amount  of  signal  is  still  col¬ 
lected  from  each  pixel,  (C),  then  the  amount  of  damage/pixel 
increases.  There  is  no  free  lunch! 
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advantages.  Surely  there  must  be  a  “best”  strategy  for  setting  the 
zoom  correctly  to  produce  the  best  pixel  size. 

Fortunately  there  is! 

THE  NYQUIST  CRITERION 

It  was  not  until  1929  that  Harry  Nyquist,  who  worked  for  the 
telegraph  company,  gave  much  thought  to  the  optimal  strategy  for 
digitally  sampling  an  analog  signal  (Nyquist,  1928).  When  such 
sampling  first  became  technically  possible,  the  signal  in  question 
was  electronic,  perhaps  the  audio  signal  of  a  radio  program.  The 
process  envisioned,  as  diagramed  in  Figure  4.9,  requires  six  com¬ 
ponents:  a  pre-amplifier  feeding  the  signal  to  the  analog- to- 
digital  converter  (ADC),  a  digital  memory  system  for  storing  the 
digital  data  from  the  ADC,  a  digital-to-analog  converter  (DAC) 
that  reassembles  the  digital  information  into  a  continuous  analog 
signal  that  can  be  passed  to  the  output  amplifier,  and,  finally,  a 
clock  to  synchronize  the  whole  process.  The  clock  determines  the 
time  interval  between  samples  (i.e.,  the  sampling  frequency,  in 
samples/s). 

The  information  content  of  any  electronic  signal  is  limited  by 
the  electronic  bandwidth  of  the  amplifier  used  to  transmit  it.6  In 


6  Think  of  this  as  the  frequency  response  of  your  stereo  system.  Good  high  fre¬ 
quency  response  will  let  you  hear  your  music  more  accurately.  The  frequency 
response  of  your  stereo  is  usually  plotted  in  decibels  (a  measure  of  relative 
power)  on  the  y-axis  against  the  log  of  the  frequency  on  the  x-axis.  Note  the 
similarities  to  Figure  4. 1 . 


1949,  Claude  Shannon  was  able  to  prove  Nyquist’s  theorem  and 
show  that,  if  the  interval  between  the  intensity  measurements  is 
less  than  half  the  period  of  the  highest  frequency  in  the  signal,  it 
will  then  be  possible  to  faithfully  reconstruct  the  original  signal 
from  the  digital  values  recorded  (Shannon,  1949).  The  Shannon 
sampling  frequency,  which  is  the  inverse  of  the  Shannon  sam¬ 
pling  interval,  is  also  known  as  the  Nyquist  frequency,  especially 
in  the  imaging  community. 

It  is  often  forgotten  that  there  is  a  second  part  of  the 
Shannon/Nyquist  theorem:  the  part  about  reconstructing  the  orig¬ 
inal  data.  The  theorem  states  that  the  output  “amplifier”  through 
which  you  play  back  the  reconstructed  signal  from  the  DAC  must 
have  the  same  bandwidth  as  the  pre-amplifier  that  originally  fed 
the  signal  to  the  ADC.  This  is  an  important  condition,  one  that  is 
often  not  satisfied  in  current  confocal  microscopes  unless  their 
images  are  deconvolved  before  presentation  (as  will  be  discussed 
later). 

Attempting  to  apply  Nyquist  sampling  to  2D  or  three- 
dimensional  (3D)  images  gives  rise  to  the  question:  How  do  we 
measure  the  “bandwidth”  of  the  “amplifiers”  when  faced  with  the 
problem  of  digitizing  2D  or  3D  microscopical  image  data? 

Electronic  bandwidth  is  not  a  simple  concept.  The  frequency 
response  of  any  real  amplifier  does  not  remain  flat  until  some  fre¬ 
quency  and  then  go  abruptly  to  zero  at  any  higher  frequency. 
Rather,  limitations  imposed  by  the  components  of  which  the  circuit 
is  made  cause  its  power  response  to  decrease  gradually  as  the  fre¬ 
quency  increases,  usually  dropping  to  one  half  or  one  quarter  the 
original  output  power  as  the  frequency  goes  up  each  octave  above 
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FIGURE  4.9.  The  components  needed  to  digitize 
and  reconstruct  an  analog  signal.  The  “post-amp”  is 
essential  to  remove  the  “single-pixel  noise”  that  is 
added  to  the  original  analog  signal  by  Poisson  sta¬ 
tistics.  Because  real,  Nyquist-sampled  data  can 
never  have  features  smaller  than  4  pixels  across, 
single-pixel  noise  can  be  removed  by  limiting  the 
bandwidth  of  the  post-amplifier.  In  microscopy,  this 
limiting  function  is  implemented  by  either  Gauss¬ 
ian  filtering  or  deconvolving  the  raw  3D  data. 


Original  signal  Digitized  signal  Reconstructed  signal 


Original  NOISY  signal  Digitized  signal  .  Reconstructed  signal 

^vvb 


some  “cut-off  frequency.”7  As  in  optical  systems,  higher  electronic 
frequencies  are  still  transmitted,  but  at  lower  intensity.  In  elec¬ 
tronics,  the  convention  is  to  define  the  bandwidth  by  the  frequency 
at  which  the  power  response  drops  to  50%  of  the  linear  response, 
a  frequency  called  the  “3  dB  point.”  This  defines  the  bandwidth 
Shannon  used.  In  optical  terms,  we  usually  think  of  the  image 
being  useful  until  it  drops  to  about  25%  of  its  peak  contrast  (i.e., 
the  Abbe  criterion  noted  above),  although  this  too  is  an  arbitrary 
choice. 

If  we  think  of  an  analog  electronic  signal  as  a  one-dimensional 
image,  it  is  not  hard  to  think  of  an  image  as  a  2D  (or  3D)  version. 
Except  that  image  data  varies  in  space  rather  than  time,  the  rest 
of  the  analysis  applies.  The  “bandwidth”  of  an  image  must  be 
somehow  related  to  its  “sharpness,”  and  this  is  related  to  the 
highest  spatial  frequencies  it  contains. 

Now  if  we  were  applying  this  analysis  to  the  CCD  sensor 
used  in  a  consumer  snapshot  camera,  we  would  have  a  problem. 
Although  the  “world”  out  there  may  be  composed  of  objects  of 
every  size,  we  really  have  little  knowledge  of  the  CTF  of  the  lens, 
let  alone  whether  or  not  it  is  focused  correctly  or  whether  you  are 
capable  of  holding  it  motionless  during  the  exposure  period.  As  a 
result,  we  really  don’t  know  the  bandwidth  of  the  data  and  conse¬ 
quently  we  don't  know  whether  or  not  the  pixels  are  small  enough 
to  meet  the  Nyquist  criterion.  “More  is  better”  is  the  slogan  that 
sells. 

Fortunately,  this  is  not  the  case  in  microscopy.  Here  we  do 
know  that,  at  the  very  least,  diffraction  limits  the  maximum  sharp¬ 
ness  of  the  data  that  can  be  recorded,  and  that  the  “spatial  fre¬ 
quency  response”  of  the  microscope  can  be  defined  by  a  suitably 
calibrated  version  of  Figure  4.3. 

Therefore,  the  convention  is  to  choose  the  size  of  the  pixel 
to  be  equal  to  one  half  of  the  Abbe  criterion  resolution  of  the 
optical  system. 

There  are  some  caveats.  The  structural  features  of  a  ID  image 
can  only  vary  in  that  dimension.  The  structural  features  of  a  2D 
image  can  vary  in  more  than  two  possible  directions.  Although 
signals  defining  features  such  as  vertical  or  horizontal  lines,  vary 


7  As  in  music,  an  octave  represents  a  factor  of  2  in  signal  frequency. 


in  only  the  x-  or  v-directions,  respectively,  what  about  a  set  of  lines 
oriented  at  45°  to  these  axes?  It  would  seem  that  sampling  points 
along  a  45°  line  would  be  spaced  1.4 lx  as  far  apart  as  sampling 
points  along  features  that  vary  along  the  x-  or  y-axes.  Pixels  just 
small  enough  to  sample  a  given  small  spacing  when  it  is  oriented 
vertically  or  horizontally  would  be  1.41x  too  big  to  sample  this 
same  structure  were  it  to  be  oriented  at  45°.  However,  this  analy¬ 
sis  neglects  the  fact  that  all  image  “features”  extend  in  2D.  As  a 
result,  lines  running  at  45°  will  also  be  sampled  by  other  pixels  in 
the  array  and  if  we  count  all  the  pixels  that  sample  the  blurred 
features  along  a  line  at  45°,  one  finds  that  the  sampling  interval 
isn’t  1.41x  larger  but  in  fact  only  0.707  as  large  as  the  sampling 
interval  in  thex-  or  y-directions  (Fig.  4.10).  Clearly  we  want  to  be 
able  to  see  structures  oriented  in  any  direction.  To  be  on  the  safe 
side,  it  may  be  better  to  use  pixels  ~2.8x  smaller  than  the  finest 
spacing  you  expect  to  record  in  your  image.8 

Estimating  the  Expected  Resolution  of  an  Image 

Assuming  that  the  shape  of  the  CTF  curve  describing  the  optics  of 
the  microscope  depends  only  on  the  NA  and  the  wavelength,  it  is 
also  a  plot  of  power  level  versus  the  logarithm  of  the  frequency, 
just  like  the  frequency  response  curve  of  a  stereo.  Although  the  CTF 
defines  the  best  that  one  can  hope  for,  it  does  not  guarantee  it. 

Performance  can  be  worse,  and  if,  in  fact,  it  is  worse,  does  it 
make  sense  to  use  smaller  pixels  than  we  need? 

Let  us  take  some  concrete  examples.  The  calculation  of  the 
Abbe  criterion  resolution  assumes  that  two  point  objects  of 
similar  intensity  are  represented  in  the  image  as  Airy  disks, 
spaced  so  that  the  peak  of  each  is  located  over  the  first  dark  ring 
of  the  other.  If  we  sum  the  light  intensity  of  these  two  Airy  disks, 
there  will  be  a  valley  between  the  two  peaks  in  the  summed  image. 
At  the  exact  mathematical  bottom  of  this  valley,  the  intensity  is 


8  A  similar  line  of  argument  could  be  used  to  suggest  that  one  use  even  smaller 
pixels  when  sampling  3D  data  because  the  diagonal  of  a  cube  is  1 ,732x  longer 
than  its  side.  However,  we  will  soon  see  that,  as  the  ^-resolution  of  the  con- 
focal  microscope  is  always  at  least  3x  lower  than  the  xv-resolution,  ignoring 
this  factor  does  not  cause  any  problem  in  practice. 
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FIGURE  4.10.  Spatial  frequency  and  geometry.  The  3x3  array  of  squares 
represents  a  small  raster  and  the  dots  in  the  center  of  each  represent  the  sam¬ 
pling  points.  Although  one  might  be  tempted  to  think  that  these  sampling  points 
would  be  too  far  apart  along  the  diagonal  to  be  able  to  properly  sample  any 
signal  that  just  meets  the  Nyquist  sampling  criterion  when  oriented  either  hori¬ 
zontally  or  vertically,  this  is  not  so  because  the  sampling  points  of  the  adjacent 
diagonal  rows  of  pixels  actually  sample  at  0.71  of  the  x-  or  y-raster  pitch. 


about  25%  lower  than  the  intensity  of  either  peak.  This  is  the  basis 
of  the  idea  that  25%  contrast  is  equal  to  the  Abbe  criterion  reso¬ 
lution  (Fig.  4.11). 

Under  these  circumstances,  the  smallest  resolvable  spacing  is 
defined  as  the  distance  between  the  center  of  an  Airy  disk  and  the 
center  of  its  first  dark  ring.  To  be  properly  sampled,  pixels  should 
be  less  than  one  half  this  distance  in  size.9 

Suppose  that,  along  a  line  joining  centers  of  the  images  of  the 
two  points,  one  pixel  just  happens  to  be  centered  on  the  brightest 
part  of  one  Airy  disk.  The  adjacent  pixel  would  then  be  centered 
over  the  valley  between  the  peaks  and  the  third  pixel  will  be  over 
the  second  Airy  peak.  If  we  sample  the  brightness  at  the  center  of 
these  three  pixels,  the  digital  data  will  reflect  the  trough  in  inten¬ 
sity  between  them. 

On  the  other  hand,  if  the  “valley  pixel"  has  a  value  propor¬ 
tional  not  to  the  intensity  at  the  exact  center  of  the  pixel  but  to  the 
average  intensity  over  the  whole  pixel.10  the  value  stored  for  the 
center  pixel  will  be  much  more  than  75%  of  the  peak  intensity: 


9  Or  perhaps  a  bit  less  if  we  use  the  2.3  or  2.8  samples/resolvable  element 
(resel)  suggested  above.  For  simplicity,  I  will  stick  to  2  samples/resel  in  this 
discussion,  because,  as  discussed  below,  in  the  case  of  the  fluorescent  images 
of  most  interest,  lack  of  signal  usually  prevents  one  from  realizing  Abbe  cri¬ 
terion  resolution  and  consequently  the  “actual”  resolution  is  lower  than  Abbe 
and  using  somewhat  fewer/larger  pixels  is  appropriate. 

10  In  microscopy  terms,  the  CCD  samples  the  average  value  of  a  pixel  while 

the  ADC  sampling  the  PMT  signal  in  most  single-beam  confocals  acts  more 
as  the  center-sampling  device. 


FIGURE  4.11.  Nyquist  sampling  of  an  image  of  two  points  separated  by  the 
Rayleigh  resolution. 


that  is,  the  contrast  recorded  between  the  three  pixels  will  now  be 
much  lower  than  25%  (Fig.  4.12). 

If  the  two  features  that  produced  the  two  Airy  disk  images  are 
not  of  equal  brightness  (surely  the  more  likely  occurrence)  then 
the  contrast  along  a  line  joining  the  peaks  will  again  be  much  less 
than  25%. 

Worse  still,  what  if  the  peaks  are  uncooperative  and  are  not 
squarely  centered  on  two  pixels,  nicely  spaced  on  either  side  of  the 
central,  darker  pixel?  If  the  value  recorded  at  each  pixel  is  the 
average  of  the  intensity  across  the  pixel,  the  contrast  along  a  line 
between  the  features  can  be  substantially  reduced  or  even 
eliminated  (Fig.  4.13). 

Now  it  is  fair  to  say  that  while  these  considerations  are  prob¬ 
lems,  to  some  extent,  they  only  represent  a  serious  problem  if  we 
ignore  the  second  part  of  the  Nyquist  sampling  theorem,  the  part 
having  to  do  with  reconstruction.  If  the  image  is  properly  recon¬ 
structed  (deconvolved),  in  most  cases,  information  from  adjoining 
pixels  (those  in  the  rows  in  front  or  behind  the  printed  page  in  Fig. 
4.13)  will  allow  one  to  smooth  the  image  to  form  a  good  estimate 
of  the  structure  of  the  original  object  as  is  discussed  later  in  the 
chapter.11 

Deconvolving  or  filtering  the  image  data  eliminates  high 
spatial  frequencies.  Effectively,  such  filtering  causes  the  signal  to 
overshoot  the  contrast  present  in  the  digital  signal.  This  process 
substantially  reverses  the  apparent  reduction  in  contrast  that  occurs 
on  digitization. 


11  Periodic  structures  having  a  size  near  the  resolution  limit  also  present  sam¬ 
pling  problems.  Suppose  that  the  object  is  a  sinusoidally  varying  structure 
with  a  period  equal  to  the  Abbe  spacing.  If  the  two  samples  required  by 
Nyquist  coincide  with  the  plus  and  minus  excursions  of  the  sine  wave,  then 
we  will  have  some  measure  of  its  magnitude  and  the  position  of  its  zero¬ 
crossing  [Fig.  4.5(B)],  However,  if  the  two  samples  happen  to  be  taken  as 
the  sine  wave  crosses  its  origin,  all  the  sampled  values  will  be  zero  and  hence 
can  contain  no  information  about  the  sine  wave  [Fig.  4.5(C)].  This  apparent 
exception  to  Nyquist  sampling  success  is  not  actually  an  exception  in  terms 
of  the  original  application  of  the  theorem:  information  theory.  According  to 
information  theory,  a  sine  wave  contains  no  information  beyond  its  frequency 
and  magnitude.  As  long  as  you  have  slightly  more  than  two  samples/period, 
the  sampling  intervals  will  “beat”  with  the  data  to  create  a  sort  of  moire 
effect,  from  which  one  can  estimate  the  magnitude  and  period  of  the  sine- 
wave  object.  All  this  does  not  change  the  fact  that  an  image  of  a  periodic 
object  must  be  at  least  2x  over-sampled  if  it  is  to  be  recorded  with  reason¬ 
able  fidelity  [Fig.  4.5(A)],  This  is  particularly  important  when  imaging  the 
regular  patterns  found  on  resolution  test  targets. 
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FIGURE  4. 12.  Two  methods  of  sampling:  at  the 
center  point  and  as  the  average  value. 


Nyquist  digitizing  of  two  point  objects  at  Abbe  separation. 


Average  sample 


FIGURE  4.13.  How  can  this  possibly  work?  "Lucky” 
and  "unlucky"  Nyquist  sampling  of  the  image  of  two 
points  separated  by  one  Rayleigh  resolution. 


Nyquist  digitizing  of  two  point  objects  at  Abbe  separation  with  center  sampling. 


On  the  other  hand,  this  reduction  in  contrast  is  entirely  rele¬ 
vant  if  one  tries  to  assess  raw  digital  image  data  from  a  Nyquist- 
sampled  confocal  microscope  directly  from  the  cathode -ray  tube 
(CRT)  or  liquid-crystal  display  (LCD)  screen  or  when  viewing 
unprocessed  data  as  a  hardcopy  from  a  digital  printer. 

There  is  another  problem  that  even  proper  reconstruction  will 
not  solve.  Recall  the  example  above  in  which  a  “bright”  signal 
(perhaps  the  peak  of  an  Airy  disk?)  was  only  16  ±  4  photons. 
Clearly  the  ±4  represents  a  25%  average  error,  that  is  the  same 
order  of  uncertainty  as  the  maximum  expected  contrast  we  hope 
to  see  between  the  peaks  (Fig.  4.14).  In  other  words,  even  though 
diffraction  theory  says  that  we  should  record  a  lower  signal  in  the 


pixel  between  two  peaks  of  equal  intensity,  at  these  low  signal 
levels,  Poisson  statistics  says  that,  about  30%  of  the  time,  the  inter¬ 
vening  pixel  will  actually  be  measured  as  brighter  than  at  least  one 
of  the  two  peaks.  [As  each  peak  pixel  is  subject  to  its  own  in¬ 
dependent  statistical  variations,  in  a  given  image,  it  is  unlikely  that 
all  3  pixels  (or  9  pixels  if  we  consider  the  2D  image)  will  be 
recorded  as  the  same  brightness.] 

Artifactual  “features”  such  as  those  diagramed  in  Figure 
4.14(B)  and  produced  by  “single-pixel”  Poisson  noise,  will  be 
removed  if  the  dataset  is  deconvolved  or  even  3D-Gaussian 
smoothed  as  discussed  below. 


FIGURE  4.14.  The  effect  of  Poisson  noise.  While 
a  Nyquist-sampled  signal  of  Rayleigh-separated 
features  seems  to  work  well  when  the  signal  is  com¬ 
posed  of  many  photons  and  has  little  noise  (A), 
when  the  number  of  photons  counted  drops  by  a 
factor  of  100,  and  the  signal-to-noise  ratio  (S/N) 
drops  by  a  factor  of  10,  then  random  variations  in 
the  signal  can  play  havoc  with  the  data  (B)  allow¬ 
ing  “single-pixel”  noise  features  to  masquerade  as 
very  small  features. 
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The  Story  So  Far 

Once  we  know  the  size  of  the  smallest  data  we  hope  to  record,  we 
can  adjust  the  zoom  magnification  on  a  confocal  microscope  or  the 
CCD  camera  coupling  tube  magnification  on  a  widefield  micro¬ 
scope  to  make  the  pixels  the  right  size. 

But  is  Figure  4.3  really  a  good  way  to  estimate  this  maximum 
spatial  frequency? 

REALITY  CHECK? 

Are  we  kidding  ourselves  in  thinking  we  will  be  able  to  see  indi¬ 
vidual  point  features  separated  by  Abbe  criterion  resolution 

when  viewing  faint,  fluorescent  specimens?  In  fact,  under  these 
conditions,  we  may  be  lucky  to  separate  features  that  are  even 
twice  this  far  apart  and  we  now  recognize  that  we  could  record 
such  data  using  pixels  that  were  twice  as  big  and  4x  less  numer¬ 
ous  (in  a  2D  image;  8x  fewer  in  a  3D  image). 

On  the  other  hand,  our  human  ability  to  “see”  (recognize?) 
extended  features,  such  as  fibers  or  membranes,  is  enhanced  by  the 
ability  of  our  mind  to  extract  structural  information  from  noisy 
data.  We  do  this  “magic”  by  integrating  our  visual  analysis  over 
many  more  pixels  (100x?).  While  viewing  noisy,  extended  objects 
doesn’t  improve  the  quality  of  the  data,  it  allows  the  mind  the  illu¬ 
sion  of  averaging  out  the  statistical  noise  over  more  pixels  because 
each  is  an  independent  measurement.  In  this  case,  Nyquist/Abbe 
sampling  may  be  more  worthwhile  after  all. 

Is  Over-Sampling  Ever  Wise? 

Yes!  When  viewing  a  specimen  that  is  not  damaged  by  interacting 
with  light,  over-sampling  can  improve  visibility  by  recording  more 
data  and  hence  reducing  the  effect  of  Poisson  noise.  Video- 
enhanced  contrast  microscopy  has  been  utilized  to  image  isolated 
features  much  smaller  than  the  Abbe  limit.  When  imaging  struc¬ 
tures  such  as  isolated  microtubules,  one  often  employs  “empty 
magnification,”  sampling  much  more  finely  than  is  required  by 
Nyquist.  This  is  effective  because  such  structures  produce  only  a 
very  small  amount  of  image  contrast. 

As  a  simplified  example,  assume  that  the  signal  from  the 
feature  is  only  1%  greater  than  that  from  the  gray  background. 
Turning  the  light  signal  into  an  electronic  signal  permits  one  to 
adjust  the  contrast  arbitrarily.  However,  if  the  electronic  signal  is 
too  noisy,  the  result  will  just  be  more  contrasty  noise. 

To  detect  a  1%  difference  using  photons,  we  must  ensure 
that  the  contrast  produced  by  Poisson  noise  variations  in  the 
background  gray  are  less  than  that  between  the  background 
and  the  feature.  At  the  minimum,  this  involves  counting  at  least 
10,000  photons/pixel  because  the  Poisson  noise  is  y  1 0,000  and 
100/10,000  =  1%.  One  could  produce  an  even  more  easily 
interpretable  image  if  the  intensity  of  the  feature  differs  from  the 
background  by  more  than  one  standard  deviation.  Recording 
100,000  photons/pixel  would  make  the  1%  signal  become  3x  more 
than  the  Poisson  noise. 

As  most  image  sensors  saturate  (become  non-linear)  when 
exposed  to  more  than  100,000  photons/pixel,  the  only  way  to  “see” 
such  a  low  contrast  feature  is  to  make  many  different  measure¬ 
ments  (i.e.,  use  more  pixels).  A  single  pixel  might  be  bright 
because  of  statistics  but  it  is  less  likely  that  four  adjacent  pixels 
will  all  be  recorded  as  bright.  Using  more  pixels  produces  even 
greater  visibility  by  further  separating  the  signal  representing  the 
feature  from  that  representing  the  background.12 


Under-Sampling? 

In  some  cases,  the  useful  resolution  of  the  image  is  set  by  non- 
optical  limits.  An  example  might  be  a  fluorescence  image  of  a  cell 
containing  a  dye  that  changes  its  properties  in  response  to  the 
concentration  of  certain  ions.  If  the  diffusion  of  ions  and  dye 
molecules  precludes  the  existence  of  small-scale  variations  in  the 
fluorescence  signal  from  such  a  cell  (i.e.,  no  small  features),  there 
is  no  need  to  divide  the  data  into  small  pixels.  Measuring  each  of 
fewer,  larger  pixels  for  a  longer  time  may  give  more  accurate 
results,  especially  when  the  expected  changes  in  ion  concentration 
produce  only  small  changes  in  the  fluorescent  properties  of  the  dye 
used  (i.e.,  a  low-contrast  image)  or  when  two  noisy  images  must 
be  ratioed  to  obtain  the  final  result. 

In  such  specimens,  high  spatial  resolution  is  impossible 
because  of  diffusion,  while  high  intensity  resolution  is  required  to 
make  small  changes  visible.  In  this  case,  it  is  particularly  impor¬ 
tant  to  spatially  filter  the  raw  digital  data  before  attempting  to 
display  or  ratio  the  data  (see  Chapter  42,  this  volume). 

DIGITIZING  TRADE-OFFS 

We  have  now  discussed  how  most  of  the  relevant  factors:  pixel 
size,  optical  resolution,  and  photon  signal  strength  all  interact.  The 
best  choice  will  almost  always  depend  primarily  on  the  robustness 
of  your  sample:  Assuming  careful  adjustment  of  the  optics,  more 
counted  photons  will  always  give  a  better  estimate  of  the  distri¬ 
bution  of  fluorescent  molecules  within  the  specimen. 

You  must  decide  when  the  need  for  better  spatial  or  intensity 
resolution  justifies  increasing  the  signal  level  and  when  it  cannot 
be  tolerated  because  to  do  so  would  reduce  the  “biological  reli¬ 
ability”  of  the  data  (i.e.,  kill  or  injure  the  cells,  see  Chapters  38 
and  39,  this  volume).  Data  with  higher  spatial  resolution  may  not 
be  useful  if  they  represent  structural  features  of  a  cell  that  is  dead 
or  dying. 

NYQUIST  RECONSTRUCTION: 
"DECONVOLUTION  LITE" 

Elsewhere  in  this  volume  the  technique  for  recording  3D  data  sets 
of  both  point  objects  and  fluorescent  specimens  using  a  widefield 
microscope  and  a  CCD  camera  and  then  computer-processing  the 
resulting  data  to  produce  3D  images  much  like  those  produced  by 
the  confocal  microscope  are  discussed  in  detail  (Chapters  23,  24, 
and  25).  The  most  advanced  form  of  this  processing  is  called  iter¬ 
ative,  constrained  3D  deconvolution  and  uses  the  image  of  the 
point  object  to  determine  the  3D  point-spread  function  (PSF)  for 
the  imaging  system.  Here,  I  will  discuss  only  one  part  of  this 
process,  a  process  that  can  be  thought  of  as  filtering  or  smoothing. 


12  It  is  important  to  state  here  that  I  am  not  talking  about  limitations  in  the  image 
that  could  be  overcome  by  resetting  the  "contrast”  and  “brightness”  of  the 
image  display  system  in  order  to  make  any  image  contrast  more  visible  to 
the  observer.  These  are  assumed  to  be  set  in  the  best  possible  manner  for  the 
individual  concerned.  The  limitation  on  visibility  discussed  here  relates 
solely  to  the  fact  that  the  data  in  the  recorded  image  is  insufficiently  precise 
for  any  observer  (or  even  a  computer!)  to  determine  the  presence  or  absence 
of  the  structure.  For  more  about  visibility  and  the  Rose  criterion,  see  Chap¬ 
ters  2  and  8,  this  volume. 
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As  noted  above,  sampling  the  analog  data  to  produce  the 
digital  record  was  only  half  of  the  process.  The  second  part 
involves  “passing  the  reconstructed  signal  through  an  amplifier 
having  the  same  bandwidth  as  that  from  which  the  original  data 
was  received.” 

To  see  why  this  is  necessary  it  may  help  if  we  imagine  a  recon¬ 
struction  of  the  digital  data  as  being  sort  of  a  bar  graph,  in  which 
each  bar  represents  the  intensity  value  stored  for  this  pixel  [Fig. 
4.15(A)].  Clearly  a  “signal”  represented  by  the  boxy  contour  line 
going  along  the  tops  of  the  bars  will  generally  change  much  more 
abruptly  than  the  original  data.  As  a  result,  it  is  not  a  faithful  recon¬ 
struction  of  the  original  signal. 

How  can  it  be  made  more  similar?  In  terms  of  Fourier  optics 
a  “square-wave  object,”  such  as  a  bar,  can  be  thought  of  as  being 
composed  of  the  sum  of  a  number  of  sine-wave  objects,  each 
having  a  periodicity  that  is  an  integer-multiple  (harmonic)  of  the 
square  wave  frequency.  The  first  sine  term  in  this  series  converts 
each  “square”  of  the  square  wave  into  a  rounded  curve.  As  subse¬ 
quent  terms  are  added,  they  add  the  “ears”  to  the  hump  that  make 
the  sum  resemble  the  original  boxy  square  wave  ever  more  accu¬ 
rately  (Fig.  4.16). 

If  we  apply  this  logic  to  the  top  line  of  our  bar  graph,  we  can 
think  of  it  as  the  sum  of  a  lot  of  sine  waves.  If  we  leave  out  the 
higher  harmonic  terms  before  reconstructing  the  original  line,  the 
boxy  corners  will  be  rounded.  Passing  the  boxy  reconstruction 
through  an  amplifier  of  limited  bandwidth  prevents  the  higher 
order  terms  (higher  frequencies)  in  the  sine-wave  series  from  being 
included  in  the  reconstructed  signal  [Fig.  4.15(C)]. 

This  is  important  when  viewing  a  digital  image  because  our 
eye/brain  system  is  designed  to  emphasize  the  sharp  edges  that 
define  the  boundary  of  each  pixel  on  the  liquid-crystal  display 
(LCD)  screen  and  this  is  more  likely  to  happen  when  a  single  noisy 
pixel  stands  out  from  a  darker  background. 

The  same  thing  is  true  when  we  reconstruct  an  image  from 
digital  data.  However,  in  the  case  of  fluorescence  or  other  low- 
intensity  data,  there  is  an  additional  complication.  The  Nyquist 
theorem  assumes  that  the  signal  digitized  is  continuous,  that  is,  that 


FIGURE  4.1 5.  How  “limiting  the  bandwidth  of  the  output  amplifier”  smooths 
off  the  rough  comers  (B)  and  improves  the  reconstruction  (C). 


original 

\  order  1 
and  3 

wave 

FIGURE  4.16.  Fourier  analysis.  The  Fourier  theorem  says  that  any  periodic 
structure  (such  as  the  "square-wave”  in  the  top  left)  can  be  represented  as  the 
sum  of  a  number  of  sine  waves,  each  of  which  is  harmonic  of  the  frequency 
of  the  stmcture.  Think  of  these  frequencies  as  the  spatial  frequencies  introduced 
in  Figure  4.2.  As  more  components  are  added  to  the  sum,  the  result  looks  more 
and  more  like  the  original.  The  same  thing  happens  in  microscopy,  where  using 
a  lens  with  a  higher  NA  allows  more  “terms”  that  are  carrying  high-frequency 
information  (and  therefore  diffract  at  higher  angles)  to  contribute  to  the  image. 


determining  the  intensity  to  be  stored  for  each  pixel  does  not 
involve  measuring  small  numbers  of  quantum-mechanical  events. 
A  continuous  signal  is  not  capable  of  changing  by  a  large  amount 
from  one  pixel  to  the  next  because  the  pre-amplifier  bandwidth  was 
too  narrow  to  permit  such  a  rapid  change. 

In  the  microscopic  case,  the  Abbe  bandwidth  limits  the  amount 
of  change  possible  between  adjacent  Nyquist  pixels.  However,  in 
the  confocal,  Poisson  noise  can  effectively  sneak  past  the  “pre¬ 
amp”13  and  get  digitized  as  part  of  the  signal.  As  a  result,  such 
abrupt  changes  can  be  recorded. 

Consider  the  following  example:  Suppose  we  record  the  image 
of  a  bright  point  object  on  a  black  background  using  Nyquist  sam¬ 
pling.  A  one-dimensional  (ID)  transect  across  the  center  of  this 
feature  might  include  5  pixels.  If  sampled  many  times,  the  average 
recorded  intensities  in  the  central  pixels  might  represent  10 
photons,  8  in  the  pixels  on  either  sides,  and  3  for  the  two  pixels 
next  farther  out. 

Had  we  recorded  these  averaged  values,  we  would  only  have 
to  worry  about  the  “boxy  corners”  artifact  noted  above.  However, 
if  we  only  record  a  single  set  of  values,  Poisson  noise  introduces 
another  factor.  On  any  particular  sampling  of  this  line  of  data,  we 
will  generally  not  get  the  average  values  but  something  else.  Were 
we  to  record  not  3,  8,  10,  8,  3  but  2,  7,  13,  10,  4,  the  resulting 
reconstruction  would  now  be  very  different.  In  particular,  the 
center  of  the  feature  would  have  moved  right  and  it  would  now 
appear  narrower.  The  transients  caused  by  the  statistical  nature  of 
the  signal  have  made  a  proper  reconstruction  more  difficult. 

In  fact,  one  would  be  correct  in  saying  that,  as  the  accuracy  of 
the  values  stored  in  the  computer  are  always  limited  by  the  statis¬ 
tics  involved  in  counting  quantum  mechanical  events,  we  can 
never  know  their  “true”  intensity  of  any  pixel  and  our  efforts 


13  This  is  possible  because,  in  this  case,  it  is  the  microscope  optics  that  limits 
the  bandwidth  rather  than  an  electronic  pre-amplifier. 
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to  make  a  reconstruction  of  the  object  are  doomed  to  be  only 
approximate. 

While  this  dismal  analysis  is  correct,  we  would  like  at  least  to 
make  this  approximation  as  accurate  as  possible.  We  can  do  this 
by  applying  the  second  Nyquist  constraint:  treating  the  data  stored 
in  the  image  memory  to  make  sure  that  they  do  not  contain  spatial 
frequencies  that  are  higher  than  the  optical  system  could  have 
transmitted.  Although  the  best  way  to  do  this  is  to  subject  the  3D 
data  set  to  iterative  3D  deconvolution,  much  benefit  can  be  gained 
by  applying  a  simple  2D  or  3D  Gaussian  smoothing  filter.  The 
effect  of  such  a  filter  is  to  make  the  intensity  of  every  pixel  depend 
to  some  extent  on  the  intensity  of  63  or  124  neighboring  voxels 
(depending  on  whether  a4x4x4ora5x5x5  smoothing  kernel 
is  used).  This  filtering  effect  averages  out  much  of  the  statistical 
noise,  reducing  it  by  an  amount  proportional  to  the  number  of 
voxels  in  the  convolution  kernel. 

If  we  apply  a  smoothing  filter  that  simply  suppresses  “impos¬ 
sible”  spatial  frequencies  (i.e.,  those  higher  than  the  optical  system 
is  capable  of  producing),  the  contrast  of  small  features  that  owe 
their  (apparent)  existence  solely  to  the  presence  of  quantum  noise 
in  the  data  will  be  greatly  reduced. 

It  is  important  to  note  that  applying  such  a  filter  will  reduce 
the  apparent  contrast  of  the  image  data.  Digital  look-up  tables  can 
be  used  to  increase  the  apparent  contrast  on  the  viewing  screen  and 
the  resulting  images  will  be  just  as  contrasty  and  will  show  less 
statistical  noise  than  the  raw  data. 

MORAL:  Your  image  is  not  suitable  for  viewing  until  it  has 
been  at  least  filtered  to  remove  features  that  are  smaller  than 
the  PSF,  or,  thought  of  the  other  way,  to  remove  data  having 
spatial  frequencies  beyond  the  maximum  bandwidth  of  the 
optical  system  in  use. 

Some  Special  Cases 

In  classic  sampling  theory,  the  time  (or  space)  taken  to  measure 
or  sample  the  intensity  in  each  pixel  is  very  small  compared  to  the 
inter-pixel  sampling  interval.  Although  this  condition  is  met  for  the 
ADCs  used  in  commercial  confocal  microscopes,14  it  is  not  met  for 
the  CCD  camera,  where  it  is  common  for  the  sensitive  area  of  1 
pixel  to  be  almost  equal  to  the  area  it  covers  on  the  silicon  surface. 
Clearly,  this  means  that  the  space  taken  to  sample  the  light  flux  is 
almost  the  same  as  the  pixel  size.  The  system  works  well  enough 
as  long  as  we  stick  to  Nyquist  sampling  of  a  signal  of  known  band¬ 
width  (4-5  pixels/blob).  In  fact,  some  CCD  manufacturers  have 
gone  even  further  and  made  an  effort  to  increase  the  effective 
spatial  resolution  of  a  CCD  by  making  a  series  of  four  or  nine 
exposures,  in  which  for  each  member  of  the  series  the  sensor  is 
offset  from  the  previous  one  by  one  half  or  one  third  of  a  pixel  in 
x  and  y,  respectively.15  While  reasonable  results  can  be  obtained 
in  this  way,  problems  can  develop  when  it  is  applied  to  color 
imaging  systems  that  employ  a  color  pattern  mask  on  each  pixel 
(Fig.  4.17). 

Even  in  the  confocal  microscope,  problems  can  occur  because 
the  zoom  magnification  control  creates  a  variable  relationship 
between  the  optical  bandwidth  of  the  signal  and  the  electronic 
bandwidth  which  is  set  by  the  “time-constant”  of  the  pre-amplifier 
just  before  the  ADC.  Not  only  that,  but  the  optical  bandwidth 
creates  blurring  in  2D  while  the  pre-amplifier  time  constant  only 


14  Common  values  might  be  pixel  period  =  1  gs,  sampling  interval  =  3  ns. 

15  For  example,  the  Zeiss  Axiocam. 
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FIGURE  4.1 7.  Wiggling  the  chip  to  increase  the  resolution.  A  number  of  CCD 
cameras  increase  their  effective  spatial  resolution  by  adding  together  the  infor¬ 
mation  from  a  set  of  four  (or  nine)  images,  each  of  which  is  collected  with  the 
CCD  displaced  horizontally  by  one  half  (or  one  third)  of  a  pixel,  first  in  x,  then 
in  y,  compared  to  the  other  members  of  the  set.  The  image  in  the  top  left  was 
made  at  a  tube  magnification  of  1.5x  with  the  camera  readout  normally,  while 
that  below  it  was  made  with  3x  less  optical  magnification  and  3x  more  “reso¬ 
lution”  by  moving  the  chip.  Both  look  quite  similar  when  they  have  been  up- 
sampled  and  then  blurred  in  Photoshop  except  that  the  “wiggled”  result  has  a 
color  caste  caused  by  the  fact  that  the  color  mask  filters  on  the  chip  have  a 
“pitch”  that  is  twice  the  nominal  pixel  spacing.  The  up-sampled  and  blurred 
picture  on  the  right  is  analogous  to  the  round,  red  feature  in  Figure  4.1. 


limits  the  signal  bandwidth  in  the  fast  scan  direction  (usually  hor¬ 
izontal).  If  the  zoom  is  set  to  under-sample  high-contrast  optical 
data,  then  very  large  pixel-to-pixel  variations  are  possible  and  the 
bandwidth  should  be  wide.  The  reverse  is  true  for  over-sampling. 
In  response  to  this  problem,  some  commercial  instruments  esti¬ 
mate  the  optical  resolution  from  the  NA  of  the  objective  and  the 
wavelength  of  the  laser  and  then  use  this  information  to  set  the 
pre-amplifier  to  the  optimal  time  constant  (bandwidth)  on  the  basis 
of  the  zoom  setting.  When  such  a  system  is  functioning  properly, 
the  “apparent  noisiness”  of  the  signal  recorded  from  a  bright  but 
relatively  featureless  object  will  become  less  as  the  zoom  is 
increased:  the  signal/pixel  remains  the  same  but  the  longer  time 
constants  effectively  averages  this  noisy  signal  over  more  pixels 
in  the  horizontal  direction. 

Starting  with  the  MRC-600,  all  Bio-Rad  scanners  used  full- 
integration  digitizers.  These  were  composed  of  three  separate  sec¬ 
tions.  At  any  instant,  one  is  integrating  the  total  DC  signal  current 
from  the  PMT  during  a  pixel,  the  second  is  being  read  out,  and 
the  third  is  being  set  back  to  zero.  This  system  effectively  emu¬ 
lates  the  image  digitizing  system  of  the  CCD.  This  approach  works 
well  for  under-sampled  data  and  was  a  great  improvement  on 
earlier  systems  that  used  a  time  constant  that  was  fixed  at  (pixel 
time/4)  and  therefore  let  a  lot  of  high-frequency  noise  through  to 
the  ADC. 

If  you  don’t  want  to  worry  about  any  of  this,  stick  to 
Nyquist! 
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GRAY  LEVELS,  "NOISE,"  AND 
PHOTODETECTOR  PERFORMANCE 

When  an  image  is  digitized  it  must  be  quantized  in  intensity  as 
well  as  in  location.  The  term  gray  level  is  the  general  term  refer¬ 
ring  to  the  intensity  of  a  particular  pixel  in  the  image.  Beyond  this 
general  definition,  things  become  more  complicated.  What  kind  of 
a  measure?  Linear?  Logarithmic?  What  is  a  gray  level?  Let  us 
begin  at  the  beginning  with  a  discussion  of  how  these  matters  were 
handled  by  the  first  reliable  method  for  recording  image  intensity: 
photography. 

Optical  Density 

Early  work  on  the  quantification  of  image  intensity  was  related  to 
the  performance  of  photographic  materials.  Developed  photo¬ 
graphic  emulsions  are  darker  where  they  have  been  exposed  to 
more  light.  However,  this  darkening  process  is  not  linear  because 
the  individual  grains  of  silver  halide  that  make  up  the  emulsion 
only  become  developable  after  absorbing,  not  one,  but  two  light 
photons  within  a  short  space  of  time  (~ls).16  As  a  result,  at  low 
exposures  the  number  of  grains  exposed  is  proportional  to  the 
square  of  the  light  intensity,  a  term  we  will  use  here  to  represent 
the  number  of  photons  per  unit  area  per  unit  time  at  the  detector. 

The  photometric  response  of  photographic  emulsions  is 
quantified  in  terms  of  so-called  HD  curves.  These  plot  the  log  of 
the  light  intensity  (H)  against  the  log  of  the  darkening  (D).  Figure 
4.18  shows  the  important  features  of  such  a  curve.  The  darkening 
is  measured  as  a  ratio  compared  to  a  totally  clear  film,  using  log¬ 
arithmic  optical  density  (OD)  units:  OD  =  0  implies  no  darkening 
and  all  the  light  is  transmitted;  OD  =  1  means  that  the  emulsion 
transmits  10%  of  the  incident  light;  OD  =  2  implies  that  it  trans- 


FIGURE  4.18.  Photographic  H-D  curve. 


16  The  response  of  photographic  emulsions  exposed  to  X  rays,  electrons,  or 
other  ionizing  particles  is  quite  linear. 


mits  1%  of  the  incident  light,  etc.  The  use  of  a  log/log  scale  allows 
one  to  describe  the  HD  response  over  4  to  5  orders  of  magnitude 
on  a  single  plot.  However,  it  also  obscures  much  of  the  quantita¬ 
tive  complexity  of  the  plot  and  parts  of  it  that  seem  “linear”  would 
not  seem  so  on  a  linear  plot. 

Because  there  is  always  some  background  exposure  of  the 
emulsion,  D  is  never  “zero”  but  starts  at  the  “fog”  level.  Small 
exposures  produce  almost  no  additional  darkening  because  few 
grains  receive  two  hits.  Eventually,  however,  the  log  of  darkening 
seems  to  become  proportional  to  the  log  of  exposure  and  the 
response  curve  enters  its  “linear”  region.  At  high  intensity,  the 
response  saturates  for  two  reasons:  as  there  are  only  a  finite  number 
of  grains  in  each  pixel,  one  cannot  do  more  than  develop  all  of 
them.  In  addition,  as  more  grains  are  developed,  they  are  more 
likely  to  be  “behind”  other  developed  grains  and  so  each  new  grain 
contributes  relatively  less  to  the  darkening  of  the  emulsion.  The 
presence  of  a  background  or  noise-level  signal  and  some  sort  of 
saturation  effect  at  high  exposure  is  not  unique  to  photographic 
emulsions,  but  characterizes  all  types  of  photodetectors. 

The  response  of  a  given  emulsion  will  depend  on  the  devel¬ 
opment  conditions  (type  and  concentration  of  developer,  time, 
temperature)  as  well  as  the  exposure  level  (light  intensity  x  expo¬ 
sure  time).  The  “linear”  part  of  the  curve  becomes  steeper  (higher 
contrast)  and  starts  at  a  lower  exposure  level  if  the  development 
time  or  temperature  is  increased. 

As  the  best  photographic  negatives  are  recorded  using  expo¬ 
sures  representing  H  values  below  the  center  of  the  lineal'  portion 
of  the  H-D  curve,  the  transition  region  from  the  fog  level  to  the 
linear  region  (called  the  “Toe”  response)  is  of  prime  importance  to 
the  final  result.  In  this  region,  the  density  is  roughly  proportional 
to  the  exposure  squared.  Of  course,  the  photographic  paper  used 
to  print  the  final  result  also  has  a  photographic  emulsion.  Although 
print  development  conditions  are  more  standardized,  printing 
papers  can  be  purchased  in  different  contrasts. 

In  principle,  one  might  suppose  that  the  ideal  situation  would 
be  for  the  paper  to  have  an  H-D  response  that  just  complemented 
that  of  the  negative.  The  resulting  print  would  represent  a  close 
approximation  of  the  intensity  values  of  the  various  parts  of  the 
image  originally  passing  through  the  camera  lens.  In  practice,  a 
perfect  match  of  these  two  square-law  curves  is  very  hard  to 
achieve  but  this  sort  of  compensation  still  occurs  to  some  extent. 
For  example,  every  camera  lens  transmits  more  light/unit  area  (and 
hence  produces  more  darkening  of  the  negative)  in  the  center  of 
the  field  than  at  the  periphery.  However,  as  this  is  also  true  of  the 
optical  system  used  to  print  the  negative,  the  two  non-linearities 
partially  cancel  out  because  the  denser  center  of  the  negative 
serves  as  a  sort  of  local  neutral  density  filter. 

The  Zone  System:  Quantified  Photography 

Ansel  Adams  is  justly  famous  not  only  for  the  beautiful  images  he 
recorded  of  nature  but  also  for  inventing  The  Zone  System  for 
quantifying  photographic  exposures  and  the  response  of  various 
different  emulsions.  Each  zone  represents  a  brightness  in  the  image 
being  recorded  that  differs  in  intensity  from  neighboring  zones  by 
a  factor  of  2. 

Adams  believed  that  a  good  print  could  transmit  information 
over  a  range  of  seven  zones17  and  that  it  was  important  to  match 


17  That  is.  that  the  brightest  part  of  the  print  would  reflect  about  (2)6  =  64  times 
more  than  the  darkest.  This  was  a  bit  optimistic  as  a  black  print  emulsion 
still  reflects  -2%-3%  and  white  only  -97%,  a  ratio  of  only  about  1 :30. 
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the  range  of  brightness  in  the  scene  (which  might  be  either  more 
or  less  than  seven  zones)  to  the  64 : 1  range  of  brightness  levels 
that  could  be  seen  in  the  print.  This  could  be  done  by  making  a 
judicious  choice  of  emulsion,  exposure,  and  development  condi¬ 
tions.  While  it  is  not  appropriate  here  to  go  into  all  the  details  of 
this  process,  two  aspects  of  this  system  deserve  mention: 

•  The  size  of  each  inter-zone  intensity  steps  relates  to  its 
neighbor  logarithmically,18  much  like  the  eye/brain  system  (see 
below). 

•  The  system  is  non-linear  like  the  square-law  response  of  a  pho¬ 
tographic  emulsion  exposed  to  light.19 

Although  this  logarithmic  response  served  well  in  photog¬ 
raphy,  modern  scientific  imaging  tends  to  prefer  image  recording 
systems  with  a  linear  response. 

Linearity:  Do  We  Need  It? 

There  is  obvious  appeal  to  the  idea  that  the  intensity  value  detected 
in  a  given  pixel  should  be  linearly  related  to  both  the  numerical 
value  stored  in  the  image  memory  and  to  the  brightness  of  the  same 
pixel  when  the  image  is  finally  displayed.  It  seems  that  this  should 
be  easy:  most  electronic  photodetectors  and  ADCs  are  linear. 
It  is  also  logical:  how  else  could  one  represent  what  has  been 
measured? 

Although  linearity  does  indeed  have  these  advantages,  there 
are  some  practical  complications  when  applying  it  to  electronic 
imaging,  especially  when  viewing  the  sort  of  image  data  often 
encountered  in  fluoresecence  microscopy.  These  complications 
have  two  sources: 

1.  Non-linearity  is  inherent  in  all  the  common  methods  whereby 
one  can  view  digital  image  data:  computer  displays  and 
grayscale  or  color  hardcopy.  In  addition,  there  is  the  problem 
of  how,  or  even  if,  one  should  try  to  account  for  the  fact  that 
the  retinal/brain  response  of  the  eye  is  more  logarithmic  than 
linear. 

2.  Because  of  Poisson  statistics,  intensity  values  representing 
only  a  small  number  of  photons  are  inherently  imprecise;  dis¬ 
playing  as  different  tones,  intensity  steps  that  are  smaller 
than  this  imprecision  is  pointless  and  can  even  be  mislead¬ 
ing.  Worse  still,  the  absolute  imprecision  is  not  constant 
but  increases  with  the  square -root  of  the  intensity:  the  errors 
are  greatest  in  the  brightest  parts  of  the  image,  “where  the  dye 
is.” 

The  Goal 

We  start  with  the  idea  that  the  over-riding  purpose  of  microscopy 
is  to  create  in  the  mind  of  the  observer  the  best  possible  estimate 
of  the  spatial  distribution  of  the  light  intensities  representing 
the  structural  features  of  the  specimen.  The  question  then  arises  as 
to  whether  or  not  one  should  bias  the  digitization  or  display 
processes  away  from  “linearity”  to  compensate  for  the  inherent 
statistical  and  physiological  factors.  We  will  try  to  answer  this 
question  with  a  (very!)  quick  review  of  some  relevant  aspects  of 
human  vision. 


18  The  increment  in  darkening  present  between  zones  6  and  7  represents  the 
effect  of  recording  32  times  more  “additional  photons/area"  than  the  incre¬ 
ment  between  zones  1  and  2. 

19  The  steps  in  the  brighter  parts  of  the  final  image  represent  a  larger  increment 
of  exposure  than  in  the  darker  parts. 


Problems  Posed  by  Non-Linearity  of  the 
Visual  System  and  Image  Display  Devices 

Studies  of  the  photometric  response  of  the  eye  to  light  concentrate 
on  the  just  noticeable  difference  (JND).  It  has  been  found  that  most 
people  can  recognize  a  feature  that  differs  in  brightness  from  its 
background  by  a  fixed  fraction  of  this  background  light  intensity 
level,  for  example,  10%.  Although  there  are  some  limitations  on 
this  response,20  it  can  easily  be  seen  to  be  inherently  logarithmic. 
In  addition,  the  number  of  “gray-levels”  that  a  human  eye  can  see 
is  fixed  by  the  size  of  the  JND  and  the  dynamic  range  of  the  image 
it  is  viewing. 

Suppose  that  the  background  has  10  units  of  intensity,  then  if 
the  feature  is  to  be  visible  it  will  have  to  have  either  >9  or  <11 
units,  a  change  of  10%  or  1  unit.  However,  if  the  background  is 
100  units  then  the  10%  JND  step  will  be  10  units:  10  times  bigger. 
No  smaller  increment  or  decrement  will  be  visible  to  the  eye. 

How  might  you  go  about  displaying  the  intensities  9,  10,  11, 
90,  100,  110  units  on  a  computer  screen?  Most  computer  video 
memories  are  8  bits  deep.  This  means  that  (notionally,  at  least)  they 
can  store  and  display  a  maximum  of  28  or  256  different  signal 
intensities.21 

Suppose  that  we  load  our  six  intensity  values  into  the  video 
memory  without  change  as  the  numbers  9,  10,  11,  90,  100,  and 
110.  This  will  mean  that  the  brightest  part  of  our  image  uses  less 
than  half  (110/256)  of  the  numeric  display  range  of  which  the 
display  is  capable.  It  also  means  that  we  do  not  utilize  any  of  the 
memory  levels  between  11  and  89,  91  and  99  or  101  and  109,  etc. 
Does  this  mean  that  we  now  have  an  image  of  256  gray  levels,  of 
only  the  six  levels  of  our  object,  or  some  other  number? 

Alternatively,  to  better  utilize  the  dynamic  range  of  the  video 
memory,  we  might  multiply  the  original  numbers  by  ~2.3  before 
putting  them  into  the  video  memory.  The  brightest  patch  would 
then  be  2.3  X  110  =  253,  almost  at  the  top  of  the  possible  range. 
Do  we  now  have  an  “8-bit”  image? 

What  of  the  computer  display  itself?  Is  there  a  linear  relation¬ 
ship  between  the  number  stored  in  the  video  memory  and  the 
number  of  photons/second  emitted  from  the  corresponding  pixel 
of  the  CRT  screen  or  LCD  image  display? 

In  a  word:  No!  The  exact  relationship  between  these  two  is 
determined  by  the  display  manufacturer  and  the  settings  of  the  con¬ 
trast,  brightness,  gamma,  hue,  and  saturation  controls.  Although 
display  standardization  is  possible  for  those  working  in  the  color 
printing  industry,  it  is  a  very  complex  process  seldom  attempted 
by  working  microscopists,  at  least  in  part  because  it  requires  stan¬ 
dardized  room  lighting.  The  fundamental  problem  for  CRT  dis¬ 
plays  is  that,  while  the  brightness  of  a  pixel  on  the  screen  is  directly 


20  We  are  very  good  at  interpreting  small  changes  in  brightness  at  edges,  but 
see  the  uniform  areas  on  either  side  of  the  edge  as  shaded,  even  though  they 
are  not  shaded.  In  fact,  given  the  large  number  of  recognized  optical  illu¬ 
sions.  one  must  treat  the  eye/brain  as  more  suitable  for  measuring  contrast 
than  intensity. 

21  In  this  discussion  we  will  ignore  the  inconvenient  fact  that  the  performance 
of  most  display  systems  is  itself  limited  by  Poisson  statistics.  For  instance, 
each  “pixel”  on  the  CRT  contains  only  a  small  number  of  phosphor  crystals, 
each  of  which  may  be  more  or  less  efficient  at  converting  energy  from  the 
three  beams  of  electrons  into  light.  Only  a  very  small  fraction  of  these 
photons  will  pass  through  the  pupil  and  be  detected  by  the  retina.  How  many 
actually  do  is  subject  to  statistical  variations.  In  addition,  each  of  these 
three  electron  beams  deposits  only  a  small  number  (1000s?)  of  quantum- 
mechanical  particles  (electrons)  into  a  small  area  of  the  tube  surface  during 
the  “pixel-dwell”  time.  The  exact  number  deposited  is  limited  by  Poisson 
statistics.  Just  because  ignoring  these  complexities  makes  analysis  easier 
does  not  mean  that  they  are  unimportant. 
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proportional  to  the  amount  of  current  in  the  electron  beam  during 
the  time  it  is  illuminated  (i.e.,  the  total  charge  deposited),  this 
current  is  in  turn  proportional  to  the  “3/2”  power  of  the  voltage 
applied  to  the  control  grid  of  the  tube.  Therefore,  even  if  the  digital 
number  in  the  video  memory  is  turned  into  a  linearly  proportional 
voltage  at  the  grid  of  the  CRT,  changes  in  this  value  will  produce 
a  “more  than  proportional”  brightness  change  from  the  phosphor.22 

Cathode-ray  tube  manufacturers  are  aware  of  this  problem  and 
have  developed  a  variety  of  countermeasures.  The  umbrella  term 
for  these  efforts  to  introduce  a  compensating  non-linearity  in 
image  brightness  is  gamma.  In  electronic  imaging,  gamma  is  a 
measure  of  the  non-linearity  of  the  relationship  between  stored 
intensity  information  (Fig.  4.19)  and  displayed  brightness.  It  can 
be  more  or  less  than  unity.  Positive  gamma  stretches  out  changes 
in  intensity  occurring  at  the  lower  end  of  the  intensity  scale  and 
compresses  those  occurring  at  the  top  of  the  scale.  Negative 
gamma  does  the  reverse.  If  nothing  else,  the  presence  of  a  gamma 
other  than  one  means  that  shifting  the  average  signal  level  up  or 
down  (using  the  black-level  or  “brightness”  control)  will  also 
change  the  “gain”  (or  contrast)  of  the  final  result. 

Given  the  uncertainty  regarding  the  correction  software  used 
on  a  particular  CRT,  the  setup  adjustment  of  the  individual  elec¬ 
tron  guns  themselves,  not  to  mention  differences  introduced  by 
user  settings  of  the  controls  or  the  use  of  different  phosphors  on 
the  face  of  the  CRT,  variations  in  the  level  and  color  of  ambient 
lighting,  the  average  user  can  have  little  confidence  in  the  inten¬ 
sity  linearity  of  most  CRT  displays. 

The  same  situation  is  even  more  true  for  displays  that  incor¬ 
porate  LCDs,  where  viewing  angle  to  the  screen  is  an  additional 
and  important  variable.  Non-linearities  also  abound  in  all  of  the 
types  of  hard-copy  renderings  made  using  digital  image  printers: 


Number  stored  in  image  memory 


FIGURE  4.19.  How  the  gamma  control  varies  the  relationship  between  input 
signal  level  and  output  brightness  on  the  screen  or  the  printed  page. 


22  There  are  also  other  variables  that  affect  pixel  brightness:  beam  voltage  (this 
may  dip  if  the  whole  screen  is  bright  vs.  dark)  or  blooming  (if  the  beam  con¬ 
tains  too  much  current,  it  will  become  larger,  i.e.,  “bloom”).  When  this 
happens,  central  pixel  brightness  remains  almost  constant  while  adjacent 
pixels  become  brighter.  This  is  not  a  complete  list. 


spectral  properties  of  dyes,  dither  patterns,  paper  reflectance 
and  dye  absorption,  etc.  This  topic  is  covered  in  more  detail  in 
Chapter  32. 

Once  one  realizes  that  strict  linearity  is  neither  possible  nor 
perhaps  even  desirable,  one  can  move  on  to  “distorting  the  gamma 
in  a  way  that  allows  the  observer  to  see  the  biological  information 
that  the  image  contains"  while  trying  to  be  careful  not  to  introduce 
irresponsible  or  misleading  artifacts.  Clearly,  this  is  a  hazy  area  in 
which  much  discretion  is  needed.  The  topic  of  responsibility  when 
processing  images  is  discussed  in  Chapter  14. 

Matching  the  Gray  Levels  to  the  Information 
Content  of  the  Image  Data 

When  Ansel  Adams  developed  the  Zone  System,  no  consideration 
was  given  to  limitations  on  the  recorded  intensity  other  than  the 
exposure  conditions  (exposure  time  and  lens  aperture),  and  the 
intrinsic  illumination  and  reflectivity  of  object.  This  attitude  was 
justified  because  the  exposure  of  a  single  pixel  on  the  large-format 
negatives  he  used  involves  the  absorption  of  thousands  of  photons 
by  a  small  fraction  of  the  millions  of  silver-halide  grains  located 
there.  As  a  result,  statistical  variations  in  this  number  (i.e.,  square 
root  of  the  number  of  developed  grains)  were  likely  to  be  small 
compared  to  the  10%  JND.  The  same  was  true  of  the  printing 
papers. 

In  live-cell  microscopy  generally,  and  confocal  fluorescence 
microscopy  in  particular,  this  condition  is  often  not  met.  The  flu¬ 
orescence  signal  is  inherently  weak — about  a  million  times  less 
intense  than  the  excitation  light  used  to  produce  it.  Although  this 
limits  the  rate  at  which  data  can  be  produced,  bleaching  and 
phototoxicity  may  impose  even  more  stringent  limits  to  the  total 
recorded  intensity.  In  other  words,  the  imaging  modality 
imposes  absolute  limits  on  the  total  number  of  photons  that 
can  be  detected.  As  a  result,  in  biological  fluorescence  micro¬ 
scopy,  we  are  usually  starved  for  photons.  In  laser  confocal 
microscopy,  it  is  not  uncommon  to  collect  only  10  to  20  photons 
in  the  brightest  pixels  and  zero  or  one  photon  in  the  unstained 
regions  that  often  constitute  a  large  majority  (<99%)  of  the  pixels 
in  a  particular  scan. 

Suppose  that  the  signal  in  the  brightest  pixel  of  a  confocal  flu¬ 
orescence  image  represents  only  16  photons  (not  an  unusual 
figure).  As  we  do  not  have  negative  photons,  and  even  though  we 
are  collecting  these  data  into  an  “8-  or  12-bit”  image  memory 
having  256  or  4096  possible  intensity  intervals,  respectively,  one 
cannot  imagine  that  an  image  in  which  the  highest  intensity  was 
only  16  detected  photons  could  possibly  have  more  than  16  mean¬ 
ingful  gray  levels  corresponding  to  1,  2,  3,  .  .  .  16  photons. 

However,  because  the  counting  of  photons  is  a  quantum- 
mechanical  event  and  hence  limited  by  Poisson  statistics,  the 
number  of  “meaningful”  intensity  steps  in  this  signal  is  even  less. 
The  brightest  recorded  signal  is  really  16  +  4.  The  next  dimmer 
signal  level  that  can  be  discriminated  from  it  by  at  least  one  stan¬ 
dard  deviation  (o),  is  9  ±  3.  With  a  16-photon  peak  signal,  we  can 
discriminate  only  four  “real”  signal  levels.  These  correspond  to  the 
levels  1  ±  1,  4  ±  2,  9  +  3,  and  16  ±  4. 

This  is  really  quite  inconvenient.  What  can  be  done  if  the  stain¬ 
ing  levels  of  our  specimen,  as  modified  by  the  CTF  of  our  micro¬ 
scope,  do  not  coincide  with  this  square-law  of  statistical 
detectability?  There  is  only  one  option:  to  collect  more  signal 
(more  dye,  longer  exposure,  etc.)  or  average  the  data  in  space  over 
the  <64  voxels  that  represent  the  whole,  Nyquist-sampled,  3D  PSF 
by  deconvolving  it  as  discussed  above  and,  in  more  detail,  in 
Chapter  25. 
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Beyond  this,  the  only  strategy  is  humility:  Don’t  base  great 
claims  on  the  detected  brightness  of  one  or  even  a  few  pixels  but 
on  patterns  visible  in  a  number  of  images  from  many  specimens. 

GRAY  LEVELS  IN  IMAGES  RECORDED  USING 
CHARGE-COUPLED  DEVICES:  THE  INTENSITY 
SPREAD  FUNCTION 

The  data  recorded  using  CCD  detectors  for  widefield  /deconvolu¬ 
tion  are  subject  to  similar  limitations.  Conventional  CCDs  have 
higher  quantum  efficiency  (QE),  but  much  higher  readout  noise, 
than  the  photomultiplier  tube  (PMT)  detectors  used  in  most  single¬ 
beam  confocals.  The  higher  QE  increases  the  number  of  detected 
photons  (thereby  reducing  the  effect  of  Poisson  noise)  but  the  pres¬ 
ence  of  substantial  read  noise  reduces  the  number  of  useful  gray 
levels  substantially  below  what  one  would  estimate  from  Poisson 
noise  alone.  Read  noise  becomes  even  more  important  when  exci¬ 
tation  levels  are  lowered  to  reduce  phototoxicity.  Because  both  the 
sources  and  the  form  of  the  noise  signals  are  quite  different  in  each 
detector,  it  has  been  difficult  to  make  comparisons  that  are  both 
simple  and  informative  and  the  recent  introduction  of  the  “elec¬ 
tron  multiplier”  CCD  readout  amplifier  (EM-CCD)  has  made  com¬ 
parisons  even  more  complex.  The  discussion  that  follows  describes 
an  effort  to  define  a  suitable  measure  of  overall  photodetector 
performance. 

What  Counts  as  Noise? 

Just  what  counts  as  “noise”  in  the  context  of  fluorescence  micro¬ 
scopy  is  far  from  settled.  Should  one  count  as  noise  the  signal  from 
non-specific  staining?  From  stray  or  reflected  light  in  the  micro¬ 
scope?  Fixed-pattern  noise  traceable  to  stray  magnetic  fields  or 
electronic  interference?  Even  among  practicing  microscopists,  it 
is  not  uncommon  for  “noise”  to  become  an  umbrella  term  for 
anything  that  makes  an  image  resemble  the  “snowy”  output  of  a 
television  receiver  displaying  the  signal  from  a  distant  station. 
Although  a  variety  of  very  different  physical  processes  can 
produce  such  a  “noisy”  signal,  only  some  of  these  can  be  related 
to  defects  in  the  performance  of  the  detector/digitizer  system.  For 
example,  it  is  common  to  hear  that  turning  up  the  gain  of  the  PMT 
makes  the  confocal  image  “noisier.”  It  would  be  more  proper  to 
say  that  the  fact  that  the  PMT  needs  to  be  so  high  is  an  indication 
that  the  signal  itself  must  be  very  weak  and  hence  must  contain  a 
very  high  level  of  Poisson  noise.23 

In  the  discussion  that  follows,  three  types  of  noise  will  be 
considered: 

•  Poisson  noise:  the  irreducible  minimum  uncertainty  due  to 
quantum  mechanics. 

•  Readout  noise:  assumed  to  be  random  fluctuations  in  the 
electronics  and  virtually  absent  in  the  PMT  and  the  EM-CCD. 

•  Quantum  efficiency:  Although  many  people  think  of  QE  as 
totally  separate  from  noise,  because  it  reduces  the  number  of 
quantum  mechanical  events  sensed,  it  increases  the  effect  of 
Poisson  noise. 

One  can  define  noise  in  any  imaging  system  as  that  part  of  the 
electronic  output  of  the  detector  that  is  not  related  to  the  number 


23  Unless,  of  course,  the  PMT  is  faulty  and  actually  generates  noise  when  high 
voltages  are  applied. 


FIGURE  4.20.  The  ISF  for  a  signal  of  16  ±4  photons/measurement.  (A)  About 
700  identical  exposures  are  acquired.  The  values  collected  at  few  particular 
pixels  (red  and  green  lines)  are  then  converted  from  ADU  units  in  the  com¬ 
puter,  to  photoelectrons  (ne)  and  assembled  to  form  a  histogram  (B).  This 
example  shows  the  number  of  photons  absorbed  at  pixel  p,  assuming  that,  nc, 
was  16  electrons/pixel  and  there  is  no  read  noise.  Approximately  63%  of  the 
trials  yield  a  value  for  ;ie  in  the  range  of  16  ±  4  or  between  12  to  20  (pink- 
shaded  box).  The  halfwidth  of  this  distribution  (red  arrows)  equals  the  RMS 
noise  of  this  measurement.  The  remaining  37%  of  trials  yields  a  value  outside 
this  band. 

of  photons  detected  per  unit  time  and/or  space.  However,  as  the 
electronic  signal  from  the  PMT  is  digitized  in  a  very  different  way 
from  that  of  the  CCD,  it  is  much  more  useful  to  compare  the 
performance  of  these  detector  systems  not  in  terms  of  the  signal 
from  the  detector  but  in  terms  of  the  number  that  is  measured  by 
the  ADC  and  stored  in  the  image  memory  to  represent  the  bright¬ 
ness  of  a  single  pixel. 

Suppose  that  the  average  number  of  photons  striking  pixel,  p, 
of  a  CCD  during  a  number  of  identical  exposure  periods  is  nr  This 
exposure  will  excite  a  number  of  electrons,  nc,  into  the  valence 
band  in  the  location  of  p,  where  nc  is  smaller  that  than  np  because 
the  QE  is  always  less  than  1.  In  fact: 

—  =  QE  (1) 

nv 

One  might  imagine  that  the  best  we  can  do  is  to  measure  nc. 
However,  as  noted  above,  even  this  is  impossible  because  the 
absorption  of  a  photon  is  a  quantum  mechanical  event  and  there¬ 
fore  the  number  absorbed  on  any  given  trial  will  not  be  constant 
but  will  vary  according  to  Poisson  statistics.  If  the  average  number 
of  photons  is  16,  the  histogram  of  numbers  of  photons  actually 
absorbed  on  a  given  trial  versus  the  number  of  times  when  this 
number  was  detected  will  look  like  Figure  4.20. 

The  hatched  area  denotes  the  ±  4  electrons  band  of  values  that 
corresponds  to  the  ±Vl6  imposed  by  Poisson  statistics.  On 
average,  63%  of  the  trials  should  give  values  that  lie  within  this 
band.  Although  only  a  small  fraction  of  these  trials  (about  100) 
will  yield  what  we  have  defined  to  be  the  average  value  (16),  it  is 
important  to  recognize  that  even  a  perfect  photodetector  (i.e.,  one 
with  a  QE  =  1  and  no  measurement  noise),  could  never  record  data 
any  more  accurate  than  this.24 


24  The  analysis  also  holds  for  a  PMT  used  in  the  photon-counting  mode  and 
assuming  that  the  number  16  refers  to  the  average  number  of  photons  actu¬ 
ally  counted.  As  there  is  essentially  no  readout  noise  in  such  a  PMT,  the 
analysis  stops  here. 
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Other  characteristics  of  the  photodetector,  such  as  the  presence 
of  measurement  noise  or  imperfect  digitization,  can  only  move  the 
distribution  to  the  left  and  also  widen  it  compared  to  its  mean 
value.  For  example,  if  the  QE  were  only  25%  rather  than  100%, 
the  recorded  values  would  cluster  about  four  detected  photons 
rather  than  16  and  the  error  bar  would  be  +2  photons  —  a  50% 
likely  error  that  is  twice  that  of  the  perfect  detector  (16  +  4  repre¬ 
sents  a  25%  error).  Indeed,  because  the  ratio  of  the  peak  value  of 
this  histogram  to  its  width  is  a  function  of  both  the  QE  and  the 
measurement  noise,  and  also  because  it  measures  directly  the  accu¬ 
racy  of  the  detector  in  determining  the  number  of  photons  associ¬ 
ated  with  pixel  p,  this  ratio  of  peak  (also  the  mean)  to  its  standard 
deviation  (SD)  provides  a  perfect  metric  for  comparing  the  per¬ 
formance  of  the  different  types  of  photodetectors  used  in  fluores¬ 
cence  light  microscopy. 

In  analogy  to  the  term  point  spread  function  (PSF),  this  metric 
is  called  the  intensity  spread  function  (ISF).  Both  concepts  have 
an  ideal  result:  the  ideal  PSF  is  the  3D  Airy  figure  for  a  given  NA 
and  wavelength.  The  ideal  ISF  is  the  Poisson  distribution  for  a 
given  number  of  quantum  events.  In  each  case,  it  is  easy  to 
compare  the  actual  result  with  the  ideal. 

The  ISF  is  the  ratio  of  the  halfwidth  at  half  maximum  of 
the  histogram  of  the  intensities  recorded  from  one  pixel,  A hp, 
on  sequential  "‘reads”  of  a  constant  signal,  to  the  mean  value 
of  this  signal,  np,  all  calibrated  in  photoelectrons.  The  ratio  of 
number  of  electrons  actually  counted  is  converted  to  photons 
using  published  QE  curves. 

n„ 
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MEASURING  THE  INTENSITY 
SPREAD  FUNCTION 

It  is  important  to  understand  that  the  ISF  is  only  a  meaningful 
measure  of  detector  performance  if  the  graph  is  calibrated  properly 
in  terms  of  photoelectrons  rather  than  arbitrary  computer  units. 
Only  quantum  mechanical  events  follow  the  Poisson  distribution. 
This  next  section  discusses  how  such  calibration  can  be  carried  out. 

Calibrating  a  Charge-Coupled  Device  to 
Measure  the  ISF 

Because  the  readout  noise  of  the  conventional  scientific  CCDs 
used  in  microscopy  is  in  the  range  of  ±  3  electrons  RMS  to  ±  15 
electrons  RMS,  there  is  no  way  to  discriminate  the  signal  from  a 
single  real  photoelectron  from  that  of  none.  As  a  result,  the  gain 
of  the  amplifiers  leading  up  to  the  ADC  is  usually  adjusted  so  that 
the  smallest  digitizing  interval  (analog-digital  unit  or  ADU)  is 
equal  to  somewhere  between  half  and  all  of  the  RMS  noise  value 
(sort  of  Nyquist  sampling  in  “intensity  space”).25  The  specification 
defining  #electron/ADU  is  called  the  gain  setting.  In  other  words, 
if  the  read  noise  is  quoted  as  ±6  electrons  RMS,  then  the  gain 
setting  should  be  in  the  range  of  3  to  6  electrons/ADU.  On  the  best 
cameras,  this  gain  setting  is  measured  quite  accurately  at  the 
factory  as  part  of  the  initial  CCD  setup  process  and  is  usually 
written  on  the  inside  cover  of  the  user  manual. 

If  this  is  not  the  case,  a  fair  approximation  of  the  gain  setting 
can  be  calculated  if  one  knows  the  full-well  (maximum 


signal/pixel)  of  the  CCD  and  the  dynamic  range,  in  bits,  of  the 
camera  system  as  a  whole.  Suppose  that  the  full-well  signal  is 
40,000  electrons  and  the  camera  uses  a  12-bit  digitizing  system. 
As  12-bits  implies  4096  digitizing  intervals,  and,  assuming  that  the 
pre-ADC,  electronic,  gain  has  been  adjusted  so  that  a  40,000  elec¬ 
tron/  pixel  signal  will  be  stored  as  a  value  slightly  less  than  4096, 
one  can  see  that  the  an  increment  of  1  ADU  corresponds  to  ~10 
electrons/  pixel  (Fig.  4.21).26 


FIGURE  4.21.  Bit  depth  and  CCD  camera  performance.  The  top  image  was 
recorded  using  a  “12-bit”  CCD  camera  with  a  "full-well”  (brightest)  signal 
level  of  40  k  electrons/pixel.  Subsequent  images  were  recorded  with  the  same 
light  level  but  steadily  shorter  exposure  times.  Although  one  might  expect  the 
camera  to  have  a  S/N  of  about  4000 : 1  (i.e.,  12-bits),  the  image  disappears  into 
the  noise  when  the  peak  signal  is  reduced  by  a  factor  of  only  1000  (10-bits). 


25  Another  factor  is  that  ADCs  tend  to  be  made  with  certain  fixed  levels  of  res¬ 
olution,  12-bit,  14-bit,  etc.,  and  as  this  feature  can  be  “sold,"  it  is  sometimes 
the  case  that  the  CCD  noise  level  spans  8  or  even  16  ADU. 


26  The  uncertainty  is  due  to  the  practice  of  setting  up  the  system  so  that  a  zero- 
photon  signal  is  recorded  in  the  image  memory  not  as  zero  but  as  some  small 
positive  value.  This  prevents  lost  data  in  the  event  that  the  zero  signal  level 
drifts  downwards. 
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Once  one  knows  the  scaling  factor  between  digitally  recorded 
intensities  and  electrons,  one  merely  has  to  acquire  some  image 
data  and  plot  an  intensity  histogram,  as  shown  in  Figure  4.20.  Con¬ 
tinuing  the  example  above,  one  can  convert  the  digital  image  data 
to  “electron  data”  by  multiplying  the  value  in  every  pixel  by  10. 

"Fixed- Pattern"  Noise 

Although  one  might  assume  that  one  could  measure  the  read  noise 
by  calculating  a  histogram  of  all  the  pixel  intensities  (converted  to 
electrons)  in  a  single  image  made  with  no  light  striking  the  CCD, 
this  is  not  so.  All  the  pixels  on  a  CCD  are  not  equal.  Charge  packets 
from  pixels  on  the  corner  of  the  array  farthest  from  the  read  node 
must  be  transferred  many  more  times  than  charge  packets  from 
pixels  near  the  read  node.  More  transfers  take  a  longer  time  and 
this  increases  the  dark  current  in  far  pixels  versus  near  pixels.  Vari¬ 
ation  in  dark  current  is  just  one  of  a  number  of  factors  that  can 
produce  the  fixed-pattern  noise  signal.  Fixed-pattern  noise  differs 
from  read  noise  in  that  its  form  is  somewhat  predictable  over  time 
and  consequently  it  can  be  virtually  eliminated  by  “flat-fielding.” 
Flat-fielding  is  a  digital  normalization  process  that  compensates 
for  variations  in  the  dark  current  (offset  or  dark  level)  and  sensi¬ 
tivity  on  a  pixel-by-pixel  basis.  For  weak  images,  in  which  pixel- 
to-pixel  sensitivity  variations  are  small  compared  to  Poisson  noise, 
this  can  be  accomplished  by  subtracting  a  dark  image,  made  by 
averaging  many  dark  fields,  from  each  recorded  image. 

To  generate  the  data  needed  to  plot  an  ISF  histogram  similar 
to  that  in  Figure  4.20,  one  should  not  use  intensity  values  from 
all  the  pixels  of  a  CCD  exposed  to  a  “uniform”  light  signal  for 
two  reasons.  Not  only  is  it  very  difficult  to  create  such  a  uniform 
signal,  but  the  fixed-pattern  noise  will  usually  produce  much 
more  spread  in  the  ISF  than  the  read  noise.  Therefore,  one  must 
make  the  ISF  histogram  using  data  from  sequential  brightness 
measurements  obtained  by  reading  a  specific  pixel  many  times  in 
succession. 

On  a  3D  deconvolution  system,  this  can  be  accomplished  by 
recording  as  a  3D  data  stack  a  long  time-series  (-100  frames)  of 
“black”  images,  each  using  the  shortest  possible  exposure  time.  To 
obtain  the  set  of  intensities  recorded  from  a  single  pixel,  extract 
the  values  along  a  “line”  oriented  in  the  time  direction  in  the  X, 
Y,  T  data  stack.  As  no  signal  photons  reached  the  detector  during 
these  exposures  and  the  short  exposure  time  keeps  the  dark  current 
low,  the  signal  is  shown  centered  on  zero  intensity.27  The  horizon¬ 
tal  scale  has  been  calibrated  using  the  value  of  2.5  electrons/ACU 
provided  by  the  manufacturer.  The  read  noise  specification  for  this 
camera  is  ±5  electrons  and  this  tallies  well  with  the  width  of 
the  peak  in  the  histogram.  The  bottom  horizontal  scale  is  calibrated 
in  photons,  using  the  quoted  QE  of  this  chip  at  this  wavelength: 
40%. 

The  histogram  should  vary  only  slightly  as  the  x-  and  y- 
coordinates  of  the  vertical  line  are  varied.  In  general,  the  noise 
measured  in  this  way  will  be  lowest  in  the  corner  of  the  image 
closest  to  the  read  node  (usually  the  top  left  corner  of  the  image 
on  the  screen),  and  highest  in  the  opposite  corner. 

As  the  ISF  includes  the  effect  of  errors  caused  by  both  read 
noise  and  Poisson  noise,  it  can  be  used  to  estimate  the  error  at  any 
signal  level.  Those  interested  in  live-cell  light  microscopy  often 


27  CCD  electronics  are  usually  set  up  so  that  the  “no-light”  signal  is  centered 
on  a  value  somewhat  above  zero,  usually  about  50  ADU.  This  assures  that 
signal  will  not  be  lost  even  if  amplifier  drift  were  to  cause  the  peak  of  this 
distribution  to  drift  to  a  lower  value. 


operate  their  system  so  that  the  maximum  recorded  signal  is  less 
that  10%  of  the  CCD  full-well  capacity  (i.e.,  only  4000  ±  63  elec¬ 
trons/pixel).  Under  these  conditions,  ISF  performance  at  low 
signal  levels  becomes  important.  Those  using  a  CCD  camera  to 
record  the  image  from  a  disk-scanning  confocal  may  have  a  peak 
signal  of  only  10  to  100  photoelectrons. 

To  simulate  this  situation,  expose  the  camera  to  uniform  illu¬ 
mination,  sufficient  to  fill  all  the  pixels  with  a  signal  that  is  (on 
average!)  just  10  times  the  read  noise  level  (50  photons  or  20  ADU, 
in  our  example).  This  can  be  done  by  setting  up  a  microscope  for 
transmitted  light  illumination  with  a  clear  specimen  and  using  ND 
filters  and  the  CCD  camera  shutter  control  to  reduce  the  recorded 
intensity  sufficiently.  This  may  require  a  darkened  room  and  it  is 
also  important  to  use  a  regulated  DC  supply  on  the  transmitted 
light  source  so  that  all  the  exposures  in  the  series  are  exposed  to 
a  constant  light  level. 

This  second  histogram  should  cluster  around  an  intensity  that 
is  50  electrons.  At  2.5  electron/channel,  this  would  be  20  channels 
to  the  right  of  the  center  of  the  zero-signal  histogram.  It  should  be 
wider  than  the  first  histogram  because  now  it  contains  ±7  electrons 
of  Poisson  noise  in  addition  to  the  read  noise.  To  the  extent  that 
the  read  noise  signal  really  is  random  or  “white”  (i.e.,  that  it  does 
not  have  some  fixed  pattern  caused,  for  example,  by  flickering  in 
the  light  source),  these  two  RMS  noise  signals  should  be  added  as 
the  square-root  of  the  sum  of  their  squares.  Therefore,  when  scaled 
in  electrons,  the  width  of  the  distribution  will  now  be  V(25  +  49) 
or  about  ±8.1  electrons  (or  ±3.2  channels). 

Although  it  is  not  possible  to  use  this  setup  to  measure  the  QE 
directly,  comparative  measurements  between  different  cameras  are 
possible.  As  QE  is  a  strong  function  of  wavelength,  one  should  use 
a  narrow  bandpass  filter  [such  as  the  interference  green  filter  used 
for  critical  differential-interference-contrast  (DIC)  imaging]  in 
addition  to  the  ND  filters.28  It  is  also  necessary  to  compensate  for 
the  pixel  dimensions:  larger  CCD  pixels  collect  more  photons, 
have  higher  full-well  capacities,  and  somewhat  higher  read  noise. 
However,  as  the  CCD  must  be  coupled  to  the  focus  plane  in  the 
object  by  an  optical  system  having  the  magnification  required  to 
make  the  CCD  pixels  satisfy  Nyquist,  pixel  size  per  se  is  not  of 
fundamental  importance.  In  general  the  QE  curves  published  by 
the  chip  manufacturers  are  hard  to  improve  on. 


GAIN-REGISTER  CHARGE-COUPLED  DEVICES 

In  January  2001,  Marconi  introduced  the  CAM  65,  a  CCD  camera 
incorporating  a  new  type  of  charge  amplifier  (Lamontagne,  2004; 
Robbins  and  Hadwin,  2003).29  The  heart  of  this  device  is  a  new 
solid-state  electron-multiplier  amplifier.  The  electron  multiplier  is 
essentially  a  second  horizontal  register,  located  between  the  output 
of  the  normal  serial  (horizontal)  register  and  the  input  of  the  charge 
amplifier.  Because  it  has  an  extra,  grounded  phase  between  charge- 
transfer  phases  1  and  2,  and  a  higher  and  variable  voltage  (35-45 
volts  vs.  the  normal  5-15  volts)  on  phase  2,  electrons  must  pass 
through  a  high  field  region  when  passing  from  phase  1  to  phase  2. 
As  a  result,  there  is  a  small  (-1%)  chance  that  collision  amplifi- 


28  All  these  instructions  assume  use  of  a  monochrome  CCD.  Because  of  the 
complex  way  that  signals  from  adjacent  pixels,  each  equipped  with  a  differ¬ 
ent  colored  filter,  are  interpolated  to  produce  the  final  color  image,  noise  in 
single-chip  color  CCDs  is  quite  complex.  As  a  first  approximation,  one  could 
expose  a  color  CCD  to  one  color  at  a  time,  and  calculate  the  ISF  for  each  one. 

29  More  info  at:  http://e2vtechnologies.com/technologies/13vision.htm 
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cation  will  occur.  Therefore,  the  electron  multiplier  operates  as  a 
series  of  500  to  700  very-low-gain  avalanche  diodes.30 

When  the  light  signal  is  low,  the  phase  2  voltage  can  be 
increased  to  multiply  each  charge  packet  by  from  200x  to  4000x. 
This  process  increases  the  signal  from  even  a  single  PE  well  above 
the  noise  floor  of  the  normal  FET  read  amplifier,  and  does  so 
even  this  amplifier  is  operating  at  a  very  high  readout  speed  (10 
to  35  MHz). 

Because  the  EM-CCD  reads  out  fast  without  high  read  noise, 
it  makes  focusing  and  searching  quick  and  easy.  Because  each 
charge  packet  always  remains  separated  from  all  others,  EM-CCDs 
have  full  spatial  resolution,  and  as  the  gain  of  the  register  can  be 
varied,  the  device  “can  be  made  as  sensitive  as  needed.”  Of  course, 
if  one  reads  out  fast,  there  may  not  be  enough  time  to  accumulate 
much  signal  but  this  cannot  be  blamed  on  the  camera. 

This  system  breaks  the  old  rule  that  scientific  CCDs  have  more 
readout  noise  when  they  read  out  faster. 

There  is  one  snag. 

Multiplicative  Noise 

The  charge  amplification  of  the  gain  register  is  not  quite  “noise 
free”  because  the  exact  amount  that  each  electron  in  the  packet  is 
amplified  depends  on  probability. 

The  gain-per-charge-transfer  is  very  low:  about  1  %/stage.  A 
charge  packet  containing  100  electrons,  would,  on  average,  contain 
101  electrons  after  one  transfer.  Big  deal!  Why  bother?  Because, 
after  500  or  600  transfers,  the  average  total  gain  can  reach  more 
than  lOOOx.  However,  some  electrons  do  not  get  amplified  very 
much  while  others  get  amplified  a  lot  (more  like  the  stock  market 
than  a  bank!!).  This  gives  rise  to  “multiplicative  noise.”  Given  a 
series  of  charge  packets,  each  containing  only  a  single  electron, 
the  spectrum  of  pulse  heights  at  the  gain  register  output  follows 
a  decreasing  exponential  with  many  small  pulses  and  fewer 
large  ones  (several  useful,  online  references  are  available 
at:  http://e2vtechnologies.com/technologies/13vision.htm).  (See 
Figure  4.22.) 

The  multiplicative  noise  generated  by  this  type  of  pulse-height 
distribution  has  the  same  form  and  magnitude  as  Poisson  noise.  As 
RMS  noise  terms  are  added  as  the  square-root  of  the  sum  of  the 
squares,  the  output  of  the  gain  register  has  1.41x  the  uncertainty 
present  in  the  input  PE  signal.  Because  the  only  way  to  overcome 
this  loss  in  precision  is  to  count  twice  as  much  signal,  one  can 
think  of  the  entire  system  acting  as  if  the  amplifier  were  noise- 
free  but  the  sensor  QE  is  only  half  its  actual  QE.31 

The  back-illuminated  sensors  now  available  have  an  intrinsic 
QE  of  about  85%,  or  effectively  42%  when  used  in  the  EM-mode. 
This  is  still  very  good  compared  to  the  PMT,  especially  in  the  red 
end  of  the  spectrum. 

The  50%  reduction  in  effective  QE  only  occurs  when  the  gain- 
register  amplifier  is  used.  If  it  is  turned  off,  the  CCD  QE  operates 
as  advertised,  but  the  read  noise  now  depends  on  the  FET  ampli¬ 
fier  and  will  increase  with  readout  speed. 

Multiplicative  noise  also  occurs  in  the  PMT  because  the  gain 
experienced  by  each  electron  at  each  dynode  is  also  subject  to  sta¬ 
tistical  noise.  Suppose  that  an  electron  from  the  photocathode  (PE) 


30  See  Chapter  6  and  Appendix  3  for  definitions  of  phase  I,  etc. 

31  A  more  complete  analysis  of  the  noise  produced  by  this  detector  can  be  found 
at  http://www.marconitech.com/ccds/lllccd/technology.html.  Try  the  third 
article:  “Sub-Electron  Read  Noise  at  MHz  Pixel  Rates:  University  of  Cam¬ 
bridge  •  Date:  Jan  2001  •  Filesize:  650 kb. 
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FIGURE  4.22.  Single  PE  pulse-height  spectrum. 


produces  an  average  of  16  secondary  electrons  (SE)/pixel  on  strik¬ 
ing  the  first  dynode.  In  fact,  the  number  of  SE  arriving  at  the 
second  dynode  for  a  particular  PE  is  governed  by  Poisson  statis¬ 
tics:  that  is,  it  will  be  16  ±  4.  This  uncertainty  occurs  at  every 
dynode  but  it  is  most  important  at  the  first  dynode  because  the 
number  of  quantum  events  is  smallest  there.  Not  surprisingly,  after 
the  pulse  has  propagated  down  the  rest  of  the  multiplier  section,  a 
PE  that  sent  20  SE  to  the  second  dynode  is  likely  to  produce  a 
larger  final  pulse  height  than  a  PE  that  only  produced  12  SE.  The 
result  is  that  single-PE  pulses  from  the  PMT  may  vary  in  size  over 
a  range  of  10: 1,  although  they  will  cluster  around  a  mean  value.32 
Because  the  gain/stage  is  much  higher  in  a  good  PMT  than  in 
the  gain  register  (more  like  5x-10x  than  l.Olx),  the  single-PE 
pulse-height  distribution  has  a  distinct  peak  (Fig.  4.6).  As  a 
result,  multiplicative  noise  adds  only  about  20%  to  the  Poisson 
noise  and  the  effective  QE  of  a  PMT  is  consequently  reduced  to 
about  70%  (1/1.41)  of  that  claimed  by  the  manufacturer’s  QE 
curves.33 

As  used  in  the  laser  confocal,  PMT  multiplicative  noise 
has  another,  more  insidious,  essentially  psychological,  effect. 
Although  the  histogram  of  the  pixel  intensities  in  a  “normal”  stored 
confocal  image  may  show  some  pixels  recorded  at  virtually  every 
ADU  level  from  0  to  255,  the  image  commonly  represents  a  peak 


32  PMT  suppliers  can  provide  a  single  PE  pulse-height  distribution  graph  for 
any  type  of  tube.  A  good  tube  will  have  a  “valley”  between  the  single-PE 
peak  and  the  electronic  noise  peak  (near  zero  size)  that  is  less  than  33%  of 
the  peak  height. 

33  Actually,  it  is  worse  than  this,  because  the  published  QE  curves  are  “best 
performance”  not  average  performance,  and  they  refer  only  to  photoelectrons 
leaving  the  photocathode,  per  incoming  photon,  not  photoelectrons  that  actu¬ 
ally  reach  the  first  dynode  and  result  in  charge  multiplication.  About  30% 
of  the  electrons  leaving  the  photocathode  fail  to  reach  the  first  dynode  and 
propagate.  This  loss  further  reduces  the  effective  QE. 
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TABLE  4.1.  Theoretical  Intensity  Function  (ISF)  Values  for  Various  Photon  Signals  to  Compare  Performance  of  a 
"Conventional"  Scientific  CCD,  a  "Gain-Register"  CCD,  and  a  PMT,  Calculated  as  Per  Discussion  in  Text.  Bold  Numbers 
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signal  of  only  about  10  photoelectrons.  Were  it  not  for  the  multi¬ 
plicative  noise  in  the  PMT,34  the  operator  would  be  alerted  to  the 
low  signal  level  by  the  “posterizing”  effect  of  having  at  most  10 
gray  levels  displayed.  The  presence  of  multiplicative  noise  gives 
the  illusion  of  a  full  gamut  of  “gray  levels”  when  in  fact  the  data 
are  insufficient  to  define  more  than  three  or  four  levels  at  one- 
standard-deviation  precision. 

In  large  part,  the  ISF  was  developed  to  compare  the  photode¬ 
tector  performance  of  confocal  microscopes  employing  EM-CCDs 
with  those  using  PMTs. 

At  zero  light  level,  the  halfwidth  of  the  ISF  is  equal  to  the  read 
noise  level.  Although  the  ISF  itself  will  vary  with  the  signal  level, 
a  few  standard  signal  levels  could  be  chosen  (say  10  or  100 
photons/pixel)  to  specify  detector  performance. 

Alternatively,  as  there  seems  to  be  a  general  trade-off  between 
QE  and  read  noise,  one  could  calculate  or  measure  the  “crossover” 
signal  level  at  which  equal  ISF  performance  should  occur  on  two 
detectors.  Given  a  100  photon/pixel  input,  a  perfect  detector  would 
have  an  ISF  of  10  (i.e.,  QE  =  1  and  only  Poisson  noise,  and  ISF  = 
VTOO).  Areal  gain-register  CCD  with  an  effective  QE  of  40%  and 
no  read  noise,  would  have  an  ISF  of  about  40/V40,  or  6.3.  A  con¬ 
ventional  scientific  CCD  with  a  QE  of  80%  and  a  “white”  read 
noise  of  ±6  electrons  would  have  a  Poisson  noise  of  ±9  electrons, 
and  an  ISF  of  (80/V(36  +  80)  =  80/11,  or  about  7.3.  Other  values 
are  listed  in  Table  4. 1 ,  above. 

Unless  high  read  speed  is  important,  Table  4.1  shows  that  a 
normal  CCD  with  the  parameters  stated  will  yield  more  accurate 
data  (higher  ISF)  for  any  signal  greater  than  about  45 
photons/pixel. 

The  situation  is  graphed  in  Figure  4.23  which  shows  a  “mean- 
variance”  plot  of  the  noise  of  an  EM-CCD  camera  operated  with 
no  gain,  low-gain,  and  high  gain  (Janesick,  2001).  It  shows  how 
the  “noise”  (as  measured  by  the  variance)  varies  with  the  signal 
level  (in  PE).  The  low-gain  is  the  noisiest  overall.  The  no-gain  is 
least  noisy  above  a  signal  level  of  ~20  PE  (or  ~45  photons) 
because  the  effective  QE  is  higher  with  no  EM  amplification. 
However,  below  this  level,  the  high-gain  camera  becomes  increas¬ 
ingly  superior  as  the  signal  level  approaches  zero.  Because  the 


34  Or  if  it  were  used  in  the  photon-counting  mode,  which  eliminates  multi¬ 
plicative  noise  by  counting  all  single-PE  pulses  the  same.  Unfortunately, 
pulse-counting  systems  are  usually  by  pulse-pileup  limited  to  peak  count 
rates  of  -30 MHz.  This  means  that  in  a  lgs  pixel  time,  50%  of  a  15 
counts/pixel  signal  would  be  lost  to  pileup. 


EM-CCD  effective  QE  is  about  three  times  higher  than  that  of  a 
PMT,  this  represents  the  high  end  (!)  of  the  signal  range  of  a  con- 
focal  microscope. 

Although  this  may  sound  like  bad  news  for  the  EM-CCD,  in 
fact,  for  a  CCD  camera  on  a  disk-scanning  confocal,  the  peak 
signal  level  may  indeed  be  <45  photons/pixel/frame  time  (or 
indeed,  less  than  4!).  Although  45  photons/pixel  may  not  sound 
like  much,  remember  that  as  long  as  the  pixels  (or  voxels)  are  the 
correct  size  to  meet  the  Nyquist  sampling  criterion  (see  Chapter  4, 
this  volume),  the  3D  image  of  a  single  point  object  will  produce 
significant  counts  in  at  least  64  to  125  voxels.  Assuming  proper 
digital  filtering  of  the  stored  data  before  it  is  displayed,  this  means 
that  at  least  45  x  64  =  2880  photons  will  reach  the  photodetector 
from  each  “bright”  point.  Between  1152  and  2304  of  these  will  be 
“detected,”  giving  statistical  accuracy  in  the  3%  to  4%  range. 

In  this  simulation,  the  PMT  never  provides  the  highest  ISF.  In 
fact,  it  will  do  better  than  a  gain-register  CCD  at  very  low  average 
photon  fluxes  (~1  PE/  pixel)  because  its  dark  current  and  read 
noise  is  actually  about  lOx  lower  than  the  EM-CCD.  Both  the  PMT 
and  the  EM-CCD  are  well-suited  for  confocal  imaging  because 
they  measure  zero  light  very  accurately  and  this  ability  matches 
well  with  the  fact  that  most  voxels  contain  no  stain. 
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FIGURE  4.23.  Mean-variance  curves  of  an  EM-CCD  camera  operated  with 
no  gain  (green),  low  gain  (red)  and  high  gain  (blue).  The  variance  is  a  measure 
of  the  uncertaintly  of  the  brightness  measurement,  obtained  by  subtracting  two 
sequential  CCD  images  of  a  flat  “white”  scene.  The  mean  signal  is  the  average 
value  of  the  scene,  calibrated  in  PE.  One  can  easily  see  the  greater  accuracy 
provided  by  high-gain  operation  when  detecting  signal  lower  than  ~20  PE. 
These  plots  kindly  collected  and  provided  by  Colin  Monks,  Brian  Bodensteiner 
and  Pavani  Korada  (Intelligent  Imaging  Innovations,  Denver,  CO). 
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TRADE-OFFS 

Digital  microscopy  performance  is  dependent  on  three  factors  that 
have  nothing  directly  to  do  with  the  specimen:  optics/diffraction, 
photon  statistics,  and  digitization.  Unfortunately  these  three  factors 
always  interact.  Although  it  is  possible  to  make  very  poor  digiti¬ 
zation  choices  that  substantially  reduce  the  quality  of  the  data  col¬ 
lected  (e.g.,  using  less  than  Nyquist  sampling  or  failing  to  count 
as  much  of  the  photon  signal  produced  as  possible),  it  is  unfortu¬ 
nately  the  case  that,  even  when  one  does  everything  right,  there 
are  still  trade-offs.  Assuming  that  the  resolution  is  determined  by 
the  Abbe  equation,  this  determines  the  pixel  size  needed  to  record 
data  up  to  the  Abbe  resolution  level.  However,  one  will  still  not  be 
able  to  visualize  features  separated  by  this  distance  unless  the 
sample  can  provide  enough  photons.  The  intensities  recorded  in 
the  pixels  defining  any  small  structure  can  only  be  determined 
approximately.  If  the  false  “contrast”  produced  by  Poisson  noise 
and  read  noise  becomes  comparable  to  the  contrast  of  the  image 
feature,  the  feature  will  become  invisible.  The  only  possible  escape 
from  this  conundrum  is  ALWAYS  to  deconvolve  3D  data  and  filter 
2D  data  as  this  will  eliminate  artifactual  “out-of-bandwidth”  fea¬ 
tures  and  effectively  average  out  the  Poisson  noise  over  the 
number  of  Nyquist  voxels  needed  to  define  the  central  peak  of  the 
3D  Airy  figure  (see  Chapter  25,  this  volume ). 

If  low  signal  levels  make  seeing  small  details  impossible,  use 
bigger  pixels  (lower  the  zoom  magnification,  “bin”  the  CCD,  or 
change  its  coupling  tube  magnification)  and  count  more  photons 
in  each,  bigger  pixel.  This  will  give  you  better  statistical  accuracy 
for  the  same  damage  level  and  it  won’t  really  reduce  the  spatial 
resolution  because  resolution  is  now  limited  more  by  counting  sta¬ 


tistics  than  by  diffraction,  or  under-sampling.  It  will  also  have  the 
added  bonus  of  either  a  shorter  scan  time  or  a  larger  recorded  field 
of  view. 

In  live-cell  microscopy,  you  can’t  have  it  all!! 
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Laser  Sources  for  Confocal  Microscopy 

Enrico  Gratton  and  Martin  J.  vandeVen 


INTRODUCTION 

Laser  assisted  confocal  microscopy  has  made  a  lot  of  progress  over 
the  past  few  years.  Laser  systems  have  become  more  modular  and 
compact.  There  is  an  ever-increasing  number  of  available  laser 
excitation  lines  as  well  as  an  improvement  in  user  friendliness 
and  ease  of  use.  At  the  same  time,  expansion  of  Web  resources 
has  provided  easy  access  to  a  wealth  of  information.  Our  goal  is 
both  to  aid  the  experienced  and  novice  microscopist  in  quickly 
locating  and  sorting  through  the  relevant  laser  information  and  to 
provide  a  means  of  avoiding  common  problems  and  pitfalls  in  the 
use  of  laser  excitation  in  the  various  fluorescence  techniques 
such  as  fluorescence  correlation  spectroscopy  (FCS),  fluorescence 
lifetime  imaging  microscopy  (FLIM),  fluorescence  loss  in  photo- 
bleaching  (FLIP),  fluorescence  recovery  after  photobleaching 
(FRAP),  optical  coherence  tomography  (OCT),  second  harmonic 
generation  (SHG),  single  molecule  detection  (SMD),  and  single 
particle  tracking  (SPT).  In  this  chapter  we  describe  the  character¬ 
istic  properties  of  a  number  of  lasers  commonly  used  in  fluores¬ 
cence  microscopy.  We  concentrate  on  the  characteristics  of  lasers 
in  relation  to  their  use  as  an  illumination  source  for  microscopy. 
Compared  to  other  sources  emitting  electro-magnetic  radiation, 
such  as  hot  filaments,  arc  lamps,  and  light-emitting  diodes  (LEDs), 
lasers  have  a  number  of  unique  properties,  which  make  them  an 
almost  ideal  light  source  for  use  in  confocal  microscopy.  These 
properties  are: 

•  high  degree  of  monochromaticity 

•  small  divergence  angle 

•  high  brightness 

•  high  degree  of  spatial  and  temporal  coherence 

•  plane  polarized  emission  (for  many  types) 

•  a  Gaussian  beam  profile  (in  some  cases  this  requires  special 
optics). 

In  the  40  years  since  the  realization  of  the  first  experimental 
laser,  a  wide  and  still  rapidly  expanding  variety  of  lasers  has  been 
developed.  Currently  very  rapid  development  of  miniaturized, 
easy-to-use,  tunable  “pocket”  lasers  is  taking  place.  These  small 
convenient  lasers  are  in  the  process  of  replacing  many  of  the  large 
laser  systems  still  in  use. 

Available  laser  systems  cover  an  extremely  wide  range,  dif¬ 
fering  from  each  other  in  physical  size,  principle  of  operation,  and 
optical,  temporal,  and  mechanical  properties,  such  as  beam  diver¬ 


gence,  output  power,  polarization  properties,  duty  cycle,  stability 
of  the  beam,  and  vibration  sensitivity.  These  characteristics  are 
related  to  the  mechanical  design,  emission  wavelengths  and  tun- 
ability,  ease  of  operation,  maintenance  costs,  reliability,  and  safety 
aspects.  This  chapter  introduces  the  microscopist  to  the  operation 
of  the  laser,  the  most  important  laser  parameters,  their  influence 
on  the  quality  of  the  confocal  image,  and  methods  to  create  wave- 
length-tunable  light  sources.  In  addition,  laser  systems  for  second 
harmonic  generation  and  optical  tweezers  are  described. 

LASER  POWER  REQUIREMENTS 

First,  we  need  an  order  of  magnitude  estimate  of  the  emission 
intensity  that  can  be  obtained  in  fluorescence  microscopy  using 
1  mW  of  input  light.  The  amount  of  laser  power  needed  depends 
crucially  on  the  quantum  efficiency  of  the  contrast  medium  being 
studied.  The  most  common  contrast  factors  are  sample  fluores¬ 
cence  and  backscatter. 

It  is  convenient  to  express  the  quantities  in  terms  of  photons/ 
(s  *  pixel  *  mW)  of  incident  light  at  a  given  wavelength  because 
the  intrinsic  dark  noise  of  modern  detectors  is  often  specified  in 
similar  units.  Also,  expressing  the  flux  per  pixel  provides  a  quan¬ 
tity  that  is  independent  of  the  illuminated  area.  The  following  are 
useful  relationships: 

•  Energy  of  one  photon:  hv  =  hdX  =  4  x  10~19J  at  X  =  500  nm 

•  lmW  of  light  intensity  at  500  nm  represents  2  x  10+I5 

photons/s 

On  a  widefield  image  of  1000  x  1000  pixels,  1  mW  of  incident 
light,  uniformly  distributed,  is  equivalent  to 

•  flux  per  pixel  =  2  x  109  photons/(s  *  pixel  *  mW)  at  500  nm 

There  are  two  considerations.  First,  how  many  photons  will 
be  emitted  per  pixel?  Secondly,  how  many  photons  can  be  toler¬ 
ated  per  pixel  before  saturation  of  the  fluorescent  molecules 
occurs? 

Let  us  analyze  the  first  question.  Using  fluorescein,  one  of  the 
most  common  probes,  the  molar  extinction  coefficient  is  about 
100,000/cm  of  optical  path.  Assuming  an  effective  optical  path  of 
about  1  pm  (the  depth  of  field),  the  molar  extinction  is  about  10. 
The  local  concentration  of  fluorescein  can  vary,  depending  on  the 
spatial  location  and  the  degree  of  labeling.  Assuming  that  a 
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concentration  of  1 0  5M  is  reasonable,  the  optical  density  (OD)  of 
a  1  pm  path  length  is  ~  1CT4.  The  number  of  photons  absorbed  is 
then 

photons  absorbed  =  (flux  per  pixel)  x  (OD)  = 

2  x  105/(s  *  pixel  *  mW)  at  500nm. 

Assuming  a  quantum  yield  of  0.8  and  a  collection  efficiency 
of  10%,  the  detector  receives 

photons  at  the  detector  =  1.6  x  104/ 

(s  *  pixel  *  mW  of  incident  light). 

Given  the  quantum  efficiency  for  a  good  detector  (10%  at 
500  nm),  the  final  detected  photon  flux  should  be  about 

flux  detected  =  1600  photons/(s  *  pixel  *  mW  of  light). 

This  flux  should  be  compared  with  the  dark  noise  of  most 
detectors,  which  can  vary  between  10  and  100  equivalent 
photons/(s  *  pixel).  In  our  estimation,  the  only  quantities  that  can 
vary  over  a  wide  range  are  the  power  of  the  laser  and  the  effective 
concentration  of  the  probe.  Lasers  can  have  up  to  watts  of  power 
and  the  concentration  of  the  probe  can  be  higher  than  we  have 
assumed.  The  efficiency  of  detection  is  usually  smaller  than  we 
estimate  and  the  noise  can  be  larger.  The  purpose  of  our  calcula¬ 
tion  is  to  give  a  rough  idea  of  the  kind  of  power  that  a  laser  must 
furnish  to  be  usable  for  fluorescence  detection  in  confocal  laser 
scanning  microscopy  (CLSM).  Tsien  and  Waggoner  (Chapter  16, 
this  volume)  find  an  optimal  power  with  the  best  signal-to-noise 
ratio  (S/N)  with  respect  to  autofluorescence  and  elastic  and  inelas¬ 
tic  scattering  of  76pW  at  488 nm  and  590pW,  as  long  as  triplet 
formation  is  neglected.  Therefore,  a  laser  power  of  1  to  2mW 
spread  over  106  pixels  at  the  specimen  position  should  be  more 
than  sufficient  for  most  applications.  Effectively,  10  to  lOOpW  is 
common  in  confocal.  Assuming  a  10%  optical  path  efficiency  a 
laserhead  output  power  of  >~1  mW  suffices. 

There  are  two  different  types  of  saturation  effects.  One  is 
related  to  the  number  of  molecules  that  can  absorb  light  in  a  given 
area  for  a  certain  incident  flux.  In  a  given  pixel,  assuming  a  volume 
of  1  pm3,  the  volume  is  I O-15  L.  At  a  molar  concentration  of  1CT5, 
we  should  have  approximately  6000  molecules/pixel.  Since  the 
number  of  photons  absorbed  per  milliwatt  of  incident  light  is  about 
2.5  X  10+5/s  on  a  single  pixel  in  widefield,  each  molecule  is  excited 
about  40  times  per  second.  From  the  photophysical  point  of  view, 
the  decay  of  fluorescein  (and  in  general  any  singlet  single  state 
decay)  is  very  fast  (4  x  10-9  s),  so  that  the  ground  state  should  be 
repopulated  very  rapidly.  However,  in  the  confocal  microscope  for 
a  pixel  dwell  time  of  about  1  ps,  the  40  x  4  ns  =  160  ns  dead  time 
represents  16%  of  the  pixel  period. 

There  are  many  possible  photochemical  processes  that  are 
either  irreversible  or  have  a  cycle  time  of  several  milliseconds  to 
seconds.  In  this  latter  case,  even  if  the  quantum  yield  for  these 
effects  is  very  low  (below  0.001),  and  the  exposure  time  is  on  the 
order  of  seconds,  molecules  lost  to  the  long-lived  state  will 
severely  limit  the  overall  peak  excitation  intensity  that  can  be  used 
before  the  output  loses  its  linear  relationship  with  the  input. 
For  quantitative  microscopy  this  is  the  most  important  limitation. 
Hess  and  Webb  (2002)  found  that  their  FCS  data  implied  a  non- 
Gaussian  three-dimensional  (3D)  volume  and  distortion  of  the 
calibration  of  the  excitation  volume  at  a  power  level  of  10  to  100 
pW  at  488  nm,  for  one  photon  Rhodamine  Green  excitation  and 
5  to  lOmW  at  980  nm,  for  the  two-photon  case  (Rhodamine  Green 
or  Alexa  488,  Molecular  Probes). 

Having  discussed  the  power  requirements,  we  continue  with 
a  concise  description  of  the  basic  elements  of  a  laser,  its  principle 


of  operation,  and  other  important  practical  aspects,  such  as  heat 
removal  and  mechanical  and  optical  stability. 

In  general,  confocal  microscopes  work  best  at  <1  mW  of  con¬ 
tinuous  wave  (CW)  beam  power  at  the  specimen  and  useful  images 
of  living  cells  have  been  made  with  <100  nW  (see  Chapter  19,  this 
volume).  When  pulsed  lasers  are  used  for  two-photon  excitation 
it  is  important  for  the  pulse  frequency  to  be  high  enough  (80— 
100  MHz)  so  that  many  pulses  occur  during  the  1-  to  4-ps  pixel 
dwell  time.  The  intensity  instability  of  this  light  when  it  reaches 
the  specimen  must  be  small  compared  to  the  statistical  uncertainty 
that  will  occur  when  the  signal  photons  are  detected;  if  the  signal 
level  is  only  100  photons/pixel,  then  Poisson  statistics  will  limit 
accuracy  to  10%,  while  104  detected  photons  will  yield  1% 
accuracy. 

THE  BASIC  LASER 

The  acronym  laser  stands  for  light  amplification  by  stimulated 
emission  of  radiation.  Laser  action,  that  is,  the  emission  of  coher¬ 
ent  radiation,  has  been  observed  for  a  large  number  of  different 
media,  but  all  lasers  have  several  features  in  common  (see 
Fig.  5.1). 

•  Active  laser  medium:  Atoms,  molecules,  and  atomic  ions  in 
pure  or  mixed  gases,  vapors,  liquids,  or  solids,  confined  to  the 
gain  medium  volume  where  they  can  be  excited  by  an  exter¬ 
nal  source  of  energy. 

•  Excitation  source:  An  external  source  of  energy  used  to  pump 
the  laser  medium.  This  can  be  another  laser,  arc  lamp  or  flash 
lamp,  electron  beam,  proton  beam,  electrical  current,  electri¬ 
cal  discharge  or  radio  frequency  (RF)  excitation,  etc.  The 
choice  of  pump  source  is  determined  by  the  optical,  thermal, 
and  mechanical  properties  of  the  active  medium  and  the  wave¬ 
length  region  of  interest. 

•  Optical  resonator:  The  laser  medium  is  enclosed  between  two 
parallel  mirrors,  which  form  a  Fabry-Perot  interferometer.  The 
mirrors  are  placed  at  each  end  of  the  laser  medium.  One  mirror, 
the  high  reflector  (HR),  totally  reflects  ( R  =  99.9%);  the  other 
mirror,  the  output  coupler  (OC),  only  partially  reflects.  To  a 
first  approximation,  the  ratio  of  these  two  reflectivities  is  the 
gain  of  the  cavity  that  they  form.  For  instance,  if  the  reflec¬ 
tivity  of  the  OC  is  95%,  on  average  a  given  photon  will  pass 
through  it  1  time  in  20  and,  in  the  absence  of  other  loss  mech¬ 
anisms,  the  cavity  gain  will  be  20.  In  small  semiconductor 
lasers  the  physical  polished  sides  of  the  devices  may  act  like 
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FIGURE  5.1.  An  optical  resonator  is  formed  between  a  highly  reflective 
mirror  (high  reflector.  R  =  99.99%),  HR.  and  a  mirror  with  a  reflectivity  of,  for 
example,  80%,  the  output  coupler  (OC).  Within  this  resonator  we  find  the  active 
lasing  medium  (a  crystal,  semiconductor,  liquid  or  gas)  and  its  energy  supply: 
the  pump  source,  an  arc  lamp  or  flash  lamp,  another  laser. 
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effective  mirrors.  Laser  emission  generated  by  electronic  tran¬ 
sitions  that  are  relatively  rare  require  cavities  with  higher  gain. 


Principle  of  Operation 

A  particle  of  the  active  laser  gain  medium  becomes  excited  when 
it  absorbs  pump  energy  and  goes  to  an  excited  level  [Fig.  5.2(A)]. 
It  then  returns  to  the  ground  state  via  non-radiative  relaxation 
processes  and  also  by  emission  of  radiation  [Fig.  5.2(B)].  Under 
normal  conditions,  a  Boltzmann  equilibrium  describes  the  popula¬ 
tion  of  the  various  energy  levels:  the  higher  the  energy  level  the 
lower  the  population  of  that  level.  When  an  excited,  metastable 
level  with  a  long  lifetime  exists  in  the  laser  medium,  energy  will 
accumulate  in  this  level.  If  the  excitation  is  intense  enough,  the 
Boltzmann  distribution  normally  present  will  “invert”  for  the  pop¬ 
ulation  of  this  metastable  state  (i.e.,  there  will  be  many  more  elec- 
tons  in  the  excited  state  than  Boltzmann  would  predict).  In  the  laser 
cavity,  photons  emitted  from  this  energy  level  will  strongly  inter¬ 
act  with  the  population  of  the  metastable  level  [Fig.  5.2(C)], 
forcing  it  to  release  energy  and  return  back  to  the  lower  level  [Fig. 
5.2(D)].  This  process  is  called  stimulated  emission  of  radiation, 
that  is,  the  interaction  of  the  light  with  the  excited  particle 
increases  the  likelihood  that  the  particle  will  return  to  the  ground 
state.  The  stimulated  emitted  light  has  a  high  degree  of  mono¬ 
chromaticity,  because  emission  occurs  from  a  well-defined  transi¬ 
tion.  In  addition,  the  photon  that  results  from  this  stimulated 
de-excitation  process  is  in  phase  with  the  electromagnetic  wave 
traveling  in  the  laser  medium.  As  a  result,  the  emitted  radiation  has 
excellent  spatial  and  temporal  coherence  and  is  highly  directional. 

A  convenient  way  to  let  the  electromagnetic  radiation  interact 
with  the  laser  medium  is  a  resonant  cavity  (Fig.  5.1).  At  optical 
wavelengths,  this  is  achieved  by  a  Fabry-Perot  type  interferome¬ 
ter.  Two  plane-parallel  mirrors,  one  highly  reflective,  the  other 
semi-transparent,  are  separated  by  a  distance  equal  to  an  integral 
multiple  of  half  the  lasing  wavelength.  Because  the  electro¬ 
magnetic  radiation  interacts  repeatedly  with  the  laser  medium,  this 
resonant  cavity  increases  the  probability  of  stimulating  de¬ 
excitation.  It  also  provides  the  necessary  feedback  to  make  the 
emission  grow  coherently.  Because  the  emission  of  the  first  photon 
going  in  precisely  the  correct  direction  to  reflect  back  and  forth  in 
the  cavity  is  a  low  probability  event,  getting  the  laser  to  fire  is  a 
chaotic  phenomenon  that  exhibits  a  threshold  effect;  lasers  won’t 
start  or  work  stably  below  a  certain  output  power  level. 
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FIGURE  5.2.  Optical  (de)excitation  processes.  (A)  Transition  from  ground 
state  to  excitated  state  upon  absorption  of  a  photon,  an  electron  moves  to  a 
more  outward  shell,  timescale  <  femtoseconds.  (B)  Relaxation  to  a  lower  level, 
for  example,  ground  state  under  spontaneous  emission  of  a  photon,  electron 
returns  to  a  lower  level,  timescale  nanoseconds.  (C)  Light  driven  interaction 
inside  laser  gain  medium  of  a  photon  with  electrons  in  excited  metastable  state 
with  a  long  lifetime.  (D)  Stimulated  coherent  emission  and  light  amplification. 


A  3-STATE  LASER 


B  4-STATE  LASER 


FIGURE  5.3.  Fast  non-radiative  relaxation  processes  couple  an  excited  state 
with  a  lower  metastable  energy  level.  Laser  emission  occurs  in  the  gain  medium 
from  the  metastable  state.  Dependent  on  the  gain  medium  properties  this  laser 
process  is  described  by  a  three-level  or  four-level  lasing  scheme. 


So  far  we  have  only  discussed  a  three-level  laser,  that  is, 
ground-state,  upper  excited-state,  and  lower  excited-state  [Fig. 
5.3(A)].  In  a  four-level  laser  [Fig  5.3(B)],  as  in  the  helium-neon 
(He-Ne)  laser,  the  population  inversion  can  be  obtained  more 
easily  (Arecchi  and  Schultz-Dubois,  1972).  The  titanium-sapphire 
(Ti :  Sa)  vibronic  laser  is  effectively  also  a  four-level  laser  due  to 
its  broad  energy  bands  (see  later  section).  Other  improvements 
relate  to  the  replacement  of  the  Fabry-Perot  mirrors  by  corner  cube 
reflectors  or  crossed-roof  prisms  to  increase  mechanical  and 
thermal  stability.  For  further  information,  see  Arecchi  and  Schultz- 
Dubois,  1972;  Stitch,  1979;  Bertolotti,  1983;  Bass  and  Stitch, 
1985;  Kudryashov  and  Weber,  1999;  Webb  and  Jones,  2004; 
Hodgson  and  Weber,  2005. 

An  interesting  approach  is  the  increasing  use  of  optical  fibers 
that  act  as  both  lasing  medium  and  cavity  at  the  same  time;  a  very 
compact  design  is  described  later  in  this  chapter. 


Pumping  Power  Requirements 

In  order  to  sustain  laser  action,  the  gain  of  the  optical  resonator 
needs  to  be  larger  than  the  losses  due  to  resonator  walls  and  other 
optical  elements.  The  minimum  necessary  pumping  power,  P,  is 
proportional  to  v3.  This  means  that,  as  one  shifts  from  the  infrared 
(IR)  through  the  visible  (VIS)  towards  the  ultraviolet  (UV),  an  ever 
increasing  amount  of  energy  is  needed  to  obtain  laser  action.  This 
limits  the  possible  pumping  mechanisms. 


Laser  Modes;  Longitudinal  (Axial)  and  Transverse 

•  Axial  or  longitudinal  modes:  Separated  by  a  distance,  L,  the 
two  plane-parallel  mirrors  of  the  Fabry-Perot  interferometer 
cavity  form  an  optical  resonator.  This  separation  distance  can 
be  long  (meters,  as  in  big  frame  ion  lasers)  or  very  small 
(micrometers,  as  in  very  compact  diode  lasers).  A  number  of 
standing  wave  patterns  each  consisting  of  an  integer  multiple, 
m,  of  half  wavelengths,  M2,  exists  in  an  optical  cavity  of  length 
L:  m  =  L/(M2).  The  frequency,  v,  for  the  mth  vibration  along 
the  long  axis  of  the  laser  is,  therefore,  v  =  mc/2  L,  where  c  is 
the  speed  of  light  in  the  laser  cavity.  The  frequency  spacing 
between  adjacent  longitudinal  modes  is  c/2L,  that  is,  the 
inverse  of  the  laser  cavity  round-trip  time.  A  very  large  number 
of  longitudinal  modes  can  exist  in  the  laser  cavity  unless 
bandwidth-limiting  devices  such  as  Fabry-Perot  etalons  are 
installed  in  the  cavity. 
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•  Transverse  modes:  These  modes  vibrate  perpendicular  to  the 
long  axis  of  the  laser  and  are  known  as  transverse  electro¬ 
magnetic  modes,  TEM„„„  where  m  and  n  are  integers  describ¬ 
ing  the  nodal  points  in  the  two  directions  perpendicular  to  the 
laser  axis.  For  each  transverse  mode,  many  longitudinal  modes 
can  exist.  (For  an  in-depth  derivation,  see  laser  handbooks  by 
Arecchi  and  Schultz-Dubois,  1972;  Stitch,  1979;  Demtroder; 
1996;  Bass  and  Stitch;  1985;  Hecht  and  Zajac,  2003;  Silfvast, 
2004).  An  adjustable  diaphragm  or  pinhole  inside  the  laser 
cavity  is  sometimes  used  to  select  the  TEM00  lasing  mode  and 
control  its  intensity.  No  transverse  modes  exist  in  single  (lon¬ 
gitudinal)  mode  fiber  lasers. 

The  TEM0o  mode  is  desired  for  most  light  microscopy  exper¬ 
iments.  It  has  a  Gaussian  beam  profile  with  no  phase  shifts  across 
the  beam.  The  maximum  intensity  is  in  the  center  of  the  beam,  it 
has  complete  spatial  coherence,  and  the  smallest  possible  beam 
divergence.  Only  this  mode  can  be  focused  to  a  diffraction-limited 
spot.  Doughnut-shaped  transverse  modes  TEM.01  (TEM10  in 
overlap  with  TEM0i),  such  as  produced  by  some  helium-cadmium 
(He-Cd)  lasers  operating  at  325  nm,  are  not  desirable  because  they 
possess  no  intensity  in  the  center  of  the  beam.  In  this  case,  spatial 
filtering  is  necessary  (see  later  section). 

Polarization 

The  output  of  many  lasers  is  linearly  polarized  with  the  polariza¬ 
tion  vector  vertical. 

•  Lasers  with  Brewster  surfaces:  A  convenient  and  inexpen¬ 
sive  way  to  minimize  reflection  losses  and  to  generate  linearly 
polarized  light  is  the  installation  of  a  Brewster  surface  (i.e., 
one  that  is  tilted  so  that  the  normal  of  the  plane  and  the  incom¬ 
ing  beam  form  a  specific  angle,  0Brewster)  at  the  end  of  the  res¬ 
onator.  Horizontally  polarized  light  incident  on  such  a 
plane-parallel  plate  will  be  completely  reflected.  Vertically 
polarized  light  will  be  completely  transmitted  without  any 
reflection  losses  (Fig.  5.4).  In  laser  resonator  cavities,  this  plate 
is  usually  a  quartz  or  a  fused-silica  plate.  In  solid-state  lasers, 
the  semiconductor  rod  itself  is  sometimes  cut  at  the  Brewster 
angle  to  minimize  reflection  losses.  In  gas  lasers,  the  exit 
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windows  are  usually  oriented  at  the  Brewster  angle  to  obtain 
vertically  polarized  light.  Dust  often  collects  on  Brewster 
windows  that  point  upward,  damaging  the  coating.  To  operate 
properly,  they  must  be  kept  absolutely  clean  using  a  dust  cover 
and  a  slightly  positive  dry  air  or  nitrogen  pressure  in  the  laser 
head.  In  the  jet-stream  dye  laser,  the  jet  is  placed  at  Brewster 
angle  in  order  to  minimize  losses  due  to  reflection  of  the 
pumping  beam,  thereby  maximizing  the  pumping  efficiency. 
For  the  same  reason,  the  tuning  element  of  the  dye  laser  is  also 
placed  at  Brewster  angle.  Brewster  surfaces  can  also  be  the 
origin  of  very  dangerous  reflections.  Eye  protection  should 
always  be  worn  when  protective  covers  are  removed  from  a 
laser. 

•  Lasers  built  without  Brewster  windows  will  still  show  some 
polarized  output  due  to  birefringence  within  several  optical 
components.  However,  the  plane  of  polarization  may  change 
with  time  and,  even  though  the  total  intensity  stays  constant, 
should  this  light  pass  through  a  polarizer,  large  output  inten¬ 
sity  fluctuations  may  be  observed. 

Randomly  polarized  beams  contain  two  orthogonal  compo¬ 
nents  but  the  phase  and  ratio  of  the  two  components  vary  rapidly 
in  time,  creating  polarization  noise.  Dichroic  mirrors,  and,  in  fact, 
any  mirror  or  lens  surface,  not  at  right  angles  with  the  incoming 
radiation  will  reflect  vertically  and  horizontally  polarized  light  dif¬ 
ferently  (directions  are  taken  with  respect  to  the  plane  of  incidence 
formed  by  the  reflected  and  transmitted  beams). 

A  convenient  way  to  depolarize  laser  emission  is  to  install  a 
polarizer  or  polarizing  beam-splitter  and  a  1/4-wave  plate.  When 
placed  at  an  angle  of  45°  with  respect  to  the  incoming  linear  polar¬ 
ization,  this  1/4-wave  plate  converts  linearly  polarized  light  into 
circularly  polarized  light.  These  waveplates  are  usually  designed 
for  a  specific  wavelength.  Achromatic  retarders,  such  as  Fresnel 
rhombs,  are  sometimes  preferred,  but  are  quite  expensive  (Driscoll 
and  Vaughan,  1977;  Hecht  and  Zajac,  2003).  Another  advantage 
of  this  arrangement  in  optical  microscopy  is  that  it  prevents 
reflected  or  backscattered  light  from  reaching  the  detector.  The 
reflected  light  passes  through  the  1/4-wave  plate  in  such  a  way  that, 
on  return,  it  is  blocked  by  the  input  polarizer  because  its  plane 
polarization  is  now  orthogonal  to  the  main  transmission  direction 
of  the  input  polarizer.  Phase  randomizing  will  briefly  be  discussed 
in  a  later  section.  For  an  in-depth  treatment  see  Chapter  31  by 
Harris  and  Delaney.  In  fiber  lasers,  optical  isolators  are  inserted 
for  the  same  reason,  that  is,  to  prevent  backreflections.  The 
polarization-independent  isolator  consists  of  a  small  cylindrical 
package  containing  a  fiber  pigtail,  non-spherical  collimating 
lenslet,  a  slightly  wedged  birefringent  platelet,  a  magneto-optic 
Faraday  rotator  at  45°,  exit  birefringent  platelet  also  at  45°,  and 
refocusing  lenslet  followed  by  another  fiber  pigtail. 


Coherent  Properties  of  Laser  Light 


FIGURE  5.4.  Emission  of  linearly  (vertically)  polarized  laser  light  from  an 
ion  laser  resonator  cavity  equipped  with  Brewster  angle  plasma-tube  windows. 
Only  vertically  polarized  light  is  amplihed.  It  experiences  no  reflection  losses 
at  the  Brewster  angle:  J5Brewster  =  tan-1rc,  with  n  the  refractive  index  of  the  pure 
crystalline  quartz  window.  Horizontally  polarized  light  suffers  reflection  losses 
and  stays  below  the  lasing  threshold.  Windows  normal  to  the  optical  axis  of 
the  laser  would  introduce  reflection  losses  and  could  prevent  the  laser  from 
operating  at  ah  or  introduce  a  set  of  reduced  length  resonator  cavities.  Tung¬ 
sten  disks  or  BeO  tubes  with  a  central  bore  with  or  without  focusing  magnetic 
fields  keep  the  lasing  plasma  volume  centered  in  the  cavity.  Gas  return  paths 
are  also  provided  in  the  disks.  HR,  highly  reflecting  mirror;  OC,  output  cou¬ 
pling  mirror. 


Laser  beams  illuminate  objects  coherently.  The  process  of 
stimulated  emission  imposes  coherence  effects  on  the  emitted  laser 
light  waves.  All  parts  of  the  electromagnetic  wave  reach  a  crest, 
that  is,  are  in  phase,  at  a  given  point  in  space  at  the  same  time. 
Spatial  and  temporal  coherence  is  present.  This  coherence 
stretches  for  a  certain  distance  and  time  and  depends  on  the  spec¬ 
tral  width  or  pulse  duration  of  the  laser  light.  Beyond  a  certain  dis¬ 
tance  or  time  interval  synchronization  differences  arise.  When  a 
choice  exists,  a  short  coherence  length  laser  should  be  used  for 
CLSM  or  scrambling  devices  should  be  inserted  into  the  optical 
path. 
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•  Temporal  coherence:  The  coherence  time  is  the  time  interval 
during  which  light  traveling  over  a  certain  distance  maintains 
a  phase  difference  of  less  than  n  (or  180°),  Tcoh.  =  1/Av. 

•  Coherence  length:  The  path  traveled  by  the  wave  during  the 
coherence  time  is  called  the  coherence  length,  Lc~  c/Aco,  where 
Atn  is  the  spectral  width  of  the  laser.  Longitudinal  coherence 
length  Lcoh  =  c/Av.  Lateral  coherence  length,  Zcoh  =  /i/(|),  with  § 
the  beam  divergence.  For  a  typical  gas  laser,  for  example,  a 
He-Ne  laser  with  a  bandwidth  of  1.5  GHz,  leads  to  Lcoh  of 
~20  cm.  In  single-mode  operation  with  Ato  ~  1  MHz,  Lcoh  is 
~50m.  A  diode  laser  with  a  spectral  width  of  0.1  nm  at  780  nm 
possesses  a  coherence  length  of  1  mm.  Dye  lasers  equipped 
with  tuning  elements,  diode  lasers,  and  other  lasers  such  as  the 
Ti :  Sa  systems,  usually  have  bandwidths  on  the  order  of  tens 
of  gigahertz  and  an  Lcoh  of  a  few  millimeters  or  less.  Depend¬ 
ing  on  the  type  of  line  narrowing  element  installed,  Lcoh  can 
be  increased  by  a  factor  of  103  to  106.  For  practical  confocal 
microscopy,  a  shorter  coherence  length  is  preferred  to  elimi¬ 
nate  the  influence  from  out-of-focus  defects.  For  example,  dust 
on  semi-transparent  surfaces,  lenses  and  mirrors  creates  inter¬ 
ference  fringes  when  a  laser  with  long  coherence  length  is  used 
(Hell  et  al.,  1991).  (See  Tables  6.1  and  6.2  which  contain 
columns  with  spectral  width  and  pulse  length,  from  which  the 
coherence  length  follows.) 

•  Spatial  coherence:  Spatial  coherence  occurs  when  a  constant, 
time-independent,  phase  difference  exists  for  the  optical  field 
amplitude  over  a  given  distance  at  two  different  points  in 
space. 

•  Coherence  surface:  The  coherence  surface  is  the  region  of 
space  for  which  the  absolute  phase  difference  of  the  optical 
field  is  less  than  n.  The  well-known  “speckle”  pattern  consist¬ 
ing  of  a  pattern  of  darker  and  brighter  light  spots  is  visible 
when  a  matte  wall  surface  is  illuminated  by  a  distant  He-Ne 
laser.  The  interference  effects  caused  by  irregularities  in  the 
surface  will  be  noticeable. 

•  Coherence  volume:  The  coherence  volume  is  the  product  of 
coherence  length  and  the  coherence  surface.  Interference 
between  superimposed  coherent  waves  will  only  be  visible 
within  this  volume. 


Phase  Randomization:  Scrambling 
the  Coherence  Properties  of  Laser  Light 

A  long  coherence  length  of  lasers  will  cause  laser  speckles  (i.e., 
interference  effects)  and  scatter  from  out-of-focus  particles  and 
these  will  interfere  with  the  image.  Normally  in  a  fluorescence 
CLSM,  this  scattered  light  is  removed  by  the  dichroic  mirror  as 
the  emitted  fluorescence  has  a  different  wavelength  from  the  illu¬ 
minating  light  source.  However,  a  polarizing  beam-splitter  and  1/4- 
wave  retardation  plate  can  be  equally  effective  for  the  removal  of 
light  backscattered  by  dust  or  optical  surfaces.  The  angular  posi¬ 
tion  of  this  1/4-wave  plate  and  the  polarizer  is  important 
(Szarowski  et  al.,  1992).  For  generating  circularly  polarized  light, 
it  is  only  necessary  that  the  incoming  plane  of  polarization  and  the 
principal  axis  of  the  1/4-wave  plate  be  at  a  45°  angle.  Light 
reflected  or  scattered  from  an  illuminated  dielectric  surface  has  its 
direction  of  rotation  reversed.  On  its  return  passage  through  the 
plate,  the  light  becomes  linearly  polarized  again,  but  its  plane  of 
polarization  is  now  rotated  by  90°  and  it  is  removed  from  the 
optical  path  by  the  polarizing  beam-splitter. 

Another  method  to  reduce  speckle  comes  from  color  confocal 
microscopy  where  one  uses  several  lasers.  As  the  coherence  prop¬ 


erties  of  the  various  laser  light  sources  are  not  the  same,  averag¬ 
ing  over  the  three  wavelengths  and  over  several  planes  will  reduce 
the  effect  of  the  laser  speckle  (Cogwell  et  al.,  1992b). 

Measures  to  Reduce  the  Coherence  Length  of 
Laser  Light 

Many  applications  work  better  and  image  quality  improves  if  the 
coherence  of  the  laser  light  is  reduced.  One  way  to  achieve  this  is 
to  place  a  rotating  diffuser  wedge  in  front  of  the  beam  expander 
(Hard  et  al.,  1977).  Rotation  or  vibration  of  this  wedge  is  essen¬ 
tial  to  average  out  local  diffuser  properties.  Alternatively,  focusing 
the  laser  beam  into  a  multi-mode  optical  fiber  bounces  the  light 
around  inside  the  fiber  and  mixes  the  various  propagation  modes 
to  scramble  the  Gaussian  intensity  pattern  (TEM00)  to  a  multi- 
mode  pattern  with  a  homogeneous  spot  intensity.  This  eliminates 
or  reduces  the  coherence  surface.  A  short  focal  length  lens  or 
graded  index  lens  (GRIN)  recollimates  the  light  at  the  fiber 
exit. 

To  eliminate  the  speckle  pattern  due  to  temporal  coherence 
properties,  a  piezoelectric  driver  can  be  used  to  induce  vibrations 
in  a  section  of  the  fiber  (Ellis,  1979).  In  the  context  of  medical 
imaging,  Connor,  Davenport  and  Gmitro  (1992)  used  a  rotating 
diffuser  to  eliminate  the  speckle  pattern  by  coupling  the  laser  beam 
into  a  bundle  of  small,  multi-mode  fibers.  The  latest  method  in  this 
area  is  the  introduction  of  ultra-thin  single-mode  image  fibers.  A 
fiber-optic  bundle  with  a  total  diameter  of  200  to  500  pm  consist¬ 
ing  of  2000  to  5000  micro  image  fibers,  each  with  a  core  size  of 
1.0  to  1.4  pm  (Kiat  et  al.,  1992),  can  be  used  as  a  phase  random¬ 
izer  in  conjunction  with  a  rotating  diffuser.  Although  these  ultra- 
thin  fiber  bundles  propagate  shorter  wavelengths  much  more 
strongly  than  longer,  leaky  ones,  this  is  not  a  problem  as  long  as 
monochromatic  laser  light  is  used. 

As  light  is  scattered  in  all  directions  by  most  diffusers,  they 
reduce  the  intensity  of  the  laser  light.  However,  holographic  dif¬ 
fusers  have  been  designed  to  carry  strong  forward  intensity  lobes. 
Among  others,  Lasiris,  Inc.  and  Physical  Optics  Corp.  (see  Table 
5.3)  produce  holographic  laser  beam  homogenizers,  having  a 
conversion  efficiency  of  80%  to  90%  and  low  backscatter  (Bains, 
1993). 

Laser  coherence  effects  can  also  be  eliminated  by  spot  scan¬ 
ning  the  field  of  view  with  a  tiny,  focused,  single-mode  laser  beam. 
Alternatively,  the  condenser  plane  can  be  scanned  with  the  same 
tiny,  focused,  single-mode  laser  beam  while  illuminating  the  whole 
field  of  view.  Coherence  scrambling  occurs  because  the  laser  spot 
rapidly  scans  the  back-focal  plane  and  the  field  is  continuously 
illuminated  with  a  mix  of  beam  angles  (Ellis,  1988). 

Heat  Removal 

Most  of  the  laser  excitation  energy  is  converted  into  heat.  This 
must  be  removed  to  prevent  thermal  destruction  of  the  active  laser 
medium.  Small  laser  systems  can  use  convective  air  cooling,  but 
larger  laser  systems  need  forced-air  or  water  cooling.  Especially 
for  the  largest  systems,  fans  and  turbulent  water-flows  may  intro¬ 
duce  vibrations  in  the  system  (microphonics).  These  unwanted 
mechanical  vibrations  are  inevitably  coupled  to  the  mirror  mounts 
of  the  resonator  cavity,  causing  increased  noise  in  the  optical 
output.  Pumps  that  recycle  liquid  laser-media  are  another  source 
for  vibration.  In  this  case,  the  vibration  is  transferred  via  hoses  to 
the  active  medium  in  the  resonator.  To  minimize  these  effects,  the 
hoses  should  be  clamped  or  fastened  to  the  support  table  as  near 
as  possible  to  the  laser  head. 
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The  heat  generated  will  also  put  thermal  stress  on  the  mechan¬ 
ical  parts  of  the  resonator.  Poorly  designed  laser  systems  quickly 
lose  their  proper  alignment  or  may  need  an  unacceptably  long 
warm-up  time.  The  installation  of  a  laser  system  in  a  room  with 
large  daily  or  annual  temperature  fluctuations  or  a  ceiling  fan  con¬ 
tinuously  blowing  cold,  dusty,  air  directly  onto  a  laser  will  also 
hamper  operation  and  cause  performance  to  deteriorate.  The  use 
of  thermo-electrically  (TEC)  cooled  diode  lasers  that  are  in 
thermal  contact  with  their  enclosures  may  create  hot  surfaces 
that  should  be  labeled  as  such.  Diode  arrays  may  require  water 
cooling.  Because  of  their  monolithic  design,  they  are  less  suscep¬ 
tible  to  mechanical  instabilities  but  thermal  stress  has  to  be  avoided 
at  all  cost.  The  same  is  true  for  fiber  lasers,  which,  due  to  the  fiber 
length,  are  sensitive  to  temperature  fluctuations  unless  counter¬ 
measures  such  as  negeative  temperature  coefficient  materials  are 
used. 

Other  Installation  Requirements 

Manufacturers  usually  describe  the  electrical  power  requirements 
and  flow  rates  for  cooling  water  (Gibson,  1988,  1989;  Rapp,  1988). 
Before  installation  of  a  laser  system,  adequate  control  of  room  tem¬ 
perature  and  air  conditioning  should  be  available.  Walls  should  be 
painted  to  eliminate  concrete  dust  from  settling  on  delicate  optical 
surfaces.  For  emergency  situations,  a  master  switch  for  the  laser 
power  should  be  installed  with  easy  access  at  eye  height  and 
labeled  as  such.  Heat  from  large  fan-cooled  laser  power  supplies 
can  sometimes  be  plumbed  directly  into  the  intake  of  the  air  con¬ 
ditioning  system,  with  backflow  protection,  if  this  is  allowed.  Sim¬ 
ilarly,  acoustic  noise  can  be  very  tiring  and  noisy  power  supplies 
should  be  placed  outside  the  experiment  room. 

An  exhaust  for  noxious  fumes  such  as  ozone  or  dye  vapor 
should  exist  and  backup  systems  should  be  established  to  prevent 
interruptions  of  power  or  coolant  flow.  Mechanical  vibrations  due 
to  nearby  traffic  can  be  eliminated  by  installing  the  system  on  com¬ 
mercially  available  vibration-free,  air-cushioned  laser  tables  or  on 
tables  isolated  using  sand-filled  containers,  tennis  balls,  or  motor¬ 
cycle  inner  tubes.  When  equipment  used  for  radio-frequency 
crystal-growth,  nuclear  magnetic  resonance  (NMR)  equipment, 
building  air-conditioning  machines,  large  elevators,  or  other  large 
laser  systems  are  present,  it  is  often  wise  to  equip  each  laser  with 
its  own  stabilized  power  line  to  prevent  large  voltage  spikes  from 
reaching  laser  power  supplies.  Hidden  cables  drawing  a  large 
current  (for  elevators,  for  example)  in  ducts  near  your  facility  may 
ruin  a  carefully  planned  sensitive  system  by  disruptive  induction 
currents.  Stray  magnetic  fields  may  affect  the  flow  of  ions  in  gas 
lasers. 

Movement  of  fiber-optic  components  changes  the  beam  prop¬ 
agation  properties  of  the  fiber  (bundle)  and  should  be  prevented 
for  best  stability  (unless  the  device  is  used  intentionally  to 
scramble  the  laser  light  polarization  and  reduce  the  coherence 
length). 

Attenuation  of  Laser  Beams 

Laser  damage  levels  for  materials  are  usually  given  in  mW/cm2 
while  laser  output  is  given  as  total  average  power  (mW)  for  a 
small,  for  example,  2-mm  diameter  beam.  Although  intensities  in 
the  milliwatt  range  can  be  attenuated  with  neutral  density  filters, 
high-power  lasers  will  easily  destroy  this  type  of  filter.  The 
absorbed  light  may  even  bleach  or  heat  them  so  much  that  they 
fracture  or  explode.  Polaroid  material  can  be  used  only  for  low 
light  intensities  but  this  material  usually  has  a  peak  transmission 


of  only  30%  to  50%.  For  intensities  above  about  10 mW,  better 
attenuators  are  Glan-Thompson  polarizers.  When  a  laser  emits 
only  polarized  light,  one  rotatable  polarizer,  set  at  a  proper  angle 
will  suffice.  For  randomly  polarized  lasers  a  set  of  two  polarizers 
can  be  used.  The  first  one  passes  50%  of  the  total  intensity.  The 
second  polarizing  analyzer  transmits  a  continuously  adjustable 
amount  of  light  depending  on  its  orientation  angle.  In  all  these 
cases  one  must  be  careful  to  ensure  that  light  reflected  from  the 
various  crystal  surfaces  is  also  absorbed  by  beam  stops.  Initial 
alignment  should  be  done  with  only  the  lowest  laser  power  avail¬ 
able.  If  a  polarizer  is  used  to  attenuate  an  initially  randomly 
polarized  laser  beam,  fluctuations  in  the  intensity  may  be  observed. 
These  are  due  to  the  varying  nature  of  the  interaction  between 
the  cavity  modes  and  therefore  the  polarization  state  inside  the 
laser. 

Glan-Thompson  polarizers  contain  Canada  balsam  to  hold 
them  together.  This  material  may  absorb  slightly,  especially  in  the 
UV.  Power  levels  above  several  watts  (power  =  energy/time  =  area 
of  pulse)  will  destroy  this  type  of  polarizer.  For  the  highest 
expected  power  levels,  air-spaced  Glan-Taylor  polarizers  are 
strongly  recommended.  A  new  type  of  polarizing  element  is 
the  Microwire  polarizer  commercialized  by  Moxtek  Inc.  (see 
Table  5.3).  It  fits  in  very  tight  spaces  but  is  somewhat  more 
fragile. 


STABILIZATION  OF  INTENSITY,  WAVELENGTH, 
AND  BEAM  POSITION  IN  LASERS 

Sources  of  Noise  in  Lasers 

Stability,  both  temporal  and  positional  (beam  pointing),  is  a  very 
important  aspect  of  any  CLSM  laser  light  source.  As  beam  point¬ 
ing  affects  the  amount  of  light  coupled  into  the  delivery  fiber, 
pointing  instability  can  cause  intensity  instability.  Other  intensity 
instabilities  are  related  to  backreflections  from  the  fiber  that  can 
be  avoided  (Cog well  et  al.,  1992a)  by  employing  a  wedged  fiber 
input  (e.g.,  Point  Source,  see  Table  5.3).  Figure  5.5  gives  examples 
of  noise  in  the  output  of  several  laser  systems.  The  most  stable  are 
the  diode  laser  and  the  forced-air  cooled  (fan)  small-frame 
argon-ion  laser.  The  argon-ion  laser-pumped  titanium :  sapphire 
laser  output  was  coupled  into  a  fiber-optic  attached  to  the  detec¬ 
tor.  The  average  relative  intensity  fluctuations  are  defined  as 
N~l  £i(l/j  -  (7)l/(7)),  where  N  is  the  number  of  points.  For  the  diode 
laser  this  quantity  is  about  0.1%  (Franceschini  et  al.,  1994),  and 
for  the  Ti :  Sapphire  laser  at  the  fiber  output,  about  2%.  Both  would 
be  acceptable  for  microcopical  applications  in  which  fewer  than 
50  photons  were  detected  per  pixel. 

It  is  imperative  to  keep  lasers  well-aligned  with  very  clean 
windows.  Misaligned  mirrors,  a  half-illuminated  reference  monitor 
diode,  or  a  dust  speck  may  create  a  very  unstable  output.  As  mis¬ 
alignment  can  cause  unusual  modes  to  appear,  these  laser  lines  will 
fluctuate  independently,  and  the  total  output  becomes  unstable.  For 
instance,  the  two  UV  lines  of  an  old  water-cooled  argon-ion  laser 
were  observed  to  alternate  between  bright  and  dim  with  a  period 
of  a  few  seconds  to  a  minute.  Realignment  helped  but  the  tube  had 
to  be  replaced.  Transverse  mode  swapping  is  seen  in  He-Cd  tubes 
that  need  replacement. 

Noise  in  the  optical  output  of  a  laser  can  be  created  by  a  variety 
of  phenomena.  Several  factors  produce  high-frequency  (noise)  or 
slow  variations  (drift)  in  beam  power.  These  include  power-supply 
variations,  thermal  drift  of  the  cavity,  and  mode  competition. 
Examples  include: 
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FIGURE  5.5.  Measured  laser  emission  stability  (arbitrary  units).  Normalized 
and  shifted  intensities  for  (A)  a  diode  laser  (Sony  SLD104AV,  /.  =  780  nrn;  top 
curve);  (B)  a  forced-air  cooled  small  frame  argon-ion  laser.  Omnichrome  model 
532,  running  at  488  nm  and  7mW  power  ( center  curve);  (C)  Mira  (Coherent) 
Ti:Sa  laser  running  at  780  nm  and  1.2  W  ( bottom  curve).  The  normalized  noise 
for  the  diode  laser  is  0.1%  and  for  the  Ti:Sa  laser  about  2%.  The  attenuated 
Ti:Sa  laser  was  focused  into  a  fiber-optic. 


•  Dye  lasers:  Noise  and  drift  in  dye  lasers  can  be  caused  by  air 
bubbles  and  inhomogeneity  in  the  dye,  the  dye  pump,  and  by 
the  laser  pump  source. 

•  Gas  lasers:  A  major  source  of  noise  comes  from  plasma  oscil¬ 
lations  and  from  microphonics  generated  by  cooling  water 
turbulence.  Forced-air  cooling  for  smaller  lasers  can  avoid 
most  microphonics,  but  the  fan  may  introduce  some  new  noise. 
Commercial  confocals  often  mount  the  lasers  on  a  heavy  stone 
slab  (Zeiss)  or  place  them  in  a  separate,  shielded  drawer 
(Leica).  Also  any  optical  pump  source  can  introduce  noise. 

•  He-Cd  lasers:  Strong  plasma  oscillation  between  225  and  400 
kHz  may  create  an  output  variation  of  up  to  12%. 

•  Semiconductor  or  diode  lasers:  Semiconductor  laser  diodes 
require  a  current  source  with  the  highest  stability  and  the 
lowest  electronic  noise  (Malzahn,  2004).  Furthermore,  tem¬ 
perature  stability  is  imperative  for  adequate  intensity  and 
wavelength  stability  (Hodgson,  1994).  Forward-biased  semi¬ 
conductor  junctions  are  inherently  thermally  unstable;  an 
increase  in  current  increases  the  temperature,  which,  in  turn, 
increases  the  current  even  more.  Therefore,  current  supply 
design  is  very  important  and  must  be  suited  to  the  specific 
laser.  Internal  damage  in  laser  diode  junctions  due  to  heat 
buildup  ages  the  device,  resulting  in  a  reduced  output  and  slope 
efficiency.  1/f  noise  is  due  to  trapping  of  carriers  in  the  device 
(Mooradian,  1993).  Drive-current  fluctuations  also  degrade  the 
performance  and  will  modulate  the  beam  intensity.  Mistreated 
fiber-optics  can  also  increase  the  amount  of  noise.  On  the  other 
hand,  an  imposed  high  frequency  (MHz)  modulation  allows 
the  laser  to  be  used  for  time-resolved  and  frequency-domain 
imaging  applications. 


•  Solid-state  lasers:  The  important  noise  sources  are  micro- 
phonic  noise  and  1/f  noise  related  to  thermal  fluctuations  in  the 
lasing  rod.  Furthermore  the  Ti :  Sa  rod  temperature  bath  should 
be  set  at  the  correct  temperature,  for  example,  18°C.  Failing 
to  do  so  creates  condensation  droplets  on  the  pump 
surface.  Fiber  lasers  should  be  kept  in  a  stable  temperature 
environment. 

All  lasers  suffer  from  noise  introduced  by  their  power  supplies. 
Switching-mode  power  supplies  introduce  switching  ripple,  typi¬ 
cally  at  tens  of  kilohertz.  This  type  of  power  supply  was  devel¬ 
oped  to  take  advantage  of  better  transformer  efficiencies  and 
smaller  physical  sizes  (Forrester,  1994a, b). 

Planar  optical  elements  and  filters  should  be  inserted  at  a  small 
angle  with  respect  to  the  optical  axis  to  prevent  beam  reflections 
from  returning  to  the  laser  cavity  where  they  might  increase  the 
noise  level  of  the  laser  (Cogswell  et  al.,  1992a). 

Images  created  on  commercial  CLSM  instruments  may  show 
a  fine  pattern  of  irregular  vertical  lines.  Possible  sources  include 
resonances  in  the  scanning  mirror  systems  and  high  frequency 
ripple  in  the  switching  power  supplies.  However,  sometimes  the 
problem  is  not  the  light  source  but  the  detector  system  (French  and 
Gratton,  personal  communication).  Last  but  not  least,  the  main 
laboratory  power  supply  lines  may  be  unstable. 

Other  sources  of  noise  are  caused  by  external  influences:  traffic 
vibrations,  etc.  A  simple  means  to  reduce  its  effect  is  a  well- 
damped,  stable  support  platform  (see  above). 

Laser  Beam  Intensity  Stabilization  in 
Current-  or  Power-Control  Mode 

Continuous  wave  gas  lasers  can  easily  be  intensity  stabilized  using 
either  tube  current  stabilization  or  external  modulation  of  the  light 
intensity  (Miller  and  Hoyt,  1986). 

•  Constant-current  mode:  An  electronic  feedback  loop  directly 
controls  the  tube  current  and  minimizes  current  drift.  As  this 
system  takes  no  account  of  how  temperature  may  affect  cavity 
gain,  best  results  are  obtained  after  the  laser  temperature  sta¬ 
bilizes. 

•  Constant  output  power  mode:  The  drive  current  is  controlled 
via  a  signal  obtained  from  a  built-in  monitor  diode.  The 
stabilization  circuit  typically  consists  of  a  beam-splitter- 
photodiode  assembly  mounted  behind  the  output  coupler.  The 
monitor  diode  picks  off  a  small  amount  of  laser  intensity. 
Changes  in  intensity  are  compensated  for  by  opposing  changes 
in  the  drive  current.  When  not  properly  aligned,  large  intensity 
fluctuations  may  result.  Multi-line  lasing  depends  on  an  inter¬ 
play  of  different  gain  and  mode  patterns  especially  when  the 
bore  holes  in  the  internal  BeO  laser  disks  become  larger 
with  time  (Fig.  5.4).  A  constant  output  power  supply  in  com¬ 
bination  with  an  aging  plasma  tube  will  increase  in  tube 
voltage  and  current  supplied,  damaging  the  tube  further  and 
even  more  rapidly.  Both  effects  are  even  stronger  in  mixed  gas 
lasers. 

Even  if  the  laser  beam  is  stable,  external  noise  can  be  intro¬ 
duced  to  the  system  by  room  dust,  traffic  vibrations,  or  movement 
of  fiber-optic  components  and  may  still  adversely  affect  intensity 
stability.  The  long-term  stability  of  the  largest  argon-ion  models  is 
better  than  1%  in  this  control  mode.  But  this  light  control  mode 
does  not  work  so  well  when  multiple  emission  lines  are  monitored 
simultaneously  (Brelje  et  al.,  1993). 


Laser  Sources  for  Confocal  Microscopy  •  Chapter  5  87 


Laser  Beam  Intensity  Stabilization  with  External 
Pockels  Cell  Modulator 

Some  manufacturers  provide  special  accessories  to  improve  the 
intensity  stability  of  CW  lasers.  These  devices  are  external  modu¬ 
lators  incorporating  a  fast  feedback  system  (Miller  and  Hoyt,  1986; 
Miller,  1991;  see  Fig.  5.6),  and,  of  course,  they  reduce  the  total 
output  power.  Intensity  fluctuations  of  up  to  50%  of  the  maximum 
power  can  be  corrected  in  this  way,  but  only  with  a  50%  reduction 
in  the  total  available  power.  Laser  intensity  stabilizers  can  regu¬ 
late  the  output  of  CW  and  mode-locked  lasers  to  within  0.025%, 
with  a  noise  attenuation  of  400 :  1  and  a  bandwidth  from  direct 
current  (DC)  to  several  megahertz  in  some  systems. 

Depending  on  the  polarization  state  of  the  laser  light,  up  to 
80%  transmission  is  obtained.  In  this  system  (Fig.  5.6),  a  photo¬ 
diode  is  illuminated  with  a  fraction  of  the  laser  light  deflected  by 
a  beam-splitter.  A  servo-control  unit  compares  the  detector  signal 
with  a  user-selected  set  point.  The  difference  signal  is  amplified 
and  drives  an  electro-optic  (Pockels  cell)  modulator.  The  ampli¬ 
fied  electrical  signal  causes  a  rotation  of  the  plane  of  polarization. 
Depending  on  the  voltage  applied,  more  or  less  light  can  pass 
through  the  modulator  creating  a  variable  beam  attenuator.  Ideally 
the  laser  beam  should  be  stabilized  as  close  as  possible  to  the 
sample  position  rather  than  at  the  exit  of  the  laser  head.  Otherwise, 
vibrations  or  dust  may  alter  the  stability  of  the  beam  after  it  leaves 
the  source. 

Stabilization  units  should  be  able  to  operate  at  a  variety  of 
wavelengths  with  UV  capability  when  necessary.  They  should  be 
protected  against  driver  signals  beyond  a  certain  maximum,  so  that 
a  runaway  situation  does  not  occur,  for  example,  if  the  beam  is 
blocked  or  temporarily  interrupted.  When  the  stabilizer  is  driven 
by  an  external  control  signal,  complex  intensity  sequences  may 
illuminate  the  sample.  Commercial  laser  power  controller  (LPC) 
stabilizers  can  be  obtained  from  BEOC  and  several  other  manu¬ 
facturers  (see  Table  5.3). 

When  randomly  polarized  laser  light  has  to  be  stabilized,  the 
external  Pockels  cell  modulator  must  be  placed  between  two 
crossed  polarizers.  Unfortunately,  such  a  modulator  cannot  work 
with  low-repetition-rate  pulsed  lasers.  Another  simple  way  to 
remove  laser  noise  is  similar  to  that  typically  used  in  steady-state 
fluorescence  spectroscopy.  The  effect  of  source  fluctuations  is  can¬ 
celled  by  dividing  the  detected  signal  from  the  sample  by  a  signal 
derived  from  the  laser  sampling  monitor  detector. 


LASER  HEAD  ELECTRO-  POLARIZER  BEAM 

OPTIC  SPLITTER 

MODULATOR 

FIGURE  5.6.  Example  of  a  laser  beam  intensity  stabilizer  and  laser  noise  sup¬ 
pressor.  A  beam-splitter  delivers  a  fraction  of  the  laser  output  beam  to  a  fast 
photodiode  detector.  Electronic  feedback  loops  for  low-  and  high-frequency 
signals  control  the  electro-optic  modulator  (EOM)  intensity  regulator.  The 
rotated  plane  of  polarization  exiting  from  the  EOM  passes  through  an  angle- 
adjustable  Glan-Taylor  polarizer.  It  defines  a  single  plane  of  linear  polariza¬ 
tion.  A  detected  intensity  fluctuation  alters  the  drive  voltage  to  the  crystal. 


Diode  Laser  Intensity  and  Wavelength 
Stabilization 

Incorporating  a  Fiber  Bragg  Reflector  (FBR)  or  Grating  (FBG) 
in  a  (pump)  diode  laser  stabilizes  wavelength,  improves  coupling 
efficiency,  reduces  unwanted  reflections  and  the  effects  of 
temperature  and  drive  current  fluctuations,  as  well  as  simplifying 
manufacturing  (Guy  and  Painchaud,  2004).  The  FBR  consists  of  a 
stack  of  thin  coating  layers  forming  a  bandpass  reflectivity  filter  at 
either  the  high  reflector  or  output  coupler  end.  Even  simpler  is  lens 
coupling  the  diode-laser  output  into  a  section  of  FBG  fiber  pigtail. 
A  periodic  or  non-periodic  (chirped)  refractive  index  variation  of 
the  Bragg  grating  can  be  present  in  its  core,  creating  a  narrow  band 
low-reflectivity  filter  that  reflects  a  small  fraction  of  the  light  from 
the  diode  laser  back  into  it.  However,  due  to  interference  effects, 
only  a  certain  wavelength  —  the  designed  lasing  wavelength  —  is 
actually  fed  back  into  the  laser  diode.  All  other  wavelengths  are  out 
of  phase  and  die  off.  In  this  way,  the  attachment  of  the  proper  FBG 
fiber  pigtail  creates  a  diode  laser  that  is  stabilized  in  wavelength 
and  intensity.  Naturally,  if  the  FBG  temperature  or  internal  strain 
changes,  the  lasing  wavelength  will  vary  slightly.  A  typical  tem¬ 
perature  coefficient  is  0.012nm/°C.  A  temperature  increase  from 
18°C  to  40°C  will  cause  the  lasing  wavelength  to  shift  0.26  nm, 
but  in  practice  it  may  well  be  5-fold  larger.  With  temperature  com¬ 
pensating  packaging  and  a  strain-inducing  material  with  a  negative 
temperature  coefficient,  this  effect  can  be  reduced  to  0.001  nm/°C. 

Beam  Pointing  Improvement  Via  Active  Laser 
Cavity  Stabilization 

The  mirror  orientation  changes  when  the  laser  cavity  warms  up. 
This  causes  the  laser  beam  to  wander  and  the  beam  intensity  to 
vary.  Active  cavity  stabilization  corrects  for  misalignment  of  the 
resonator  via  a  feedback  mechanism  (Peuse,  1988).  The  advantage 
is  that  the  resonator  structure  becomes  independent  of  changes  in 
the  environment,  for  example,  temperature,  and  as  a  result  pro¬ 
vides  an  extremely  short  warm-up  time  (seconds),  enabling  hands- 
off  operation.  In  addition,  there  is  less  chance  that  setting  the 
power  supply  for  a  constant  optical  output  will  provide  damaging 
power  levels  to  a  tube  that  has  low  gain  because  it  is  misaligned. 
Large  frame  ion  lasers  currently  come  standard  with  active  res¬ 
onator  stabilization  systems  such  as  PowerTrack,  ModeTrack  and 
ModeTune  (Coherent)  or  BeamLok  Z-Lok  (for  mode-hop-free 
single-frequency  operation)  and  J-Lok  (for  jitter  reduction;  Spectra 
Physics).  These  systems  result  in  1-min  warm-up  times,  hands-off 
operation,  and  long-term  intensity  stability  of  better  than  1%. 
Beam-pointing  stability  is  obtained  using  a  quadrant  photodiode 
detector. 

The  horizontal  and  vertical  positions  of  the  high  reflecting 
mirror  are  continuously  tuned  for  optimum  laser  power  with  inde¬ 
pendent  (magnetic  or  piezo)  actuators.  A  small  oscillation  (dither 
signal)  is  superimposed  on  both  actuators.  A  sensor  photodiode 
signal  reflects  the  power  level  of  the  beam.  The  filtered  dither 
signal  (lock-in  detection)  creates  vertical  and  horizontal  error 
signals.  This  oscillation  is  compared  with  a  microprocessor-stored 
reference.  The  horizontal  and  vertical  difference  signals  drive  the 
mirror  positions  independently  to  obtain  maximum  power.  The 
minimum  detected  oscillation  (dither)  signal  corresponds  with 
optimum  alignment  (Miller,  1991). 

Beam  Delivery  and  Positioning 
Fiber-optic  Coupling 

Continous  wave  laser  beams  can  be  delivered  to  the  scan  head  or 
microscope  frame  via  single-mode,  polarization-preserving,  dual- 
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circular  stress-rod  design,  fiber-optic  (the  so-called  “Panda-eye 
pattern  type,”  see  Chapter  26,  this  volume).  One  should  use  fiber- 
optics  whenever  laser  light  has  to  be  transported,  eye-safe,  over 
large  distances,  especially  when  the  laser  is  located  in  a  dusty  envi¬ 
ronment  (i.e.,  not  a  clean  room).  A  single-mode  fiber  core  typically 
has  a  diameter  of  3  to  6pm  for  VIS  wavelengths.  This  creates  a 
point-like  excitation  spot  for  confocal  microscopy.  Multi-mode 
fibers  with  core  diameters  ranging  from  0. 1  to  1  mm  may  also  be 
used  at  times  but  beam  characteristics  such  as  beam  width  and 
polarization  state  are  altered.  However,  fibers  have  been  found  less 
suitable  for  carrying  UV  light  or  for  preserving  the  pulse  width  of 
femtosecond  pulsed  lasers.  For  femtosecond  pulse  lasers,  the  beam 
should  be  enclosed  in  a  dust-free  tube  and  reflected  from  disper¬ 
sion  controlling  mirrors  (e.g.,  from  Newport  Corp.).  Picosecond 
systems  are  less  affected  (see  Chapter  26,  this  volume).  Handy 
beam  couplers  are  available  from  many  suppliers  (OZ  Optics  Ltd., 
see  Table  5.3).  The  Cell-viZio  from  Mauna  Kea  Technologies  (see 
Table  5.3)  uses  a  bundle  of  30,000  fibers  of  2pm  each. 

Ultrafast  pulses  can  currently  be  delivered  by  fiber-optic 
without  dispersion  using  two  methods:  by  pre-compensation  or  by 
photonic-bandgap  fibers.  Ultrafast  pulses  create  non-linear  effects 
in  glass  fibers  and  the  fibers  suffer  from  dispersion  effects.  Even 
at  low  pulse  energies,  a  non-linear  Brillouin  and  Raman  back¬ 
ground  is  generated.  Pulse  trains  can  only  be  transported  without 
detrimental  effects  from  chromatic  dispersion  under  special  con¬ 
ditions.  By  precompensating  for  the  expected  fiber  pulse  chirp  with 
a  dual  prism  or  grating  configuration  (Fork  et  al.,  1984;  Zeek 
et  al.,  1999),  the  initial  shape  of  the  pulse  can  be  preserved  after 
it  exits  the  fiber,  but  this  adds  to  the  complexity  of  the  system  and 
creates  power  loss.  This  precompensation  uses  the  fact  that  a  short 
pulse  consists  of  many  frequencies  described  by  a  Fourier  sum, 
with  proper  amplitudes  and  phases. 

In  holey  fiber  coupling,  single-mode,  hollow-core  (=  air-filled) 
photonic-bandgap  or  crystal  fibers  (PCF),  non-linear  effects  occur 
at  a  much  higher  threshold  than  in  glass.  As  long  as  the  zero- 
dispersion  region  of  ~812nm  for  the  waveguide  matches  the  laser 
wavelength,  a  perfect  femtosecond  pulse  is  transmitted  without 
distortion  (Tai  et  al.,  2004).  Tuning  outside  this  range  gives  pulse 
broadening  again  (Hitz,  2004c).  The  spectral  width  of  the  0  to 
350mW  Ti:Sa  laser  stays  unchanged  at  812nm  after  passing 
through  a  1.5-m  length  of  HC-800-01  bandgap  fiber  (Crystal  Fibre 
A/S,  see  Table  5.3).  Dunsby  and  colleagues  (2004)  describe  an 
electronically  tunable  white  light  continuum,  435  to  1150nm, 
generated  by  injecting  80MHz  120  femtosecond  light  into  a 
micro-structured  fiber. 

Direct  Mirror  Coupling  and  Pulse  Width  and 
Pulse  Shape  Control 

Combinations  of  CW  Lasers  All  commercial  confocal  systems 
these  days  offer  facilities  to  combine  the  output  of  several  lasers 
using  dichroic  mirrors  and  intensity-balancing  neutral-density 
filters.  A  device  for  selecting  wavelength  and  intensity,  such  as  an 
acousto-optic  tuning  filter  (AOTF)  or  acousto-optic  beam  splitter 
(AOBS),  is  then  added.  One  of  the  advantages  of  having  many 
separate  lasers  is  that  the  intensity  of  each  line  is  servo-controlled 
independently.  In  the  competing  Kr/Ar  ion  laser,  it  is  only  the  total 
optical  output  that  is  stabilized.  The  portion  of  this  total  associated 
with  each  individual  line  can  vary  widely  with  time.  For  example, 
Cogswell  and  colleagues  (1992a, b)  combined  a  632.8  nm  He-Ne 
with  a  532  nm  frequency-doubled  Nd-YAG  laser  and  the  442-nm 
line  from  a  He-Cd  laser  for  true-color  confocal  reflection 
microscopy.  Issues  that  may  arise  are  spectral  purity  and  polariza¬ 
tion  of  the  excitation. 


Ultrafast  Lasers  Assuming  the  proper  coatings,  direct  mirror 
coupling  into  the  scan  head  can  maintain  the  femtosecond  pulse 
width  and  multi-photon  intensity  for  ultrafast  lasers  such  as  Ti :  Sa 
systems  (Fermann  et  al.,  2002).  On  the  other  hand  the  dispersion 
material  used  in  acousto-optic  modulator  (AOM)  or  acousto-optic 
deflector  (AOD)  devices  or  the  glass  in  thick  objectives  may  cause 
inappropriate  pulse  broadening  due  to  group  velocity  dispersion 
(GVD).  A  GVD  compensating  prism  or  grating  pair  or  a 
Gires-Tournois  interferometer  (GTI)  can  control  the  GVD  and 
compensate  the  widening  to  some  extent.  Instead  of  prism  and 
grating  pairs,  chirped  mirrors  could  be  used.  These  limit  tunabil- 
ity  but  improve  stability  against  environmental  effects  such  as 
temperature  changes  and  reduce  high-frequency  noise.  In  a  chirped 
mirror,  the  Bragg  wavelength  increases  with  increasing  penetra¬ 
tion  depth.  Double-chirped  mirrors  that  produce  an  additional 
chirp  in  the  coupling  of  the  incoming  and  reflected  wave  reach 
almost  perfect  dispersion  compensation. 

Multi-photon  ultrafast  (lOOfs)  lasers  possess  a  phase  that 
depends  on  the  spectral  region.  To  match  the  slightly  different  two- 
photon  absorption  properties  of  fluorophores  in  different  environ¬ 
ments  one  could  tune  the  laser  but  this  changes  the  beam  properties 
such  as  intensity,  phase  distortions,  etc.  A  better  approach  is  to  use 
a  charge-coupled  device  (CCD)  computer-assisted  pulse  shaper 
either  to  create  a  transform-limited  flat  phase  across  the  spectrum 
or  to  modify  the  phases  within  the  pulse  via  multi-photon  intra¬ 
pulse  interference  (Mil),  making  selective  excitation  possible 
(Dantus,  2003;  Dela  Cruz,  2004).  Mil  can  also  limit  the  amount 
of  three-photon  processes,  thereby  limiting  deep  UV  damage  to 
living  cells. 

Wavelength  Selection  and  Intensity  Control 
Wavelength  Selection 

In  commercial  confocal  systems,  several  laser  beams  are  often 
combined  via  dichroic  mirrors  onto  the  same  optical  path  leading 
towards  an  AOTF  or  AOBS  (Chapter  3,  this  volume).  Both  allow 
very  fast  wavelength  and  intensity  control  on  a  microsecond 
timescale.  We  found,  however,  that  the  spectral  purity  of  the 
AOTF-fiber  combination  is  sometimes  insufficient.  When  viewing 
mirror-like  solid  surfaces  covered  with  labeled  biosensor  mole¬ 
cules  excited  at  488  nm,  a  little  bit  of  514-nm  laser  light  was 
observed  passing  through  the  fiber-optic  and  masquerading  as  flu- 
oroscein  isothiocyanate  (FITC)  509  nm  emission.  A  thin  488  nm 
interference  filter  placed  in  front  of  the  fiber-optic  entrance  in  the 
AOTF  module  remedies  this  problem. 

Intensity  Control 

Continuous  wave  laser  beams  can  be  deflected  with  acousto-optic 
devices/deflectors/modulators  (AOD,  AOM)  for  intensity  and 
position  control.  The  functioning  and  practical  use  of  these  devices 
has  been  discussed  in  great  detail  in  Art  and  Goodman  (1993). 
Draaijer  and  Houpt  (1988)  used  these  devices  for  video-rate 
scanning  in  confocal  microscopy  and  their  system  was  commer¬ 
cially  available  through  Noran  and  still  is  supported  by  Visitech 
Inti.  Ltd.  and  by  Prairie  Technologies,  Inc.  (see  Table  5.3).  Inten¬ 
sity  control  via  an  electro-optic  modulator  is  described  in  Chapter 
3.  For  use  of  polarizers  to  control  laser  intensity,  see  above  under 
Attenuation  of  Laser  Beams. 

Polarization  of  the  Laser  Light 

We  measured  a  degree  of  polarization  of  the  light  entering  the 
Zeiss  AOFT  at  better  than  1000: 1.  Using  a  compact  home-built 
polarization  sensitive  power  meter  at  the  sample  position  the  laser 
light  was  still  very  well  polarized,  better  than  200 : 1  with  the  prin- 
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cipal  axis  5°  counterclockwise  from  the  positive  jc-axis  of  the  stage 
(i.e.,  left  to  right  direction). 

SPATIAL  BEAM  CHARACTERISTICS 

Although  lasers  provide  very  small  beam  divergence,  the  unifor¬ 
mity  of  the  intensity  across  the  beam  may  be  relatively  poor. 
Fortunately,  this  is  generally  not  a  serious  problem  because  the  uni¬ 
formity  of  illumination  can  be  easily  improved  by  spatial  filtering. 
Filtering  is  very  efficient  when  there  is  good  spatial  coherence  of 
the  beam. 

Several  devices  are  available  for  spatial  filtering;  the  most 
common  is  the  Gaussian  spatial  filter  (Melles  Griot,  see  Table  5.3), 
which  consists  of  a  focusing  lens  that  converges  the  beam  toward 
a  very  small  pinhole.  After  this  pinhole,  a  second  lens,  one  focal 
length  away,  is  used  to  regenerate  a  parallel  beam.  By  choosing 
the  focal  lengths  of  the  two  lenses  carefully,  this  configuration  can 
also  be  used  to  expand  or  decrease  the  beam  diameter  but  a  pinhole 
is  always  essential  for  spatial  filtering.  Spatial  filtering  takes  place 
because  diffraction  effects  at  the  pinhole  produce  an  out-going 
wavefront,  which  is  the  Fourier  transform  of  the  pinhole  and  is  not 
affected  by  the  spatial  properties  of  the  light  impinging  on  the 
pinhole  (Fig.  5.7).  It  should  be  noted  that  dust  particles  moving  in 
the  vicinity  of  the  pinhole  can  modulate  the  transmitted  intensity, 
so  great  care  must  be  taken  to  ensure  that  this  does  not  occur.  For 
most  practical  purposes,  a  single-mode  optical  fiber  functions  as  a 
spatial  filter. 

For  large  size  beams  from,  for  example,  excimer  lasers,  an 
external  beam  homogenizer  consisting  of  a  prism/lens  combina¬ 
tion  can  be  used.  The  optics  recombine  sections  of  the  beam  to 
produce  a  homogeneous  rectangular  shape  (Austin  et  al.,  1989). 

Edge-emitting  diode  lasers  typically  require  extensive  astig¬ 
matic  and  anamorphic  corrective  optics  to  obtain  a  circular,  paral¬ 
lel  beam.  Commercial  packages  are  available  to  implement  this 
(Ingeneric  GmbH,  see  Table  5.3).  However,  the  use  of  a  spatial 
filter,  a  single-mode  fiber-optic,  or  an  overfilled  back  aperture 
plane  can  also  produce  a  symmetric  beam. 

LASER  REQUIREMENTS  FOR  BIOLOGICAL 
CONFOCAL  LASER  SCANNING 
MICROSCOPY-RELATED  TECHNIQUES 


Optical  Tweezers 

Single-cell  manipulation  is  feasible  by  trapping  the  cell  in  an 
optical  box  formed  by  one  or  more  focused  laser  beams.  Each  focal 
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FIGURE  5.7.  Diagram  of  a  spatial  filter  by  a  pair  of  lenses  and  a  small  pinhole. 
High  frequency  noise  is  removed  form  the  laser  beam.  The  spatial  intensity  is 
cleaned  up. 


spot  has  a  diameter  of  about  1  pm.  Taken  to  the  extreme,  optical 
trapping  can  take  place  in  a  completely  enclosed  environment. 
Ashkin  and  colleagues  (1987)  reported  the  optical  trapping  of 
particles  and  viruses  although  the  technique  was  also  used  decades 
earlier  for  levitation  of  particles  in  light-scattering  experiments.  A 
simple  design  is  given,  for  example,  in  Smith  and  colleagues 
(1999).  A  beam  power  of  20  to  lOOmW  TEM00  at  the  specimen  is 
sufficient.  For  biological  specimens  infrared  light  should  be  used. 
A  careful  choice  of  the  trapping  wavelength  is  essential  to  reduce 
optical  damage  to  biological  samples  as  much  as  possible.  Shorter 
wavelengths  are  absorbed  by  the  specimen,  longer  wavelengths  by 
water.  Therefore,  traps  commonly  use  a  Nd-YAG  (Hoffmann  et  al., 
2000;  Reichle  et  al.,  2001)  or  a  Nd-YLF  lOOmW  laser  operating 
at  1064 nm.  Other  useful  lasers  may  be  a  632. 8nm,  25 mW  He-Ne 
laser,  or  a  diode  laser  beam.  An  improved  design  is  given  by 
Bechhoefer  and  Wilson  (2002).  Fiber  lasers  with  their  diffraction- 
limited  output  and  excellent  beam  quality  are  also  very  suitable  for 
laser  tweezers  applications  (Woods,  2003). 

Optical  trapping  of  particles  larger  than  the  illumination  wave¬ 
length  can  be  explained  with  geometric  optics.  When  a  trans¬ 
parent  spherical  particle,  having  a  refractive  index  larger  than  the 
surrounding  medium,  refracts  a  ray  of  light,  a  change  in  momen¬ 
tum  of  the  refracted  photon  occurs.  Due  to  the  principle  of  con¬ 
servation  of  momentum,  the  particle  receives  an  equal  but  opposite 
change  in  momentum.  This  force  pushes  the  particle  away  from 
the  direction  of  the  refracted  photon. 

When  a  particle,  somewhat  smaller  than  the  focal  spot  of  the 
laser  beam,  is  suspended  in  the  intensity  gradient  of  a  Gaussian 
beam  profile,  then  the  intensity  of  the  light  on  one  side  of  the  par¬ 
ticle  is  larger  than  on  its  opposite  side.  This  makes  the  particle 
move  to  the  side  with  the  higher  intensity  (i.e.,  the  center  of  the 
laser  beam)  and  it  seems  as  if  the  laser  beam  attracts  the  particle. 

Heating  and  Damage  Effects  in 
the  Cellular  Environment 

Cells  attached  to  biosensor  surfaces  may  suffer  additional  heating 
effects  because  of  one-sided  contact  with  a  buffer  solution  and 
close  contact  with  coated  or  absorbing  semiconductor  surfaces. 
Objects  caught  in  an  optical  trap  are  also  subjected  to  more  severe 
heat  stress.  For  instance,  a  100  mW  Nd-YV04  laser  focused  into 
watery  media  heats  it  by  0. 8 K/ lOOmW,  but,  add  a  polystyrene 
bead  in  glycerol,  and  it  jumps  to  4K/100mW.  Even  when  this 
amount  of  heat  is  not  damaging  to  the  cell,  it  influences  the  cali¬ 
bration  of  the  trap,  that  is,  the  force-displacement  relationship 
(Boas,  2003;  Peterman,  2003).  Several  studies  report  cell  damage 
conditions  (Konig  et  al.,  1996)  of  NIR  multi-mode  optical  traps. 
Neumann  (1999)  shows  that  for  the  spectral  region  from  790  to 
1064nm,  an  oxygen-mediated,  one-photon  process  is  causing  the 
damage.  Wavelength  and  power  level  effects  monitored  over  time 
by  Leitz  and  colleagues  (2002),  indicate  that  the  700  to  760  nm 
spectral  region  is  unsuitable  due  to  induced  photochemical  effects, 
but  that  810nm  is  better  because  only  photothermal  effects  occur. 
These  can  be  controlled  by  reducing  the  trap  energy  to  36  mW.  Cel¬ 
lular  response  to  NIR  light  with  varying  pulse  length  is  reported 
by  Konig  and  colleagues  (1999  and  Chapter  38,  this  volume). 
Zhang  and  colleagues  (1999)  look  at  cell  viability  in  CW  Al :  GaAs 
diode  laser  traps. 

Total  Internal  Reflection  Microscopy 

For  imaging  cell  adhesion  and  other  biological  processes  such 
as  vesicle  transport  and  observing  thin  self-assembled  layers 
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for  biosensor  applications,  total  internal  reflection  provides  a 
background-free  fluorescent  imaging  technique.  Penetration  of  the 
evanescent  wave  beyond  the  boundary  plane  is  limited  to  roughly 
lOOnm.  Objects  farther  from  the  interface  are  not  excited  and  stay 
dark.  Two  methods  exist  to  create  total  internation  reflection  fluo¬ 
rescence  (TIRF)  conditions.  One  is  to  inject  the  light  through  a 
very  high,  >1.45,  numerical  aperture  (NA)  objective  lens  and  the 
other  utilizes  an  external  prism  coupling  to  the  slide  on  the  far  side 
of  the  specimen.  The  through-the-objective  lens  method  requires 
only  40pW  CW  at  488 nm  for  green  fluorescent  protein  (GFP) 
imaging  with  reduced  bleaching  effects.  For  the  prism  coupling 
technique,  20  mW  CW  was  barely  sufficient  and  would  have 
benefitted  from  higher  power.  Other  suitable  lasers  include  50  mW 
green  532  nm;  70 mW  Fle-Ne  laser,  or  200 mW  488  water-cooled 
argon-ion  lasers.  It  must  be  noted  that,  at  these  power  levels,  some 
published  configurations  of  the  external-beam  prism  method  can 
be  very  hazardous  to  the  eye. 

Confocal  Raman  Confocal  Laser  Scanning 
Microscopy  for  Chemical  Imaging 

Raman  spectroscopy  and  imaging  is  used  in  research  on  immobi¬ 
lized  molecules  and  biosensor  interfaces.  It  requires  a  mode- 
hop-free  tuning  range  of  several  gigahertz,  a  narrow  line  width  of 
1  MHz,  and  output  power  of  several  hundred  milliwatts.  Cellular 
autofluorescence  can  present  a  large  background.  In  order  to 
reduce  its  influence,  red  excitation  from  a  diode  laser  is  a  good 
choice.  Stry  and  colleagues  (2004)  utilized  a  Tiger  ECDL  laser 
system  (see  Table  5.2),  with  grating  tuning  from  775  to  785  nm, 
and  also  from  730  to  1085  nm  with  1 W  output  power. 

Coherent  anti-Stokes  Raman  scattering  (CARS),  complemen¬ 
tary  to  CLSM,  maps  via  vibrational  contrast  the  intracellular  water 
distribution.  It  uses  the  beat  frequency  of  two  lasers,  preferably 
operating  in  the  IR  to  avoid  damage  to  cells  as  much  as  possible. 
Multi-photon  lasers  emitting  picosecond  pulses  with  widths  of  2 
to  5ps  are  better  for  CARS  because  their  spectral  linewidth 
matches  the  typical  Raman  bandwidth  of  ~20cnf'  much  better 
than  femtosecond  pulses  (Nan  et  al.,  2004).  They  were  able  to 
visualize  the  CH2  stretch  vibrations  from  lipid  tails  (see  also 
Chapter  33,  this  volume). 

Non-Linear  Confocal  Microscopy 

Second  harmonic  imaging  is  a  very  good  method  to  probe  exter¬ 
nal  membranes  of  living  cells.  Laser  system  considerations  and 
characteristics  are  given  below  under  Wavelength  Expansion  Tech¬ 
niques.  The  SHG  images  show  little  speckle  background  and  can 
be  created  with  about  the  same  laser  intensity  as  used  for  fluores¬ 
cence  imaging.  One  can  track  microtubules  with  SHG  and  intra¬ 
cellular  inhomogeneities  with  third  harmonic  generation  (THG) 
(Hogan,  2004).  Yang  and  Mertz  (2002)  used,  for  example,  a  mode- 
locked  860nm  Ti:Sa  laser  (see  below)  with  a  ~100fs.,  82MHz 
optical  pulse  train  that  delivered  lOmW  at  the  sample  (see  also 
Chapter  40,  this  volume). 

Nanosurgery  and  Microdissection 

Multi-photon  imaging  systems  can  be  used  for  multi-photon  abla¬ 
tion  of  cellular  structures  and  tissue.  It  requires  high  (kHz)  repeti¬ 
tion  rate  lasers  with  1  to  10  |iJ  pulse  energies  (Arrigoni,  2004a; 
also  see  Chapter  38,  this  volume). 


TYPES  OF  LASERS 

The  earliest  lasers  were  solid-state  lasers  using  ruby  as  the  active 
laser  medium  (Bertolotti,  1983).  Subsequently,  a  wide  variety  of 
lasers  were  developed  (Weast  and  Tuve,  1971;  Arecchi,  1972; 
Brown,  1981;  Bass  and  Stitch,  1985;  Eden,  1988;  Fermann  et  al., 
2002;  Silfvast,  2004;  Webb  and  Jones,  2004).  Essentially,  all  CW 
gas  lasers  (Bloom,  1968)  and  some  solid-state  lasers  with  emis¬ 
sion  in  the  visible  part  of  the  electromagnetic  spectrum,  meet  the 
minimum  intensity  requirements  estimated  above  for  fluorescence 
microscopy.  The  list  of  available  wavelengths  continues  to  expand 
(Weber,  1999).  For  trapping  and  label-free  SHG  microscopy,  IR 
wavelengths  are  the  most  suitable.  Some  other  important  parame¬ 
ters  are  the  output  power  at  each  wavelength,  efficiency,  and  sta¬ 
bility.  Table  5.1  lists  the  major  types  of  CW  lasers.  Table  5.2  lists 
the  major  options  for  pulsed-laser  systems. 


CONTINUOUS  WAVE  LASERS 

Continuous  wave  (CW)  lasers  can  be  divided  into  several  classes: 

•  gas  lasers 

•  dye  lasers 

•  solid-state  lasers. 


Gas  Lasers 

Three  major  types  of  CW  gas  lasers  are  available  (Bloom,  1968): 

•  argon-ion,  krypton-ion,  and  a  mixture  of  argon  and  krypton 
(mixed  gas) 

•  helium-neon 

•  helium-cadmium 

•  alkali  metallic  vapor  lasers. 

The  CO  and  C02  lasers  that  emit  around  5  pm  and  at  10.6  pm 
are  not  discussed  here.  Commercially  available,  but  not  widely 
used  and  therefore  also  omitted  from  further  discussion,  are  the 
neon  laser  that  can  emit  1 W  of  multi-line  UV  (CW)  at  339.2 
(0.3W),  337.8  (0.2W),  and  332.4nm  (0.5W)  and  the  xenon 
laser  emitting  between  488  and  540 nm  but  usually  only  operated 
in  pulsed  mode.  In  general,  krypton  and  helium-neon  lasers 
are  used  only  if  red  excitation  is  necessary,  but  the  latter  also 
now  provide  lines  in  yellow,  green  (GreNe),  and  orange  (594 nm). 
With  the  increasing  availability  of  red-absorbing  dyes,  their 
importance  is  growing.  Several  leading  confocal  microscope 
manufacturers  still  equip  their  systems  with  red  and  green 
helium-neon  lasers  emitting  a  few  milliwatts  and  air-cooled 
argon-ion  lasers.  Argon-krypton  mixed-gas  ion  lasers  often 
form  a  cheaper  alternative  to  the  purchase  of  separate  argon  and 
krypton  systems  plus  the  additional  requirements  for  power  and 
beam-steering  optics.  It  is  worth  realizing  however,  that  the 
lifetime  of  a  specific  red  laser  line  in  a  mixed  gas  laser  may  be 
much  less  than  the  stated  lifetime  of  the  strongest  laser  line  (green, 
514nm). 

Argon-ion 

Because  its  emission  wavelength  matches  the  absorption  peak  of 
fluorescein  and  other  popular  dyes,  the  argon-ion  laser  is  still  by 
far  the  most  common  laser  in  microscopy.  Lately,  however,  highly 
efficient  488  nm  small-footprint,  solid-state  lasers  have  appeared. 


Laser  Sources  for  Confocal  Microscopy  •  Chapter  5  91 


o  #  > 

Cl 


§  g  s 

Q  E  o 


.  .  43 

S'  -9  "S 

USX 

pH  CO  5 


>  s  a 

-5  <u  S 

iS  a A 


CS  £ 
w  O 
£  &. 

<z>  rv 

(D  & 

c*  ^  £ 

\2  C  3 
S'  o  Dh 

GO  e3  ^ 
C  N  CO 
U'CDh 

—  «  Q 
«d  2=r 

.S  Oh  2 
^  <D  >> 

>■» ,Q  c 


-  O  £ 
O  O  3G 

22! 

rl®S 


s*j-osa 

^»»  = 
S  o  •£  os  | 

2  o  -  os  ® 

3  -H  >^00  CN 


«5  c  o 
^  a  os 


QJ 

a  s  ’a 

P  O  9 

0-  Oh 


£>  Q  oo  >» 

Q  Lp.  HH  Q  dj 

HH  |S  C  HH 

C  r  )  C  Q 

.2  ,  fc  .2  ,  «J 

73  O^j  £  73  ^ 

3  0  3  .34 


3  »  r*  O 

u  n  a,-G  . 

-  K  g  -a  S3 

1  2  *  §  .§ 

2  d  c  <u  g 

3  rs  s  §  1 

3 1  §>£5  S 

n  -s  £2  2  P 

2  ^  i>  "5  00  c3 

2s  T3  O  r-  :> 

m  “  oo  ^ 

3  <D  -§  O  >  o 

^.22  e  "O  . 

;goSg,i 

JlSsSO 

J  M  «  cd  3  _ _ 

h  C  cd  ><  n. 

2  -a  M  W  y  § 

=  “  c  g  %  s 

"  g  ^  .2  3  w 
">  <L>  +  e3  Tn’> 
L>  5P  <D  g  2^-S 
>  .2  >»  §  .£  ^ 

3  •_  "73  -is  *-■  ■— I 

5  *P  X3  §  P  N 

5Q  .33  1)  HiH 

^.S  C  uQ* 
3  D  5  <y  ~c5  2 

Soh-O  £  c  -o 

o  o  H  e 

m  ol  On  03 


TABLE  5.1.  Continuous  Wave  Lasers  ( Continued ) 


92  Chapter  5  •  E.  Gratton  and  M.J.  vandeVen 


—  s  b  u 

m  B  3 


£  I  % 

u  >  S 
<  w 


o  o  o 
oo  oo  o 

-h  rn  to 


o  o  o 
o  o  o 

O  00  t-- 


oooooooooo 

oooooooooo 

ooinoocooooooooo 


lOM'tiOiOOiO-Hff) 

r-'Ocot'-oocnvivo 

cNcococncococococo 


lOONOO(N'nOlOt^;‘Ot^; 

^■t^vici'dodvd-^^-oodM 

ioin'OhhoooNOHCs2> 


£  I  % 

U  >  g 
<  w 


ooooooooooogoog 

oOininin'^-^ooo'^-inSinooO 
r,  tn  (N  (N  h  -1  IT)  -  M  O'  \  On  ^ 


-Hoo^ina;ocojir)qioh'oooh 

in\OJiOTi'OtstsOO^OHnM 


&  ^  ^ 
U  >  £ 

<  w 


U  _ 

o  hi  ° 

t-~  o  lO 


u  >  £ 
<  w 


^  t  $ 

u  >  £ 
<  w 


OOoOOO^’OO'^1 
ooor-ooOo«nO 
O  O  ro  < — i  so  ^  'O  CO  O 

in  vo  1-*  c<f 


o  o  o  o  o 
oo  ©  m  ©  -?t 
(N  cO  © 


•  ^avoocNinoinr-inoor-ONOi 
®w(^irici'Oo6,o-H^:dcodo6 
taj'5>n'sOi^r-oocvO^-cNcTco'0 


w  £  r-  ©  <n 
j  >n 

S  —I  VO  ©>  f- 

^  2 


oo  Tt 


* a  £ 


<03 

■S  CQ 

O  © 


§  <B  c«s 
§  O  £ 


T3  M  ^ 

a  JJ  S  1 

!  §  m  t 

i  9  O  m 


who 

60  S  g 


§  m  o  1 

$  al s 

«  S  C  T3 
>  3  <D 


go  a  ^  -3  a 

cS  c3  3 

J  ^  Q  ^ 


a  >  s  -g  s 

^  o  h  ^  CQ 


£  B-  -j-. 

(D  <L>  © 


&  Argon-ion5  514.5  500:1 

SpectraPhysics  <20A  458  40mW  <0.1  0.69  <0.95  TEMTO  V  <1.2 

&  Advantage6  488  all  lines  <1  pp  100 : 1 

163Caircooled  514 


Laser  Sources  for  Confocal  Microscopy  •  Chapter  5  93 


-  e 

>o-§ 

a  I 


OO  to  00 

oo  oo 
o  o  o 


H  I  M 


°  * 
<->  a  m 
oj  7  a 
M  JJ  S 
O  X  eo 


z  ^  I  ^ 

£  ffi  M  £  £ 

^  >  <u  £  « 

G  2  i  ffi 

S  o  s  J  ^ 

U  3  M 

O  >  O  |  Pi 


Diode  lasers:  Continuous  Wave 


94  Chapter  5  •  E.  Gratton  and  M.J.  vandeVen 


SSo^ooooooo 

oogo°oooooo 

>>-h-aaaa>>> 


8  8  8 

w  w  w 

H  H  H 


co  co 

cNcN°°r-r-t^r- 


rqCS|C-}C_>t--t-^r^[^^rqOOlOV-)iniO  (N  (N 

~^xxd>d>d>d>\/~d>d>d>d>d>  ©  © 

VVr-r-VVVVvVVVVVV  vv 


<d  <u  y 

.2  B  8 

*  -a  a 


1)  I  N 

•3  O  ffi 


O  U  ffi 

S  £  S 


r, 

£  _2  ^ 


■''t  ’'t  -<1- 

t-^  r-^  X  co  x  X 

o  o  o  to  to  io 

c4  oi  <N 


c  lb* 


^  <T)  o  o 
V  V  V  V 


o  o  o 

CO  CO  CO 

V  V  V 


^  (N 
O  O  H 
V  V  ° 


00OiOOOvO*O'sDv>DCNc0 
•0  <N  ©  ©  —  <N  CO  i— 1 
<N  <N 


M  M  ici  h  in 


h  +|  +|  M  +1  +|  +|  +|  +|  +|  +|  +|  +|  +|  +|  +|  +|  +|  +|  +| 

0'/")»o2oooO'oinooocoooocN|iov~i©cov~> 
r-  ©  'Coo^cohO^hvooocohO^hO 
CO  Tt^t^-v0C0^t^-^^^-iOC0^-^-\|-'t 


D  c/3 

I  §  * 

CL.  U 


o  o  o 

MOM 
V  V  V 


n  i,  in  in  n  ©  »o 

<2  |  %  *  K  S  I  "o  5  S 


^  -c  1  g  !  !  §  r?  J  J 

I  c/d  u  vC’TD’Oj-oo’O'do  a. 

^  2  ^  1  coo  ^  ^  i-  u 


>>  _  o 

fcd)  ©  © 

O  c/3  ©  © 

o  d*  -S  #  10  00 

C  °°  C/D  C/D 

•g  «>.  “  g  j  j 

g  ro  a  o-  •=  >  > 

^  h  m  m  a  a.  Cl. 

e2 


v.  <  y 
g>  >  -a 


-S  ^  ^  o 

V.  _«  J  7 

a)  O  ^ 


— i  z 


UMFL491  <0.3 


Continuous  Wave  DPSS  Lasers 


Laser  Sources  for  Confocal  Microscopy  •  Chapter  5  95 


<N 

V 


w  w 

H  H 


w  w  w  w 

H  H  H  H 


W 

H 


co  m 


o  o  o  o  o 

o  o  ©  o  o 

i— i  »o  o  in  04 


r-  -5t 
^  so 
os  o 

— 1  — «  Tf 


=8  N 

4  M 


(N  <N  CN 

V  V  V 


(N  — H 

V  V  V 


S  n 

w  w  w 

H  H  H 


WWW 
H  H  H 


so  so  so 

o  o  o 


X  X 

^  ° 1  ■'sl- 


Oh  w-.  CL 
o-  ^  CL 

M  9  fO 
\/  V  \/ 


+1  <N  +| 


O 

LT) 


cn  +| 

^  §§s 

'xf 


+1  O 
00  dg 
00  ^ 
Tf  CO 

d 


o 

W  .«§  r 

H  .5  1 
rv  cl 
^  0) 


3  § 

CQ  c/5 


JC  r 

cl  w 

S  >< 


£  dg 


a 


<  u 

H  H 
CQ  CQ 


.S  dg 


o 

A 


=<5  oo 

VD 


S  0-  £ 


•s 


3  O  3  O 


CQ  CQ 


-  •* 
a-  W 


3 


1  CL,  -  <n 


w  >  3 


TABLE  5.2.  Pulsed  Lasers 


96  Chapter  5  •  E.  Gratton  and  M.J.  vandeVen 


Ph  g 


o  o  o  o  o  o 


I  2  I 

j4  >  O  A 

Oh  |  ^ 


<u  f= 

"T3  oo  ^ 

e£  g  °*  ^ 


<D  c3  2 

oi  as  § 


O  O  O  I 
CS  U-5  00  N 

X 

<N 


>  8  a 

a)  a 


00  ON  O 
O  o  2  P~ 


°  ^ 's 

°ll 


•O  »n  O  io 

o  n  2?  o 

^l"  -O  P"-  'Tf 
in  in  o  in 

r-^.  OO  r~^ 

m  'xf  ^  CO 


|  £?  a 
ItS  c 


S  g 
o  r9  ^ 
a-  U 


Q-  fc 

s  £  ^ 

3  o  « 

0.  CL 


in  m  ±1  n 
in  in  K  K 
<n  @  o  o 
®  o  "*  P 
!g  £  @  - 

V  V  M  ® 


>n  o  S  9?  in 

cn  oo  ^n  co  ^ 

1  I  IV  1 

in  IT)  '  '  © 


I!a  g<5* 


.a  t~- 

£  co 

_  ^3 
•8  %  m 

o  is  -> 

*  0u 
°o  00 


o  3 

o  <  ^ 


CL-  fc 

s 

£  J  d 


PLP-10  440,470,  20,20,  13-22, 

635, 650,  30,  30,  10-35, 

670,  780,  30,  30,  10-30, 

850  All  50  pk  12-32 

±10  power 


Pulsed  diode  laser 


Laser  Sources  for  Confocal  Microscopy  •  Chapter  5  97 


o  o  o  o  o  o  o 

O'  O'  ^  ^  O' 

A  A  A  A  A  A  A 


o  o  o 
o  o  o 

VO  O  O 

r-  vo 


R  g  i  V  g  O  o  o  .  -  !  v  v  v 


o  o  o  o  I 
o  o  o  o  I 


o  O  ^  o'  q  o'  o'  o'  q 

H  H,  !2  ^  °°  H.  ®  !S  o  S  ° 

o'  o'  '%  cd  o'  q  o  in  9  ^  H  H 

i— <’  ,—l  w  oo  oo  d  cn  °°  w 


M  M  M  M 
o  O  O  m 

— <  (N  <N  CO 


^  io  in  o 

co  o  o 

vo  oo  ^ 

O  O  \/~i  o 
o  r-  r-.  d- 

"t  VO  ro 


. r-~ 

io  >n  o  vo  +i 
i  in  vo  ov  in  o 

VO  VO  vD  h  OO  c^j 

I  u-T  i>  o  o  in  >n  o 

CO  CO  VO  C''~  CO  00  Zh 

vo  vo  vo  vo  r-  r-  So 


O  o  S 

g  £  2 

w  A  o 

o  00  Q 
CO  f-.  O 


M  M  ^4  M 
co  co  co  co 
V  V  V  V 


O  hh  |_H 
C  co  CS  W 
5  j  SQ 

a  Q  a  -j 

O  ^  O  hJ 
O  ^  _0  Q 

Oh  Oh  Oh 


><  d  co  co  cn  co 
O  J  J  J  J  j| 


<400ps.(wide  pulse  listed),  selected  50ps  devices  ,  narrow  distributed  feedback  bandwidth  available,  440  nm  also  available  with  3mW  average  output. 

1080,  40,  20,  10  and  5 MHz  pulse  repetition  rates  and  adjustable  rates,  ASE  <50mW  @  780nm,  <60mW  @  1060nm,  fiber  coupling  passes  <5%  IR  in  multimode  and  <0.1%  for  single-mode  fiber-optic,  spectral  linewidth  4nm 
or  0. 1  nm  (narrow). 

"Water  consumption  21/min,  80%  of  plane-plane  cavity  output  within  0.6mrad,  Timing  jitter  (ps):  LSI  0-1 0/20- 10:  5ps,  LS  20-50:  2ps,  LS  35:  3ps. 

( Continued ) 


TABLE  5.2.  Pulsed  Lasers  ( Continued ) 


98  Chapter  5  •  E.  Gratton  and  M.J.  vandeVen 


XT’  ’T"  t* 

HH  HH  HH  HH 


S  3  s 

W  3  pj 

H  S  H 


o  o  o  o 
o  o  o  o 

(N  <N  <N  <N 


z  o  8 

o  d,  5 

-H  ^ 


■g  v  a 
s  —  s 


s;  o 

Is# 


_  *+ 

v  S  v 


M  M  M  M 
10  0*0  0 


^4  M  M  ^ 

vo  o  o 

— I  (O  !Q 

(N 


+1  5  (N 

O  rn  (N 
5  H  'O  'O 
o 


£  g  £ 

O  rR  ^ 

CU  u 


g  2  * 

V  T/  V 


h  &  s 

o  fl  ■o 

si  3  O 

A  £■  3 


H  ^  a  ij 

2  2  |  I 

A  A  0-0 


1  g 

X  ^ 

Q><  hJ  C4_< 

^  Q  C 

S3  CQ  £ 
^  2 

o  oo  -5 
D  O 

CQ  U 


v±  SH 

III  1 1 
Issllglgl 

KjCkjOKjOKiDkO 

C<5  Coo  D^o  D  ®0  D°0 

3  Q-  Cl  O. 


O 

Sh 

Oh  t- 
i  c« 

.a  D  73 


5  o 
i  > 
<  * 
X  S 

S  °1  M 

tS  <  >< 


Optronics8  1,064/532 


Laser  Sources  for  Confocal  Microscopy  •  Chapter  5  99 


w  w 
H  H 


<N  OO 
O 


ffi  C\  © 

Os  o 


O  O 

~ 

— I  <N  ^ 

V  V  V 


in  o 

©  CM 

V  V 


^  O  o  ^  o  ^ 

^  O  ^  r^i  O  1- 

_j  oo  -  _j  oo 


d  [v 

d  ©  q 


_  o  o 

©  o  oo 

00  o  o 


^  <D 

^  ^  OU 

3  Oh  g 


g  c  ^ 

9  Si  2 

43  U  d 

u  -g  S 

U 


8  8  <8  |  M 
OO  00  c  ,  S 

SS^I 


r-  ^ 
•§  CLh 
»h  2  Q 
<D  H  - 

^0  3 


Sall-gjo'g'sg  & 


Q  -§  >  > 


B  £  O 


U  43  o  -d  S 
00  bo  Oh  2 


'b  =  = 

|  s  s 


*J  o  > 
0)  ^  C3 


.y  d  S 
a-^  a 

°  •§  s 

J  o  . 
X  Q  £ 


>  M  ^ 

03 


oi  ^  c  d  a 

O  O  cS  7™  6fl 

a  c  ffl  o  ,2 

S  §  a  ,SP  13 

tL,  [Ih  £  H  CLh 


TABLE  5.2.  Pulsed  Lasers  ( Continued ) 


100  Chapter  5  •  E.  Gratton  and  M.J.  vandeVen 


£  ?  5S 
6 


—  -o  £ 

3  IT  CG 

0-  ^ 


O' 

«  S  N 

o  K 

s  -g  S 

o 


g  •  u 

s  ■§  -g 


~.§  E  « 

0  3  y  ' 

rv  °  o  l- 
2  &  3L  o 


o  g  £ 

Cl  C  ^ 

W 


<o  o  ^ 
^  o-  c 

^  C/3  G 


10^10  I 

cs  71  or  o 
V  V  V  JS 


©  O  o  O  ^  1 ©  G 

<»q\-<ooai^0?c?o 

cticiciciScici^ 

^  m  ^  oo  Tj-  - HOOlO 


"G  'S  -B  O  £ 
o  «  r  «  c 
O  £  5  •• 
cn  ca  .n  c 

2  G  O'  s  -I 

’*  .£  o  o. o 
<  .£  O  ; 

2  §*  £  £  < 

3  i  i 

S  S  a|  s 
;aosi 
.a  s  <  %  ' 

3  _>H  .3  c 

aifl  §-; 
a’  &  1  i 
?  2,1  s° 


aoi  5,E« 

G  C/3  a)  3  t-1 


-  >  -  ^ 

D  £  -xL  fe 

:so£ 

3  •>  u 

S3  C  O 

-  i>  ^  52 

-  d  jj  3 

^<L)/-.Gh 
“  73  <L>  £  ' 

G-r-  TD  3 
3>°°- 
3  O  £  tL 
^  „s  CD  I-J 


*0  £  g 
£  >  1/3  CL 
hC  Sh  ^  u 
O  •;  3  <D 

C  Z  >>  O 

<D  r  o  CL 

I  f?  £  ■§ 
|§.g> 

O  !?  i  j» 

.5°  §  o 

o  o  §  05 

c  O  c  G 


53^ 
&  2.“  °  ;>, 
s  |  a  S 

3  >  5  -a 

G-_  >  tu 

G  B  >-  O. 

»  oS  E 

P  S  z  <2 


o  o  a. 

•T1  "G  33 
G  •—  2 
o  S  jp 
W)  w  > 

<  j-  C  hJ 

>  o  -3  u 
2  cq  s  9 
a  J  _  S 

—  c«  *G 

00  -G  ’>  o 

>  s  c  ^ 

13  o  2  9. 

•  o  2  =  S 
e  a  o  o 

r;  c/5  C/3 

.§  a>°  § 

s,-l  »?  I 

>  oCb 
2  sT- 


g  2 

o  a«  > 

111  t 

&  °  s  a 

*“  2  ao  *- 
3^  C  L> 


"o  -  c  n 

‘S  <3  'o  f 

^  43  O-.G 
%  B  G  B 
—  <D  £ 

asjs 

4  jfe  “ 

a  «  d  " 
o  |l  | 

3  «l-  C 

1^41 

|z 

O  o 

vi  S  v) 

P  -8 -a  S3 

I  3  %  3 

COO—1 

8  -S.  J  £ 

t  <C  £  5 

x  0-  C  ^ 

K°8X  i 
-6^“1 ! 
~  (U  a  a  * 

1  i  ! 

i  1  1 1  i 

c  _  G  c  '! 

G  2-_x  O  4 

oq  £  is  J=  « 

/h  G  £L  2  D 


3-1 OW  DPSS  or  ion  laser  pump  power. 

Pump  power  6-10  W  Ytterbium  fiber  pump,  options  available  w.r.t.  emission  center,  bandwidth  and  average  power,  FemtoRainbow  100  OPO  KTP  based  delivers  ~  100  fs  pulses. 


CWAv.  &  Time-jitter  Rep  Ratel  _ Pulse  E  &  Width  (FWHM) _ Beam  Parameters _ 

Manufactory  Wavelength  Pk  Power  &  Noise  ML  ML  Q-sw  ML  &  Q-sw  Diam.  Div.  Mode  Pol.  Qual. 

&  Model  nm  mW  &  kW  rms,  ns/  %  MHz  mJ  &  ps  pJ  &  nsec  mJ  &  nsec  mm  mrad  Dir/%  M2 
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Their  contrasting  characteristics  are  described  near  the  end  of  this 
section.  As  a  pump  source  for  Ti :  Sa  lasers,  the  argon-ion  laser  has 
been  completely  replaced  by  solid-state  lasers. 

In  general,  argon-ion  laser  stability  is  better  than  that  of  a 
krypton  laser,  with  less  gas  slushing.  The  output  power  of  krypton 
is  at  best  about  30%  that  of  argon  under  identical  lasing  condi¬ 
tions.  Commercial  systems  provide  a  large  variety  of  emission 
wavelengths  and  output  powers  (Fig.  5.8;  Ginouves,  2002). 

Fragile  quartz  plasma  tubes  have  been  replaced  by  versions 
with  rugged  metal/ceramic  (beryllium  oxide,  BeO)  envelopes  for 
the  small-  and  medium-frame  lasers,  which  usually  do  not  use  a 
magnet  to  confine  the  plasma  to  the  central  core  of  the  tube.  Exter¬ 
nal  copper  disks  remove  the  heat. 

Large  systems  with  high  output  power  use  alumina  (A1203) 
ceramic  tubes  with  brazed  tungsten/copper  disks  inside,  crystalline 
quartz-coated  windows,  and  quartz-metal  hard  seals  (Fig.  5.4). 
This  significantly  increases  their  reliability.  Other  systems  use 
rugged  metal/ceramic  BeO  tubes.  The  central  bore  defines  the  laser 
beam.  The  plasma  discharge  pumps  ions  towards  the  cathode  end 
of  the  tube,  creating  a  pressure  gradient  between  the  anode  and 
cathode.  Return  holes  around  the  outer  edge  of  the  disks  form  an 
internal  gas  return  path  to  maintain  a  uniform  gas  pressure  along 
the  length  of  the  plasma  tube.  Typical  gas  pressure  inside  a  gas 
laser  is  about  1  Torr. 

The  formation  of  color  centers  in  the  Brewster  windows  has 
been  reduced  by  the  use  of  better  quality  window  materials, 
increasing  the  life  span  of  this  type  of  laser  to  3000  to  10,000  h. 
Flowever,  tube  lifetime  may  only  reflect  the  life  of  the  strongest 
laser  line  and  not  that  of  the  weaker  lines  needed  on  a  daily  basis. 
The  stability  of  these  lasers,  especially  the  small-  and  medium¬ 
sized  models,  is  good  because  they  can  be  cooled  by  convection 
(passive  cooling)  or  forced  air  and  their  electric  power  require¬ 
ments  are  modest.  Some  models  are  portable  and  easy  to  use.  The 
larger  frame  lasers  provide  active  resonator  stabilization. 

The  wall-plug  efficiency  of  large  argon-ion  models  is  very  low 
(0. 1  %  typical)  and  the  heat  generated  must  be  removed  with  a  large 
amount  of  cooling  water.  The  turbulent  flow  causes  some  vibra¬ 
tion  in  these  systems  while  insufficient  water  flow  may  cause  the 
cooling  water  to  boil  and  lead  to  the  destruction  of  the  plasma  tube. 
The  large  amount  of  heat  generated  also  puts  more  strain  on  the 
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FIGURE  5.8.  Schematic  energy  diagram  for  an  argon-ion  laser. 


resonator  cavity.  Longer  warm-up  times  are  necessary  when 
no  active  stabilization  is  incorporated  in  the  design.  The  opera¬ 
tional  lifetime  of  the  higher  power  laser  tubes  is  now  well  above 
2000  h. 

Using  proper  interference  filters  (inserted  with  their  shiny  side 
towards  the  laser  head  to  prevent  damage)  or  emission  line  selec¬ 
tion  by  means  of  an  AOTF  or  AOBS,  single,  strong  laser  lines  can 
be  selected.  Dual  line  emitting  argon-ion  lasers  simultaneously 
emitting  at  454  to  457 nm  and  514nm  are  also  offered  (Table  5.1). 

When  the  experiment  requires  a  reliable,  stable  source  of  deep 
UV  radiation  at  power  levels  of  tens  to  hundreds  of  milliwatts, 
these  ion  laser  systems  provide  a  better  alternative  than  any  current 
solid-state  UV  laser  system,  as  these  require  more  steps  to  produce 
UV  light.  The  area  where  the  argon-ion  laser  was  very  valuable, 
as  a  5  to  10W  pump  source  for  solid-state  Ti:Sa  lasers,  has  been 
completely  overtaken  by  solid-state  lasers. 

Argon-ion  Laser  Lines  in  the  Ultraviolet 

Commercially  available  argon  lasers  have  emission  wavelengths 
extending  from  275  to  528  nm  (Hecht,  1992).  Higher  discharge 
currents  are  necessary  to  populate  the  doubly-ionized  argon  levels 
sufficiently  for  deep  UV  emission.  Wavelength  selection  is 
obtained  by  installation  of  an  angle-tuned  quartz  prism  or  other 
appropriate  optical  elements.  The  wavelength  range  is  extended 
down  to  229  nm  by  inserting  a  frequency-doubling  crystal  in  the 
laser  cavity  and  thereby  producing  a  CW  intra-cavity  frequency- 
doubled  argon-ion  laser  system  (FreD,  Coherent,  see  Tables  5.1, 
5.3).  Reported  power  levels  range  from  40mW  at  228. 9nm  to  1 W 
at  257  nm  for  the  model  Innova  Sabre  MotoFreD. 

Argon-ion  Laser  Lines  in  the  Visible  Region  of 
the  Spectrum 

The  most  common  spectral  region  for  lasers  in  microscopy  is 
between  488  and  514nm,  although  argon-ion  laser  emission  can 
be  obtained  at  454.5,  457.9,  465.8,  472.7,  476.5,  488.0,  496.5, 
503.7,  514.5,  and  528.7  nm. 

Krypton-ion 

When  strong  red  emission  at  647.1  nm  is  needed  to  expand  the 
spectral  coverage  of  an  argon-ion  laser,  a  krypton  laser  is  the 
system  of  choice  (Brelje  et  al.,  1993).  Its  stability  is  slightly  less 
than  that  of  a  comparably  sized  argon-ion  system  and  the  gas  reten¬ 
tion  in  the  graphite  disks  is  slightly  larger.  However,  with  active 
stabilization  now  available  and  with  ceramic  tube  technology 
firmly  in  place,  both  drawbacks  have  now  been  largely  overcome. 
Based  on  the  FreD  argon-ion  laser,  see  above,  we  may  see  the 
introduction  of  an  intracavity  frequency-doubled  krypton-ion  laser 
as  a  source  for  UV  light  at  376,  338,  323,  284,  and  266 nm. 

Mixed  Gas  Argon-Krypton 

Combining  the  best  properties  of  the  argon  and  krypton  ion  lasers 
creates  a  reliable  laser  with  stable  output  and  the  broad  spectral 
coverage  needed  for  multicolor  CLSMs.  Brelje  and  colleagues 
(1993)  used  an  air-cooled  15mW  Kr-Ar  ion  laser  for  multicolor 
immunofluorescence  microscopy  and  described  a  range  of  appli¬ 
cations.  Several  companies  produce  systems  delivering  several 
tens  of  milliwatts,  for  example,  Melles  Griot  (Table  5.3). 

Helium-Neon 

The  use  of  these  stable,  inexpensive  lasers  in  microscopy  has  been 
somewhat  limited  by  their  relatively  low  intensity  and  predomi¬ 
nantly  632.8  nm  red  emission.  Several  manufacturers  have  recently 
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introduced  He-Ne  lasers  with  emission  at  534,  594,  612,  and 
632. 8nm  and  lines  in  the  infrared,  for  example,  1152nm  (Fig.  5.9). 
To  better  excite  Texas  Red,  Alexa  594,  and  several  other  dyes,  the 
red  He-Ne  at  632.8 nm  and  the  green  He-Ne  (GreNe)  at  543  nm 
have  been  joined  by  the  orange  He-Ne  at  594 nm.  Typical  powers 
for  the  632.8  nm  line  range  from  0.5  to  lOmW  with  a  maximum  of 
75  mW.  Although  increased  competition  is  expected  from  solid- 
state,  semiconductor,  and  diode  lasers,  the  He-Ne  laser  still  com¬ 
petes  well  thanks  to  its  good  beam  quality,  shorter  wavelengths, 
and  long  lifetime.  Invar-stabilized  He-Ne  frames  are  now  available 
from  Research  Electro-Optics  Inc.  (Table  5.3).  The  632.8 nm  can 
be  used  to  create  an  autofocus  feedback  circuitry  that  automatically 
adjusts  for  specimen  vertical  movement  (ASI,  Zeiss,  Table  5.3). 

Helium-Cadmium 

Helium-cadmium  (He-Cd)  lasers  have  found  several  applications 
in  microscopy.  Usually  they  operate  at  three  emission  wave¬ 
lengths:  325,  354,  and  442nm  (Fig.  5.10),  with  power  levels  up  to 
several  hundred  milliwatts  (Hecht,  1993c).  The  shortest  wave¬ 
length  requires  special  optics  and  is  very  rarely  used  in  fluores¬ 
cence  microscopy.  Membrane  probes  such  as  Indo-1  and  Fura-2 
can  be  conveniently  excited  with  the  354 nm  line.  The  emission  at 
442  nm  is  ideal  for  excitation  of  cyan  fluorescent  protein,  flavins, 
and  other  fluorescent  molecules.  The  power  available  at  325  and 
354  nm  is  generally  on  the  order  of  a  few  to  tens  of  milliwatts, 
while  at  442nm  up  to  150mW  is  now  available. 

The  stability  of  these  lasers  is  substantially  less  than  that  of  the 
argon-ion  lasers.  In  the  325  nm  region,  intensity  fluctuations  of  10% 
to  20%  are  not  uncommon  but  the  emission  tends  to  be  more  stable 
at  442  nm.  Although  most  He-Cd  systems  had  an  operational  life 
of  1  to  1.5  years  (1000  to  2000h),  a  new  mirror  technology  has 
increased  the  lifetime  of  the  coatings  to  >5000  h  for  325  nm 
mirrors.  The  less  powerful,  smaller  frame  lasers,  operating  at 
325  nm,  may  still  have  a  limited  life  span.  One  manufacturer  now 
supplies  a  sealed,  multiple-wavelength  version  with  up  to  10,000  h 
lifetime  (see  Table  5.2). 

The  He-Cd  laser  experiences  heavy  competition  from  the 
numerous  blue  diode  lasers  coming  to  market  in  the  wavelength 
region  spanning  roughly  400  to  450  nm. 

Alkali  Vapor  Lasers 

Diode-pumped,  alkali  lasers  (DPAL)  can  generate  blue  laser 
emission  by  a  two  step  up-conversion  scheme  (Beach  et  al.,  2004; 
Krupke,  2004).  Cesium  and  rubidium  vapor  CW  lasers  pumped  by 
diode  lasers  either  at  852  or  859  or  with  876  or  921  nm  (Cs)  or  at 
780  or  795  or  with  761  or  776nm  (Rb),  emit  at  455  or  459  (Cs) 
and  420  or  422 nm  (Rb).  With  their  low  cost  and  compact  design, 
potential  50%  efficiency,  high  average  power  (>100mW),  easy 
scalability,  and  good  beam  quality,  they  will  be  competition  for 
thin  disk  and  fiber  lasers  and  show  a  great  potential. 

Dye  Lasers 

As  all  dye  lasers  for  confocal  microscopy  are  optically  pumped  by 
other  powerful  lasers,  they  are  not  so  much  lasers  as  wavelength 
shifters.  Suitable  pump  sources  are  excimer,  ion,  neodymium- 
yttrium  aluminum  garnet  (Nd-YAG),  neodymium-yttrium  lithium 
fluoride  (Nd-YLF),  and  metal  vapor  lasers.  For  CW  operation,  the 
dye  is  circulated  to  prevent  heating  and  bleaching,  to  reduce  the 
competing  triplet-state  population,  and  to  remove  dye  molecule 
aggregates.  Compared  with  gas  lasers,  the  emission  spectra  are 
quite  broad  and  this  permits  them  to  be  tuned  to  lase  over  a  fairly 


broad  wavelength  band.  This  band  can  be  extended  by  changing 
dyes.  Tuning  of  the  laser  is  accomplished  by  a  prism,  a  diffraction 
grating,  and  a  stack  of  birefringent  plates  inserted  at  the  Brewster 
angle  or  by  etalons  (Demtroder,  1996;  Mollenauer  and  White, 
1987).  The  laser  medium  is  easy  to  handle,  inexpensive,  and  con¬ 
sists  of  a  fluorescent  dye  dissolved  in  a  solvent.  Because  of  so- 
called  "forbidden  transitions,”  the  molecules  tend  to  pile  up  in  the 
first  excited  triplet  state  and  thereby  become  unavailable  for  lasing. 
Rapid  pumping  or  the  use  of  a  dye-jet  stream  reduces  the  popula¬ 
tion  of  this  triplet  state.  In  order  to  keep  the  quantum  yield  high, 
the  dye  is  usually  cooled  to  minimize  competing  processes,  such 
as  collisional  and  vibrational  de-excitation.  Intrinsic  optical  effi¬ 
ciencies  as  high  as  20%  or  30%  can  be  obtained  (not  counting  the 
efficiency  of  the  pump  laser)  reaching  an  output  power  of  several 
hundred  milliwatts. 

Operating  a  dye  laser  is  sometimes  rather  cumbersome,  involv¬ 
ing  bottles  of  spent  laser  dye  and  waste  materials,  dye  spillage,  and 
often  large  quantities  of  dye  solvent  that  must  be  stored  in  a  safe 
storage  area.  Cuvettes  holding  laser  dye  have  to  be  cleaned  regu¬ 
larly  because  the  pumping  beam  bakes  dye  molecules  onto  the 
optical  surfaces.  Although  solid-state  lasers  have  largely  replaced 
the  dye  laser,  they  still  can  fill  several  spectral  gaps  between 
390  nm  and  the  IR. 

The  intensity  stability  of  dye  lasers  is  lower  than  that  of  its 
lamp  or  laser  pump  source.  Two  percent  to  5%  intensity  stability 
is  considered  good.  Beam  pointing  stability  is  heavily  influenced 
by  the  regular  tuning  done  on  these  systems.  Distances  should  be 
kept  as  short  as  possible  or  fiber-optic  beam  transport  should  be 
considered. 

The  spectral  regions  covered,  tuning  curves,  and  the  output 
power  offered  can  be  found  in  the  manufacturer’s  documentation. 

Solid-State  Lasers 

An  ever-growing  number  of  power-efficient  solid-state  lasers  is 
being  offered  for  use  in  confocal  microscopy.  They  are  particularly 
successful  in  replacing  inherently  inefficient  gas  lasers.  Their  ver¬ 
satility  and  ease  of  operation  either  for  use  as  CW  or  pulsed  direct 
light  sources  or  for  use  as  pump  sources  for  other  lasers  guaran¬ 
tees  continued  rapid  development.  There  is  a  trend  towards  smaller 
more  efficient  all-in-one-box  designs  and  a  drive  to  increase  total 
average  power  by  creating  laser  arrays. 

•  Semiconductor  lasers:  Most  are  based  on  gallium  mixtures, 
emit  a  single  line  in  the  red  or  NIR  and  can  be  pumped  elec¬ 
trically  with  very  high  efficiency.  SHG  generation  in  an  appro¬ 
priate  crystal  creates  UV,  blue,  and  green  emission.  The 
number  and  versatility  of  these  lasers  increases  daily. 

•  Non-tunable  solid-state  laser:  Another  class  of  laser  is 
formed  by  the  Nd-YAG,  Nd-YLF,  and  neodymium-yttrium 
orthovanadate  (Nd-YV04)  systems  operating  at  1064,  1047, 
and  1064  nm,  respectively.  These  can  be  used  as  a  stand-alone 
light  source  or  serve  as  a  pump  for  dye-lasers  and  titanium: 
sapphire  amplifier  systems. 

•  Tunable  solid-state  ultrafast  lasers:  These  are  epitomized  by 
the  titanium :  sapphire  laser  systems  generating  very  short 
pulses  down  to  a  few  to  tens  of  femtosecond.  They  provide  a 
very  broad  tuning  range,  comparable  with  that  of  dye  lasers, 
but  they  are  much  easier  to  operate,  generate  a  comparable  or 
higher  output,  provide  significantly  better  stability,  and  can  be 
built  in  a  range  of  configurations.  They  can  also  generate 
stable,  high-power  picosecond  and  femtosecond  pulses  as  we 
will  discuss  later. 
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TABLE  5.3.  Sample  Listing  of  Manufacturers 

ALPHALAS  GmbH,  Gottingen,  Germany,  +49-(0)55 1-770-6146,  http://www.alphalas.com/ 

Amplitude  Systemes,  Talenc,  France,  +33-(0)5-4000-3447,  http://www.amplitude-systemes.com 
AVESTA  PROJECTS  Ltd.,  Troitsk/Moscow  region,  Russia,  +7-(0)95-334-0078,  http://www.avesta.ru/ 

ASI,  Applied  Scientific  Instrum.,  Eugene,  OR,  (800)-706-2284,  http://www.ASIimaging.com 

BATOP  GmbH,  Weimar,  Germany,  +49-(0)  172-3520029,  http://www.batop.com/index.html 

BEOC,  Brocton  Electro-Optics  Corp.,  Brockton,  MA,  (508)-559-9958,  http://www.brocktoneo.com/services.htm 

Blue  Sky  Research,  Miltipas,  CA,  (408)  941-6068,  http://www.blueskyresearch.com/index.htm 

Bromba  GmbH,  Munich,  Germany,  +49-(0)89-3266-8930,  http://www.bromba.com/indicard.htm 

Cambridge  Laser  Laboratories,  Fremont,  CA,  (5 10)-65 1-01 10,  http://www.cambridgelasers.com/ 

CDP  Inc.  or  Systems,  represented  by  Edinburgh  Instrum.  Ltd.  Inc,  http://www.edinst.com/agencylaser.htm  and  PicoQuant  GmbH. 

Clark-MXR,  Inc.,  Dexter,  MI,  (734)-426-2803,  http://www.cmxr.com/gate.htm 
Cobolt  AB,  Stockholm,  Sweden,  +46-(0)8-545-9 1-230,  http://www.cobolt.se/ 

Coherent  Inc.,  Laser  Group,  Santa  Clara,  CA,  (800)-527-3786,  http://www.cohr.com/  and  http://search.coherentinc.com/ 
cgi-bin/MsmFind.exe?RESMASK=MssRes.msk&CFGNAME=MssFind.cfg&QUERY=brochures 
Continuum,  Santa  Clara,  CA,  A  division  of  Excel  Technology  Comp.,  (888)-532-1064  http://www.continuumlasers.com/mainswf.html 
The  Cooke  Corp.,  Auburn  Hills,  MI,  (248)-276-8820,  http://www.cookecorp.com/ 

CSK  Optronics,  Torrance,  CA,  (310)-574-8181,  csk@usa.com 

Crystal  Fibre,  A/S,  Birkerpd,  Denmark,  +45-(0)4348-2820,  http://www.crystal-fibre.com/ 

CrystaLaser  Inc.,  Reno,  NV,  (775)-348-4820,  http://www.crystalaser.com/homenew.html 

Del  Mar  Ventures,  San  Diego,  CA,  (858)-481-9523,  http://www.femtosecondsystems.com/index.php 

Edmund  Optics  Inc.,  Barrington,  NJ,  (800)-363-1992,  http://www.edmundoptics.com/ 

Edmund  Scientific,  Tonawanda,  NY,  (800)-728-6999,  http://scientificsonline.com/Default.asp?bhcd2=1099033491 
Electronik  Laser  Systems  GmbH,  http://www.els.de/Products/vdisk.html 

Electroi  Optical  Components,  Inc.,  Santa  Rosa,  CA,  (707)-568-1642,  http://www.eoc-inc.com/ir_sensor_cards.htm 
Evergreen  Laser  Corp.,  Durham,  CT,  (860)-349-1797,  http://www.evergreenlaser.com/Default.htm 
Excel/Quantronix  Corp.,  East  Setauket,  NY,  (631)-784-6100,  http://www.quantron.com/ 

Femtolasers  Productions  GmbH,  Vienna,  Austria,  +43-(0)  1-503-70020,  and  Femtolasers  Inc.,  Harvard,  MA,  (978)-456-9920,  http://www.femtolasers.com/ 

Fianium  Ltd.,  Southampton,  UK,  +44-(0)2380-458776,  http://www.fianium.com 

Guided  Color  Technologies  GmbH,  Jena,  Germany,  +49-(0)364 1-675350,  http://www.gc-tec.com/ 

GWU-Lasertechnik  GmbH,  Erftstadt,  Germany,  +49-(0)2235-955220,  http://www.gwu-group.de/laser/index.html 

High  Q  Laser,  Hohenems,  Austria,  +43  (0)5576-43040,  and  High  Q  Laser  (US)  Inc.,  Watertown,  MA,  (617)-924-1441,  http://www.highqlaser.at/  and 
http://www.HIGHQ-US.com/ 

Holo-Spectra  Inc.,  Van  Nuys,  CA,  (818)-994-9577,  http://www.lasershs.com/index.html 

IMRA  America,  Inc.  (IMRA:  Institut  Minoru  de  Recherche  Avancee,  now  IMRA  Europe  S.A.),  Ann  Arbor,  MI,  (734)-930-2590,  http://www.imra.com/ 
Ingeneric  Gmbh,  Aachen,  Germany,  +49-(0)241-963-1343,  http://www.ingeneric.com/ 

INNOLIGHT  GmbH,  Hannover,  Germany,  +49-(0)5 11-760-7270,  http://www.innolight.de/products/start.htm 
Kapteyn-Mumane  Laboratories,  Inc.,  Boulder,  CO,  (303)-544-9068,  http://www.kmlabs.com/ 

Kimmon  Electric  USA.,  Englewood,  CO,  (303)-754-0401,  Edinburgh  Instrum.  Ltd.  rep  http://www.edinst.com/agencylaser.htm 
Lambda  Physik  AG  (sub.  of  Coherent  Inc.),  Gottingen,  Germany,  +49-(0)55 1-69380,  http://www.lambdaphysik.com/ 

Laser  Diode,  Inc.  (part  of  Tyco  Electronics  Co.),  Edison,  NJ,  (732)-549-9001,  http://www.laserdiode.com/ 

Laser  Innovations,  Santa  Paula,  CA,  (805)-933-0015,  http://www.laserinnovations.com/coherent_lasers.htm 
Laser  Quantum  Ltd,  Stockport,  Cheshire,  UK,  +44-(0)  161-975-5300,  http://www.laserquantum.com/ 

Laser  Resale  Inc.,  Sudbury,  MA,  (978)-443-8484,  http://www.laserresale.com/ 

LaserMax  Inc.,  Rochester,  NY,  (585)-272-5420,  http://www.lasermax-inc.com/ 

Lasermet  Ltd.,  Laser  Safety  Solutions,  Bournemouth,  UK,  +44-(0)  1202-770740,  http://www.lasermet.com/ 

Lasiris  Inc.  (Div.  of  StockerYale),  Stocker  Yale  Canada,  Montreal,  Quebec,  Canada,  (800)-8 14-9552 

National  Laser  Corp.,  Salt  Lake  City,  UT,  (801)-467-3391,  http://www.nationallaser.com 

Lasos  LaserTechnik  GmbH,  Jena,  Germany,  +49-(0)364 1-29440,  http://www.lasos.com/index.htm 

Lexel  Laser  Inc.  (div.  of  Cambridge  Laser  Laboratories),  Fremont,  CA,  (5 1 0)-65 1  -0 110,  http://www.lexellaser.com/ 

LFW,  Laser  Focus  World  Buyers  Guide,  Penn  Well  Corp.,  2004,  Nashua,  NH,  (603)-819-0123,  http://lfw.pennnet.com/home.cfm 
LG  Laser  Technologies  GmbH,  Kleinostheim,  Germany,  +49-(0)6027-46620,  http://www.lg-lasertechnologies.com/ 

LIA,  Laser  Institute  of  America,  Orlando,  FL,  (800)-345-2737,  http://www.laserinstitute.org/ 

Limo  Lissotschenko  Mikrooptik  GmbH,  Dortmund,  Germany,  +49-(0)23 1-2224 10,  http://www.limo.de/ 

LUMERA  LASER  GmbH,  Kaiserslautern,  Germany,  +49-(0)630 1-703-1 81,  http://www.lumera-laser.de/index2.html 
Lumics  GmbH,  Berlin,  Germany,  +49-(0)30-6780-6760,  http://www.lumics.de/ 

Lumitek  Inti.  Inc.,  Ijamsville,  MD,  (301)-83 1-1001,  http://www.lumitek.com/Sensorcatalog.PDF 
Mauna  Kea  Technologies,  Paris,  France,  +33-(0)  1-4824-0345,  http://www.maunakeatech.com/ 

Melles  Griot,  Laser  Div.,  Carlsbad,  CA,  (800)-645-2737,  http://www.mellesgriot.com/contactus/default.asp 
Micro  Laser  Systems,  Garden  Grove,  CA,  (714)-898-6001,  http://www.microlaser.com/blueConfocal.html 
MWK  Industries,  Corona,  CA,  (909)-278-0563,  http://www.mwkindustries.com/ 

Newport  Corp.,  Irvine,  CA,  (800)-222-6440,  http://www.newport.com/ 

New  Focus,  Inc.  (a  division  of  Bookham),  San  Jose,  CA,  (408)-919-1500,  http://www.newfocus.com/ 

Noran  Instruments  Inc.,  see  Prairie  Technologies,  Inc. 

Novalux  Inc.,  Sunnyvale,  CA,  (408)-730-3800,  http://www.novalux.com/contact.html 
Oxford  Lasers  Inc.,  Littleton,  MA,  (978)-742-9000,  http://www.oxfordlasers.com/ 

Optronics  Technologies  S.A.,  Athens,  Greece,  +30-(0)210-983-7121,  http://www.optronics.gr/ 

OZ  Optics,  Ltd.,  Corp,  Ontario,  Canada,  (800)-361-5415,  http://www.ozoptics.com/ 
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TABLE  5.3.  ( Continued ) 

P.A.L.M  Microlaser  Technologies  AG,  Bernried,  Germany,  +49-(0)8158)-99710,  http://www.palm-microlaser.com/ 

Photonic  Products  Ltd.,  Bishops  Storford,  Hertfordshire,  UK,  +44-(0)  1279-7 19 190,  http://www.photonic-products.com/company_ov.htm 
Picoquant  GmbH,  Berlin,  Germany,  +49-(0)30-6392-6560,  http://www.tcspc.com/ 

POC,  Physical  Optics  Corp.,  Torrance,  CA,  (310)-320-3088,  http://www.poc.com/ 

Point  Source,  Hamble,  UK,  +44-23-80-744500,  http://www.point-source.com/products.asp 

Power  Technology  Inc.,  Little  Rock,  AR,  (501)-407-0712,  http://www.powertechnology.com/pti/index.asp 

Prairie  Technologies  Inc.,  services  Noran  microscopes,  Middleton,  WI,  (608)-662-0022,  http://www.prairie-technologies.com/index_files/page0005.htm 
Quantronix  (see  Excel/Quantronix) 

R&D  Ultrafast  Lasers  Kft.,  Budapest,  Hungary,  +36-(0)  1-392-2582,  http://www.szipocs.com/products.php 
Rainbow  Photonics  AG,  Zurich,  Switzerland,  +41-0)1-445-2030,  http://www.rainbowphotonics.com/ 

Research  Electro-Optics,  Boulder,  CO,  (303)-938-1960,  http://www.reoinc.com/ 

Rockwell  Laser  Industries,  Inc.,  Cincinnati,  OH,  (800)-945-2737,  http://www.rli.com/ 

Roithner  LaserTechnik,  Vienna,  Austria,  +43-(0)  1-586-52430,  http://www.roithner-laser.com/ 

Sacher  Lasertechnik  Group,  Marburg,  Germany,  +49-(0)642 1-3040,  http://www.sacher-laser.com/ 

Spectra-Physics  Lasers  and  Photonics,  Inc.,  Mountain  View,  CA,  (800)-775-5273,  http://www.spectra-physics.com/  and  Darmstadt,  Germany,  +49-(0)6151- 
7080 

StockerYale,  Inc.,  Salem,  NH,  (603)-893-8778,  http://www.stockeryale.eom/i/lasers/index.htm 

Teckhnoscan  Laser  Systems,  Novosibirsk,  Russia,  +7  (0)3832-397224,  http://www.tekhnoscan.com/english/index.htm 

Toptica  Photonics  AG.,  Martinsried,  Germany,  +49-(0)8989-99690,  http://www.toptica.com/index.php 

Torsana  Laser  Technologies  A/S,  Skodborg,  Denmark,  +45-(0)4556-0056,  http://www.torsanalaser.com/index.html 

Time-Bandwidth  Products,  Inc.  Zurich,  Switzerland,  +41  (0)44-445-3120,  http://www.tbwp.com/Time_Bandwidth/Home/FrameSetHome.htm 
TuiLaser  AG,  Germering/Miinich,  Germany,  +49-(0)89-894070,  http://www.tuilaser.com/products/lasers/index.htm 
Uvex,  Smithfield,  RI,  (800)-343-3411,  http://www.uvex.com/ 

Visitech  International  Ltd.,  Sunderland,  UK,  +44-(0)191 -5 16-6255,  http://www.visitech.co.uk/index.html 


Semiconductor  or  Diode  Injection  Lasers 

Charge  carriers  in  a  semiconductor  material  can  be  pumped  either 
optically  or  by  an  electric  current  (Fig.  5.11).  Electron- 
beam-pumped  semiconductor  lasers  can  emit  3  to  5  W  in  the  480 
to  650  nm  range  (Hobbs,  1993),  but  the  most  common  excitation 
method  simply  starts  with  a  diode  not  unlike  that  in  a  light- 
emitting  diode  (LED)  and  uses  an  externally  applied  current  in  the 
forward-biased  direction.  This  process  can  have  an  efficiency  as 
high  as  80%.  To  do  so,  the  electrons  must  be  confined  not  to  a  bulk 
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HELIUM  NEON 

FIGURE  5.9.  Schematic  energy  diagram  for  a  helium-neon  laser.  Helium 
functions  as  a  pump  energy  transferring  buffer  for  neon. 


+  + 

HELIUM  CADMIUM 

FIGURE  5.1 0.  Schematic  energy  diagram  for  a  helium-cadmium  laser.  Green 
and  red  emission  is  also  possible  but  not  commercialized. 
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FIGURE  5.11.  Schematic  energy  diagram  for  a  semiconductor  laser.  It  is 
based  on  laser  emission  from  a  forward  biased  p-n  junction  when  driven  by  a 
well-regulated  drive  current. 


volume  (Fig.  5.12),  but  to  a  stack  of  very  thin  layers  forming  mul¬ 
tiple  quantum  wells  (MQW),  or  planes  of  quantum  wires  or  dots 
(superlattice).  Due  to  the  small  confined  volume  a  high  radiative 
efficiency  exists  as  well  as  a  low  lasing  threshold.  Most  devices 
still  operate  in  NIR  but  the  trend  is  to  develop  diode  lasers  using 
wide  bandgap  materials  that  have  an  output  below  the  red.  Blue 
diode  lasers  based  either  on  ZnSe  or  doubling  860  nm  light,  emit 
at  around  430  nm  and  are  about  to  enter  the  commercial  market. 
At  higher  power  levels  (Figueroa,  2002),  direct  frequency¬ 
doubling  in  the  diode  laser  forms  an  alternative  route  to  the  blue 
region.  For  example,  the  D3  (Direct  Doubled  Diode  laser,  Coher¬ 
ent)  delivers  10  mW  at  430  nm.  When  selecting  a  drive  current 
source,  it  is  important  to  select  one  with  a  low  noise,  good  stabil¬ 
ity,  and  including  a  temperature  controller  on  the  diode  laser  head. 
Diode  lasers  can  change  their  emission  wavelength  over  a  limited 
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range  (10-20nm)  by  varying  their  drive  current  and  junction  tem¬ 
perature  (Hodgson,  1994).  The  approximate  tuning  rate  is  about 
0.1  nm  /°C.  Cooling  brings  the  lasing  wavelength  down.  Beam 
quality  suffers  from  astigmatism  that  has  to  be  optically  corrected 
(Snyder  and  Cable,  1993).  Edge  emitters  now  provide  high  power, 
up  to  several  watts  CW  in  NIR. 

Semiconductor  lasers  are  very  appealing  because  they  are 
small  [Fig.  5.13(B)],  highly  efficient,  easy  to  use,  and  relatively 
cheap.  Integrated  fiber-optic  output  is  another  feature  available 
from  many  manufacturers  [Fig.  5.13(A)].  However,  it  should  be 
stressed  that  this  small  package  comes  with  important  special 
requirements.  The  devices  can  be  rapidly  destroyed  if  both  current 
transients  or  nanosecond  current  spikes  at  start-up  and  internal 
heating  are  not  kept  under  control  by  the  power  supply  electron¬ 
ics.  Static  discharges  (SD)  from  a  person  or  an  ungrounded  sol¬ 
dering  iron,  or  the  use  of  solder  that  is  too  hot  or  remains  in  contact 
for  too  long  may  instantly  destroy  the  laser.  A  mechanical  shunt¬ 
ing  device  (Unger,  1994)  may  prevent  SD  damage  during  han¬ 
dling.  Alternating  current  (AC)  line  filters  are  recommended 
(Hodgson,  1994).  This  market  is  strongly  driven  by  the  digital 
video  disk  (DVD)  and  audio  compact  disk  (CD)  industry  where 
the  goal  is  to  increase  information  storage  densities.  Most,  if  not 
all,  of  the  following  examples  are  also  wavelength  stabilized  by 
stacks  of  multi-layer  coatings  usually  deposited  at  the  HR  side  and 
producing  bandpass  filter  reflectivity  only  for  the  lasing  wavlength 
[Fig.  5.14(B,C)].  Combining  a  proper  OC  coating  with  a  fiber 
pigtail  having  an  inscribed  Bragg  grating  has  the  same  stabilizing 
effect. 


A 


LASER  DIODE  WITH 
DBR  FIBER  OPTIC  PIGTAIL 


B 


2  mm  LASER  DIODE 


CIRCULAR  BEAM 
WITH 

ANAMORPHIC  OPTICS 


ELLIPTICAL,  COLLIMATED  BEAM 
WITH 

ASTIGMATISM  CORRECTING  OPTICS 


FIGURE  5.12.  Dimensions  and  cross-section  through  an  edge  emitting  diode 
laser.  The  polished  sides  act  as  highly  reflecting  mirror  and  output  coupling 
mirror.  Thermo-electric  cooling  to  remove  heat  is  required. 


FIGURE  5.13.  Example  of  (A)  optional  pigtail  fiber-optics,  (B)  physical 
dimensions  of  a  diode  laser,  and  (C)  corrective  optics  to  create  a  circular  beam 
profile. 
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FREQUENCY  SELECTING  BRAGG  GRATING  INTEGRATED  IN  ACTIVE  MEDIUM 
DISTRIBUTED  FEEDBACK  (DFB)  DIODE  LASER 


SHG  doubling  of  ~980nm  IR  light.  Similar  specifications  are 
found  for  the  Starbright  488,  lOmW  pumped  at  40 W  (Torsana 
Laser  Technologies  A/S);  the  BluePoint  488  at  2  to  5mW  from 
Rainbow  Photonics  AG;  and  the  tunable  488  nm  laser  from  Toptica 
Photonics  AG.  Coherent  introduced  the  Sapphire  (NOT  to  be 
confused  with  a  Ti :  Sa  system).  It  is  a  VECSEL  GaAs  diode  laser 
consuming  30  to  60 W  (mostly  in  the  thermo-electric  cooler), 
optically  pumped  at  976  nm,  and  intra-cavity  frequency-doubled  to 
provide  20  mW  of  electrical  input. 

The  Novalux  Inc.  Protera  is  based  on  an  ECSEL  design  with 
a  three-mirror  folded  cavity  for  better  mode  control  and  uses  a 
KNb03  doubling  crystal  inside  the  laser  cavity. 


c 


REFLECTOR:  COUPLER  GAIN  SECTION 
•PHASE  ADJ.  SECTION 

DISTRIBUTED  BRAGG  REFLECTOR  (DBR)  DIODE  LASER 


FIGURE  5.14.  Diode  lasers  with  wavelength  tunability  via  an  angle- 
adjustable  grating  in  an  extended  cavity  configuration  (A);  improved  mechan¬ 
ical  and  thermal  stability  as  well  as  wavelength  and  intensity  control  via  a 
bandpass  reflective  filter  formed  by  a  stack  of  thin  coating  layers  (Bragg 
grating).  When  the  grating  is  incorporated  into  the  gain  medium  the  device  is 
a  distributed  feedback  (DFB)  diode  laser  (B),  otherwise  it  is  a  distributed  Bragg 
reflector  (DBR)  diode  laser.  For  ultrafast  pulsed  operation  semiconductor  sat¬ 
urable  absorber  (SESAM)  layers  can  be  added  to  the  high  reflecting  mirror 
stack  or  to  the  saturable  output  coupler  (SOC)  to  the  output  coupling  face.  Edge 
emitting  diodes  require  corrective  optics  because  the  emission  is  elliptic. 


Violet  and  Deep  Blue  Diode  Lasers 

Because  these  lasers  produce  lOmW  of  deep  blue  emission 
(395-440  nm)  from  an  input  power  of  less  than  10  W,  they  do  not 
require  water  cooling.  Compare  this  with  an  argon-ion  laser  pro¬ 
ducing  the  same  10  mW  from  1  kW  of  electrical  input,  gallons  of 
cooling  water  per  minute,  and  filling  a  good  part  of  a  laser  table. 
Most  of  the  diode  lasers  have  a  single  fixed- wavelength,  which  may 
vary  somewhat  (see  above  under  FBG  Wavelength  Stabilization). 
Violet  405  nm  (Photonic  Products  Ltd.,  Table  5.3)  lasers  efficiently 
couple  into  3  pun  core  single-mode  fibers.  Diode  lasers  presently 
operate  at  395  to  440  nm  (LG  Laser  Technologies  GmbH, 
Coherent;  Table  5.3),  415  and  430nm  (Crystal  Laser  Inc.),  440 nm 
(PicoQuant  model  LDH  440,  Power  Technology  Inc.),  457  nm 
(Melles  Griot),  and  473  nm  (Crystal  Laser  Inc.,  Power  Technology 
Inc.).  But  they  are  not  nearly  as  cheap  as  laser-pointers!  Prices 
range  from  a  few  thousand  to  US$25,000,  depending  on  complex¬ 
ity  and  features  such  as  thermo-electric  cooling. 

Visible  and  Red  Diode  Lasers 

488  nm  Diode  Lasers  An  alternative  to  the  argon-ion  laser  has 
appeared  in  the  form  of  the  Protera  488  system  from  Novalux  Inc. 
(Table  5.3).  Listed  advantages  over  the  argon-ion  laser  include 
twice  the  life  expectancy  (20,000  h),  1%  of  the  power  consump¬ 
tion,  up  to  20  mW  output  power,  2%  of  the  size,  0.2%  rms  noise 
level,  and  built-in  thermo-electric  Peltier  cooling  with  conductive 
heat  removal  via  the  enclosure.  The  ChromaLase  488  from  Blue 
Sky  Research  has  a  typical  power  consumption  of  2  W  and  output 
power  from  1  to  25  mW  output  and  creates  488  nm  laser  light  by 


491  nm  Diode  Lasers  Some  designs  are  based  on  up-conversion 
of  a  fiber  laser  to  produce  lOmW  at  491  nm  (Guided  Color  Tech¬ 
nologies  and  Lumics  GmbH).  Others  use  a  different  principle.  The 
DPSS  Dual  Calypso  laser  (Nordborg  and  Karlsson,  2004)  from 
Cobolt  AB  (Table  5.3)  offers  491  nm  at  20mW  and  532nm  at 
50  mW  simultaneously.  It  uses  a  periodically  polled  potassium  tita¬ 
nium  oxide  phosphate  (KTP)  crystal,  partly  inside  the  laser  cavity, 
for  non-linear  optical  frequency  conversion.  The  491  nm  line  stems 
from  sum  frequency  mixing  of  Nd:  YV04  with  the  1064  nm  line 
from  the  Nd:YAG  laser.  Four  hundred  fifty-seven  and  1340nm 
lines  can  be  added  using  similar  methods.  This  laser  has  a  noise 
level  of  <0.3%  and  can  directly  replace  the  argon-ion  laser  oper¬ 
ating  at  488  and  514nm. 

606  and  635  nm  Diode  Lasers  The  He-Ne  594  nm  orange  and 
the  red  632.8  nm  lines  can  be  replaced  by  the  606 nm  (orange) 
VECSEL  Eksel  110  diode  laser  (Toptica  Photonics  AG;  Haring  and 
Gerster,  2003)  and  the  ChromaLase  635  25  mW  (Blue  Sky 
Research).  The  Radius  635  nm  CW  25  mW  (Coherent)  has  a 
16-h  beam  pointing  stability  of  about  60mrad  compared  with 
~160mrad  for  a  17 mW  He-Ne.  A  637 nm  red  diode  laser  He-Ne 
laser  substitute  has  been  incorporated  into  the  Bio-Rad  Radi- 
ance2100CLSM  (currently  Carl  Zeiss  CellScience  Ltd.). 

685  nm  Diode  Laser  A  685  nm  30  mW  CW  diode  laser  is  offered 
by  Lasos  Laser  Technik  GmbH. 

Recent  Developments  in  Diode  Lasers 

In  this  expanding  field,  many  features  are  rapidly  improving  to  the 
benefit  of  the  user. 

Emerging  Tunability  By  incorporating  a  diode  laser  into  an 
external  cavity  equipped  with  a  tunable  grating  (a  Littman-Metcalf 
cavity)  wavelength  tunability  from  632.5  to  1630nm  within  a 
50-150  GHz  bandwidth  is  now  available  (New  Focus;  Table  5.3; 
Scifres,  1994).  These  devices  are  listed  as  external  or  extended 
cavity  diode  lasers  [ECDL;  Fig.  5.14(A)].  A  narrow-line  width 
version  has  been  described  in  Day  and  Dessau  (1994).  Although 
for  the  moment  manufacturers  seem  to  focus  on  single  wavelength 
devices,  mechanical  and  thermal  stability  should  not  be  compro¬ 
mised. 

Beam  Quality  and  Delivery  Diode  lasers  typically  require 
extensive  astigmatic  and  anamorphic  corrective  optics  to  obtain 
a  circular,  parallel  beam  [Fig.  5.13(C)].  Other  methods  include 
using  a  spatial  filter  or  a  single-mode  fiber-optic  as  well  as  beam 
homogenizers,  such  as  the  monolithic  V-step  design  by  Ingeneric 
GmbH  (Table  5.3),  arrays  of  symmetrically  arranged  lenslets,  the 
so-called  “fly’s-eye”  lenses  or  light  integrators  (Homburg  et  al., 
2003),  available,  for  example,  from  Limo  Lissotschenko  Mikroop- 


108  Chapter  5  •  E.  Gratton  and  M.J.  vandeVen 


CIRCULAR  CROSS-SECTION 
OUTPUT 

METAL  CONTACT 
p-TYPE  DBR  OC 

MQW  LASING  LAYERS 

n-TYPE  DBR  HR 

SUBSTRATE 
METAL  CONTACT 


MULTI-QUANTUM  WELL  (MQW) 

VERTICAL  CAVITY  SURFACE  EMITTING  LASER  DIODE  (VCSEL) 

FIGURE  5.15.  Dimensions  and  layout  of  a  vertical  cavity  semiconductor 
diode  laser  (VCSEL).  Stacks  of  thin  coating  layers  form  the  distributed  Bragg 
reflector  (DBR)  highly  reflective  mirror  (HR)  and  the  output  coupling  mirror 
(OC).  The  emission  pattern  is  circular  and  can  be  near-Gaussian. 


tik  GmbH  (Table  5.3).  Optical  elements,  called  graded  index 
(GRIN)  lenses,  have  also  been  introduced  to  eliminate  spherical 
aberration  in  an  elegant  way  (Carts,  1994).  The  vertical-cavity, 
surface-emitting  semiconductor  laser  (VCSEL;  Fig.  5.15),  is  a 
semiconductor  laser  with  good  circular-pattern  beam  quality 
(Cunningham,  1993).  Power  levels  are  now  reaching  100 mW  with 
a  wall-plug  efficiency  of  20%,  high  temperature  stability,  and  good 
beam  quality. 

Output  Power  and  Cooling  Output  power  has  been  increased 
by  building  diode  lasers  as  banks  or  arrays  but  these  often  require 
additional  water  cooling  that  adds  to  their  complexity.  Quasi-CW 
output  levels  can  reach  300  ml  and  1.5  kW  peak  power.  Tens  to 
100  s  of  watts  of  CW  power  and  long  lifetimes  are  becoming 
common.  Cooling  requirements  depend  on  the  output  power  gen¬ 
erated  and  they  range  from  passive  air  cooling  via  thermo-electric 
(Peltier)  cooling  to  water  cooling,  the  latter  often  using  microchan¬ 
nels  for  optimum  control  of  the  junction  temperature. 

Temperature  Tuning  There  is  no  longer  a  need  to  have  a  large 
number  of  different  diode  lasers  nor  a  large  collection  of  emssion 
filters  for  CLSM.  Sivers  and  colleagues  (2004)  showed  that, 
with  a  3  mW  low-power  superlattice  (multi-quantum  well)  635  nm 
diode  laser  attached  to  a  Bio-Rad  MRC600  CLSM,  many  cell 
stains  that  absorb  in  a  broad  range  around  640  nm  can  be  reached 
by  cryogenically  cooling  the  laser  down  to  —  196°C.  This  caused 
the  emission  wavelength  to  shift  linearly  ~25  nm  down,  while  the 
output  power  went  up  about  five  times.  For  equal  fluorescence 
signal,  the  noise  level  in  the  image  caused  by  reflected  laser  light 
decreased  five  times  because  it  was  possible  to  optimize  the  laser 
wavelength  to  the  optical  filters  more  precisely. 

Output  Modulation  Another  important  feature  of  semiconduc¬ 
tor  lasers  is  that  their  output  can  easily  be  modulated  to  well 
above  100  MHz  and  some  can  reach  tens  of  gigahertz.  This  makes 
them  prime  candidates  for  use  in  frequency-domain  fluorescence 
lifetime  imaging  microscopy  (FLIM;  see  Chapter  27,  this 
volume). 


Diode-  and  Lamp-Pumped  Solid-State  Lasers 

A  second  class  of  solid-state  lasers  includes  those  using 
neodymium  yttrium  aluminum  garnet  (Nd-YAG)  and  lasing  in  the 
IR  at  1064.2  nm  with  very  high  fracture  resistivity  and  good 
thermal  conductance  (Fig.  5.16);  Yttrium  lithium  fluoride  (Nd- 
YLF)  with  low  thermal  lensing  (Arrigoni,  2004b);  Nd-BaYF  with 
better  subpicosecond  generation  and  amplification  properties; 
yttrium  orthovanadate  lasing  at  1064.3  nm  having  a  lower  lasing 
threshold  than  Nd-YAG  (Kaminskii,  1981);  GdV04  lasers  and 
Nd:LuV04,  also  having  a  lower  lasing  threshold.  All  of  the  mate¬ 
rials  listed  above  are  pumped  by  FBG  diode  lasers  [Fig.  5.14(C)]. 
Nd-YAG  and  Nd-YLF  provide  high  overall  efficiency  and  the  good 
beam  quality  necessary  for  frequency  doubling  to  the  visible  pro¬ 
ducing  lines  at  532  and  523  nm.  Microchip  lasers  are  formed  by 
sandwiching  laser  and  frequency  doubling  crystals  into  a  millime¬ 
ter  size  compact  structure.  Due  to  their  small  size  they  benefit  from 
ceramic  laser  materials  with  high  thermal  conductivity  (Wisdom 
and  Digonnet,  2004). 

A  simpler  design  is  possible  with  ytterbium  tungstate 
(Yb:KGW)  absorbing  between  900  and  lOOOnm  that  can  be 
directly  pumped  by  a  red  diode  laser. 

Early  models  used  flash  lamp  or  CW  krypton  lamp  pump 
sources  but  the  latest  models  tend  to  be  equipped  with  a  diode  laser 
[diode-pumped  solid-state  lasers  (DPSS);  Hobbs,  1994].  The 
advantages  of  diode  lasers  over  conventional  pump  sources,  such 
as  lamps,  are  reduced  cooling  requirements  because  of  higher  effi¬ 
ciencies  due  to  a  collimated  and  focused  output,  a  perfect  match 
of  pump-source  emission  wavelength  with  the  absorption  spectrum 
of  the  lasing  medium,  and  enhanced  frequency  stability  (Baer, 
1986).  A  typical  improvement  in  electrical-optical  conversion  effi¬ 
ciency  is  from  0.5%  (TEM00  mode)  to  6%. 
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Nd  :  YAG  and  Nd  :YV04 

FIGURE  5.16.  Schematic  energy  diagram  for  a  neodymium-yttrium- 
aluminum  garnet  (Nd-YAG)  laser  and  a  neodymium-yttrium  orthovanadate 
(Nd:YV04)  laser. 
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The  infrared  output  at  1064  (Nd-YAG)  or  1054nm  (Nd-YLF) 
is  easily  frequency-doubled,  tripled,  or  quadrupled  with  appropri¬ 
ate  non-linear  crystals  such  as  lithium  triborate  (LBO)  and  beta 
barium  borate  (BBO)  to  provide  radiation  at  532  and  527  nrn 
(SHG),  355  and  349  (THG),  266  and  262  nm  [fourth  harmonic  gen¬ 
eration  (FHG)],  respectively.  BBO  even  allows  generation  of  the 
fifth  harmonic  at  213  nm.  The  stability  of  these  lasers  is  very  good 
and  the  green  emission  wavelength  is  ideal  for  the  excitation  of 
rhodamine  dyes  or  for  pumping  the  tunable  solid-state  lasers 
described  below.  Nd-YAG  also  has  lower-gain  wavelengths  at 
1440,  1320,  and  946nm,  which  when  doubled  provide  720,  660, 
and  473  nm.  Power  levels  sufficient  to  obtain  frequency  doubling 
create  a  blue  laser  suitable  to  replace  the  ion  laser  lines.  Argon-ion 
514nm  emission  can  be  replaced  with  a  frequency-doubled 
532  nm  DPSS  design  when  the  mode  and  beam  characteristics  are 
optimized.  A  473  nm  DPSS  laser  (CrystaLaser  Inc.  and  National 
Laser  Corp.  with  5-10mW,  Table  5.3)  is  created  by  intracavity- 
KNb03  doubling  the  946  nm  IR  light  of  a  diode-pumped  Nd:YAG 
laser.  Typical  cost  is  about  US$8000. 

As  a  pump  source,  the  514nm  line  of  a  large  frame  argon-ion 
lasers  may  well  be  replaced  with  a  frequency-doubled  VersaDisk- 
515  laser  from  Electronik  Laser  Systems  GmbH  (Table  5.3)  emit¬ 
ting,  depending  on  the  model.  2.5  to  15  W  of  515  nm  light  in  a 
small  footprint  without  water  cooling,  and  improving  the  Ti:Sa 
pump  efficiency  by  20%  compared  with  the  more  common  532 nm 
DPSS  pump  source.  Several  companies  now  offer  DPSS  systems 
in  kit  form.  Examples  are  the  DPSS  educational  kit  from  Optron¬ 
ics  Technologies  S.A.  and  the  Nd:YV04  and  Nd:YAG  kits  with 
Cr:YAG  SESAM  from  ALPHALAS  GmbH.  These  lasers  are 
extremely  compact,  stable,  and  efficient  with  good  beam  quality 
and  offer  turn-key  operation.  Electrical  power  requirements  are 
low:  tens  of  watts.  Beam  quality  can  be  improved  with  Gaussian 
resonator  mirrors  and  phase  plates  (Casperson,  1994). 

Thin  Disk  Lasers 

Very  intense  lines  with  much  reduced  thermal  lensing  and  bire¬ 
fringence  can  be  generated  by  thin  disk  lasers.  Radial  thermal 
effects  are  most  severe  in  the  rod-type  lasing  materials  used  in  the 
original  Nd-YAG  designs.  Slab-type  designs  largely  lift  this 
thermal  problem.  Scientific  designs  are  profitting  from  Yb:YAG  or 
similar  based  thin  disk  systems  such  as  the  VersaDisc  by  Elec¬ 
tronik  Laser  Systems  GmbH,  which  can  attain  tens  to  hundreds  of 
watts  of  output  power  with  minimal  heating  effects.  It  is  based  on 
a  100  pm  thin  disk  of  Nd:YAG,  Yb:YAG,  or  other  material  bonded 
to  a  heat  sink  and  optically  pumped  on  the  opposite  side.  Because 
the  heat  gradient  is  almost  perfectly  planar  (i.e.,  parallel  to  the 
bonding  surface),  the  thermal  lensing  strongly  present  in  high- 
power,  rod-type  media  is  drastically  reduced  along  with  the 
amount  of  birefringence.  Optical  aberrations  are  not  introduced. 
As  the  disk  is  small  and  thin,  a  special  mechanical  mirror  arrange¬ 
ment  makes  the  pumplight  impinge  on  it  many  times.  Lasers  such 
as  the  VersaDisk  currently  deliver  up  to  about  100W  CW  at 
1030nm  and  about  15  W  at  515nm  (Hitz,  2004a). 

Tunable  All  Solid-State  Laser 

The  release  of  the  CW  titanium-sapphire  laser  was  an  important 
event  (Hammerling  et  al.,  1985).  These  lasers  can  be  pumped 
by  a  CW  ion,  Nd-YLF,  Nd-YAG,  or  diode  laser.  Because  their 
vibrational-electronic  levels  are  spread  in  a  broad  band,  laser 
transitions  can  take  place  over  a  wide  range,  and,  given  the  right 
mirrors,  the  tuning  range  extends  from  700  to  1000 nm  (Fig.  5.17). 
Power  output  is  3.5  W  at  790nm  when  pumped  with  an  18  W  Nd- 
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FIGURE  5.17.  Schematic  energy  diagram  for  the  Ti:Sa  four-level  vibronic 
laser. 


YV04  or  20  W  all-line  argon-ion  laser.  The  Ti:Sa  laser  has  virtu¬ 
ally  replaced  the  IR  dye  lasers;  it  is  much  easier  to  operate,  no  dyes 
have  to  be  changed,  and  the  long-term  stability  is  better  than  3% 
due  to  the  elimination  of  flowing  dye.  The  RMS  noise  between 
10Hz  and  2MHz  is  less  than  2%,  a  factor  of  about  10  less  than 
for  dye  lasers.  The  main  characteristics  of  these  systems  are  the 
generation  of  short  pulses  of  about  lOOfs  (FWHM),  and  high  peak 
power  for  frequency-doubling  to  the  blue  and  near-UV  spectral 
range  and  for  use  in  two-photon  CLSM.  Several  configurations 
have  appeared  resembling  either  CW  ring  or  standing-wave  dye 
lasers.  To  cover  the  complete  spectral  range  of  this  laser,  special 
mirror  sets  are  required. 

A  similar  wide-tuning  range  and  high  average  power  is  offered 
by  the  Alexandrite  laser  (Cr3+  in  BeAl204  host),  which  covers  the 
730  to  826 nm  region  when  pumped  CW  at  room  temperature.  This 
laser  can  also  be  flash  lamp  pumped.  The  Forsterite  laser  (Cr4+  ions 
in  a  Mg2Si04  host  material)  emits  between  1167  and  1345nm  and 
can  be  used  in  the  1200  to  1250nm  tissue  penetration  window. 
Its  three  main  pumping  bands  are  centered  around  350  to  550,  600 
to  850,  and  850  to  1200nm.  Both  CW  and  pulsed  models  are 
available.  Its  second  harmonic  is  tunable  from  600  to  650  nm 
(Mortensen,  1994).  Liu  and  colleagues  (2001)  report  multi-photon 
femtosecond  excitation  of  plant  tissue  at  about  1250nm  with  a 
Cr:forsterite  laser  (see  also  Chapter  21,  this  volume).  Potential 
members  of  this  family  of  tunable  solid-state  lasers  are  the  LiSAF 
laser  (Cr  in  LiSrAlF6)  with  a  tuning  range  from  780  to  1060nm 
and  a  peak  emission  at  825  nm  (Perry  et  al.,  1993),  and  the  LiCAF 
laser  (Cr  in  LiCaAlF6)  tunable  from  720  to  840  nm.  Tunable  UV 
could  be  created  by  Ce:LiSAF  (Anderson,  1994b). 

Continuous  Wave  Fiber  Lasers 
and  Up-Conversion 

Confusingly,  fiber-coupled  lasers  in  which  a  fiber-optic  merely 
guides  the  emission  are  also  called  fiber  lasers.  Although  also 
capable  of  generating  a  white  continuum,  micro-structured  pho¬ 
tonic  crystal  fiber  (PCF)  laser  delivery  systems  should  not  be  clas¬ 
sified  as  fiber  lasers.  Unlike  single-mode  fibers,  highly  multi-mode 
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core  fibers  are  insensitive  to  twisting  and  bending  but  otherwise 
behave  like  single-mode  fibers.  PCF  fibers  designed  to  have  better 
higher  order  dispersion  compensation  include  the  crystal  fiber 
large  mode  area  (LMA)  PCF. 

We  will  only  use  the  term  fiber  laser  for  systems  where  the 
laser  gain  medium  is  the  fiber  itself.  IMRA  America  (Table  5.3) 
shows  that  the  design  of  a  fiber  laser  can  be  very  simple:  Two  res¬ 
onator  mirrors  with  the  fiber-optic  in  between  (Fig.  5.18).  Double- 
clad  fibers  are  preferred  for  fiber  lasers  because  the  pump  light  can 
be  efficiently  coupled  into  the  high  NA,  >0.46,  inner  cladding 
while  only  a  single,  transverse-mode-doped  core,  having  a  low  NA 
of  about  0.06,  is  excited.  They  also  possess  excellent  heat  dissi¬ 
pation  (Limpert  et  al.,  2004).  Furthermore,  using  LMA  double- 
clad  fiber  (for  example,  20/400  pm  core/outer  cladding)  reduces 
non-linear  Brillouin  and  Raman  effects  (Galvanauskas  and 
Samson,  2004).  PCF  fibers  can  accommodate  high  powers 
(Folkenberg  and  Broeng,  2004),  with  260  W  CW  output  from  a 
short  piece  of  4  pm  double-clad  fiber,  thereby  also  limiting  non¬ 
linear  effects. 

Up-Conversion  Fiber  Lasers 

Theses  devices  are  based  on  excited  state  absoption  of  rare  earth 
ions  in  fluoride  “ZBLAN”  glass  fibers  (Fig.  5.18).  ZBLAN  stands 
for  zirconium,  barium,  lanthanum,  aluminum,  sodium  fluoride. 
When  pumped  by  a  pair  of  diode  lasers,  each  operating  at  a  dif¬ 
ferent  red  wavelength,  several  milliwatts  of  green  light  can  be 
created  (Piehler,  1993).  In  this  typical  up-conversion  scheme,  low- 
energy  photons  are  sequentially  absorbed  and  create  a  population 
inversion  in  the  excited  state  (Chenard,  1994).  By  doping  the  fiber 
with  more  than  one  rare-earth  material  (Fig.  5.19),  the  spectral 
region  longer  than  ~500nm  can  be  covered.  Examples  include 
doping  with  Er(bium)  for  546  nm,  with  Pr(aseodymium)  to  obtain 
491,  520,  605,  635,  and  720nm  emission  at  10  to  40mW  (Guided 
Color  Technologies,  GmbFI;  Table  5.3),  with  PrYb  for  840 nm, 
with  T(huliu)m  for  482  and  800nm  as  well  as  470nm  and  1735  ± 
1  nm  from  a  single-frequency  Tm-doped  DFB  fiber  laser  (Hitz, 
2004b),  with  Tm  for  980  nm,  and  with  Yb  for  1060  nm.  One 
more  example  is  the  700  blue  fiber  laser  series  from  LasNova 
(Table  5.3). 

Lumics  GmbH  has  developed  a  fluorozirconate  (ZBLAN 
glass)  fiber  laser  that  is  a  potential  replacement  for  argon-ion 
lasers.  The  spectral  linewidth  of  a  fiber  laser  is  typically  very  small, 
about  0.02  nm  (Gabler,  2004).  One  notes  that  the  exact  wave¬ 
lengths  at  times  differ  by  1  to  2  nm  between  various  designs,  a  fact 
which  may  be  related  to  doping  and  operating  conditions. 

In  the  IR,  fluoride  glass  fiber  can  be  doped  with  rare-earth  ions, 
such  as  erbium,  which  emits  at  1540nm  (Hecht,  1993a).  The 
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FIGURE  5.18.  Schematic  diagram  of  a  dual  ion  doped  CW  ZBLAN  up- 
conversion  fiber  laser.  Pumped  by  a  distributed  feedback  (DFB)  diode  laser 
emits  visible  laser  light.  The  connectors  contain  the  high  reflecting  mirror  (HR) 
and  the  output  coupling  mirror  (OC). 
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FIGURE  5.19.  Schematic  diagram  for  a  dual  ion  doped  up-conversion  laser. 


energy  stored  in  the  single-mode  laser  is  about  500  pJ  and,  in  a 
multi-mode  laser,  lOmJ.  The  maximum  optical  peak  power 
amounts  to  about  lOOkW.  Although  initial  tests  were  apparently 
carried  out  with  an  IMRA  775  nm  lOOfs  laser  on  a  Zeiss  micro¬ 
scope,  no  commercial  system  has  yet  been  offered  by  Zeiss. 

In  a  single-longitudinal-mode  fiber  laser  with  a  fiber  core  of  2 
to  4  pm,  the  interaction  area  is  very  small  (a  few  square  microm¬ 
eters).  The  interaction  length  is  very  long,  from  millimeters  to  even 
kilometers.  This  results  in  a  very  high  single-pass  optical  gain  at 
very  low  threshold  (sub-mW)  pumping  power  levels  with  pumping 
efficiencies  to  75%.  Fibers  consist  of  amorphous  material  that  give 
very  broad  absorption  bands  (Chenard,  1994).  The  single-pass  gain 
is  practically  unity.  Output  is  always  diffraction  limited  with 
excellent  beam  quality.  In  combination  with  an  easily  attainable 
average  power  level  of  a  few  watts  this  makes  them  very  suitable 
for  laser-tweezer  applications.  These  advantages  of  a  fiber  laser 
were  recognized  as  early  as  1964.  However,  at  that  time  there  was 
little  capacity  to  produce  good  quality,  low-loss,  single-axial-mode 
fiber. 

PULSED  LASERS 

Why  are  pulsed  lasers  useful?  The  simple  answer  is  higher  peak 
power.  Sufficiently  powerful  CW  pump  sources  are  often  not  avail¬ 
able.  Often  the  wavelength  and  intensity  offered  by  available  lasers 
are  not  appropriate.  Therefore,  wavelength  conversion  schemes 
have  been  developed  and  many  of  them  are  based  on  non-linear 
phenomena  that  occur  only  at  very  high  intensity.  Because  of  their 
high  peak  power,  pulsed  systems  provide  efficient  wavelength  con¬ 
version  and  tuning  capabilities.  In  general  terms,  the  peak  inten¬ 
sity  is  greater  than  the  average  intensity  by  a  factor  that  is  the 
inverse  of  the  duty  cycle.  The  duty  cycle  is  the  ratio  of  pulse  width 
to  pulse  interval  and  can  often  be  10-5. 

It  is  important  to  remember  that  pulsed  lasers  are  only  useful 
in  scanned  fluorescence  microscopy  if  the  pulses  occur  at  a  high 
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enough  rate  that  many  pulses  occur  in  each  digitizing  period  and 
intensity  integration  over  the  pixel  period  can  provide  a  per¬ 
formance  similar  to  a  CW  laser.  Pulsed  lasers  are  characterized  by 
the  high  peak  power  (see  Table  5.2). 

Classification  of  Pulsed  Laser  Systems 

A  logical  way  to  classify  pulsed  laser  systems  is  according  to  their 
temporal  pulse  behavior.  Four  methods  for  pulsed  operation  exist: 
normal  mode,  Q-switching,  mode-locking,  and  cavity-dumping. 

•  Normal  mode,  free-running:  When  observed  with  a  fast 
diode  and  an  oscilloscope,  solid-state  lasers  generate  a  contin¬ 
uous  train  of  random  spikes  with  varying  intensity  caused  by 
the  interaction  of  various  modes  in  their  very  small  cavities. 
This  type  of  operation  is  not  very  useful. 

•  Q-switched  lasers:  The  term  Q- switched  is  derived  from  the 
radio  and  microwave  terminology,  and  it  relates  to  a  forced 
change  in  the  quality  factor,  Q,  of  the  resonant  cavity  (i.e.,  a 
bell  is  “high-Q,”  a  lump  of  wet  clay  is  “low  Q”).  A  low  Q  res¬ 
onator  is  one  which  hardly  supports  laser  action.  What  is 
needed  is  a  method  of  changing  the  gain,  or  resonance,  of  the 
laser  cavity.  A  simple  implementation  of  a  Q-switching  device 
is  the  installation  of  an  optical  shutter  in  the  resonator.  The 
sudden  opening  of  the  shutter  releases  all  the  energy  stored  in 
the  excited  electrons  of  the  medium  in  one  giant  optical  pulse. 

•  Active,  mode-locked:  In  a  resonant  cavity,  many  simultane¬ 
ously  oscillating  optical  modes  are  present  at  the  same  time 
(Arecchi,  1972;  Mollenauer  and  White,  1987).  Usually  these 
modes  interact  with  each  other  in  a  random  way  and  no  phase 
relationship  exists  between  them.  The  output  power  fluctuates 
randomly  according  to  a  time-dependent  interaction  between 
the  various  modes.  Installation  of  an  intra-cavity  modulator 
(i.e.,  a  high-speed  shutter)  with  a  resonance  frequency  exactly 
matching  the  round  trip  time,  2 Lie,  in  the  laser  resonator  will 
lock  all  modes  to  that  particular  resonance  frequency.  A  certain 
phase  relationship  is  maintained.  Throughout  the  cavity,  a 
single,  intra-cavity  pulse  is  created  at  a  pulse  rate  of  70  to 
100  MHz.  Mode-locking  frequencies  can  be  changed  by  using 
a  crystal  with  different  mode-locking  frequency  and  simulta¬ 
neously  adjusting  the  laser  cavity  length.  Thermal  stability  has 
to  be  excellent.  Folded  compact  cavity  design,  further  minia¬ 
turization,  a  one-box  design  strategy,  direct  IR  diode  pumping 
using  ytterbium  tungstate,  and  standard  thermo-electric  Peltier 
cooling  help  maintain  proper  mode-locking.  For  fiber  lasers, 
smaller  size  LiNb03  Mach-Zehnder  modulators  are  used 
because  they  possess  a  lower  insertion  loss  and  better  control 
of  chirp. 

•  Passively  mode-locked  lasers:  In  passive  mode-locking  a 
very  high  loss  is  introduced  to  interrupt  CW  operation. 

Kerr  Lens  Mode-Locking 

One  way  to  generate  short  pulses  from  a  solid-state  Ti:Sa  laser  is 
using  Kerr  lens  mode-locking.  The  Kerr  effect  refers  to  the 
quadratic  non-linear  dependence  of  the  refractive  index  of  a 
non-linear  material  on  the  applied  electrical  field.  The  intensity 
dependent  refractive  index  induces  a  linear  frequency  sweep  on 
any  short  optical  pulse  leading  to  spectral  broadening.  This  effect 
can  be  compensated  by  sending  the  pulse  through  a  few  millime¬ 
ters  of  glass  or  short  piece  of  fiber-optic.  High  intensity  optical 
pulses  exert  strong  forces  on  the  valence  electrons  of  the  laser 
medium.  The  transverse  gradient  in  a  Gaussian  beam  profile  deliv¬ 
ers  the  highest  intensity  near  the  center  of  the  beam,  less  so  near 


the  perimeter.  Kerr-induced  refractive  index  effects  cause  the  beam 
to  contract  producing  self-focusing.  This  regular  focusing  pattern 
in  the  gain  medium  imposes  a  specific  set  of  modes  in  the  Ti:Sa 
laser  and  eliminates  other  non-matching  modes.  An  important  con¬ 
sequence  is  that  the  gain  medium  works  like  a  master  oscillator 
with  its  own  set  frequency.  This  method  requires  an  intracavity  slit 
or  pinhole  to  suppress  CW  contributions  and  needs  an  active  starter 
such  as  a  noise  spike  created  by  tapping  a  mirror  mount  and 
otherwise  perturbing  the  cavity. 

Saturable  Bragg  Reflector  and  Semiconductor 
Saturable  Absorber  Mirrors 

Creation  of  self-starting  intense  optical  pulse  trains  in,  for 
example,  a  Nd-YV04  or  fiber  laser  running  at  a  high  frequency  of, 
for  example,  80  MHz,  is  implemented  by  introducing  into  the 
cavity  a  low-loss,  superlattice,  saturable-crystal  absorber  such  as 
InGaAs.  The  material  possesses  a  low  reflectance  for  weak  optical 
signals  (such  as  noise)  and  a  high  reflectance  for  high  power 
signals  (such  as  optical  pulses)  and  has  a  very  short  recovery  time 
(saturable  absorber  mirror,  SAM).  It  is  incorporated  into  a 
high-reflectivity  Bragg  mirror.  Quantum  wells  can  be  used  in  place 
of  the  InGaAs.  The  wider  the  spectral  bandwidth  of  the  gain 
medium,  the  shorter  the  optical  pulse  generated  and,  in  addition, 
shorter  pulses  are  created  when  a  shorter  rod  is  used  because 
dispersion  can  be  better  compensated.  For  normal  dispersion,  red 
is  ahead  of  blue  light.  For  anomalous  dispersion  the  opposite  is 
true.  Transparent  materials  usually  possess  normal  dispersion. 
Because  a  Ti:Sa  rod  introduces  normal  dispersion,  it  must  be 
compensated  by  inserting  anomalous  dispersion  components 
such  as  prisms,  grating  pairs,  or  chirped  mirrors  into  the  cavity.  In 
ultrafast  Ti:Sa,  CnLiSAF,  or  fiber  lasers,  pulse  durations  of  a  few 
tens  of  femtosecond  can  readily  be  created.  Mode-locked  oscilla¬ 
tors  such  as  an  ultrafast  Ti:Sa  laser  create  nanojoule  pulse 
energies. 

The  latest  oscillator  addition,  Yb:KGW  (tungstate),  is  directly 
pumped  by  two  fiber-delivered  high-power  940  or  980  nm  IR  diode 
pumps  (Eclipse,  Spectra  Physics).  No  prism  pair  is  required  for 
compensation  of  the  positive  GVD  because  the  cavity  contains 
dispersion-compensating  coatings.  In  addition,  an  saturable  bragg 
reflector  (SBR)  Bragg  high  reflector  provides  self-starting  and 
passive  mode-locking.  At  1048 nm,  lOOmW  average  power  is  gen¬ 
erated  with  a  repetition  rate  of  80  MHz  and  a  pulse  width  of  150 
fs.  Components  are  readily  available  (BATOP  GmbH;  Table  5.3). 
Similarly,  a  microchip  Nd:YAG/Cr:YAG  gain  medium/absorber 
can  be  designed  (Kraft,  2004). 

Saturable  Output  Coupler 

The  SAM  can  be  replaced  by  a  saturable  output  coupler  (SOC) 
incorporated  into  the  cavity.  A  passively  mode-locked  DPSS  laser 
creates  a  very  simple  design  because  the  pump  power  can  enter 
through  the  high  reflector  into  the  laser  medium. 

Cavity-Dumped  Lasers 

Cavity  dumping  is  a  common  technique  for  reducing  the  laser  rep¬ 
etition  rate.  For  example,  for  frequency-domain  fluorescence  life¬ 
time  imaging  microscopy  (FLIM)  the  80  or  76  MHz  laser  pulse 
repetition  rate  is  too  high.  The  repetition  rate  can  be  brought  down 
to  2  or  4  MHz  by  applying  a  momentary  radiofrequency  (RF) 
signal  to  an  appropriate  modulator  crystal.  This  allows  more  har¬ 
monics  to  be  used  for  FLIM  imaging  and  optimizes  the  system  for 
the  nanosecond  or  or  picosecond  lifetimes  encountered. 
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Nitrogen  Lasers 

Among  the  more  common  pulsed  lasers  are  nitrogen  lasers. 
Because  of  their  low  pulse  repetition  rate  (20-100pps),  nitrogen 
lasers  are  only  used  in  microscopy  for  intentionally  wounding  cells 
(Burgess,  2004).  The  average  power  is  between  1  mW  to  1 W, 
and  the  emission  wavelength  is  337.1  nm  (Hecht,  1993b).  The 
pulse  width  of  these  lasers  varies  between  a  fraction  of  a  micro¬ 
second  to  less  than  a  nanosecond.  The  time  characteristics  of 
the  pulsed  nitrogen  laser  emission  at  337  nm  was  exploited  in  flu¬ 
orescence  microscopy  by  Wang  and  colleagues  (Wang,  1990; 
Wang  et  al. ,  1990,  1992).  Their  conclusion  was,  however,  that  the 
pulse-to-pulse  intensity  stability  of  this  type  of  laser  (500  ps 
FWHM  pulse  at  25  Hz)  is  not  presently  sufficient  and  long  inte¬ 
gration  times  were  necessary.  For  this  reason  the  nitrogen  laser  is 
unlikely  to  be  useful  in  microscopy,  particularly  now  that  deep  UV 
diode  lasers  (Crystal  Laser  Inc.;  Table  5.3)  possess  much  more 
favorable  lasing  characteristics.  Bioeffects  on  living  cells  and 
tissues  of  UV  laser  irradiation  can  be  severe  but  for  microdissec¬ 
tion  applications  the  nitrogen  337  nm  line  is  very  suitable  (Burgess, 
2004). 

Excimer  Lasers 

Although  the  vacuum-UV  and  UV  output  of  these  excimer  lasers 
has  given  them  a  strong  foothold  in  tissue  ablation,  this  type  of 
laser  is  not  often  used  for  CLSM.  Air-cooled,  compact  models  are 
available,  such  as  LEX tra  and  COMpex  models  (Lambda  Physik; 
Table  5.3).  This  company  also  introduced  a  metal-ceramic  based 
NovaTube  design,  which  lasts  for  7  X  108  pulses  on  a  single  gas 
fill. 


Metal  Vapor  Lasers 

Metal  vapor  lasers  (Lewis  et  al.,  1988;  Hecht,  1993d)  are  also 
extremely  powerful  and  can  be  operated  at  much  higher  repetition 
rates  than  excimer  lasers.  The  average  power  can  exceed  100W 
and  the  repetition  rate  can  be  as  high  as  20  to  50  kHz,  which  is  still 
a  bit  slow  for  most  microscope  applications.  Wall  plug  efficiencies 
reach  1%.  Their  characteristic  emission  wavelengths  are  also 
reported  in  Table  5.2. 

Dye  Lasers 

In  cancer  treatment,  high  fluency  rates  are  sometimes  required. 
Real-time  CLSM  following  the  effects  of  high-energy  7.5J/cm2 
585  nm  pulsed  dye  laser  irradiation  on  hyperplasia  in  vivo  was 
reported  by  Aghassi  and  colleagues  (2000).  Otherwise,  the  need 
today  for  stable  well-characterized  tunable  sources  is  largely  filled 
by  optical  parametric  oscillators  (OPO)  and  optical  parametric 
amplifiers  (OPA;  see  below)  in  combination  with  solid-state 
pumping  lasers.  Not  only  is  this  technology  far  less  messy  than 
dye  laser  technology,  it  also  adds  IR  coverage  via  difference  fre¬ 
quency  mixing  (Radunsky,  2002). 

Dye  Laser  with  Intra-Cavity  Absorber 

Synchronously  pumped  dye  lasers  equipped  with  saturable 
absorbers  have  been  used  extensively  in  the  past  to  obtain  short 
pulses  to  cover  the  gap  between  525  and  700  nm,  which  exists 
for  Ti:Sa  lasers  (see  below;  Herrmann  and  Wilhelmi,  1987; 
Muckenheim  et  a].,  1988). 


Colliding-Pulse  Dye  Laser 

Colliding-pulse  mode-locked  dye  lasers  can  emit  optical  pulses 
of  <100fs  pulse  width  at  repetition  rates  of  100 MHz.  They  can 
produce  an  average  power  of  15  mW  at  630nm  when  pumped  with 
at  least  a  4W  all-line  argon-ion  pump.  Noise  is  less  than  1%  but 
this  figure  depends  critically  on  pump  laser  performance.  This  type 
of  laser  was  used  by  Denk  and  colleagues  (1990)  and  Piston  and 
colleagues  (1992,  1994)  for  their  first  two-photon  experiments.  It 
has  rather  limited  tunability.  Thermal  equilibrium  is  reached  within 
1  h  after  turning  the  system  on.  Ti:Sa  lasers,  which  produce  lOOfs 
pulses  at  lOOx  higher  power,  have  replaced  colliding-pulse  mode- 
locked  dye  lasers. 

Modulated  Diode  Lasers 

A  small  current  change  makes  a  large  change  in  the  output  inten¬ 
sity  of  the  diode  laser,  allowing  most  CW  diode  lasers  to  be  oper¬ 
ated  in  pulsed  mode.  When  the  internal  diode  laser  processes  are 
fast  enough  and  stray  capacitance  from  leads,  etc.,  is  kept  low, 
pulse  repetition  rates  of  several  hundred  megahertz  can  be  obtained 
as  long  as  the  required  average  output  power  is  moderate.  To  gen¬ 
erate  a  crisp  pulse  train,  the  cut-on  and  cut-off  slopes  must  be 
steep.  A  recent  review  by  Landgraf  (2001,  2003)  lists  a  variety  of 
diode  lasers  and  their  upper  frequencies.  Selected  laser  diodes  have 
reached  -700  MHz,  for  use  in  determining  fluorescence  lifetime. 
With  a  trigger  circuit,  these  devices  could  act  as  a  poor  man’s  Ti:Sa 
for  FLIM. 

Diode  Pumped  Solid  State  Laser  in  Pulsed  Mode 

Doubled  or  tripled  output  could  be  used  in  tissue  ablation  but  oth¬ 
erwise  the  pulse  repetition  rate  is  too  low  and  the  high  output 
power  a  waste  for  CLSM.  Used  as  Ti:Sa  pump  laser,  a  dual  rod, 
tandem  cavity  DPPS  laser  can  now  produce  up  to  90  W  at  527  nm. 
An  example  of  these  systems  producing  1  to  10  kHz  optical  pulse 
trains  with  a  width  of  a  few  hundred  nanoseconds  are  the  Q- 
switched  Evolution  Nd:YLF  family  from  Coherent  Inc.  (Arrigoni, 
2004b)  and  the  30  W  Empower  series  from  Spectra  Physics. 

Ultrafast  Diode  Pumped  Solid  State  Lasers 

Many  one-box  compact  and  flexible  solutions  are  appearing. 
Examples  are  the  Pallas  and  Tiger  compact  picosecond/femtosec¬ 
ond  laser  systems  from  Time-Bandwidth  (Table  5.3).  Direct  diode- 
pumped,  passively  mode-locked  DPSS  Nd-YLF  lasers  can  make 
70fs  pulses  (Tiger-picosecond  from  High  Q  Laser  Production)  and 
lOOps  (Pallas)  covering  780  to  860 nm.  with  a  repetition  rate  of  50 
to  200  MHz  and  an  average  output  power  of  500  mW. 

Titanium-Sapphire  and  Related  Ultrafast  Lasers 

This  type  of  laser  covers  660  (18  W  pump)  or  700  to  llOOnm 
(Duling,  1995;  Fermann  et  al.,  2002).  It  is  an  ideal  source  for  two- 
and  multi-photon  microscopy.  The  long  wavelength  reduces  scat¬ 
tering  and  allows  it  to  penetrate  farther  into  tissues  than  visible 
lasers.  The  fact  that  excitation  occurs  only  at  the  focus  plane 
improves  the  signal-to-background  ratio  and  reduces  phototoxity. 
Extensive  overviews  of  multi-photon  fluorescence  microscopy  and 
applications  can  be  found  at  the  excellent  Molecular  Expressions 
site  (Florida  University)  and  the  Cornell  site  of  DRBIO  as  well  as 
in  Chapters  28  and  37. 

To  cover  the  entire  wavelength  range,  several  sets  of  mirrors 
may  be  necessary.  Most  Ti:Sa  systems  are  now  pumped  with  low- 
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maintenance,  highly  efficient  5  to  10W  solid-state  diode  lasers. 
The  self-mode-locking  Kerr  effect  of  the  Ti:Sa  lasing  rod  delivers 
lOOfs  pulses  at  80  to  100  MHz  repetition  rate.  Modifications  in 
design  may  lead  to  pulses  as  short  as  20fs  or  even  5 fs.1  These  very 
short  pulses  exhibit  extreme  pulse  broadening  as  they  pass  through 
dispersive  media.  Measures  to  reduce  or  eliminate  this  effect  are 
given  above  under  beam  delivery.  The  high  average  power  com¬ 
bined  with  the  short  optical  pulse  width  makes  the  Ti:Sa  laser 
almost  ideal  for  frequency  doubling  and  tripling  to  the  visible  and 
350  to  525  nm  spectral  regions.  Following  user  demand,  com¬ 
pletely  automated  and  tunable  systems  are  now  offered.  These 
provide  a  somewhat  limited  tuning  range  to  stay  away  from  the 
edges  of  the  gain  curve.  Examples  are  the  Mai-Tai  family  from 
Spectra  Physics  and  the  Chameleon  products  from  Coherent 
(Table  5.3).  Gaussian  profile  and  broadband  mirrors  are  standard 
features. 

On  the  other  hand,  several  companies  now  offer  kits  to  allow 
users  to  build  their  own  ultrafast  or  CW  Ti:Sa  laser.  These  kits  are 
much  less  expensive  and  also  less  complex.  Still  further  simplifi¬ 
cation  is  under  way  by  having  all  components  on  a  small  footprint 
in  an  all-in-one-box  design.  First,  the  Nd:YV04  green  pump  laser 
was  upgraded  with  stronger  pump  diodes  and  higher  efficiency 
doubling  crystals.  It  now  emits  CW  18  W  of  532  nm  with  low  noise 
and  in  TEM00  mode.  One  pump  diode  suffices,  halving  replace¬ 
ment  costs.  Not  only  can  the  532  nm  output  pump  both  the  Ti:Sa 
oscillator  and  a  regenerative  Nd:YLF-based  pulse  amplifier, 
halving  the  number  of  green  pump  sources,  but  also  the  edges  of 
the  gain  curve  tuning  range  are  more  easily  accessible.  Thermal 
lensing  gets  worse  with  increased  pump  power  but  has  been  over¬ 
come  by  adjusting  the  focal  length  of  the  cavity  mirrors.  Never¬ 
theless  cryogenic  cooling  may  be  required  if  systems  are  scaled  up 
further  (Arrigoni,  2004b). 

Heat  dissipation  and  efficiency  are  areas  where  fiber  and  thin 
disk  lasers  have  a  clear  edge.  A  further  simplification  is  under  way 
by  using  ytterbium  oscillators,  which  can  be  pumped  directly  with 
IR  diode  lasers,  bypassing  the  green  pump  step,  but  their  pulse 
width,  etc.,  is  still  not  as  short.  Table  5.2  lists  the  specifications  of 
several  commercially  available  examples  also  in  kit  form.  Most, 
if  not  all,  use  semiconductor  saturable-absorber  mirrors  (SAMs 
or  SESAMs,  see  above  under  mode-locking)  to  generate  a  self¬ 
starting,  high-repetition-rate  femtosecond  or  picosecond  pulse 
train  (Hogan,  2004). 


White  Light  Continuum  Lasers 

A  rather  inexpensive  source  of  quasi-white  light  is  the  CW  He-Cd 
laser,  which  seems  to  the  eye  to  emit  white  light.  Some  with  a  pos¬ 
itive  column  discharge  actually  emit  only  three  wavelengths  simul¬ 
taneously  (325,  354,  and  442  nm),  while  the  hollow  cathode  design 
emits  from  UV  to  NIR.  Selection  of  five  lines  creates  an  RGB 
white  light  laser  emission  (White  Knight  RGB  Series,  The  Cooke 
Corp.;  Table  5.3). 

An  elegant  and  wider  supercontinuum  white  light  source,  with 
generation  stretching  from  below  450  nm  to  above  950  nm,  is  fea¬ 
sible  using  aim  long  photonic  crystal  fiber  (PCF)  with  a  1 .7  pm 
core  diameter  illuminated  by  the  3  to  5ps  800  nm  pulses  from  a 
5kW  Ti:Sa  laser  (Espinase,  2004).  This  is  an  important  step 
towards  a  white  light  femtosecond  light  source.  A  similar  PCF 
white  continuum  source  with  a  length  of  38  cm  was  used  by 


1  One  can  predict  from  the  relationship  Ax.Av  >1  that  for  a  spectral  width  of 
200 nm  one  obtains  a  lOfs  pulse  at  800 nm. 


McConnell  (2004)  on  a  Bio-Rad  1024ES  scan  head.  Coherent 
introduced  a  CW-pumped,  cavity-dumped,  regenerative  amplifier 
with  white  light  continuum  generation  at  repetition  rates  up  to 
250kHz  and  lOOfs  pulses.  However,  even  this  much  improved 
repetition  rate  is  too  slow  for  the  usual  CLSM  applications.  This 
problem  is  elegantly  solved  by  a  cavity  dumped  OPO,  pumped  by 
the  second  harmonic  of  a  Ti:Sa  laser.  It  delivers  30fs  pulses  with 
a  repetition  rate  of  up  to  4  MHz  between  570  and  660  nm  with  a 
pulse  energy  of  13  nj  (Potma  et  al.,  1998). 

Ultrafast  Fiber  Lasers 

To  generate  short  pulses  with  high  peak  power,  a  ring  cavity  can 
extract  more  energy  from  the  pump.  Bragg  feedback  (Friebele  and 
Kersey,  1994)  stabilizes  the  laser  wavelength.  Mode-locked  single¬ 
mode  fibers  produce  reliable,  high  repetition  rate,  short  pulses  with 
a  small  wavelength  tuning  capability  (Duling,  1993;  Smith  and 
Lucek,  1993).  Diode  pumping  and  proper  optics  are  necessary  for 
best  performance.  The  pump  DFB  diode  laser,  output,  and  ampli¬ 
fier  stages  are  simply  coupled  by  splicing  devices  (Fig.  5.20).  This 
type  of  laser  creates  a  very  compact,  efficient,  and  cheap  alterna¬ 
tive  source  for  femtosecond  pulses  at  gigahertz  repetition  rates  and 
may  soon  replace  the  dye  laser  and  some  Ti:Sa  systems.  Hitz 
(2004d)  reports  a  diode-laser-pumped  Yb  ultrafast  fiber  laser 
oscillator-amplifier  system  equipped  with  external  grating  com¬ 
pressor  and  generating  62 MHz,  lOOfs  with  25  W  average  power. 
Stability  has  to  be  improved  against  environmental  changes  caused 
by  mechanical  stress-induced  birefringence  and  temperature 
variations.  At  the  moment,  a  fiber  laser  is  one  fifth  the  size,  has  the 
same  initial  cost,  but  only  one  third  the  operational  costs  of  a 
Nd-YAG  or  Nd-YV04  ultrafast  system.  The  FemtoMasterl060,  a 
passively  mode-locked  soliton  fiber  laser  is  offered  by  Fianium- 


ULTRAFAST,  MODE-LOCKED  FIBER  LASER 

FIGURE  5.20.  Schematic  diagram  for  a  femtosecond  pulse  mode-locked  ultra¬ 
fast  diode  laser.  The  emission  of  the  fiber  pigtailed  distributed  feedback  (DFB  ) 
diode  laser  is  coupled  to  the  main  laser  gain  medium  fiber  via  a  wavelength 
division  multiplexer  (WDM).  Laser  output  exits  from  the  laser  resonator  cavity 
in  a  similar  way.  The  resonator  consists  of  a  lens-coupled  high-reflecting  mirror 
and  a  semicondcutor  saturable  absorber  mirror  (SESAM).  The  other  end  of  the 
cavity  contains  the  grating  tuning  element.  For  polarization,  pulse  width  and 
dispersion  control,  extra  elements  are  spliced  in.  Coupling  the  SESAM  in  a  dif¬ 
ferent  way  and  lens  coupling  both  gain  medium  fiber  ends  to  the  tuning  grating 
would  create  a  fiber-ring  laser.  In  a  flexible  and  compact  way,  diode-laser- 
pumped  amplifier  stages,  pulse-width  compressors,  and  frequency-doubling 
stages  can  be  added. 
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New  Optics,  Ltd.,  a  company  that  has  also  developed  mode- 
locked,  tunable  Yb  and  Nd  fiber  lasers.  Typical  specifications  are 
tuning  from  1040  to  1120nm,  3ps  pulse  width  and  30  to  100  MHz 
repetition  rate.  With  some  further  development,  it  may  become  a 
competitor  for  the  Ti:Sa  laser  (Okhotnikov  et  al.,  2003;  Rusu  et 
al.,  2004).  Pumped  by  a  pigtailed  diode  laser,  the  mode-locked 
ultrafast  Er  fiber  laser  FemtoFiber  Scientific  (FFS)  from  Toptica 
Photonics  AG  (Table  5.3)  produces  pulses  with  a  width  of  <100fs 
at  100  ±  10MHz,  optional  tunability  over  1150  to  1400nm  and 
SHG  doubling  to  575  to  700 nm,  1550nm  is  standard  with  200mW 
average  and  1 8  kW  peak  power;  the  Femtolite  family  Er  fiber  laser 
and  Yb-fiber  based  chirped  pulsed  amplifier  FCPA  pjewel  series 
from  IMR A  America  Inc. 

WAVELENGTH  EXPANSION  THROUGH 
NON-LINEAR  TECHNIQUES 

The  spectroscopist  wishing  to  excite  with  laser  light  finds  several 
gaps  in  the  table  of  available  wavelengths.  Although  dye  lasers 
cover  an  extended  portion  of  the  spectrum,  they  require  too  much 
maintenance  to  be  useful  in  standard  confocal  microscopy.  Opti¬ 
cally  anisotropic  crystals  provide  a  good  alternative  towards 
tunable  light  as  long  as  they  are  pumped  with  lasers  having  high 
(pulsed)  peak  power  at  a  high  repetition  frequency. 

Second  and  Higher  Harmonic  Generation: 
Second  Harmonic  Generation,  Third  Harmonic 
Generation,  Fourth  Harmonic  Generation 
Label-Free  Microscopy 

Non-linear  optical  (NLO)  effects  occur  in  certain  classes  of  opti¬ 
cally  anisotropic  crystals  (Lin  and  Chen,  1987;  Tebo,  1988). 
Focusing  intense  laser  light  into  these  crystals  generates  radiation 
at  half  the  wavelength  or  double  the  frequency.  This  process,  there¬ 
fore,  is  known  as  “frequency  doubling”  or  “second  harmonic  gen¬ 
eration”  (e.g.,  Huth  and  Kuizenga,  1987;  Higgins,  1992).  The 
intensity  of  the  frequency-doubled  light  output  is  proportional  to 
the  square  of  the  incoming  light  intensity.  A  Cr:forsterite  laser 
running  at  1230nm  was  used  to  create  a  very  elegant  second  har¬ 
monic  generation  (SHG)  and  third  harmonic  generation  (THG) 
system  operating  at  615  and  410  nm  (respectively)  with  a  pulse 
repetition  rate  of  110MHz  and  a  pulse  width  of  140 ps  (Hogan, 
2004;  see  also  Chapter  40,  this  volume).  When  the  incoming  beam 
is  very  intense,  even  third,  fourth,  or  fifth  harmonics  can  be  gen¬ 
erated.  However,  the  efficiency  of  harmonic  generation  decreases 
at  higher  orders.  Most  doubling  crystals  can  be  angle  or  tempera¬ 
ture  tuned.  THG  is  very  interesting  for  label-free  monitoring  of 
inhomogeneities  at  interfaces,  for  example,  cell  membranes 
(Moreaux  et  al.,  2000).  Yelin  (2002)  and  Brocas  and  colleagues 
(2004)  compare  the  THG  performance  of  several  systems:  the 
Spectra  Physics  Tsunami-OPAL  synchronously  pumped  OPO, 
which  emits  an  80 MHz  150fs  pulsetrain  at  around  1500nm  with 
an  average  power  of  350  mW;  and  the  T-pulse  laser  (Amplitude 
Systemes)  emitting  50  MHz  200  fs  pulses  at  1030nm  with  1.1  W 
of  average  power.  THG  reaches  500  and  343  nm,  respectively,  so 
UV  transmitting  optics  may  be  required.  A  not-so-pleasant  con¬ 
sequence  of  the  doubling  process  is  the  doubling  of  the  noise  level. 
Pump  lasers  with  excellent  stability  are  a  must.  Higher  peak 
powers  and  new,  low-threshold  materials  require  a  careful  recon¬ 
sideration  of  design  strategies  (Beausoleil.  1992).  CSK  Optronics 
(Table  5.3)  sells  a  line  of  compact  doublers  and  triplers  for  Ti:Sa 
lasers.  An  important  issue  in  this  type  of  experiment  is  the  amount 


of  optical  damage  done  to  living  cells  and  tissue  by  these  power¬ 
ful  femtosecond  pulses  (Chu  et  al.,  2003;  and  also  see  Chapter  38, 
this  volume).  THG  peak  intensities  of  100  to  300GW/cm2  illumi¬ 
nation  (i.e.,  nJ/pulse)  are  not  damaging. 

Presently,  it  is  also  possible  to  combine  less  powerful  tunable 
dye  or  semiconductor  lasers  to  cover  the  wavelength  range  from 
170nm  to  18pm  almost  continuously. 

Sum  or  Difference  Mixing 

Another  technique  for  generating  different  wavelengths  from 
certain  basic  laser  wavelengths  is  sum  or  difference  mixing.  When 
two  laser  beams  of  high  and  low  intensity  and  of  frequency  CO,  and 
<»2,  respectively,  are  simultaneously  focused  into  a  non-linear 
optical  crystal,  a  sum  signal  is  generated.  The  intensity  of  the  sum 
signal,  7(0,1+02)  is  proportional  to  the  product  7(0)1)  x  l(m2).  The  higher- 
intensity  (0(  laser  helps  in  generating  enhanced  UV  output  I  (co,  + 
C02)  with  an  extended  tuning  range.  In  a  similar  way,  difference 
mixing,  /(mi_,02)  leads  to  a  tunable  IR  laser.  (For  examples,  see 
Dunning,  1978;  Adhav,  1986;  Herrmann  and  Wilhelmi,  1987; 
Kaiser,  1988;  Marshall,  1994;  Demtroder,  1996) 

Optical  Parametric  Oscillators  and  Optical 
Parametric  Amplifiers 

The  highest  repetition  rate  that  can  be  obtained  depends  on  pump 
energy  storage.  Potma  and  colleagues  (1998)  developed  a  cavity- 
dumped  optical  parametric  oscillators  (OPO)  with  13nJ  pulses 
that  reached  a  repetition  rate  of  4  MHz.  These  devices  can  gener¬ 
ate  a  continuously  tunable  output  by  non-linear  conversion  of  fixed 
wavelength  high  intensity  CW  or  pulsed  laser  light  (Butcher,  1994; 
Radunsky,  2002).  OPOs  are  a  reliable  and  easy-to-use  means  to 
cover  spectral  gaps  left  by  dye  or  Ti:Sa  lasers.  Between  the  Ti:Sa 
fundamental,  covering  700  to  lOOOnm,  and  the  second  harmonic, 
covering  350  to  500  nm,  a  spectral  gap  also  exists  from  250  to  350 
nm.  This  means  that  neither  tryptophan,  tyrosine,  coumarine,  or 
naphthalene  nor  the  caged  probes  can  be  covered  by  a  Ti:Sa.  OPO 
efficiencies  easily  reach  40%  to  60%  (Anderson,  1993),  and  prac¬ 
tical  systems  are  available  from  many  companies  (Table  5.2).  A 
continuously  tunable  output  stretching  from  UV  to  IR  is  generated 
by  non-linear  conversion,  that  is,  optical  frequency  mixing.  Para¬ 
metric  frequency  conversion  creates  two  lower-energy  photons 
(signal  and  idler)  from  each  incoming  photon.  This  process  is  the 
opposite  of  sum  mixing. 

There  are  two  methods  to  overcome  the  low  efficiency  of  the 
parametric  generation:  Method  1  —  Parametric  amplification  in  a 
non-linear  BBO  crystal  pumped  by  the  second  harmonic  of  the 
Ti:Sa  followed  by  pulse  compression.  Parametric  down-conver¬ 
sion  is  incompatible  with  long  pixel  integration  times,  and  high 
pulse  energies  may  possibly  be  damaging,  and  instrumentationally, 
it  is  quite  involved.  Method  2  —  Cavity-dumping  a  synchronously 
pumped  OPO  can  produce  pulse  repetition  rates  up  to  100  MHz. 
This  allows  FLIM  imaging  and  reduces  somewhat  the  average 
power  delivered  and  cell  damage.  In  addition,  residual  pump 
beams  at  400  and  800  nm  are  available.  Synchronous  pumping 
means  that  the  cavity  lengths  and,  therefore,  the  pulse  round-trip 
times  are  equal  in  both  pump  source  and  OPO  (Potma  et  al..  1998). 

Non-critical  phase-matching  (NCPM)  in  a  crystal  is  preferred 
because  it  allows  tight  focusing  of  the  pump  beam  and  long  inter¬ 
action  lengths,  resulting  in  low  OPO  thresholds.  Efficiency  can  be 
further  increased  by  using  periodically  polled  (PP)  waveguides, 
which  do  not  diffract  a  tightly  focused  beam  as  does  bulk  material, 
while  at  the  same  time  they  allow  long  interaction  lengths  with  a 
good  mode  profile.  For  example,  green  output  from  a  Q-switched 
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diode-pumped,  frequency-doubled  Nd-YAG  laser  pumps  a  non¬ 
linear  crystal  creating  a  broad  tuning  range  and  narrow  bandwidth 
with  1 5  mW  average  power  at  around  67 0  nm.  Typical  oscillator  and 
amplifier  crystal  combinations  cover  various  spectral  regions. 
Angle  tuning  a  proper  crystal  will  give  a  405  to  2700  nm 
tuning  range  with  45%  total  (signal  +  idler)  efficiency  (QWU- 
Lasertechnik  GmbH;  Table  5.3).  Continuum  (Table  5.3)  introduced 
the  Sunlite  OPO  (Anderson,  1994a),  which  is  currently  the  Sunlite 
EX  OPO,  with  a  very  broad  tuning  range  stretching  from  222.5  to 
1750nm  without  a  gap  and  providing  line  widths  of  0.075cm'1.  It 
is  pumped  with  100  to  160mJ  pulses  at  355  nm  from  a  Q-switched 
Nd:  YAG.  Such  a  system  is  a  good  replacement  for  pulsed  dye  lasers. 

As  mentioned  above,  there  is  a  trend  to  create  easy-to-operate 
automated  all-in-one-box  solutions.  Spectra-Physics  provides  the 
LBO-based  femtosecond  synchronously  pumped  OPAL.  Near  the 
peak  of  the  OPO  tuning  curve,  it  generates  200  to  250  mW  and 
covers  a  wavelength  range  from  1 100  to  2260  nm  with  pulse  widths 
<130fs.  By  frequency  doubling,  one  covers  the  550  nm  gap  left 
by  the  Ti:Sa  systems.  A  further  improvent  with  higher  average 
power  and  repetition  rates  is  obtained  with  ytterbium  tungstate 
(Yb:KGW)  employed  in  the  Eclipse  fs  OPA  (Spectra-Physics; 
Krueger  and  Feru,  2004). 

Several  OPO  classes  exist  depending  on  the  bandwidth  nar¬ 
rowing  optics  used.  The  narrower  the  bandwidth,  the  less  the  gain. 
In  addition,  the  number  of  passes  through  the  crystal  influences  the 
gain.  A  single-pass  device  creates  little  gain  for  a  5  ns,  355  nm, 
lOOmJ  pulse.  Enclosure  of  the  non-linear  material  in  a  resonant 
cavity  gives  optical  feedback  and  adding  a  pulsed  pump  source 
creates  extra  gain.  The  complete  design  forms  an  optical  paramet¬ 
ric  amplifier  (OPA).  The  frequency  stability  of  an  OPO  depends 
on  the  pump  source  stability  and,  with  enough  stability,  single¬ 
resonant  OPO  (SRO),  with  feedback  for  idler  or  signal,  or  a  double¬ 
resonant  OPO  (DRO)  with  feedback  for  signal  and  idler,  can  be  built 
(Radunsky,  2002).  OPOs  as  well  as  tunable  CW  dye  lasers  are  often 
used  as  narrow  bandwidth  seed  sources  for  OPAs.  For  example,  by 
SHG  doubling  the  820  nm  output  from  a  Ti:Sa  regenerative  ampli¬ 
fier  in  a  BBO  crystal,  one  obtains  410nm.  This  410nm  pumps  a 
second  BBO  crystal,  the  OPA.  An  80-fold  amplification  is  obtained 
resulting  in  lOOmJ  pulses  with  narrow  bandwidth.  Changing  the 
crystal  angle  (angle  tuning)  covers  the  wavelength  range  from 
450  to  700  nm.  These  techniques  can  be  adapted  and  implemented 
into  fiber-optic  OPO  and  OPA  designs  as  well. 

Pulse  Length  Measurement 

Continuous  on-line  monitoring  of  the  picosecond  or  femtosecond 
pulse  shape  immediately  provides  the  feedback  needed  to  align  the 
cavity  and  perform  other  adjustments.  Thermal  drift  may  require 
cavity  length  retuning  twice  a  day,  even  in  a  very  large  air-condi¬ 
tioned  dedicated  laboratory  space.  In  a  smaller  space  this  may  be 
required  far  more  often.  A  pick-off  signal  from  the  monitoring 
diode  is  fed  into  an  auto-correlator  unit  attached  to  a  400  MHz 
digital  oscilloscope.  Several  designs  exist  but  a  computer-assisted 
background-free  design  is  favored.  A  Web  site  explaining  the  oper¬ 
ation  of  a  background-free  correlator  is  http://linac.ikp.physik. 
tu-darmstadt.de/fel/optical_pulse.html.  Both  time-dependent 
amplitude  and  phase  must  be  known  for  many  experimental  pro¬ 
cedures.  Frequency-resolved  optical  gating  (FROG)  creates  a  two- 
dimensional  function  of  the  spectrally  resolved  autocorrelation  at 
a  range  of  correlation  delays.  Second  harmonic-FROG  in  a  non- 
collinear  configuration  is  background  free.  Avesta  (Del  Mar  Ven¬ 
tures;  Table  5.3)  offers  a  semiconductor-based  design  eliminating 
the  non-linear  crystal  and  the  PMT  detector.  Created  from  a  single 


pulse,  the  overlap  of  two  identical,  time-delayed  pulses  with  dif¬ 
ferent  center  frequencies  generates  a  spectral  interference  pattern 
in  spectral  phase  interferometry  for  direct  electric-field  reconstruc¬ 
tion  (SPIDER).  Pulse  spectrum  and  cross-correlations  of  a  pulse 
with  a  pulse  copy  that  has  been  altered  by  a  dispersive  plate  or 
attenuator  in  one  arm  of  the  correlator  are  used  for  phase  and  inten¬ 
sity  determination  from  correlation  and  spectrum  only  (PICASO). 

MAINTENANCE 

On  delivery,  most  laser  systems  perform  well  above  the  guaran¬ 
teed  specifications  given  by  the  manufacturer.  The  consumable 
parts  (i.e.,  laser  tubes,  lamps,  dyes,  and  amplifier  optics  in  solid- 
state  lasers  and  attached  fiber-optics)  are  usually  not  covered  by 
the  warranty.  Depending  on  the  type  of  laser  used,  these  costs  can 
be  substantial. 

A  factor  usually  forgotten  at  the  time  of  an  initial  laser  pur¬ 
chase  is  the  need  to  acquire  peripheral  equipment  for  proper  main¬ 
tenance.  Extra  dyes,  filters,  power  meters,  sampling  oscilloscopes 
and  sampling  units,  fast  diodes,  infrared  viewers,  spectrum  ana¬ 
lyzers,  beam-dumps,  radiation  shields,  warning  signs,  covers,  laser 
goggles,  explosion-  and  fire-proof  cabinets  for  solvent  storage,  and 
hazardous  waste  disposal,  etc.,  are  often  not  included. 

Maintenance  of  Active  Laser  Media 
Laser  Tubes 

The  plasma  tubes  in  ion  lasers  must  be  replaced  when  the  cathode 
sags  or  the  bore  of  the  tungsten  disks  or  BeO  tube  corrodes  to 
become  too  big.  The  “getter,”  always  installed  in  the  laser  system 
to  remove  impurities  in  the  gas,  may  become  saturated.  Cracks  that 
form  in  cooling  tubes  usually  lead  to  its  immediate  failure.  The 
introduction  of  beryllium-oxide  (BeO)  tubes  with  superb  heat  con¬ 
duction  characteristics  has  significantly  reduced  the  fragility  of 
these  laser  systems  and  increased  their  reliability.  Medium-power 
systems  should  last  for  3000  to  5000 h  (see  Tables  5.1  and  5.2)  and 
high-power  systems  for  2000  h. 

Most  laser  systems  come  with  18-month  warranties,  but  some 
self-contained,  sealed  systems  carry  a  warranty  for  up  to  5  years. 
The  technology  has  significantly  improved  over  the  last  few  years 
to  reduce  the  deposition  of  disk  material  and  other  contaminants 
on  the  inner  surface  of  the  Brewster  windows.  For  the  largest  ion 
lasers,  the  aluminum  oxide  ceramic  tube  technology  with  brazed 
tungsten/copper  disks  has  revolutionized  deep  UV  output  power 
levels  and  reliability.  For  He-Cd  lasers,  regular  tube  replacements 
are  necessary  after  lOOOh  of  operation. 

Tube  Replacement  and  the  Secondhand 
Laser  Market 

Several  companies  repair  and  resell  worn  laser  tubes.  Always  ask 
for  warranties  and  try-out.  Old  BeO  tubes  are  considered  haz¬ 
ardous  material  (HazMat)  and  can  for  a  nominal  amount  be 
returned  to  the  manufacturer. 

What  the  User  Can  Do 

To  prolong  the  life  of  a  tube,  check  mirror  alignment  so  that  the 
stabilized  output  can  be  attained  with  minimum  excitation  power 
to  minimize  tube  component  degradation.  Check  the  water  flow 
around  the  laser  head  for  bad  connections  and  look  for  internal 
deposits  at  least  once  a  year.  Keep  dust  out  of  laser  systems;  the 
laser  should  be  connected  to  a  permanent  supply  of  clean,  dry, 
room-temperature,  oil-free  air  or  nitrogen  gas.  A  slight  over- 
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pressure  helps  to  prevent  dust  from  entering  the  laser  head. 
Commercial  air  filters  that  might  introduce  dry  powders  into  the 
laser  system  should  never  be  used.  Synchronously  pumped 
systems  perform  markedly  better  and  the  intensity  stability 
improves  when  dust  is  absent.  Do  not  run  at  the  highest  power 
unless  necessary,  keep  a  user  log,  and  inform  users  on  the  cost  per 
hour  to  prevent  idling. 

Dyes 

Dye  lasers  pose  a  number  of  hidden  costs.  Rhodamine  6G  is  one 
of  the  least  expensive  dyes,  and  is  the  most  stable  with  a  lifetime 
of  2000  Wh  for  a  typical  1-L  reservoir  (i.e.,  the  output  power  of 
the  dye  is  reduced  to  50%  when  it  is  pumped  for  lOOOh  at  2  W  or 
400h  when  pumped  at  5  W).  Dye  aggregates  can  be  filtered  out  but 
evaporation  of  the  solvents  necessitates  periodic  refilling.  When 
the  dye  pump  inadvertently  starts  to  suck  in  air,  air  bubbles  are 
generated  and  cause  immediate  contamination  of  the  optical  mirror 
surfaces.  This  problem  can  be  prevented  by  careful  start-up.  Any 
faint,  sharp  clicks  at  the  nozzle  position  indicate  air  bubbles 
passing  the  nozzle  opening.  Operating  the  dye  laser  with  the  wrong 
solvent,  for  example,  incorrect  viscosity,  may  cause  the  dye  to 
foam,  which  will  quickly  clog  the  system.  In  order  to  reduce  down¬ 
time  and  contamination  of  dyes,  it  is  recommended  that  every  dye 
is  run  with  its  own  pump  module.  This  is  more  expensive,  but 
results  in  a  much  faster  hook-up.  Used  dye  and  leftover  solvent 
should  be  discarded  in  a  manner  that  does  not  pose  a  threat  to 
humans  or  the  environment.  Dye  nozzles  should  be  cleaned  regu¬ 
larly  in  a  sonicator,  especially  when  the  system  has  not  been  used 
for  a  while.  Non-laminar  flow  from  the  jet  nozzle  is  an  indication 
of  nozzle  problems. 

Gases 

Excimer  lasers  need  regular  gas  changes.  Ion  lasers  refill  their  gas 
from  a  ballast  tank  via  a  microcomputer-controlled  valve  system. 
This  reduces  the  risk  of  overfilling  the  plasma  tube  and  degrading 
the  performance  or  losing  laser  power  altogether. 

Laser  and  Amplifier  Rods 

Solid-state  media,  at  least  for  CW  lasers,  are  quite  robust. 
However,  pulsed  systems  need  a  very  good  Gaussian  beam  quality, 
otherwise  the  rod  coating  will  be  damaged.  Often  one  can  hear  this 
as  sharp  clicking  sounds  that  indicate  that  the  laser  needs  imme¬ 
diate  attention.  Improper  Ti:Sa  rod  cooling  leads  to  water  conden¬ 
sation  on  the  pump  surface  and  produces  instabilities.  In  the  worst 
case,  continued  operation  may  damage  the  bulk  of  the  laser  rod 
(Soileau,  1987).  Degradation  of  the  surface  layers  of  the  medium, 
caused  by  the  intense  illumination  from  the  pumping  system  and 
referred  to  as  solarization,  is  less  of  a  problem. 

Diode  Lasers 

These  devices  are  quite  sensitive  and  can  easily  be  damaged  or 
destroyed.  Damaged  systems  may  show  very  weak  output,  a  shift 
in  emission  wavelength,  or  a  change  in  divergence  or  beam  shape. 
Pigtail  fiber-optics,  normally  attached  in  front  of  the  laser  diode, 
also  are  suspect  when  the  output  power  drops  significantly  but  the 
drive  electronics  should  be  checked  as  well  (as  we  recently  expe¬ 
rienced!).  The  fiber-optics  may  have  been  pulled  out  of  its  socket 
or  it  may  have  broken.  At  high  power  levels  damage  to  the  fiber 
surface  may  also  occur,  especially  with  plastic  fiber-optics.  As  dis¬ 
cussed  above,  current  transients  (spikes)  and  electric  discharges 
should  be  prevented  at  all  times.  Forward-biased  semiconductor 
junctions  are  inherently  thermally  unstable;  an  increase  in  current 
increases  the  temperature,  which,  in  turn,  increases  the  current 


even  more.  Therefore,  current  supply  design  is  important  and  must 
be  suited  to  the  specific  laser. 

Maintenance  of  Pumping  Media 

For  CW  or  pulsed  lasers,  which  are  pumped  by  arc  lamps,  the 
lamp(s)  must  be  replaced  when  the  output  power  of  the  laser 
decreases.  On  the  inside  of  the  arc  lamp,  dark  deposits  may 
develop,  which,  over  time,  will  cause  an  increase  in  absorption  and 
catastrophic  failure  due  to  local  heating.  Defective  lamps  usually 
fail  within  the  first  10  to  20  h  after  start-up  (Smith,  1986; 
Fittlechild  and  Mossier,  1988).  Optical  amplifier  components  can 
be  shielded  from  damaging  UV  radiation  by  doping  the  quartz 
lamp  envelope  with  cerium.  Electrode  sputtering  can  be  reduced 
by  using  sintered  tungsten  doped  material  with  a  low-work  func¬ 
tion  to  create  a  more  uniform  cathode  current  and  lower  tempera¬ 
tures  (Erlandson  and  Powell.  1991).  In  addition,  longer  flashlamp 
lifetime  can  be  obtained  by  using  mushroom-shaped  cathodes. 

Maintenance  of  the  Optical  Resonator 

Dust  is  one  of  the  major  enemies  of  the  laser  and  tobacco  smoke 
should  be  kept  out  of  any  laser  laboratory.  Dust  covers  should  be 
in  place  at  all  times  and  any  dust  on  optical  surfaces  should  be 
removed  as  soon  as  possible,  for  example,  on  wavelength  tuning 
prisms,  birefringent  plates,  mode  locker  or  Pockels  cell  surfaces, 
or  mirrors.  As  using  the  wrong  solvent  to  clean  optics  can  destroy 
the  optical  coating,  use  only  the  ultra-pure  solvents  recommended 
by  the  manufacturer  and  follow  the  proper  procedure.  When  in 
doubt,  first  try  it  out  on  a  test  surface.  Most  laser  manuals  describe 
in  detail  the  proper  cleaning  methods.  Never  use  the  same  side  of 
a  cleaning  wipe  for  more  than  one  pass.  A  patterned  movement 
should  be  used  when  cleaning  optical  surfaces. 

The  optical  coatings  on  laser  cavity  mirrors  and  external 
mirrors  or  fibers  cannot  withstand  prolonged  exposure  to  UV  radi¬ 
ation.  Colored  rings  will  appear  on  the  coating  surfaces  or  they 
may  look  hazy  or  foggy.  The  formation  of  color  centers,  which 
cause  increased  absorption,  occurs  in  fused  quartz  materials: 
lenses,  optical  fibers,  etc.  Color  centers  are  usually  caused  by 
included  impurities.  Resulting  lattice  defects  are  mostly  single 
electrons  bound  to  negative-ion  vacancy  sites:  F-centers.  Free  elec¬ 
trons  created  by  intense  radiation  become  trapped  in  these  sites. 
Excimer  lasers  with  their  high  UV  peak  powers  are  particularly 
prone  to  creating  color  centers.  At  high  power,  too  tight  focusing, 
or  self-focusing,  conditions  can  produce  mechanical  damage 
(remember,  lasers  are  used  for  machining!;  Gunther,  1993). 
Damaged  parts  must  be  replaced  immediately  to  prevent  the  dete¬ 
riorated  beam  from  damaging  other  components.  This  is  especially 
true  for  pulsed  systems.  Frequency-doubling  elements  should  be 
inspected  regularly  or  whenever  laser  intensity  fluctuations  occur. 
A  microscope  may  be  necessary  to  see  damage  to  the  coating. 
Heating  of  some  crystals  (e.g.,  KTP)  seems  to  reduce  these  effects. 
Frequency-doubling  and  other  hydroscopic  crystals  should  be  kept 
under  optimum  dry  conditions  at  all  times. 

Maintenance  of  Other  System  Components 
Cooling  Water 

A  gradual  decrease  in  laser  power  can  be  caused  by  the  growth  of 
algae  in  the  cooling  water.  A  small  addition  of  sodium  azide  (NaN3) 
prevents  a  reoccurrence  of  the  growth.  An  annual  check  of  the 
rubber  rings  in  the  cooling  hoses  is  recommended.  At  least  once  a 
year  the  cooling  circuitry  should  be  checked  for  galvanic  and  other 
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types  of  corrosion  (Schneider  and  Williams,  1993).  Do  not  use  tap 
water  for  cooling.  Many  laser  and  temperature  bath  manufacturers 
now  offer  closed-loop  laser  cooling  systems.  Heat  removal  with 
air-to-liquid  cooling  is  also  possible  (Goldman,  1993).  For  proper 
operation  of  a  Nd-YAG  laser,  the  resistivity  of  the  cooling  water 
should  be  in  the  range  recommended  by  the  manufacturer.  When 
the  resistivity  is  too  low,  the  lamps  will  not  start.  If  it  is  too  high, 
the  plating  on  the  inside  of  the  elliptical  resonator  will  dissolve 
and  cause  a  decrease  in  laser  power.  Water  filters  and  de-ionizing 
filters  should  be  replaced  regularly  as  demanded  by  the  perfor¬ 
mance  of  the  laser.  Stacked  diode  laser  arrays,  small  in  size  but 
with  several  tens  of  watts  of  optical  output  power,  require  liquid 
cooling  with  micro-channel  technology  or  thermo-electric  (Peltier) 
cooling. 

External  Optics 

All  optical  surfaces  should  be  kept  as  clean  as  possible.  Mirror  sur¬ 
faces  exposed  to  UV  radiation  will  have  to  be  replaced  regularly, 
depending  on  the  impinging  power  density.  Coatings  may  peel  off. 
Apertures  in  spatial  filters  should  be  inspected  and  replaced  when 
damage  (burn)  occurs.  Fiber-optics,  especially  the  ones  exposed  to 
high-intensity  UV  light,  may  develop  color  centers,  which  result 
in  increased  absorption  and,  in  the  end,  failure  of  the  entrance 
section.  High-intensity  visible  or  infrared  light  may  lead  to  over¬ 
heating  at  the  input  fiber  tip.  Fortunately,  the  length  of  most  fibers 
is  sufficient  to  allow  them  to  be  cut  back,  repolished,  and  recon- 
nectorized,  though  sometimes  at  substantial  cost  and  time  delays. 
If  they  are  bent  too  tightly,  fibers  will  break,  causing  a  sudden  loss 
of  output  power.  Pigtail  fibers  attached  to  the  main  body  of  a  laser 
diode  may  dislodge  themselves.  Due  to  the  automated  fabrication 
process,  which  includes  angular  positioning  to  optimize  the  output 
and  to  use  the  most  intense  emission  spot,  manual  reattachment 
usually  does  not  re-establish  the  original  power  level.  Laser 
damage  thresholds  for  dielectric  mirrors  and  anti-reflective  coat¬ 
ings  are  roughly  250MW/cm2  at  532nm  and  500 MW/cm2  at  1064 
nm,  assuming  surfaces  are  spotless  and  clean  (Aubourg,  2002). 

TROUBLESHOOTING 

A  very  extensive  and  instructive  body  of  information  for  gas,  dye, 
and  diode  laser  maintenance,  including  optics  cleaning  and  repair, 
exists  in  Sam’s  Laser  FAQ  (see  Web  site  listings). 

SAFETY  PRECAUTIONS 

Laser  hazards  include  thermal  and  fire,  acoustic  shockwave,  and 
strongly  wavelength-dependent  photochemical  damage.  Types  of 
beam  exposure  are  direct  exposure  intrabeam,  specular,  and  diffu¬ 
sion  reflection  effects.  Eyes  and  skin  are  most  commonly  affected. 
Non-beam-related  hazards  include  electrical  shock,  capacitors, 
hose  leakage,  water  vapor  condensation,  air  contaminants,  fumes, 
aerosols  with  biological  agents,  cadmium  and  zinc  telluride  (that 
burns  in  the  presence  of  high  laser  intensity  and  oxygen),  radia¬ 
tion  damage,  fire,  compressed  gases,  gas  cylinders,  earthquake 
damage,  excimer  gases,  and  laser  dyes. 

All  lasers  are  generally  divided  into  four  classes: 

Class  1.  Embedded  lasers  and  laser  systems 

Laser  completely  enclosed,  radiation  not  accessible  during  use. 
Class  1M.  Lasers  and  laser  systems: 

CW  =  <40|iW  blue  and  =  <400[lW  red. 

Very  low  power:  Safe  for  long-term  intrabeam  viewing. 


Class  2.  Low  power  visible  lasers  and  laser  systems: 

CW  =  <lmW. 

Low  power  level.  Safe  for  brief  accidental  naked  eye  direct  expo¬ 
sure  with  blink  and  aversion  response  active. 

Class  2M.  Low-power  visible  lasers  and  laser  systems. 
Low-power  visible  collimated  or  divergent  large  beam  diame¬ 
ters.  Potential  hazard  with  magnifiers 

Class  3R.  Visible.  Low-power  lasers  and  laser  systems. 
Accidental  exposure  usually  not  hazardous  but  eye  injury  possible 
upon  intentional  long-term  viewing.  Training  required.  Equivalent 
to  Class  IIIA  “danger”  [Center  for  Devices  and  Radiological 
Health  (CDRH)]  and  ANSI  3a  (USA). 

Class  3R:  Invisible.  Low-power  lasers  and  laser  systems. 

Wavelength  dependent,  limits  are  five  times  those  of  Class  1 . 

Class  3B.  Medium-power  lasers  and  laser  systems: 

CW  =  <500  mW. 

Serious  eye  injury  even  for  brief  accidental  exposure  to  direct 
beams  possible.  Training  required. 

Class  4.  High-power  lasers  and  laser  systems. 

Even  diffuse  and  certainly  direct-eye  exposure  will  lead  to  serious 
eye  and  skin  injury.  Poses  fire  hazard  as  well.  Training  required, 
with  M:  magnifying  instruments  and  R:  relaxed  requirements. 
Limiting  values  for  small  point-like  lasers  with  angular  retinal  spot 
size  smaller  than  1.5mrad. 

General  safety  precautions  that  become  more  stringent  with 
increasing  classification,  must  be  followed  when  operating  a  laser 
[International  Commission  of  Non-Ionizing  Radiation  Protection 
(ICNIRP)  and  International  Electrotechnical  Commission  (IEC) 
60825-1  guidelines,  see  revised  ANSI-Z-136. 3  1996  classification, 
safe  use  of  lasers;  Tozer,  2001;  Schulmeister,  2003;  Sliney,  1994; 
LIAand  Rockwell  Laser  Industries  Inc.;  Table  5.3].  Separate  photo¬ 
chemical  and  thermal  retinal  exposure  limits,  including  limits  for 
damage  from  ultrashort,  <lns  pulses,  have  been  added  as  well  as 
the  *.M  classes.  Every  laboratory  should  have  a  Laser  Safety 
Officer  (LSO)  who  should  be  consulted  when  necessary.  This 
person  should  be  responsible  for  the  proper  training  of  users  of 
laser-assisted  equipment.  Safety  training  videos  are  available  from 
LIA,  OSHA,  and  Rockland  Laser  Industries  (Table  5.3)  and  often 
also  from  your  own  local  university.  Outside  the  laser  laboratory, 
a  “laser  in  use”  warning  sign  must  be  posted  and  a  red  warning  light 
should  be  positioned  at  the  entrance  to  the  laser  room. 

Inside  the  laser  laboratory,  safety  precautions  are  necessary 
even  for  users  of  enclosed  systems  attached  to  CLSMs  whenever 
the  system  is  opened  or  when  the  fiber-optics  are  aligned  and  expo¬ 
sure  to  a  laser  beam  becomes  a  possibility. 

A  brief  “Do  and  Don’t”  list: 

•  Inform  everybody  near  the  laser  setup  to  be  worked  on.  Check 
the  sign-up  agenda. 

•  Check  and  put  all  required  accessory  equipment  on  the  ready. 

•  Have  the  entrance  warning  red  light  on  when  any  laser  is  on. 

•  Remove  everybody  from  the  laser  site  whose  presence  is  not 
required. 

•  Close  the  access  doors. 

•  Close  curtains. 

•  Remove  rings,  watches,  and  ties. 

•  Put  safety  goggles  on. 

•  Never  look  directly  into  any  laser  beam. 
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•  Align  open  laser  heads  with  minimum  power. 

•  Check  for  stray  light  and  reflections  with  a  hole-punched  busi¬ 
ness  or  IR  indicator  card. 

•  Block  beams  not  being  attended. 

•  Disconnect  electrical  connections. 

•  Discharge  large  capacitors  with  a  grounded  large  screwdriver 
with  an  insulated  handle. 

Beam  Stops 

Beam  stops  should  be  made  out  of  anodized,  flat  black  aluminum 
and  positioned  in  such  a  way  that  no  radiation  is  reflected  back 
into  the  room.  Stray  reflections  must  be  prevented;  powerful  IR  or 
visible  beams  may  easily  start  a  fire  on  electrical  cables  or  other 
flammable  materials. 

Curtains 

Curtains  should  be  made  of  special  non-combustible  material, 
preferably  black  in  color.  When  a  spot  of  1  mm2  is  irradiated  with 
approximately  1  MW/m2  or  more,  the  black  side  should  emit  a  non¬ 
toxic  smoke  (that  nevertheless  may  form  damaging  deposits  on 
critical  optical  surfaces!)  and  may  glow.  See,  for  example,  the 
Rockwell  Laser  Industries,  Inc.  and  Lasermet  Ltd.  Web  sites.  Cur¬ 
tains  are  designed  for  low-power,  100  W/cm2  or  up  to  1.2kW/cm2 
lasers  (MacMullin,  2004).  Curtain  shielding  will  prevent  the 
accidental  illumination  of  other  people  while  aligning  a  laser  or 
beam-splitter. 

If  a  visual  control  is  necessary  from  outside  the  room,  a 
radiation-absorbing  plastic  window  should  be  mounted  in  the  door 
or  wall.  Tape  should  not  be  used  to  hold  curtains  together  because 
this  could  allow  the  build-up  of  poisonous  gases.  Clothing  is 
another  possible  source  of  trouble  that  is  often  disregarded. 
Famous  are  the  stories  of  halved  neckties. 

Laser  Goggles 

A  large  range  of  eye  protection  devices  is  available  (e.g.,  from 
LFW  and  Uvex;  Table  5.3).  A  telling  Ti:Sa  fovea  burn  track  is 
shown  in  Robinson  (1998).  It  is  important  to  realize  that  a  1  W/cm2 
irradiance  level  at  the  cornea  can  become  a  focused  100kW/cm2 
retina  spot  of  about  20  pm  diameter. 

Screens 

Where  possible,  anodized,  flat  black  aluminum  pipes  should  be 
used  to  enclose  all  laser  beams.  These  not  only  protect  the  opera¬ 
tor,  they  also  reduce  dust  in  the  beam  line. 

Exposure  Effects,  Warning  Signs,  and  Interlocks 

Each  laser  system  must  be  equipped  with  the  proper  warning  signs 
and  interlocks.  The  listed  references  (Sliney  and  Wolbarsht,  1980; 
Rockwell,  1983,  1986;  Winbum,  1985;  Sliney,  1986)  describe  in 
detail  electrical  hazards  and  detrimental  reflections  from  rings, 
watches,  etc.  Biological  effects  are  also  extensively  covered,  that 
is,  thermal  and  photochemical  effects  upon  exposure  to  CW  and 
pulsed  laser  radiation,  and  the  maximum  permissible  exposure  to 
eyes  and  skin. 

Chemical  hazards  may  be  caused  by  laser  dyes,  some  of  which 
are  mutagenic.  Dye  solvents  and  laser  gases  must  be  properly 
handled.  Basic  protective  measures  include  laser  goggles  (Burgin, 


1988),  cleanliness,  and  proper  handling  of  chemicals.  A  sufficient 
number  of  eye  protection  devices  should  always  be  at  hand  when 
laser  system  components  have  to  be  repositioned  or  realigned. 
Naturally,  one  should  reduce  the  power  level  as  much  as  possible 
during  alignment.  Periodic  training  meetings  and  medical  check¬ 
ups  are  recommended  and  can  be  used  as  a  certificate  of  good  laser 
usage  conduct. 

Infrared  Paper 

To  safely  find  infrared  beams,  one  can  rely  on  infrared  viewers  but 
these  are  fairly  expensive.  An  alternative  is  the  use  of  IR  indica¬ 
tor  cards.  These  emit  a  visible  luminescence  via  electron-trapped 
up-conversion.  Suppliers  include  Bromba  GmbH,  Edmund 
Scientifics,  Electro  Optical  Components,  and  Lumitek  Inti.  Inc. 
(Table  5.3). 


CONCLUSION 

In  the  previous  sections,  we  have  described  the  large  number  of 
lasers  currently  available  for  CLSM.  These  lasers  vary  widely  in 
characteristic  wavelengths,  tunability,  stability,  output  power,  ease 
of  use,  and  price.  Rapidly  expanding  areas  such  as  diode  lasers, 
optical  parametric  amplification,  and  direct-doubling  diode  lasers 
would  require  an  update  every  three  months.  Solid-state  laser 
systems  with  wide  tunability  through  the  incorporation  of  optical 
parametric  techniques  and  fiber  lasers  are  areas  showing  rapid 
development. 

Brelje  and  colleagues  (1993)  show  a  figure  displaying  the  exci¬ 
tation  wavelengths  of  various  popular  fluorophores  with  matching 
ion  laser  lines.  This  picture  has  certainly  changed  in  the  past  years 
with  the  increase  in  power  of  solid-state  and  fiber  lasers  and  the 
expansion  of  their  range  of  tunability.  For  the  moment  we  sum¬ 
marize  several  of  the  most  often  used  lasers  in  Table  5.4.  It  is  clear 
from  this  table  that  the  ion  lasers  are  being  pushed  out  the 
door. 

•  For  CLSM,  a  stable,  TEM00  mode,  easy-to-use  laser  with  short 
coherence  length  is  imperative. 

•  Wavelength  tunable  lasers  require  pulsed  pump  lasers  with 
high  peak  power,  good  mode  quality,  and  the  best  possible  sta¬ 
bility  to  reduce  noise  in  the  subsequent  wavelength  conversion 
and  amplifier  stages,  as  well  as  OPO,  OPA,  or  master  oscilla¬ 
tor  parametric  amplifier  (MOPA)  optics.  The  best  source  is  cur¬ 
rently  the  tunable  Ti:Sa  laser.  Excimer,  nitrogen,  and  similar 
pulsed  lasers  and  mode-locked  solid-state  lasers  are  still  useful 
pump  sources  for  tunable  dye  lasers  if  pulse  repetition  rates 
are  sufficient  for  your  purpose.  The  goal  should  be  to  each  have 
a  US$6000  modular,  personal  2-photon  source! 

•  Miniaturization  and  ease-of-operation  via  efficient,  low-cost, 
low-noise,  tunable  semiconductor  and  fiber  lasers  is  a  reality 
now  and  these  lasers  are  replacing  big  CW  and  pulsed  laser 
systems.  A  trend  exists  towards  modularity,  efficiency,  and 
small  footprint. 

•  With  the  advent  of  completely  automated,  self-contained  laser 
systems  (Mai-Tai  and  Chameleon)  the  ideal  turnkey  system  is 
almost  present.  Older,  bigger,  flexible  laser  systems  cannot  be 
considered  turnkey  systems  and  require  constant  attention 
from  well-trained,  qualified  persons. 

•  Laser  safety  training  is  a  must  also  for  CLSM  users  and  eye 
protection  and  other  safety  measures  should  be  at  hand  as 
required. 
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TABLE  5.4.  Lasers  Often  Used  in  LSCM 


Manufacturer  and  CLSM  system 


Visitech 

international 

Leica 

Nikon 

Olympus 

Zeiss 

Vt-eye  and 

TCP-SP2-AOBS 

Digital  Eclipse 

Fluoview 

Perkin-Elmer 

WITEC 

Axiovert  200M 

Laser  Type 

Vt-Infinity 

SP2 

Cl  Plus 

FV1000  &  FV500 

Ultraview 

CRM200 

LSM  510  META 

Fixed  wavelength  lasers 

HeCd 

UV  325  nm 

10-30  mW 

Argon-ion 

UV  351  nm 

40mW 

300  mW 

Argon-ion 

UV  364  nm 

Argon-ion 

UV  351,  364  nm 

Diode  laser 

50mW 

>20  mW 

80  mW  Enterprise 

405  nm 

6mW 

20  mW 

6mW 

7.5  mW 

25  mW 

(25 mW,  opt) 

50  mW  (510  Live) 

Diode  laser 

408  nm 

20  mW  (mod.) 

Diode  laser 

DPSS  430  nm 

8mW 

Diode  laser 

440  nm 

5  mW  (mod.) 

0.7  mW 
(5mW  opt.) 

7  mW 

16mW  (510  Live) 

HeCd 

442  nm 

20  mW 

6mW 

20  mW 

11  mW 

Argon-ion 

458,  514nm, 

Multi-line 

25  mW 

lOOmW  @  457, 

40  mW 

30  mW 

488,  514,  568. 

25  mW  @ 

30  mW 

458,  514nm 

476,  488, 

40mW  @  457, 

7.5/15  mW 

457-514.1  nm 

458,  477,  488, 

514nm  65mW 

488,  515  nm 

each 

514nm 

Blue  only 

488  nm 

+ 

10  mW 

IOmW 

+ 

Laser  diode 

488  nm 

DPSS 

lOOmW  (510  Live) 

532  nm 

75  mW  (510  Live) 

Green  HeNe 

543  nm 

Yellow  DPSS 

+ 

1.5  mW 

1.5  mW 

1  mW 

1.25  mW 

1  mW 

561  nm 

IOmW  (510-META) 

Krypton  ion 

568  nm 

25  mW 

IOmW 

Krypton/ Argon 

488,  568,  647  nm 
Orange  HeNe 

20  mW 

75  mW 

+ 

594  nm 

2mW 

2mW 

2mW  (510-META) 

Red  HeNe 

632.8  nm 

lOmW 

5  mW 

IOmW 

3.5  mW 

5mW 

Diode  laser 

33  mW 

630  nm  (ext.  cavity) 

635  nm 

638nm 

lOmW  (mod.) 

640  7.8  mW 

35mW(510  Live) 

NIR  750  nm  2mW  CW 

HeNe 

NIR  1152  nm  lmW 

(Continued) 
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TABLE  5.4.  Lasers  Often  Used  in  LSCM  (Continued) 


Manufacturer  and  CLSM  system 

Visitech 

international 
Vt-eye  and 

Laser  Type  Vt-Infinity 

Leica 

TCP-SP2-AOBS 

SP2 

Nikon 

Digital  Eclipse 
Cl  Plus 

Olympus 

Fluoview 

FV1000  &  FV500 

Perkin-Elmer 

Ultraview 

WITEC 

CRM200 

Zeiss 

Axiovert  200M 
LSM  510  META 

Broad-band  tunable  pulsed  lasers 

Dye  laser 

630  nm 

3mW 

Ti:Sa-Chameleon 

lOOOmW 

lOOOmW 

720-930  nm 

>1  ps 

<100  fs 

Ti:Sa-Mai-Tai 

<  lOOOmW 

<  lOOOmW 

750-850  or  71 0-920  nm 

>1  ps 

<100  fs 

Ti:Sa-Mira 

100-700  mW 

100-700  mW 

690-1000  nm 

>1  ps 

<100  fs 

Ti:Sa-Tsunami 

100-700  mW 

1 00-700  mW 

680-1050  nm 

>1  ps 

<100-1 30  fs 

NIR  laser 

1.2ps,  lOOOmW 

700-900  nm 

720-1000  nm 

Mod.:  modulated 
Pp  peak-to-peak 

Visitech  International,  Inc,  Sunderland,  UK  +44  (0)191  516  6255  http://www.visitech.co.uk/ 

Leica  Microsystems,  Mannheim,  Germany,  Ph.  +49-(0)62 1-70280  http://www.leica-microsystems.com/website/lms.nsf 
Nikon  Inc.,  Melville,  NY.  Ph.  (800)-526-4566,  http://www.nikonusa.com/,  http://www.nikon-instruments.jp/eng/ 

Olympus  America,  Melville,  NY.  Ph  (631)-844-5000,  http://microscope.olympus.com/,  http://www.olympusamerica.com/seg_section/seg_home.asp  and 
Perkin-Elmer,  Boston,  MA,  Ph.  (800)-762-4000,  http://las.perkinelmer.com/content/livecellimaging/index.asp 
Carl  Zeiss  AG,  Oberkochen,  Germany,  Ph.  +49-(0)7364-200  http://www.zeiss.com/ 

WITec  GmbH,  Ulm,  Germany,  Ph.  +49-(0)700-948-32366,  http://www.WITec.de  Confocal  Raman  microscope 
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Dr.  Alexander  N.  Cartwright,  http://www.ece.buffalo.edu/faculty/cartwright/ 
links.html 

Laser  Information:  Cleaning  of  Laser  Optics 

http://www.repairfaq.org/sam/laseratr.htm#atrcln 

General  Information  with  Many  Links 

http://resonator.physics.sunysb.edu/laser/laserlinks 

History  of  the  Laser 

http://www.bell-labs.com/history/laser/ 

Schawlow  and  Townes  biographies 

Titanium-Sapphire  Laser 

http://www.drbio.cornell.edu/Infrastructure/Apparatus_WWW/Ti%20 
S  apphire%20Lasers  .html 
Evolution  of  Ti:Sa  laser  systems 

Pulse  Length  Measurement  by  Optical  Means 

http://linac.ikp.physik.tu-darmstadt.de/fel/optical_pulse.html 
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Safety  Issues 

The  health  and  safety  committee  at  educational  institutions  should  have  Stan¬ 
dard  Operating  Procedures  (SOP)  for  laser  operation  and  maintenance 
available.  As  examples  may  serve: 
http://phantom.ehs.uiuc.edu/~rad/laser/ 
http://www2  .umdnj  .edu/ eohss  web/ aiha/technical/lasers  .htm 
http  ://web .  princeton .  edu/ sites/  ehs/laserguide/  appendixC .  htm 
Laser  safety  self-audit  checklist,  http://www.microscopyu.com/articles/ 
fluorescence/lasersafety.html 

Spring,  K.R.,  Parry-Hill,  M.J.,  etal..  Molecular  Expressions,  Florida  State  Uni¬ 
versity,  Tallahassee,  Florida.  Laser  classifications,  hazards,  and  bioeffects 
of  various  laser  types,  a  brief  and  concise  overview  worth  your  attention. 
http://stwi.weizmann.ac.il/Lasers/laserweb/  http://www.phys.ksu.edu/perg/ 
vqm/laserweb/Misc/ Apindex.htm 
Dr.  Rami  Arieli,  The  Laser  Adventure,  Virtual  school 

Rockwell  Laser  Industries.  Safety  materials,  regulations:  laser  safety  standards 
organizations  and  products,  e.g.,  Z136.1  ANSI  document,  laser  accident 
database,  access  control  measures,  and  courses,  http://www.rli.com/ 
Rockwell  Laser  Industries  safety  issues  and  links  laser  bioeffects,  manu¬ 
facturers,  standards,  and  laser  user  institutes  and  laboratories, 
http://www.rli. com/resources/links.asp?Link=3 
http://repairfaq.ece.drexel.edu/sam/CORD/leot/ 

Laser  Electro-Optics  Technology  (LEOT),  Center  For  Occupational  Research 
and  Development  (CORD),  Waco,  Texas,  electrical  safety  issues. 
Standards,  hazards,  and  solutions,  http://www.osha.gov/SLTC/laserhazards 

Specific  Lasers 
Argon-Ion  Laser 

One  page  argon-ion  laser  basic  construction  and  emission  properties,  one  page 
overview  of  laser  types  and  applications,  basic  level,  http://www.rli.com/ 
resources/argon. asp 

Basic  argon-ion  laser,  graph  of  relative  laser  line  intensities,  http://www.rli. 
com/resources/argon. asp 

Advanced  Level 

Ionized  gas  spectra,  construction  characteristics,  http://www.lexellaser.com/ 
techinfo_gas-ion.htm 

Laser  wavelengths  chart,  http://www.lexellaser.com/techinfo_wavelengths.htm 

Dye  Laser 

Commercial  laser  dye  as  well  as  CLSM  companies  have  colorful  wall  charts 
and  list  dyes  and  matching  laser  lightsources  on  their  sites. 

Advanced  Level 

Dr.  R.  Alexander,  http://members.aol.com/kpublish/Laser/Dye_Laser.html 
Dr.  Ladic  site  with  excitation  and  emission  spectrum  maxima  for  dyes  useful 
for  CLSM,  http://www.cs.ubc.ca/spider/ladic/images/fluoro.gif 

Expert  Level 

Instructive  site  from  LEOT,  http://repairfaq.ece.drexel.edu/sam/CORD/leot/ 
course03_mod  1 0/mod03- 1 0.html 

Colliding  Pulse  Dye  Laser 
Expert  Level 

http://utol.ecen.ceat.okstate.edu/thz%20interactive%20tour/cpmlaser.htm 

Fiber  Lasers  Tutorial 

Construction,  wavelengths,  and  optical  characteristics,  http://www.imra.com/ 
lasers-tech-tut-frame-pdf.html 

Titanium-Sapphire  Laser 
Basic  Level 

Evolution  of  Ti:Sa  laser  systems,  http://www.drbio.comell.edu/Infrastruc- 
ture/Apparatus_WWW/Ti%20Sapphire%20Lasers.html 


Advanced  Level 

http://micro.magnet.fsu.edu/primer/techniques/microscopylasers.html 
Spring,  K.R.,  Parry-Hill,  M.J.,  etal..  Molecular  Expressions,  Florida  State  Uni¬ 
versity,  Tallahassee,  Florida.  Beam  divergence,  beam  expander,  fiber-optic 
coupling  (*),  single  and  multiple  mode  fiber-optic,  laser  types. 

Expert  Level 

Dr.  Gavin  D.  Reid,  Leeds  University,  and  Klaas  Wynne,  University  of 
Strathclyde,  Glasgow,  UK,  Introduction  to  ultrafast  laser  systems  and 
optical  characterization,  dispersion  and  its  control,  pulse  broadening,  ampli¬ 
fication,  optical  Kerr  effect,  white  light  generation,  OPA,  auto  and  cross  cor¬ 
relation, http://phys.strath.ac.uk/12-503B2/introduction/introduction.html 
Construction  of  a  fs  mode-locked  laser  with  a  nice  Ti:sa  energy  diagram, 
http://www.df.unipi.it/~fisapp/Gruppi/Metrologia/spiegazioni/boris.pdf 

Troubleshooting 

http://wWW.repairfaq.Org/sam/laseratr.htm#atrcln 

Tutorial  Sites 

Classification  of  sites: 

•  Basic  level  indicates  many  images,  applets  but  no  formulae. 

•  Advanced  level  with  more  thorough  description  of  the  issues  supported 
often  with  applets. 

•  Expert  level  with  equations  and  examples. 

Basic  Level 

John  Gormalli,  stimulated  emission  explained,  Web  Science  Resources  Net 
(WSRNet),  http://members.aol.com/WSRnet/tut/tl .htm 
Stimulated  emission  explained,  laser  theory  quiz,  http://home.achilles.net/ 
~ypvsj//quiz/index.html 

Laser  theory  and  applet  on  stimulated  emission,  http://www.point-source.com/ 
LaserTheory.  asp 

Basic  atom  model,  general  laser  classification  and  safety  issues,  http://www. 
howstuffworks.com/laser.htm 

One  page,  ruby  laser  design  and  applications,  http://home.achilles.net/ 
-ypvsj/Zglossary/laser.html 

Fraunhofer  Institute’s  laser  tutorial,  http://www.ilt.fhg.de/ger/lasertutorial.html 

Advanced  Level 

http://micro.magnet.fsu.edu/primer/lightandcolor/laserhome.html 
Spring,  K.R.,  Parry-Hill,  M.J.,  etal..  Molecular  Expressions,  Florida  State  Uni¬ 
versity,  Tallahassee,  Florida.  Laser  fundamentals,  safety  issues,  micro¬ 
scopy  and  lasers  used,  intensity  modulation,  references,  many  instructive 
applets  worthwhile  to  visit. 

http://www.olympusmicro.com/primer/lightandcolor/lasersintro.html 
Samual  M.  Goldwasser’s  site,  “A  Practical  Guide  to  Lasers  for  Experimenters 
and  Hobbyists,”  Version  6.60,  very  extensive  with  safety  issues,  reference 
books  and  materials,  schematics,  construction,  safety,  repair  issues, 
http://www.repairfaq.org/sam/laserfaq.htm 
Dr.  Alexander  N.  Cartwright,  University  of  Buffalo,  laser  gain  and  pumping 
schemes,  photonics  educational  applets,  http://www.ece.buffalo.edu/ 
faculty/cartwright/photonics/rateequations.html 

Expert  Level 

Online  educational  resources  for  physics  teachers 
http://www.ba.infn.it/www/didattica.html 
Dr.  Rami  Arieli,  The  Laser  Adventure,  virtual  school  about  lasers  and  their 
applications.  Optics  aspects,  basic  laser  theory,  laser  cavities,  lasing 
modes,  fluorescence,  laser  line  shape  and  width,  laser  beam  properties, 
power,  applets,  applications,  laser  safety  issues,  questions,  reference 
books,  very  extensive  and  useful  also  for  course  development, 
http://www.phys.ksu.edu/perg/vqm/laserweb/ 

Laser  Electro-Optics  Technology  (LEOT),  Center  For  Occupational  Research 
and  Development  (CORD),  Waco,  Texas.  Introduction  to  laser  optics,  elec¬ 
tronics,  components,  systems,  electrical  safety  issues,  http://repairfaq. 
ece.  drexel .  edu/sam/  CORD/leot/ 


Laser  Sources  for  Confocal  Microscopy  •  Chapter  5  125 


Dr.  P.L.  Cross,  Optical  concepts,  laser  cavity  design,  laser  operation,  database 
of  spectral  properties  of  lasing  materials,  absorption  and  emission  spectra, 
http ://  ae  sd .  larc .  nasa .  go  v/GL/tutorial/laser/las_out .  htm 

List  Servers 

http://www.msa.microscopy.com/MicroscopyListserver/ 

MicroscopyArchives.html 

Confocal  microscopy  list  server  for  maintenance  and  trouble  shooting.  When 
there  is  a  problem  often  it  helps  to  look  for  similar  experiences  and  solu¬ 
tions  on  the  Web.  A  good  site  to  start:  http://listserv.acsu.buffalo.edu/ 
archives/confocal.html 

Microscopy  Laboratory  Sites 

Dr.  W.  Webb,  DRBIO,  Cornell  University,  Ithaca,  New  York.  Multi-photon 
excitation,  physical  principles,  excitation  cross-sections,  fluorophore 


photostability,  fluorescence  techniques,  e.g.,  FPR(photobleaching 
recovery),  FCS,  SPT,  cell  viability,  resolution  with  an  underfilled  objec¬ 
tive,  real-time  image  acquisition.  A  lot  of  specialized  information  together 
for  live  cell  imaging  and  spectroscopy,  http://www.drbio.cornell.edu/ 
Infrastructure/Infrastructure%20Index.html 

Dr.  Bruce  Jenks,  Nijmegen  University,  The  Netherlands.  Laser  principle, 
optical  scanner  principle,  optical  sectioning,  single  and  multi-photon  illu¬ 
mination  (dis)advantages,  http://www.celanphy.sci.kun.nl/Bruce%20web/ 
scanning%20microscopy.htm 

Optics 

Ultrafast  optics,  http://www.newport.com/Support/Tutorials/Optics/o4.asp 
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LED 

Light-Emitting  Diode 
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Large  Mode  Area 
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Alternating  Current 

LPC 

Laser  Power  Controller 

ASE 

Amplified  Spontaneous  Emission 

Mil 

Multi-Photon  Intrapulse  Interference 

AOTF 

Acousto-Optic  Tuning  Filter 

MOPA 

Master  Oscillator  Parametric  Amplifier 

AOBS 

Acousto-Optic  Beam  Splitter 

NA 

Numerical  Aperture 

AOD 

Acousto-Optic  Deflector 

NCPM 

Non  Critical  Phase  Matching 

AOM 

Acousto-Optic  Modulator 

Nd-YAG 

Neodymium-Yttrium-Aluminum  Garnet 

BBO 

Beta  Barium  Borate 

Nd-YLF 

Neodymium- Yttrium  Lithium  Fluoride 

CCD 

Charge-coupled  Device 

Nd-YV04 

Neodymium- Yttrium  Ortho  Vanadate 

CDRH 

Center  for  Devices  and  Radiological  Health 

NLO 

Non-Linear  Optical 

CLSM 

Confocal  Laser  Scanning  Microscopy 

NMR 

Nuclear  Magnetic  Resonance 

CTA 

Cesium  Titanyl  Arsenate 

OC 

Output  Coupler  Mirror 

CW 

Continuous  Wave 

OCT 

Optical  Coherence  Tomography 

DC 

Direct  Current 

OD 

Optical  Density 

FBG 

Fiber  Bragg  Grating 

OPA 

Optical  Parametric  Amplifier 

FCS 

Fluorescence  Correlation  Spectroscopy 

OPO 

Optical  Paramagnetic  Oscillator 

FHG 

Fourth  Harmonic  Generation 

PCF 

Photonic  Crystal  Fiber 

FITC 

Fluorescein  IsoThioCyanate 

PICASO 

Phase  and  Intensity  from  Correlation  and  Spectrum 

FLIM 

Fluorescence  Lifetime  Imaging  Microscopy 

Only 

FLIP 

Fluorescence  Loss  in  Photobleaching 

PZT 

Piezo  Tuning 

FRAP 

Fluorescence  Recovery  After  Photobleaching 

Q-switch 

Quality  (of  the  laser  resonator)  switch 

FROG 

Frequency-Resolved  Optical  Gating 

RF 

Radio  Frequency 

GFP 

Green  Fluorescent  Protein 

SAM 

Saturable  Absorber  Mirror 

GreNe 

Green  Helium-Neon 

SBR 

Saturable  Bragg  Reflector 

GRIN 

Graded  Index  Lens 

SD 

Static  Discharge 

GTI 

Gires-Tournois  Interferometer 

SESAM 

Semiconductor  Saturable  Absorber  Mirror 

GVD 

Group  Velocity  Dispersion 

SHG 

Second  Harmonic  Generation 

He-Cd 

Helium-Cadmium 

SMD 

Single  Molecule  Detection 

He-Ne 

Helium-Neon 

SOC 

Saturable  Output  Coupler 

HR 

High  Reflector  Mirror 

SPIDER 

Spectral  Phase  Interferometry  for  Direct  Electric- 

ICNIRP 

International  Commission  on  Non-Ionizing  Radia¬ 

field  Reconstruction 

tion  Protection 

SPT 

Single  Particle  Tracking 

IEC 

International  Electrotechnical  Commission 

TEC 

Thermo-Electrically  Cooled 

IR 

Infrared 

TEM 

Transverse  electromagnetic 

KLM 

Kerr-Lens  Mode-Locking 

THG 

Third  Harmonic  Generation 

KTP 

Potassium  Titanium  Oxide  Phosphate 

TIRFM 

Total  Internal  Reflection  Fluorescence  Microscopy 

LASER 

Light  Amplification  by  Stimulated  Emission  of 

UV 

Ultraviolet 

Radiation 

VECSEL 

Vertical  External  Cavity  Surface  Emitting  Laser 

LBO 

Lithium  TriBorate 

VIS 

Visible 
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Non-Laser  Light  Sources  for 
Three-Dimensional  Microscopy 

Andreas  Nolte,  James  B.  Pawley,  and  Lutz  Horing 


INTRODUCTION 

General  Remarks  on  Choice  of  Excitation 
Light  Sources 

The  first  light  source  used  for  microscopy  was  the  sun  and  the 
second  was  a  candle  flame.  Both  are  hot  plasmas  that  emit  essen¬ 
tially  black-body  radiation  (Fig.  6. 1 )  with  the  addition  of  a  few  ele¬ 
mental  lines.  The  introduction  of  light  sources  powered  by 
electricity,  both  arcs  and  incandescent  filaments,  added  a  new  level 
of  convenience  and  flexibility  but  required  improvements  in  the 
light-harvesting  optics  needed  to  illuminate  the  imaged  area  with 
light  that  was  both  intense  and  uniform. 

Brightness 

Because  the  recorded  intensity  of  the  image  drops  with  the  square 
of  the  magnification,  the  most  important  characteristic  of  a  source 
for  microscope  illumination  is  its  brightness  or  radiance.  In  this 
case,  brightness  refers  not  just  to  the  ability  of  the  source  to 
produce  many  photons/second  but  also  to  its  ability  to  produce 
these  photons  from  a  small  volume  so  that  a  fair  share  of  them  can 
be  conveyed  to  the  small  area  of  the  specimen  that  is  being  imaged. 

A  large  diffuse  source,  such  as  the  40  W  fluorescent  tubes 
used  for  lighting  houses,1  produces  about  the  same  number  of 
photons/second  as  does  the  50  W,  short  arc  HBO-50  bulb  used  in 
fluorescence  microscopy.  However,  in  the  former  case,  the  photons 
emerge  from  a  phosphor  surface  approximately  0. 1  m2  in  size, 
while  a  cross-section  through  the  brightest  part  of  the  HBO-50  has 
an  area  about  one  million  times  smaller. 

The  laws  of  optics  specify  what  fraction  of  the  light  leaving  a 
source  can  be  focused  into  an  image  of  this  source.  In  Figure  6.2, 
light  leaving  tq  that  does  not  strike  the  optical  system  clearly,  will 
not  be  focused  onto  the  image  plane  at  h2.  Although  some  of  this 
lost  light  can  be  reclaimed  by  placing  a  spherical  reflecting  mirror 
so  that  the  focal  point  f  is  centered  on  the  source,  there  are  still 
limits  on  how  bright  the  image  will  be.  If  the  optical  system  makes 
an  enlarged  image  h2  of  the  source  fq  at  the  condenser  back-focal 
plane  (BFP,  not  shown),  then  the  fixed  number  of  photons  from  the 
source  will  be  spread  over  a  larger  area  and  the  image  will  not  be 


1  Really,  a  very  diffuse  Hg  arc,  surrounded  by  a  coating  of  powdered,  inorganic 
phosphor. 


so  bright.  If  the  optical  system  demagnifies  the  source,  the  image 
will  be  smaller,  BUT  it  will  not  be  brighter,  because,  as  shown  in 
Figure  6.2,  to  demagnify  the  source,  it  must  be  farther  from  the 
optical  system  than  the  image.  As  a  result,  the  NA  (aperture)  of  the 
optical  system  collecting  light  from  the  source  will  be  smaller  by 
exactly  the  same  ratio  as  the  image  has  been  demagnified.  In  other 
words,  no  matter  how  many  lenses  are  used,  the  ability  of  an  optical 
system  to  make  a  smaller  image  of  the  source  is  inexorably  tied  to 
using  a  collector  lens  with  a  lower  NA  (as  seen  from  the  source) 
and  this  means  that  a  smaller  fraction  of  the  light  emitted  by  each 
point  on  the  source  is  actually  collected.  One  just  cannot  squeeze 
all  the  light  into  a  smaller  image.2 *  The  best  one  can  do  is  to  design 
an  optical  system  that  makes  an  image  the  same  size  as  the  source 
(magnification  =  1). 

The  moral  of  this  story  is  that  the  only  way  to  have  bright 
illumination  on  the  specimen  is  to  start  with  a  small  bright  source. 
It  follows  that  the  fraction  of  the  light  from  the  HBO-50  that  can 
be  conveyed  by  any  optical  system  to  an  area  of  the  specimen  that 
is,  for  example,  100  pm  on  a  side  is  about  one  million  times  greater 
than  could  be  achieved  using  the  phosphor  surface  of  a  40  W  flu¬ 
orescent  lighting  tube.  Source  brightness  is  measured  in  terms  of 
photons/second/square  centimeter/steradian  (radiance),  where  the 
steradian  is  a  measure  of  the  solid  angle  into  which  the  light 
projects. 

Whether  this  level  of  illumination  is  sufficient  for  the  needs  of 
microscopy  or  not  depends  on  the  contrast  method  in  use  (fluo¬ 
rescence  needs  about  106x  more  than  bright  field),  on  the  time 
available  to  accumulate  the  image,  on  the  contrast  of  the  image, 
and  on  the  accuracy  with  which  one  needs  to  measure  this  con¬ 
trast.  That  said,  although  it  is  usually  quite  easy  to  reduce  the 
brightness  of  a  light  source  by  introducing  neutral  density  (ND) 
filters,  making  it  brighter  requires  major  redesign. 

Considering  just  the  standard  configuration  in  bright-field 
microscopy,  about  5W  of  optical  power  is  emitted  by  a  100W 
halogen  lamp  at  its  rated  power  consumption.  As  the  filament  is 
4.2  x  2.3  mm  in  size,  its  cross-section  is  about  10  mm2.  The  aspher- 
ical  collector  lens  can  cover  an  aperture  of  NA  =  0.7  (45°  half 
angle)  or  about  0.15  of  the  full  solid  angle.  Using  a  back  reflector 
this  can  be  increased  to  approximately  0.30.  Because  of  the  optical 


2  Another  line  of  argument  is  to  point  out  that,  for  the  image  to  be  brighter  than 

the  source,  one  would  have  to  move  energy  from  a  cooler  source  to  a  hotter 

image.  This  is  forbidden  by  thermodynamics. 
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FIGURE  6.1.  (A)  CIE  AM  0  Solar  Spectrum,  (B)  5800K  Blackbody  spectral 
distribution  with  same  250  to  4000nm  total  irradiance.  (Spectra  kindly  pro¬ 
vided  by  Oriel/Newport,  Irvine,  CA.) 

rules  mentioned  above,  even  perfect  optics  can  only  transport 
about  0.001  of  this  optical  power  to  illuminate  a  100  x  100pm  area 
of  the  specimen.3  This  means  that  the  power  available  to  illumi¬ 
nate  the  field  of  a  high  magnification  lens  is  less  than  (5  X  0.3  X 
0.001)  =  0.0015  W  =  1.5  mW.  Actual  measurements  of  brightness 
at  the  focal  plane  are  provided  in  Table  6.2  later  in  this  chapter. 

Assuming  bright-field  imaging  for  the  moment,  almost  all  of 
this  light  passes  through  the  objective  and  intermediate  optics 
and  reaches  the  plane  of  a  perfect  charge-coupled  device  (CCD) 
camera,  composed  of  a  1000  x  1000  array  of  6  pm  pixels.4 5  As  such 
a  sensor  will  be  6  x  6  mm  in  overall  size;  it  will  just  match  the  size 
of  the  illuminated  area  on  the  specimen  if  a  60x  lens  is  used.  At  this 
point,  each  pixel  will  intercept  one  millionth  of  the  light  at  the  spec¬ 
imen  or  1.5  nW.  As  one  watt  of  light  corresponds  to  about  3  x  1018 
photons/second  and  each  36  pm2  CCD  pixel  can  absorb  about 
20,000  photoelectrons  before  saturation,  an  exposure  of  2  x  104 
photons/(1.5  x  10-9W)  x  (3  x  1018  photons/s/W)  =  3.7  x  10~6 
seconds.  In  other  words,  in  the  absence  of  other  losses,  even  an 
exposure  as  short  of  3ps  would  be  enough  to  saturate  the  CCD 
(Inoue,  1997).  Sadly,  as  the  optical  system  just  described  focuses 
the  filament  into  the  focus  plane,  a  blurred  image  of  the  filament 
will  be  present  there  to  modulate  whatever  other  contrast  the 
specimen  produces.  Correcting  this  problem  as  described  below  and 
allowing  for  reflection  and  other  losses  will  cut  the  signal  by  a  factor 
of  about  10.  Adding  an  “interference  green”  filter  and  polarizer 
for  optimal  operation  of  monochrome  differential-interference- 
contrast  (DIC)  imaging  will  cut  the  photon  flux  by  another  factor  of 
20.  Should  one  decide  instead  to  use  this  green  light  to  excite  fluo¬ 
rescence,  the  signal  will  likely  be  reduced  by  an  additional  factor 
of  one  million5  and  suddenly  exposures  of  seconds  or  minutes  are 
needed.  The  only  solution  is  to  use  a  brighter  source.6 


3  As  the  most  efficient  optics  operate  at  a  magnification  of  1 : 1 ,  only  the  light 
emerging  from  a  100  X  100  pm  area  of  the  filament  can  be  focused  into  an 
area  of  the  same  size  on  the  specimen. 

4  The  popular  Sony  ICX285AL  chip  has  a  pixel  size  of  6.45  pm2. 

5  Of  course,  this  factor  depends  strongly  on  the  dye  concentration,  and  dis¬ 
tribution  and  the  effective  thickness  of  the  sensed  layer.  Values  in  the  range 
10~3  to  10~9  would  not  be  unusual. 

6  This  discussion  relates  to  low  magnification  operation  where  intensity  is  the 

main  goal.  At  low  magnification,  it  is  necessary  to  illuminate  a  much  larger 


FIGURE  6.2.  Relationship  between  demagnification  and  NA.  When  an  optical 
system  creates  a  demagnified  image,  h2,  the  convergence  angle,  02,  is  larger 
than  the  divergence  angle,  0],  leaving  the  source,  h  and  accepted  by  the  optical 
system.  Because  the  reduction  in  area  caused  by  the  demagnification  is  exactly 
compensated  by  the  increase  in  NA2,  the  image  can  never  be  brighter  than  the 
source. 


Uniformity 

The  second  requirement  of  a  light  source  for  microscopy  is  uni¬ 
formity;  all  parts  of  the  imaged  plane  must  receive  the  same  illu¬ 
mination  density.  Initially  this  meant  that  the  illumination  level 
must  be  constant  in  any  plane,  a  condition  not  too  hard  to  arrange 
by  focusing  an  image  of  a  featureless  source,  such  as  the  sun 
(lacking  sunspots!)  or  a  flame,  directly  onto  the  specimen. 
However,  as  the  light-emitting  area  of  arcs  and  most  filaments 
cannot  be  called  uniform,  a  new  illumination  system,  Kohler  illu¬ 
mination,  was  introduced,  so  that  the  non-uniformity  was  confined 
to  aperture  planes.  While  this  optical  solution  does  provide 
uniform  intensity  in  the  image  planes,  it  means  that  there  is  still 
some  non-uniformity  in  the  angles  at  which  light  approaches  each 
point  of  the  focus  plane. 

The  Kohler  illumination  system  diagrammed  in  Figure  6.3 
(Inoue,  1997)  operates  as  follows:  Light  is  collected  from  the  near 
side  of  the  source  I  using  a  high-aperture  collector  lens  1  while  a 
spherical  mirror  returns  light  from  the  rear  side  of  the  source  to 
form  an  image  of  the  source  to  the  side  of  the  actual  source  (not 
shown  in  Fig.  6.3).  This  reflected  light  is  also  collected  by  the  col¬ 
lector  lens,  increasing  the  apparent  source  size  (Piller,  1977).  The 
collector  lens  focuses  the  source  at  the  back-focal  plane  of  the  con¬ 
denser  II  (or  of  the  objective  in  the  case  of  epi-illumination  III). 
Because  of  the  reciprocal  relationship  of  image  and  back-focal 
planes,  anything  that  is  in  focus  at  one  set  of  these  planes  is  com¬ 
pletely  out  of  focus  at  the  other.  As  a  result,  assuming  that  the  con¬ 
denser  is  correctly  aligned  and  the  source  is  planar,  the  focus  plane 
in  the  specimen  B  is  evenly  illuminated.  However,  depending  on 
the  uniformity  of  the  source  and  the  aberrations  of  the  optics,  this 
light  may  not  strike  the  specimen  from  all  possible  angles  equally.7 

The  field  aperture  A  lies  in  an  intermediate  image  plane 
between  the  collector  1  and  the  condenser  2,  from  where  it  is 


area  on  the  specimen.  This  involves  exchanging  optical  elements  to  change 
the  magnification  between  the  plane  of  the  field  diaphragm  and  the  plane  of 
the  specimen.  This  is  possible  because  it  is  no  longer  important  for  the  con¬ 
denser  to  operate  at  high  NA. 

7  Remember  from  diffraction  theory  that  the  highest  resolution  in  transmission 
is  only  obtained  when  undiffracted  light  enters  the  objective  from  all  angles 
up  to  that  set  by  its  NA. 
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A)  Kohler  Illumination 

i)  Optical  path  in  pupil  and  conjugated  planes 
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Elements: 

A  =  field  diaphragm 
B  =  specimen  plane 
C  =  intermediate  image 
D  =  retina 

I  :  Filament 

II  :  Aperture  diaphragm 


III  :  Back-focal  plane 

IV  :  Pupil  of  the  eye 

1  :  Collector 

2  :  Condensor 

3  :  Objective 

4  :  Tublens 

5  :  Eyepiece 
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B)  Critical  Illumination 


FIGURE  6.3.  Kohler  illumination  with  the  optical  elements:  (1)  collector,  (2)  condenser,  (3)  objective,  (4)  tube  lens,  and  (5)  eyepiece.  (A.i)  Ray  paths  focused 
on  pupil  and  focal  planes:  (I)  source,  (II)  aperture  diaphragm,  (III)  back-focal  plane  of  objective,  (IV)  eyepoint  above  ocular/pupil  of  the  eye.  (A.ii)  Ray  paths 
focusing  on  image  planes:  (A)  field  diaphragm,  (B)  focus  plane  in  specimen,  (C)  intermediate  image,  (D)  retina/CCD.  (B)  Critical  illumination  in  which  the 
source  is  focused  directly  on  the  focus  plane  in  the  specimen.  This  leads  to  more  light  striking  the  focus  plane  but  precludes  the  use  of  the  field  diaphragm  to 
limit  excitation  to  the  area  of  the  specimen  actually  being  imaged. 


imaged  at  plane  B  in  the  specimen  by  the  condenser.  Its  size  can 
be  adjusted  to  restrict  the  area  illuminated  to  that  being  viewed 
without  affecting  the  angle  of  the  illuminating  light. 

Because  it  makes  maximum  use  of  neither  the  full  surface  of 
the  source  nor  the  full  angular  distribution  of  the  emitted  light, 
Kohler  illumination  is  not  very  photon  efficient.  In  addition, 
because  the  condenser  lens  should  have  the  same  NA  as  the  objec¬ 
tive,  it  must  be  focused  with  equal  care  if  the  field  diaphragm  is 
to  remain  properly  focused  into  the  image  plane.  Because,  on 
most  upright  microscopes,  the  condenser  and  stage  move  together, 
moving  the  stage  to  change  focus  is  a  process  that  necessarily 
moves  the  condenser  out  of  Kohler  alignment.  This,  and  the  fact 
that  microscope  slides  and  coverslips  are  likely  to  vary  in  thick¬ 
ness  by  tens  of  micrometers,  means  that,  ideally,  one  should  read¬ 
just  the  condenser  position  whenever  the  specimen  is  changed  or 
the  focus  plane  substantially  shifted.8 

Of  course,  the  Kohler  illumination  diagram  has  some  hidden 
assumptions  that  are  really  not  justified  in  practice.  The  chief  of 


8  The  same  argument  also  applies  to  inverted  microscopes.  Ideally,  the  objec¬ 
tive  and  condenser  would  remain  fixed  to  each  other  and  only  the  stage  would 
move. 


these  is  that  the  source  is  planar  and  that,  therefore,  light  emerg¬ 
ing  from  it  can  be  focused  by  “perfect”  lenses  into  other  planes 
according  to  the  geometric  optics  of  the  diagram.  In  fact,  both  arcs 
and  most  filament  sources  are  approximately  spherical  in  shape. 
Consequently,  even  if  the  optical  system  focuses  light  from  a 
central  plane  of  this  sphere  into  the  back-focal  plane  (BFP)  of  the 
condenser  lens,  many  photons  emerging  from  planes  within  the 
luminous  sphere  that  are  either  in  front  of  or  behind  the  central 
plane  will  be  proportionally  out  of  focus  at  the  condenser  BFP,  and 
at  every  other  conjugate  BFP  in  the  diagram.  The  corollary  of  it 
being  out-of-focus  at  the  BFPs  is  that  it  will  be  partially  in  focus 
at  the  image  plane  and  therefore,  that  in  spite  of  our  best  efforts, 
the  illumination  will  still  have  some  structure  at  the  focus  plane. 

Two  more  relevant  variables  of  the  illumination  optics  are  the 
total  magnification  between  the  source  and  the  BFP  of  the  con¬ 
denser  and  what  might  be  termed  its  “length.”  All  collector  lenses 
have  large  NA  to  capture  as  many  photons  as  possible,  and  a  focal 
length  short  enough  to  keep  the  light  source  small  without  getting 
the  glass  so  close  to  the  source  that  the  heat  cracks  it.  However, 
as  the  need  to  dissipate  the  heat  dictates  that  the  source  and  col¬ 
lector  be  mounted  at  some  distance  from  the  condenser  lens,  a 
variety  of  optical  arrangements  are  used  to  convey  the  light  across 
the  intervening  space. 
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At  the  other  end  of  the  illumination  optics,  the  condenser  lens 
should  have  an  NA  as  large  as  that  of  the  largest  objective  with 
which  it  will  be  used  but  its  focal  length  is  not  specified.9  A  longer 
focal  length  implies  a  larger  BFP  and,  if  this  is  to  be  “filled”  with 
an  image  of  a  filament,  then  the  optics  up  to  that  point  may  have 
to  form  an  enlarged  image  of  the  filament.  Often  epi-illumination 
Kohler  optics  are  designed  to  make  a  relatively  small  image  of  the 
arc  at  the  BFP  of  the  objective  so  that  this  image  is  not  truncated 
(and  the  light  lost)  by  the  relatively  small  BFP  of,  for  example,  a 
lOOx  NA  1.3  objective.  However,  this  means  that  the  larger  BFP 
of  a  40x  NA  1.3  objective  will  be  severely  under-filled,  a  factor 
that  reduces  the  rate  at  which  excitation  light  diminishes  above  and 
below  the  focus  plane  and  thereby  affects  the  WF  PSF  (Hiraoka 
et  al.,  1990). 

The  length  is  important  because,  although  so-called  telan 
lenses  can  be  introduced  to  convey  all  the  light  from  the  focal  point 
of  the  collector  to  the  BFP  of  the  condenser,  light  originating  from 
planes  in  front  of  or  behind  this  plane  will  "hit  the  wall”  of  the 
optical  system  and  be  lost  or  create  reflections.  The  longer  the 
optical  conveyer  system,  the  more  it  is  true  that  the  light  reaching 
the  condenser  BFP  will  have  originated  near  to  the  focal  plane  of 
the  collector.  This  makes  such  light  both  more  coherent  and  more 
structured  because  it  conveys  the  image  of  the  inhomogeneity  of 
the  filament  at  the  focal  plane  of  the  collector. 

To  restore  some  semblance  of  order  to  the  system,  it  is 
common  to  insert  a  flat,  ground-glass  surface  just  outside  the  col¬ 
lector  lens,  and  let  this  planar  surface  become  the  effective  source. 
This  solves  the  out-of-focus  light  problem  but  substantially 
reduces  (by  >90%,  depending  on  its  roughness)  the  brightness  of 
the  source.  For  this  reason,  the  ground  glass  is  normally  used  only 
when  imaging  with  coherent,  transmitted  light  (phase  contrast, 
DIC,  etc.)  and  not  when  the  main  aim  is  just  to  get  as  many  photons 
onto  the  specimen  as  possible  (fluorescence,  dark-field). 

The  alternative  to  Kohler  illumination  is  critical  illumina¬ 
tion,  a  system  that  requires  a  highly  uniform  emitting  surface  I 
because  this  surface  is  focused  directly  into  the  imaged  plane 
within  the  specimen  B  [Fig.  6.3(B)].  Because  it  images  the  entire 
source,  a  larger  solid  angle  can  be  used  than  with  Kohler  illumi¬ 
nation  and  therefore  it  can  deliver  more  photons/second/square 
centimeter  at  the  image  plane.  The  downside  is  that  any  non¬ 
uniformity  in  the  emitter  will  appear  as  a  brightness  change  in  the 
final  image.  The  coherence  of  the  illumination  in  this  system  is 
that  of  the  source.  If  a  large  source  is  imaged,  coherence  is  quite 
low.  Flickering  variations  in  brightness  caused  by  convection 
within  the  plasma  of  the  arc  can  make  it  tiring  to  view  the  speci¬ 
men  when  the  plasma  is  imaged  directly  onto  the  image  plane,  but 
a  time-average  of  such  an  arc  image  is  quite  uniform.  A  solution 
to  this  brightness-coherence-uniformity  conundrum  was  proposed 
by  Gordon  Ellis,  who  suggested  that  the  image  of  the  arc  be 
focused  on  the  entrance  of  a  1mm  diameter  multimode  optical  fiber 
that  is  then  bent  around  two  perpendicular  axes.  Mixing  in  the  fiber 
scrambles  the  light  so  that  the  exit  surface  appears  to  be  a  uni¬ 
formly  illuminated  circle.  Illuminators  of  this  type  are  discussed 
below  in  the  section  on  light  scrambling. 

If  the  collection  mirror  used  is  elliptical  or  parabolic  rather 
than  spherical,  and  the  arc  tube  is  mounted  axially  (Fig.  6.4),  it  can 
collect  up  to  85%  of  the  total  emitted  light.  Unfortunately,  the  pro¬ 


9  Except  for  the  long  working  distance  (LWD)  condensers  used  on  inverted 
microscopes  to  allow  use  with  thick  cells  chambers  or  access  to  the  specimen 
with  micro-electrodes  and  the  like. 


FIGURE  6.4.  (A)  Elliptical  collector  with  both  the  focal  points  F,  and  F2  and 
eccentricity  e  <  1,  every  ray  (a-c)  emitted  from  focal  point  F!  will  be  imaged 
at  focal  point  F2.  (B)  Parabolic  collector  (eccentricity  e  =  1),  with  a  source  in 
the  focal  point.  The  source  will  be  imaged  into  infinity  (a  parallel  wave  front). 
(Diagrams  kindly  provided  by  Oriel/Newport,  Irvine,  CA.) 


jected  ray  bundle  has  a  dark  central  spot  where  rays  are  obscured 
by  the  end  of  the  plasma  tube.  In  addition,  because  arcs  are  more 
stable  and  uniform  when  the  axis  of  the  envelope  is  vertical,  light 
sources  using  this  type  of  collector  usually  project  their  output 
downwards. 

The  coatings  on  all  collection  mirrors  should  be  dichroic  to  let 
heat  wavelengths  pass  through.  Reflectors  designed  for  various 
wavelength  bands  are  sold  by  Oriel  Corp.  (Stratford,  CT)  and 
Optical  Radiation  Company  (Electroforming  Division,  Azusa, 
CA)  among  others,  and  sold  as  “cold  mirrors”  or  “hot  mirrors,” 
depending  on  the  wavelengths  transmitted  or  reflected.  If  some 
scrambling  is  done  and  the  source  image  is  expanded,  elliptical 
mirror  sources  can  provide  a  Kohler  illumination  scheme  suitable 
for  disk-scanning  instruments  (see  below),  though  a  somewhat 
more  uniform  output  can  be  produced  if  the  elliptical  surface  is 
slightly  deformed.  These  same  companies  and  ICL  Technologies 
(Sunnyvale,  CA)  make  modules  and  lamp  housings  that  utilize 
such  aspheric  reflectors.  Many  models  are  available  for  different 
arc  lamps. 

Wavelength 

The  third  important  variable  is  wavelength.  Although  the  plasma 
and  filament  sources  discussed  so  far  have  been  “white  light” 
sources  with  fairly  uniform  brightness  across  the  visible  wave¬ 
lengths,  the  same  is  not  true  of  arc  sources.  Photons  are  produced 
whenever  an  excited  electron  loses  energy.  They  will  be  visible 
photons  (i.e.,  photons  having  a  wavelength  of  between  380  and 
780  nm)  if  the  amount  of  energy  lost  by  the  electron  is  in  the  range 
of  3.3  to  1.59  electron  volts  (eV).  Excited  electrons  can  be  pro¬ 
duced  in  many  ways,  but  the  traditional  methods  that  produce  them 
efficiently  and  from  small  volumes  include  heating  the  electrons 
in  a  tungsten  filament  to  3200  K  or  3400  K,  or  heating  a  Hg  or  Xe 
plasma  to  a  much  higher  temperature.  These  plasma  sources  also 
have  specific  emission  bands  that  are  defined  by  the  specific  energy 
levels  of  the  atoms  involved  [Fig.  6.5(  A,B)].10 

From  Figure  6.5(B),  one  can  see  that  an  Hg  arc  is  almost  lOx 
brighter  at  365  nm  than  at  400  nm.  Not  surprisingly  the  300  W  Xe 
arc  makes  about  4x  more  total  light  than  the  100W  Xe  arc. 
However,  because  the  brightest  part  of  the  image  of  the  300  W 


10  All  irradiance  spectra  kindly  provided  by  Newport  Co.  (Irvine,  CA). 
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Wavelength  (nm) 


FIGURE  6.5.  Wavelengths  resolved  irradiance  of  various  light  sources.  (A) 
Lower  line :  30  W  deuterium  arc  lamp;  upper  line :  75  W  xenon  arc  lamp;  dashed 
line :  50  W  halogen  filament;  dashed/dotted  line :  50  W  mercury  arc  lamp.  (B) 
Dashed  line :  100  W  xenon  arc  lamp;  dashed/dotted  drawn  line :  100  W  mercury 
arc  lamp;  dotted  drawn  line :  300  W  xenon  arc  lamp.  (Spectra  kindly  provided 
by  Oriel/Newport,  Irvine,  CA.) 


source  is  almost  8x  larger  than  the  comparable  area  of  the  100W 
source,  its  brightness  is  only  half  as  high. 

Coherence 

A  fourth  important  source  parameter  is  its  coherence.  Although 
coherence  is  closely  related  to  brightness,  in  that  bright  sources  are 
likely  to  also  be  highly  coherent,  the  term  brightness  is  more  used 
to  describe  the  ability  of  a  source  to  focus  a  large  number  of 
photons  into  a  small  area,  whereas  coherence  is  more  of  a  measure 
of  the  ability  of  wave  functions  that  describe  these  photons  to  inter¬ 
fere  with  each  other,  either  at  the  focus  plane  (good)  or  between 
the  reflections  from  every  dust  particle  and  imperfection  in  the 
optical  system  (bad).  In  coherent  light,  a  large  fraction  of  the  wave 
functions  passing  a  given  point  are  in-phase  with  each  other. 

Although  light  from  a  laser  is  extremely  coherent,  even  here 
there  are  limitations.  Laser  light  emerges  not  at  a  single  wave¬ 
length  but  over  a  very  narrow  wavelength  band.  Consequently, 
even  if  all  the  photons  started  off  in  phase,  after  some  distance 
(the  coherence  length)  those  with  slightly  longer  wavelength  will 
become  out  of  phase  with  those  of  slightly  shorter  wavelength. 
What  this  means  practically  is  that  light  scattered  by  a  dust  parti¬ 
cle  on  one  side  of  a  piece  of  glass  can  only  interfere  with  light 
scattered  on  the  other  side  if  the  thickness  of  the  glass  is  less  than 
the  coherence  length.11  Interference  of  this  type  is  the  source  of 


11  The  time  required  for  light  to  move  one  coherence  length  is  the  coherence 
time. 


“laser  speckle,”  the  pattern  of  random  interferences  that  converts 
what  should  be  a  uniform  beam  into  a  pattern  of  black  and  white 
blobs.  Because  in  microscopy  it  is  preferable  that  the  contrast  seen 
in  the  image  represent  only  the  interference  that  occurred  within 
the  specimen,  light  sources  of  low  coherence  are  preferred. 

The  process  of  light  amplification  by  the  stimulated  emission 
of  radiation  that  produces  laser  light  (see  Chapter  5,  this  volume) 
necessarily  produces  very  coherent  light.  Although  it  is  often 
assumed  that  coherent  light  can  be  produced  in  no  other  way,  this 
is  untrue.  If  one  thinks  of  light  as  an  oscillating  electro-magnetic 
field  that  propagates  in  one  direction,  and  chooses  to  measure  this 
field  at  a  particular  instant  and  in  a  location  of  space  that  is  smaller 
in  dimensions  than  its  wavelength,  the  wavefields  of  all  the 
photons  passing  through  this  volume  at  any  instant  are  added  to 
produce  a  single  electromagnetic  wave  vector;  that  is,  the  light 
emerging  from  this  tiny  volume  is  completely  coherent.  Were  one 
to  focus  light  from  a  filament  onto  a  0.2  pm  diameter  aperture,  the 
light  emerging  from  the  hole  would  be  coherent  and  all  of  it  could 
be  focused  into  a  diffraction-limited  spot.12  The  problem  is  that  it 
would  also  be  a  very  dim  spot  (about  6nW  of  white  light  by  exten¬ 
sion  of  the  calculations  noted  above).  As  the  aperture  gets  bigger, 
the  coherence  is  reduced. 

Incoherent  light,  such  as  sunlight  from  a  cloudy  sky,  and  coher¬ 
ent  light  from  a  laser,  are  each  limiting  theoretical  constructs.  Even 
though,  as  presented  in  Chapter  1,  using  these  limiting  conditions 
simplifies  the  process  of  writing  equations  that  describe  the  image 
formation  process  in  the  microscope,  neither  coherence  condi¬ 
tion  can  be  realized  in  practical  microscopy.  In  conventional 
microscopy,  little  attention  is  paid  to  the  degree  of  coherence  of 
the  illuminating  light  except  when  considering  diffraction  and 
interference  effects.  Roughly  speaking,  light  is  considered  to  be 
incoherent  when  it  does  not  produce  speckle  effects  and  coherent 
when  it  does. 

Speckle  is  bright  if  the  interference  of  light  scattered  by  the 
feature  is  constructive  with  that  from  the  background  and  it  is  dark 
if  destructive  interference  occurs  [Fig.  6.6(A);  Briers,  1993].  The 
apparent  size  of  the  scattering  feature  and  that  of  the  individual 
speckles  are  related  to  the  resolution  limit  (or  NA)  of  the  optics. 
In  the  case  of  incoherent  illumination  [Fig.  6.6(B)],  overlap 
between  speckle  patterns  having  different  wavelengths  partially 
cancels  them  out  to  produce  a  lower  contrast  pattern.  Because 
speckle  is  an  interference  phenomenon,  any  movement  of  the 
optical  system  or  the  specimen  will  result  in  a  complex  change  of 
the  speckle  pattern  in  time. 

Even  in  imaging  situations  that  are  well  described  by  incoher¬ 
ent  phenomena,  coherent  effects  can  often  be  detected  if  the  results 
are  examined  with  sufficient  optical  resolution  (Reynolds  et  al., 
1989).  In  general  microscopy,  light  with  low  coherence  is  desired 
for  bright-field  and  reflection  modes,  while  light  with  higher  coher¬ 
ence  is  required  for  phase  and  interference  modes.  The  process  of 
fluorescent  emission  involves  so  many  intermediate  steps  between 
excitation  and  emission  that  any  coherence  in  the  illuminating  light 
is  usually  lost,  and  the  light  emitted  from  the  specimen  is  basically 
incoherent. 

If  the  coherence  of  the  illuminating  light  is  too  high, 
microscopy  in  the  reflected  or  backscattered  light  (BSL)  mode13 
yields  images  with  fringes  caused  by  interference  of  the  coherent 


12  Conversely,  the  extent  to  which  this  is  NOT  possible  is  a  good  measure  of 
incoherence. 

13  BSL  is  the  more  general  term  for  reflected  light,  the  term  “reflected”  should 
be  reserved  for  the  coherent  scattering  that  occurs  from  smooth  surfaces. 
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Coherent 


^  Incoherent 


FIGURE  6.6.  Speckle  patterns.  High-contrast  patterns  result  from  interference 
between  two  reflection  maxima  in  a  highly  coherent  system.  (A)  Low-contrast 
speckle  results  from  the  summation  of  interference  intensity  patterns  in  a 
system  with  incoherent  illumination.  (B)  Speckle  spot  size  is  a  function  of 
system  resolution. 


light  reflected  from  any  of  the  optical  surfaces:  lenses,  mirrors, 
dust  windows,  and,  in  particular,  the  coverslip  surface.  This 
complex  interference  can  appear  as  defined  rings,  but  more  com¬ 
monly,  it  appears  as  a  high-contrast  granular  speckle  superimposed 
upon  the  image,  making  real  image  details  hard  to  interpret.  Fur¬ 
thermore,  when  the  specimen  is  transparent  and  has  multi-layered 
microstructure,  the  speckle  spots  become  even  more  complex 
figures. 

The  coherence  of  non-laser  light  sources  can  be  modified  by 
changing  the  magnification  of  the  Kohler  illumination  system  to 
reduce  the  effective  source  size.  Doing  so  makes  the  light  less 
intense  but  more  suitable  for  interference  microscopy. 

While  the  sun  is  considered  an  incoherent  source,  under  the 
imaging  conditions  of  high-resolution  microscopy,  sunlight  has 
enough  coherence  to  impart  speckle  to  the  image.  Tungsten  ribbon 
lamps  as  well  as  light-emitting  diode  (LED)-based  lamps  have 
relatively  low  spatial  coherence  because  of  the  large  size  of  the 
emitter.  Arc  lamps  have  higher  coherence  unless  a  large  area  of  the 
plasma  is  used  as  the  source.  As  the  short-arc  Xe/I  source  men¬ 
tioned  below  uses  a  plasma  spot  that,  after  demagnification,  is  one 
third  the  area  of  the  Airy  disk,  as  a  source  for  a  confocal  micro¬ 
scope,  the  illumination  is  almost  fully  spatially  coherent  (Hell  et 
al.,  1991).  However,  Hell  and  colleagues  argue  that  because  tem¬ 
poral  coherence,  associated  with  a  bandwidth  of  360  to  570nm, 
corresponds  to  a  frequency  bandwidth  of  3.07  X  108MHz,  to  the 
extent  that  the  intensity  spectrum  of  the  arc  approximates  a  con¬ 
tinuum,  it  is  appropriate  to  use 

coherence  length  <  c/frequency  bandwidth 


FIGURE  6.7.  Phase  randomization  scheme  for  laser  use.  (After  Hard,  et  al., 
1977.) 


from  Born  and  Wolf  (1980).  This  yields  an  upper  limit  of  coher¬ 
ence  length  of  approximately  1  pm.  Points  in  the  specimen  whose 
optical  path  difference  are  greater  than  1  pm  cannot  interfere  to 
give  rise  to  noticeable  interference  artefacts  in  transparent 
microstructures. 

A  major  function  of  Kohler  illumination  systems  is  to  make 
the  illumination  homogeneous  at  the  image  planes  and  to  control 
its  coherence  somewhat.  However,  it  is  essentially  a  coherent 
system  and  does  not  “scramble”  light  to  any  extent.  Additional 
scrambling  is  sometimes  needed  to  decrease  spatial  inhomogene¬ 
ity,  spatial  coherence,  or  temporal  coherence.  Although  most 
scramblers  have  been  designed  to  work  with  highly  coherent  laser 
light,  the  same  methods  can  also  be  used  to  reduce  the  coherence 
of  light  from  other  sources. 

Scrambling  and  Filtering  the  Light 

Three  methods  have  been  proposed  to  reduce  coherence.  Hard  and 
colleagues  (1977)  proposed  a  method  to  phase-randomize  laser 
light  for  illuminating  a  conventional  microscope  by  inserting  a 
rotating  optical-wedge-and-ground-glass  combination  into  the 
light  path  (Fig.  6.7).  Because  the  wedge  and  the  ground  glass 
rotate,  any  remaining  temporal  coherence  becomes  cyclical.  This 
method  requires  very  stable  placement  and  accurate  alignment  of 
the  rotating  device  and  the  laser  relative  to  the  microscope 
(Reynolds  et  al,  1989). 

The  second  method  to  reduce  coherence  is  to  focus  the  laser 
light  into  a  flexible  length  of  single  multi-mode  optical  fiber  (Fig. 
6.8).  The  internal  reflections  in  the  bent  fiber  are  constantly  chang¬ 
ing  because  the  fiber  is  vibrated  at  up  to  100  kHz  (Ellis,  1979)  and 
this  makes  the  exit  beam  appear  uniform  in  intensity  over  time 
(rather  than  having  the  Gaussian  profile  characteristic  of  lasers). 
The  phase  is  scrambled  by  the  varying  path  lengths  of  the  light 
passing  through  the  fiber  on  different  trajectories,  but  the  high  radi¬ 
ance  and  monochromaticity  are  preserved.  Technical  Video  Ltd. 
(Woods  Hole,  MA)  markets  a  non-vibrating  version  with  a  fixed, 
single  quartz  fiber  segment  bent  to  a  specific  radius  in  two  per¬ 
pendicular  planes.  Applied  Precision  Instruments  (API,  Issaquah, 


Vibrator 

FIGURE  6.8.  Phase  randomization  scheme  for  laser  (after  Ellis,  1979). 
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WA)  also  offers  such  a  system  as  part  of  their  Deltavision  three- 
dimensional  (3D)  imaging  system. 

All  these  methods  minimize  speckle  by  making  it  change  with 
time.  Speckle  is  not  apparent  as  long  as  the  recording  system 
records  for  a  time  period  that  is  longer  than  the  period  of  the  scram¬ 
bler.  As  the  motorized  system  is  quite  slow,  it  is  only  suitable  for 
photographic  recordings.  Although  the  fiber  scrambler  can  be 
oscillated  more  rapidly,  even  this  is  too  slow  to  cause  a  significant 
reduction  in  coherence  over  the  1 .6  pis  pixel  time  that  character¬ 
izes  many  beam-scanning  confocal  instruments. 

As  light  scramblers  can  be  damaged  if  subjected  to  a  high  total 
light  flux,  provision  must  be  made  to  remove  heat  and  other 
unwanted  wavelengths  before  they  enter  the  scrambler  system. 
Ideally,  only  wavelengths  critical  to  image  formation  should  leave 
the  source.  Dichroics  that  reflect  only  specific  wavelengths  should 
be  used  to  separate  the  useful  light  and  allow  the  unwanted  heat 
to  escape.  In  single-sided  disk  scanners,  this  would  not  only  reduce 
heating  but  also  decrease  flare  and  scattering  from  the  top  surface 
of  the  disk.  Heat-absorbing  glass  is  the  most  common  heat  filter, 
but  a  filter  consisting  of  a  chamber  filled  with  a  salt  solution  chosen 
to  screen  out  infrared  light  has  much  higher  heat  capacity.  Aside 
from  heat  removal,  liquid  filters  can  also  be  made  to  function  as 
bandpass  or  cut-off  filters  by  the  careful  choice  of  the  salt 
used.  An  extensive  description  of  useful  solutions  is  described  by 
Loveland  (1970). 

TYPES  OF  SOURCES  AND  THEIR  FEATURES 

The  following  section  discusses  the  important  parameters  of 
various  common  non-laser  light  sources  with  respect  to  the  goals 
of  microscopy.  In  this  sense  “parameter”  means  a  degree  of 
freedom  a  user  can  change  to  optimize  illumination  for  the  appli¬ 
cation.  Unfortunately  these  parameters  are  not  independent  of  each 
other,  and  all  components  of  an  illumination  system  have  to  be 
treated  together  as  a  unit.  Overall  performance  depends  on  the 
geometry  of  the  source,  the  focal  length,  magnification  and  NA  of 
the  collecting  optics,  and  these  in  turn  depend  on  the  shape  and 
position  of  the  mirrors  and  lenses.  Much  can  be  learned  by  remov¬ 
ing  the  lamp  housing  and  adjusting  the  controls  used  to  position 
the  source,  the  reflector,  and  the  collector  to  project  an  image  on 
the  source  on  a  distant  wall.  (When  using  arc  sources,  take  care 
not  to  aim  it  in  such  a  way  that  the  direct  light  gives  anyone  a 
sunburn  or  strikes  anyone  in  the  eye.) 


The  important  parameters  of  non-laser  sources  are: 

1.  Structure  (spatial  distribution,  source  geometry,  coherence, 
alignment,  collecting  optics,  etc.). 

2.  Available  wavelengths. 

3.  Stability  in  time  and  wavelength. 

4.  Source  radiance  (brightness). 

5.  How  source  parameters  such  as  intensity,  color,  and  spatial  dis¬ 
tribution  are  controlled. 

Structure 

The  Actual  Source  of  the  Light 

Microscope  sources  are  optimized  to  produce  the  maximum  light 
intensity  or  brightness  (photons/s/cm2/steradian)  from  a  tungsten 
ribbon  or  the  arc  of  an  electrical  discharge.  The  filaments  of 
halogen  lamps  are  often  bent  to  resemble  disks  or  wide,  flat  bands 
to  match  the  input  aperture  of  the  light-collection  optics.  Arc  lamps 
generally  generate  light  in  a  ball  discharge  at  the  tip  of  a  pointed 
electrode  (Fig.  6.9).  The  two  electrodes  in  xenon  arcs  have  differ¬ 
ent  shapes.  The  anode  has  a  bigger  diameter  and  a  flatter  tip.  As  a 
result,  the  light  will  be  most  intense  where  electric  flux  lines  are 
closest  together  near  the  point  of  the  cathode  [Fig.  6.9(B)].  As  the 
pointed  electrode  erodes,  the  field  at  the  tip  decreases  and  the 
plasma  ball  becomes  larger  and  therefore  less  intense. 

These  sources  are  geometrically  similar  but  are  different  in  size. 
The  brightest  part  of  the  arc  in  a  common  HBO-100  arc  lamp  is 
about  0.3  x  0.5  mm  in  cross-section,  the  tungsten  filament  of  a  100 
W  halogen  lamp  is  about  4x2  mm  wide.  Both  source  dimensions 
are  set  by  the  manufacturer,  and  there  is  no  option  to  vary  them. 

Electrons  passing  through  the  depletion  region  of  any  forward- 
biased  semiconductor  diode  lose  energy  equal  to  eVg,  where  Vg  is 
the  bandgap  of  the  semiconductor.  This  energy  is  converted  into  a 
photon  having  an  energy  equal  to  the  bandgap  energy.  Silicon 
diodes  emit  in  the  near-infrared  (IR)  region,  but  diodes  made  of 
other  semiconductors  emit  in  the  visible  and  even  the  near  ultra¬ 
violet  (UV).  When  such  a  diode  is  configured  primarily  to  produce 
light,  it  is  called  a  light-emitting  diode  or  LED  (Schubert,  2003). 
Recently,  technological  developments  have  increased  available 
power  levels  and  now  LEDs  are  used  in  applications  where  their 
long  life  and  high  efficiency  are  important,  such  as  traffic  lights. 
They  are  of  interest  to  microscopists  because  they  are  compact  and 
efficient  light  sources  that  can  emit  a  high  flux  of  quasi- 
monochromatic  photons  from  a  small  area. 


DISTANCE  FROM  ARC  AXIS  (mm) 


FIGURE  6.9.  Iso-intensity  plots  from  the  discharge  of  a 
mercury  arc  lamp  (A)  and  xenon  arc  lamp  (B).  Values  in 
kcd/cm  Note  the  different  size  scales  and  the  fact  that 
the  Hg  arc  has  two  intense  plasma  balls  while  the  xenon 
has  only  one,  and  this  is  smaller  and  less  intense  than  the 
Hg.  (Plots  kindly  provided  by  Oriel/Newport,  Irvine, 
CA.) 


Non-Laser  Light  Sources  for  Three-Dimensional  Microscopy  •  Chapter  6  133 


wavelength  [nm] 


FIGURE  6.10.  Spectral  distributions  of  monochrome  LEDs.  The  compounds 
used  for  the  LED  dies  are  shown  above.  The  black  line  with  a  sharp  peak  at 
460  nm  and  a  smaller,  wider  peak  at  about  575  nm,  is  the  emission  curve  of  a 
so-called  “white”  LED  —  a  blue  emitter  surrounded  with  a  phosphor  that 
absorbs  at  575  nm  and  emits  at  longer  wavelengths. 
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An  LED  source  consists  of  an  area  of  semiconductor  crystal 
approximately  0.3  mm  x  0.3  mm  in  size  called  a  die.  The  most 
common  crystals  used  are  Ga^AlxAs^yPy,  GaN,  and  ZnSe  and 
each  emits  in  a  different  waveband  (Fig.  6.10).  Normally  one  or 
more  dies  are  embedded  in  a  larger  LED  structure  for  protection, 
light  collection,  and  electrical  and  thermal  handling.  The  advan¬ 
tage  of  LED  technology  is  that  one  can  combine  these  small  units 
to  build  up  a  light  source  of  the  shape  best  suited  to  the  needs  of 
the  application.  Possible  source  geometries  are  limited  only  by 
heat  dissipation  and  the  permitted  package  density  of  the  surface 
mount  device  (SMD)  technology  used  to  integrate  a  number  of  dies 
onto  the  printed  circuit  board.  Very  dense,  bright,  custom-designed 
sources  can  be  fabricated  in  this  way.  Figure  6.11  shows  the 
general  structure  of  an  LED  and  how  they  have  changed  over 
recent  years  (Steigerwald,  2002).  Die  dimensions  up  to  1.5  x  2mm 
are  now  available. 

How  to  Cope  with  the  Heat? 

The  most  important  aspect  of  any  microscopy  light  source  is  an 
efficient  heat  sink.  Incandescent  and  the  arc-based  lamps  produce 
a  lot  of  heat  because  of  their  low  optical  conversion  efficiency 
(5%-10%).  The  holders  and  housings  of  these  lamps  are  made  of 
a  material  resistant  to  high  temperature  and  designed  to  dissipate 
-100W  of  heat.  As  a  result,  they  cannot  be  mounted  inside  a 
microscope  system.  Although  present  LEDs  have  similar  effi¬ 
ciency,  all  the  photons  produced  are  emitted  over  a  narrow  range 
of  wavelengths  (see  below)  and  they  operate  at  a  much  lower  tem¬ 
perature.  This  means  that  less  electrical  power  is  needed  for  the 
same  optical  output  and  they  can  be  more  compact;  for  instance, 
they  can  be  bonded  to  a  small  metal  heat  sink,  cooled  by  a  small, 
computer-controlled  fan.  This  technology  makes  it  possible  to 
mount  LED  sources  inside  the  microscope  system,  closer  to  the 
specimen,  and  avoid  loss  of  light  intensity  in  transit.  Despite  this 
flexibility,  it  is  important  to  remember  that  LED-based  sources  do 
need  an  efficient  heat  sink  because  operating  much  above  room 
temperature  causes  lower  lifetime  and  a  loss  of  optical  output 
efficiency  (Perduijn  et  al.,  2004;  Fig.  9.12). 


1991  1 994  3x  flux  1 998  15x  flux  2001  22x  flux 


FIGURE  6.11.  (A)  General  structure  of  an  LED  with  the  different  layers 
marked.  The  mirror  is  used  to  reflect  the  back-emitted  light  to  the  right  direc¬ 
tion.  The  metal  bond  pad  on  the  top  is  for  the  electrical  contact.  (B)  Changes 
in  the  geometry  of  LED  dies  in  recent  years  have  led  to  larger  dies  and  have 
increased  the  efficiency  with  which  light  is  emitted.  (With  kind  permission  from 
OSRAM  Optical  Semiconductors,  Regensburg,  Germany.) 
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FIGURE  6.12.  The  dependence  of  light  flux  on  LED  temperature,  standard¬ 
ized  to  70°C  for  dies  made  using  different  types  of  LED  crystals.  Because  dies 
emitting  at  longer  wavelengths  have  a  smaller  bandgap  energy,  they  are  more 
affected  by  changes  in  temperature.  Formulas  for  the  compounds,  note  above. 
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FIGURE  6.13.  Diagram  of  lamp  housing.  The 
discharge  of  the  arc  is  located  at  the  focal  points 
of  the  rear  reflector  and  the  first  condenser  lens, 
both  of  which  can  be  moved  in  three  directions  to 
permit  alignment  with  the  optical  system. 


Reflecting  and  Collecting  the  Light: 

Source  Optics 

As  incandescent  and  arc  lamps  are  almost  isotropic  emitters  that 
radiate  equally  in  all  directions,  a  spherical  reflector  can  be  placed 
behind  the  source  to  create  the  image  of  the  source  beside  the 
actual  arc.  A  collector  lens  is  mounted  so  its  focal  point  coincides 
with  the  center  of  the  source;  it  catches  a  large  fraction  of  the  light 
and  focuses  it  at  the  BFP  of  the  condenser  (Fig.  6.13).  Due  to  the 
small  source  size  and  the  desire  for  a  short  compact  structure, 
collector  lenses  are  usually  aspheric  and  have  high  chromatic 
aberration. 

LED-based  sources  use  three  different  principles  to  reflect  and 
collect  the  light  generated  inside  the  die.  The  first  approach  was  to 
use  the  clear,  molded  plastic  LED  package  itself  to  collect  and 
focus  the  light.  Although  suitable  for  low-level  bright-field  appli¬ 
cations,  this  method  is  rarely  used  in  microscopy  because  of  the 
limited  optical  power  available  from  a  single  LED  (e.g.,  the  white 
light  LED  NEPW500,  manufactured  by  Nichia  with  a  0.3  mm2 
die).  The  second  approach  is  to  work  without  a  collector  and  print 
the  LED  dies  directly  onto  the  printed  circuit  board.  The  density 
of  the  packaging  achievable  in  this  way  is  only  limited  by  the  need 
to  bond  the  single  dies  with  the  connecting  wires  and  the  ability 
to  dissipate  heat.  The  disadvantage  is  one  loses  any  light  that  the 
LED  die  emits  to  the  side  (e.g.,  the  monochrome  arrays  available 
from  Laser2000  with  88  single  0.3  mm2  dies  packaged  on  a 
ceramic  plate).  The  third  possibility  is  to  place  the  LED  die  into  a 
mirrored  well  that  acts  as  a  reflector  (e.g.,  the  “Golden  Dragon” 
manufactured  by  ORAM)  and  then  arrange  these  units  onto  a 


printed  circuit  board.  Because  the  reflectors  are  bigger  than  the 
dies,  this  results  in  a  lower  packing  density. 

As  every  die  is  a  separate  source,  when  an  LED  array  is  built 
up  of  several  dies,  one  needs  a  different  method  to  combine  the 
light  from  them  than  one  would  use  for  a  conventional  lamp.  The 
most  efficient  way  to  collect  the  emitted  light  is  to  use  a  microlens 
array  placed  the  proper  distance  in  front  of  the  LED  array.  This 
array  can  be  made  of  molded  plastic  or  of  glass  and  must  be 
designed  so  that  every  LED  die  has  its  own  collecting  lens  (Fig. 
6. 14).  The  main  goal  of  this  configuration  is  to  catch  as  much  light 
as  possible  and  deliver  it  into  the  acceptance  angle  of  the  micro¬ 
scope  illumination  optics  so  that  it  fills  the  condenser  aperture 
diaphragm  with  axial,  parallel  light  as  homogeneously  as  possible. 

Source  Alignment 

The  rule  of  thumb  says  the  smaller  the  source,  the  more  important 
it  is  to  align  it  properly.  For  arc  lamps  and  incandescent  lamps,  one 
must  align  the  reflector  and  collector  in  such  a  way  the  source  and 
its  image  from  the  reflector  lie  side  by  side  and  are  centered  in  the 
aperture  diaphragm  of  the  illumination  path.  Only  in  this  way  will 
one  be  able  to  fill  the  aperture  diaphragm  approximately  homoge¬ 
neously  (Fig.  6.15).  Modern  systems  usually  either  incorporate 
special  viewers  so  that  one  can  check  alignment  or  use  self¬ 
aligning  sources  such  as  the  self-aligning  FIBO-100  source  avail¬ 
able  from  Carl  Zeiss  (Fig.  6.16).  The  heart  of  the  self-aligning 
source  is  a  photodiode  with  four  quadrants.  Some  of  the  light 
emitted  by  the  arc  in  the  direction  towards  the  reflector  passes 
through  a  small  hole  onto  the  optical  axis  where  it  is  captured  by 
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FIGURE  6.1 4.  Principle  of  a  micro-lens  array.  (A)  Each  mircro-lens  efficiently 
couples  the  light  from  a  single  LED  source  to  the  macroscopic  optics  of  the 
microscope.  As  a  result,  the  macroscopic  collector  optics  can  have  a  lower  aper¬ 
ture,  and  therefore,  a  larger  focal  length  at  a  reasonable  diameter.  In  addition, 
such  optics  have  lower  spherical  and  chromatic  aberrations.  (B)  Hexagonal 
micro-lens  array  made  of  quartz  glass.  This  example  is  normally  used  in 
telecommunication  to  couple  two  bundles  of  optical  fibers  together. 


the  quadrants  of  the  photodiode.  A  microcontroller  measures  the 
intensity  of  all  four  quadrants  and  moves  the  burner  on  an  jcy-stage 
until  there  is  no  difference  between  them.  As  the  right  reflector 
position  depends  on  the  particular  optics  of  the  microscope,  the  z- 
position  of  the  reflector  still  has  to  be  aligned  manually. 

For  LED-based  sources,  alignment  is  simpler.  In  bright-field 
microscopy  the  main  goal  is  to  fill  the  aperture  diaphragm  as  homo¬ 
geneously  as  possible.  This  can  be  accomplished  by  mounting  a 
large  (5-10mm)  LED  array  just  behind  the  aperture  diaphragm. 
The  problem  in  this  case  shifts  to  the  alignment  between  the 
microlens  array  and  the  LED  sources.  As  the  focal  length  of  the 
microlenses  has  to  be  relatively  short  and  the  LED  sources  are  very 
small,  the  tolerance  to  a  misalignment  is  also  small. 


Wavelength 

Historically,  many  fluorophores  were  selected  and  used  because 
they  are  excited  by  the  intense  lines  of  Hg  arc  lamps.  This  was  not 
only  because  of  the  increased  brightness  but  also  because  the 
narrow  bandwidth  made  it  more  easy  to  design  effective  dichroic 
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FIGURE  6.16.  Components  for  a  self-aligning  source.  Light  passing  through 
small  hole  in  the  reflector  strikes  a  quadrant  detector.  Signals  from  this  detec¬ 
tor  are  used  to  move  the  arc  in  y  and  z  until  the  output  of  all  four  diodes  are 
equal. 

emission  filters  and  beam-splitters.  In  addition,  microscope  objec¬ 
tives  are  often  designed  to  give  optimal  correction  at  these 
wavelengths  (Herman,  1998).  As  the  laser  wavelengths  available 
seldom  match  these  wavelengths,  the  microscopist  using  a  laser 
confocal  microscope  must  sometimes  exchange  familiar  fluo- 
rochromes  for  new,  less  familiar  ones  that  can  be  excited  at  laser 
wavelengths.  Because  histochemistry  is  complex  enough  without 
having  to  try  entirely  new  stains  and  chemistry,  it  is  important  to 
have  a  light  source  flexible  enough  to  excite  the  most  suitable  dye 
for  each  application. 

The  sun  has  very  high  brightness  and  the  continuous  spectrum 
of  a  black-body  radiator  with  a  surface  temperature  of  5800  K  (Fig. 
6.1).  A  clock-driven  heliostat  was  used  by  Petran  and  colleagues 
(1985)  to  track  it  as  a  light  source  for  confocal  microscopy.  The 
sun’s  broad  continuous  spectrum  allows  easy  selection  of  wave¬ 
lengths  for  difficult  specimens. 

Aside  from  the  sun,  only  a  synchrotron  can  provide  as  bright 
and  continuous  a  spectrum  (van  der  Oord,  1992;  Gerritsen  et  al., 
1992,  1994).  Although  using  the  sun  or  a  synchrotron  as  a  light 
source  might  be  ideal  in  many  ways,  the  nuisance  of  having  to 
depend  on  geography,  season,  cloudiness,  and  time  of  day  led  to 
the  development  of  portable  sources. 

Incandescent  lamps  also  emit  essentially  black-body  radiation, 
the  spectral  shape  of  which  depends  on  the  temperature  (Fig.  6.17). 
However,  because  the  filament  is  only  3200  K  or  3400  K,  the  light 
is  less  intense  and  has  more  red  light  than  sunlight.  Although  it  is 
possible  to  increase  the  blue  component  by  raising  the  tempera- 


FIGURE  6.1 5.  Image  of  the  filament  of  a  100W  halogen  source.  (A)  Image  of  filament  and  its  reflection  superimposed;  (B)  filament  and  its  image  side-by-side, 
right  (C)  after  insertion  of  ground  glass  to  randomize  image  of  filament.  The  illumination  optics  do  not  magnify  the  image  of  the  filament  quite  enough  to  fill 
the  BFP  completely.  Even  with  the  ground  glass  in  place  one  can  see  a  visible  drop  of  intensity  at  the  outer  edges  of  the  BFP. 
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FIGURE  6.17.  Spectral  distribution  of  black-body  radiation  at  various  tem¬ 
peratures.  Note  that  the  vertical  scale  is  logarithmic.  The  total  luminosity  of  a 
black  body  varies  with  the  fourth  power  of  the  temperature. 


ture,  doing  so  increases  the  rate  at  which  the  tungsten  sublimes. 
The  tungsten  vapor  condenses  on  the  inside  surface  of  the  quartz 
envelope  where  it  absorbs  light  from  the  source  and  heats  the 
envelope.  In  addition,  the  filament  becomes  thinner  until  it  fails. 

The  halogen  gas  in  a  quartz-halogen  source  interrupts  this 
process  by  first  reacting  with  the  evaporating  tungsten  to  form 
tungsten-halide  compounds.  These  then  decompose  when  they 
strike  the  hot  filament,  effectively  returning  the  tungsten  to  the 
hottest  (  and  thinnest  )  part  of  the  filament.  This  permits  the  filament 
to  be  operated  at  a  higher  temperature  with  little  darkening  and  a 
longer  time  before  failure.  Higher  temperatures  may  also  deform 
the  filament  structure,  causing  it  to  move  from  its  correct  location 
in  the  optical  system. 

The  optimal  wavelengths  for  exciting  classic  fluorescent  dyes 
(436 nm,  546 nm.  579 nm,  365  nm,  405  nm)  coincide  with  Hg  arc 
emissions  lines  [Fig.  6.5(A)].  The  carbon  arc  (not  shown)  has  a 
single  very  intense  line  at  400  nm.  Mixed-gas  Hg-Xe  arcs  have 
many  intense,  useful  spectral  lines  in  the  UV  and  visible.  Xenon 
and  zirconium  arcs  have  spectral  lines  in  the  near  IR. 

The  emission  spectra  of  arc  sources  can  be  classified  in  three 
ways:  (1)  continuous,  (2)  line  spectra,  and  (3)  mix  of  lines  and  con¬ 
tinuous  spectra.  Figure  6.5(A)  shows  the  spectra  of  arcs  using 
mercury  and  xenon,  as  well  as  a  tungsten  halogen  incandescent 
source,  plotted  with  the  same  horizontal  scale. 

The  super-pressure  xenon  arc  provides  intense  broadband  illu¬ 
mination  without  prominent  spectral  lines  in  either  the  UV  or  the 
visible  because  the  high  pressure  broadens  the  xenon  spectral  lines. 
Because  the  intensity  of  the  continuum  of  a  75  W  high-pressure 
xenon  lamp  is  2x  higher  than  the  continuum  of  a  100W  Hg 
arc  lamp,  the  xenon  lamp  is  better  suited  for  low  efficiency 
fluorophores  not  excited  well  by  the  prominent  Hg  lines.  The 
availability  of  broadened  spectral  lines  sometimes  allows  the 
simultaneous  activation  of  several  fluorochromes  with  differing 
emission  wavelengths,  including  those  in  the  UV. 

The  only  disadvantage  of  the  xenon  lamp  is  the  high  pressure 
in  the  bulb  (approximately  30  bar  at  room  temperature  and  approx¬ 
imately  lOx  that  at  operating  temperature).  This  makes  it  neces¬ 
sary  to  use  protective  gloves,  goggles,  and  a  shield  to  shelter  the 
chest  when  changing  a  bulb  in  case  it  should  explode. 

The  LED  technology  makes  it  possible  to  supply  the  right  exci¬ 
tation  wavelength  for  each  fluorophore.  Most  wavelengths  are  now 


available  from  UV  (365  nm)  up  to  IR  (>800 nm;  Fig.  6.10)  with 
intensities  sufficient  for  widefield  fluorescence.  The  FWHM  of  a 
quasi-monochrome  LED  varies  from  20  to  40  nm,  which  is  similar 
to  the  width  of  the  excitation  band  of  many  fluorophores.  Com¬ 
pared  to  laser  light,  the  wider  bandwidth  of  the  LEDs  make  it 
easier  to  excite  a  variety  of  dyes,  and  compared  to  the  continuous 
spectrum  of  an  arc  lamp  they  are  cooler,  smaller,  and  provide  an 
easier  way  to  choose  the  wavelength  one  wants  and  to  do  so 
rapidly.  That  said,  one  still  needs  to  use  excitation  filters  to  remove 
the  tails  of  their  emission  wavelength  distribution. 

Using  conventional  fluorescence  microscopy  as  many  as  five 
different  fluorescent  labels  have  been  imaged  simultaneously  in  the 
same  living  cell  (De  Basio  et  al.,  1987). 14  This  certainly  could  have 
been  done  as  well  using  a  disk-scanning  confocal  microscope 
having  an  efficiently  configured  arc  source. 

When  imaging  living  cells,  success  requires  using  minimum 
light  intensity  at  the  specimen  plane  and  choosing  illumination 
wavelengths  that  interfere  as  little  as  possible  with  the  life  process 
under  study.  Using  a  continuous  spectrum  source,  one  can  often 
make  small  wavelength  changes  that  reduce  specimen  mortality. 
Because  fluorophore  excitation  maxima  are  altered  by  their  molec¬ 
ular  environment,  and  because  this  is  especially  true  for  a  fluo¬ 
rophore  attached  to  a  functioning  macromolecule  in  a  living  cell, 
fine  tuning  the  excitation  wavelength  can  result  in  increased  emis¬ 
sion.  Using  strip-  or  wheel-type,  continuously  graded  interference 
filters,  such  as  those  manufactured  by  Ocean  Optics  (Dunedin, 
FL),  the  microscopist  can  often  precisely  adjust  the  illuminating 
wavelength  to  minimize  interference  with  the  process  under  study 
and  to  maximize  the  excitation  of  a  fluorophore  in  a  particular  cel¬ 
lular  environment. 

Stability  in  Time  and  Wavelength 
Ramp-Up  and  Short-Time  Stability 

Everyone  who  has  used  an  arc  lamp  knows  the  buzzing  noise  that 
occurs  when  the  lamp  is  switched  on  and  the  plasma  arc  discharge 
is  building  up  between  the  two  electrodes.  All  sources  based  on 
plasma-discharge  or  incandescence  need  a  considerable  time  to 
reach  thermal  stability. 

Figure  6.18  shows  typical  intensity  ramp-up  curves  for  the 
various  sources.15  All  lamps  that  produce  significant  heat  show  a 
dependence  of  the  emission  on  the  source  temperature.  This  even 
applies  to  LED-based  sources.  It  can  take  up  to  1  h  until  the  source 
is  sufficiently  stable  to  make  reproducible  measurements  or  to 
make  a  good  time-lapse  movie.  Once  operating  temperature  has 
been  reached,  the  halogen  lamp  is  the  most  stable  source  over  time 
periods  of  a  few  milliseconds  because  of  the  high  thermal  inertia 
of  the  tungsten  filament.  LED-based  sources  react  very  fast  (in  a 
few  microseconds)  and,  therefore,  they  are  affected  by  any  high- 
frequency  instability  in  the  power  supply.  In  general,  the  most 
unstable  source  is  the  arc  lamp.  Not  only  is  the  arc  itself  a  chaotic, 
flickering  discharge  but  its  light  output  can  also  be  affected  by 
ambient  electromagnetic  fields  or  an  unstable  power  supply. 

Stability  can  be  increased  by  using  the  signal  from  a  light 
sensor  as  a  feedback  signal  to  control  the  excitation  power.  The 
Oriel  68950  power  supply  controller  (Newport,  Inc.,  Irvine,  CA) 


14  De  Basio  and  colleagues  measured  five  cell  parameters:  (fluorescent  dye) 
nuclei  (Hoechst  33342),  mitochondria  (diIC-[5]),  endosomes  (lissamine  rho- 
damine),  B-dextran,  actin  (fluorescein),  cell  volume  (Cy7-dextran). 

15  These  curves  were  measured  with  an  upright  microscope,  using  a  source 
mounted  in  a  common  epi-illumination  lamp  housing.  A  50/50  beam-splitter 
was  used  and  a  radiospectrometer  placed  at  the  BFP  of  the  condenser. 
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FIGURE  6.18.  Ramp-up  of  brightness  and  long-term  stability  for  various 
microscope  light  sources. 

improves  the  stability  of  arc  and  halogen  sources  from  0.4%  to 
0.01%  (Fig.  6.19).  However,  these  figures  are  for  total  light  output, 
and  they  cannot  prevent  local  flickering  in  the  particular  region  of 
the  plasma  that  happens  to  be  focused  onto  your  specimen.  Some 
suppliers  of  deconvolution  systems,  such  as  Applied  Precision 
Instruments  (Issaquah,  WA),  constantly  monitor  arc  output  during 
the  CCD  exposure  time.  They  then  use  this  information  to  nor¬ 
malize  the  CCD  output  for  each  plane  of  the  z-stack. 

Long-Time  Stability,  Degradation 

When  warmed  up  and  powered  by  a  regulated  power  supply, 
tungsten  halogen  sources  are  suitable  for  making  photometric 
measurements. 

Generally  speaking,  arcs  are  less  stable  than  filament  lamps 
because  the  points  of  the  electrodes  slowly  erode.  The  larger  radius 
of  curvature  that  results  reduces  the  concentration  of  current  flow 
(and  brightness)  near  the  tip  and  also  increases  the  power  level 
needed  to  sustain  the  arc.  Eventually  the  arc  will  not  ignite.  The 
intensity  of  the  xenon  arc  can  be  deeply  and  rapidly  modulated  in 


FIGURE  6.19.  Stability  plot  of  halogen  source,  with  and  without  optical- 
feedback  power  stabilizing.  Although  quartz  halogen  sources  are  very  stable, 
temperature  rise  increases  output,  and  air-conditioning  and  other  factors  can 
produce  transients.  These  can  be  compensated  for  by  using  the  output  from  a 
monitor  photodiode  as  the  input  to  a  negative  feedback  system  controlling  the 
power  supplied  to  the  lamp.  (The  plot  was  kindly  provided  by  Oriel/Newport, 
Irvine,  CA  and  reflects  the  performance  of  their  68950  power-supply  controller.) 


time  to  keep  the  electrodes  cool,  as  is  the  case  for  the  electronic 
flash  used  for  photography.  Alternatively,  the  position  of  the  arc 
plasma  can  be  stabilized  either  by  a  periodic  magnetic  field 
imposed  by  a  rotating  permanent  magnet  or  by  the  superimposi¬ 
tion  of  a  small,  high  frequency  AC  current  on  the  main  DC  exci¬ 
tation  current  (Woodlee  et  al.,  1989). 

As  the  photoelectric  effect  is  fully  reversible,  it  is  not  surpris¬ 
ing  that  the  LED  has  the  lowest  operating  temperature  and,  as  a 
result,  is  the  most  stable  source.  In  addition,  as  long  as  it  is  oper¬ 
ated  at  the  proper  voltage/current,  the  LED  has  a  much  longer  life¬ 
time  than  all  other  sources.  One  usually  has  to  change  an  arc  about 
every  200  h  and  incandescent  lamps  about  every  500  h,  but  the 
LED  sources  have  lifetimes  in  the  range  of  a  few  thousands  hours 
without  significant  loss  of  intensity.  Some  manufacturers  promise 
a  lifetime  of  100,000h  before  the  source  intensity  drops  to  70%. 

Stability  in  Wavelength 

In  the  discharge  of  an  arc  lamp,  the  gas  pressure  affects  the  com¬ 
position  of  the  spectrum.  As  the  pressure  rises,  the  prominent  lines 
broaden  and  the  continuum  rises  [e.g.,  compare  the  continuum 
brightness  of  a  mercury  and  a  high-pressure  xenon  lamp  in  Fig 
6.5(A)].  On  the  other  hand,  the  peak  wavelengths  of  prominent 
lines  shift  just  a  few  nanometers  (<5). 

The  continuous  spectrum  of  an  incandescent  lamp  depends 
only  on  the  temperature  of  the  filament  (Planck’s  law)  and  on  the 
gas  present  in  the  bulb  (Fig.  6.17).  At  a  fixed  current,  a  change  in 
color  temperature  can  only  occur  if  evaporated  tungsten  condenses 
on  the  glass  bulb.  A  change  in  the  current  to  an  LED  causes  a  shift 
of  the  emission  peak  similar  in  magnitude  to  that  seen  in  lines  of 
the  arc  lamps.  This  may  be  due  to  the  LED  die  not  being  perfectly 
homogeneous.  The  size  of  the  shift  depends  on  the  type  and  quality 
of  the  crystal  material  used. 

As  the  shifts  are  small  compared  to  the  width  of  the  lines,  one 
can  neglect  the  effect  for  arc  lamps.  For  LEDs,  one  either  has  to 
calibrate  the  dependence  of  the  wavelength  on  the  operating 
current  or  operate  the  die  in  a  different  way  (see  sections  on 
Radiance  and  Control). 

Radiance 

Arc  lamps  are  several  orders  of  magnitude  more  radiant  than  tung¬ 
sten  filament  lamps  [Fig.  6.5(A)].  The  HBO-100  (100W  high- 
pressure  mercury  arc  lamp)  is  the  most  radiant  of  the  commonly 
used  lamps,  whatever  the  wattage,  because  it  has  a  very  small 
source  size  (compared  to  the  HBO-200,  for  example).  Because  of 
the  optics  rule  mentioned  at  the  beginning  of  this  chapter,  the  larger 
arcs  are  only  useful  to  illuminate  larger  areas  of  the  specimen 
rather  than  to  illuminate  a  single  spot  with  maximum  intensity.  The 
main  limitation  on  arc  radiance  is  that  the  electrode  tips  erode  or 
even  melt  as  the  power  level  increases  (Fig.  6.5). 

New  arc  sources  of  very  high  radiance  have  been  described  by 
Steen  and  Sorensen  (1993).  In  these  sources,  commercial  Hg,  Xe, 
or  Hg/Xe  arc  lamps  have  been  modified  to  permit  direct  water 
cooling  of  the  electrodes  and  the  superposition  of  large-amplitude 
short-duration  (20  |is)  current  pulses  on  the  DC  operating  current. 
Increases  in  output  of  up  to  10-fold  were  observed  during  each 
pulse.  Hell  and  colleagues  (1991)  describe  anew  generation  of  short 
arc  lamps  with  extremely  short  electrode  distances  (0.5  mm)  using 
a  Xe/I  fill  and  tungsten  carbide  electrodes  in  a  quartz  bulb.  The  tung¬ 
sten  iodide  T1J  dopant  gives  rise  to  a  radiating  plasma  spot  only 
150[tm  in  diameter.  Over  the  450  to  550  nm  band,  the  radiance 
exceeded  that  of  the  conventional  Xe  short  arc  by  a  factor  of  3  to 
5  and  that  of  an  HBO-100  mercury  lamp  by  12.  The  new  arc  lamp 
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has  useful  radiance  from  360  to  570nm.  These  developments  push 
the  arc  source  radiance  closer  to  the  realm  once  thought  available 
only  from  lasers.  Tables  of  the  radiance  of  various  arcs  and  lasers 
can  be  found  in  the  catalogues  of  Melles  Griot  (Rochester,  NY ) 
and  Oriel  Corp.  (Stratford,  CT,  now  a  division  of  Newport  Inc.). 

When  driven  by  a  120  Hz  square  wave,  the  modulated  (Hg-I) 
arc  lamp  introduced  by  LTM  Corp.  (Sun  Valley,  CA)  produces  a 
very  useful  spectrum  with  an  efficiency  of  110  lumens AV,  com¬ 
pared  to  301umens/W  for  a  normal  xenon  arc.  The  deep  square- 
wave  modulation  of  the  Hg-I  arc  reduces  average  heat  production, 
allowing  the  lamp  housing  to  be  compact  enough  for  placement 
near  the  microscope. 

As  the  bulk  depletion  region  inside  the  LED  die  is  an  isotropic 
emitter  (lambert),  one  might  assume  that  the  light  leaving  the  front 
surface  of  the  die  would  also  be  isotropic  in  all  directions.  However, 
as  the  light  generated  in  the  volume  of  the  crystal  must  pass  through 
the  crystal/air  interface,  any  rays  that  strike  this  surface  at  less  than 
the  critical  angle  will  be  reflected  and  reabsorbed  by  the  crystal. 
Approximately  50%  of  the  light  generated  internally  is  lost  in  this 
way  and  less  light  is  emitted  at  bigger  angles. 

The  radiance  of  the  high-brightness  LEDs  available  today  is 
still  far  less  than  that  of  the  prominent  lines  of  the  arc  lamps  (Fig. 
6.20).  In  the  continuous  operation  mode,  the  brightest  2  X  2  mm 
LED  die  (Luxeon,  5  W  Emitter,  Fa.  Lumileds,  San  Jose,  CA)  today 
delivers  around  50%  of  the  continuum  radiance  of  a  75  W  XBO 
at  the  same  wavelength,  and  is  bright  enough  to  get  an  accept¬ 
able  fluorescent  signal  from  a  well-stained  specimen  (Braun  and 
Merrin,  2003).  Unfortunately  these  high-power  dies  are  not  yet 
very  stable  and  because  of  the  high  thermal  load  they  degenerate 
very  fast.  More  to  the  point,  they  are  still  not  available  in  all  wave¬ 
lengths.  At  present,  better  results  are  obtained  using  smaller  emit¬ 
ters  (1 W  emitter,  Luxeon)  to  build  up  a  light  source.  With  a  proper 
heat  sink  these  are  now  very  stable. 

In  pulsed-mode  operation  (see  subsection  under  Control),  the 
available  radiance  can  be  a  factor  of  20  higher  than  for  the  same 
unit  used  in  continuous  mode.  When  grabbing  fluorescence  pic¬ 
tures  quickly,  this  mode  is  the  most  suitable  one,  as  one  can  trigger 
the  camera  with  the  light  pulse  to  ensure  efficient  usage  of 
the  emitted  light.  This  mode  of  operation  is  also  now  being 
used  in  the  machine  vision  area  of  industry  where  LEDs  have 
become  a  long-life  substitute  for  xenon  flash  lamps  for  illuminat¬ 
ing  moving  objects  (see,  e.g.,  available  light  sources  at 
http://www.laser2000.de).  For  other  manufacturers,  see  the  links 
in  Table  6.1. 
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TABLE  6.1.  Useful  Links 

Microscopes,  optics,  and  light  sources 
http://www.zeiss.com/ 
http://www.lot-oriel.com/ 
http://www.mellesgriot.com 
http://www.chroma.com/ 
http  ://w  w  w.  edmundoptic  s  .com 
http://www.optics.org/ 
http://www.wahl-optoparts.de/ 
http://www.oceanoptics.com 

LEDs 

http://www.luxeon.com/products/family.cfm?familyId=l 

http://www.osram-os.com/ 

http://www.nichia.com/ 

http://we.home.agilent.com/USeng/nav/-l  1 143. 0/home.html 

http://www.led.com/ 

http://www.optotech.com/ 

http://www.toyoda-gosei.com/led/index.html 

http  ://ww  w.  stockery  ale  .com/ 

http  ://ledmuseum  .home .  att.net/ledleft.htm 

http://www.laminaceramics.com 

Basics  and  history 

http ://  micro .  magnet .  f su .  edu/primer/ 
http://inventors.about.com/library/inventors/bllight.htm 


Control 

The  quartz-halogen  lamp  is  simply  driven  by  a  stabilized  DC 
power  supply  converting  the  plug  voltage  into  an  adjustable 
voltage  of  2  to  12  volts.  Varying  the  voltage  controls  the  temper¬ 
ature  of  the  filament  and  thereby  the  spectral  properties  and  inten¬ 
sity  of  the  light. 

Arc  lamps  are  usually  also  driven  by  a  current-stabilized  power 
supply.  The  current  can  be  decreased  to  70%  to  lower  the  optical 
output  and  conserve  the  electrodes  if  one  does  not  need  the  full 
optical  power.  Below  70%  the  plasma  becomes  unstable.  Because 
of  the  decreasing  temperature  of  the  discharge,  the  vapor  pressure 
drops  and  the  discharge  stops.  Modern  arc  lamps  have  a  heating 
filament  wrapped  around  the  bulb.  This  filament  heats  the  bulb, 
restoring  the  vapor  pressure,  and  allowing  the  current  to  be 
decreased  down  to  30%  without  stopping  the  discharge. 

Neither  arcs  nor  halogen  lamps  can  be  switched  on  rapidly.  To 
change  the  emitting  wavelength  or  intensity  quickly  one  has  to  use 
mechanical  shutters  and  filter  wheels  and  switching  times  are 
usually  longer  than  100  ms. 

The  stabilizing  circuitry  of  the  arc  power  supply  can  stabilize 
the  voltage,  the  current,  or  the  total  power  (voltage  x  current).  If 
the  voltage  is  stabilized,  the  current  (and  the  brightness)  will 
slowly  decrease  as  the  electrodes  become  worn  round.  If  the 
current  is  stabilized,  the  brightness  will  stay  fairly  constant16  until 
the  electrodes  become  too  rounded  for  the  arc  to  “strike.” 
However,  because  an  ever-higher  voltage  is  required  to  maintain 
the  fixed  current,  as  the  electrodes  wear  the  power  sent  to  the 
arc  slowly  increases.  As  a  result,  it  can  overheat  and  sometimes 
explode.  Although  power  supplies  that  stabilize  the  total  power 
level  will  avoid  overheating,  the  light  output  will  slowly  drop  with 
the  current  as  the  voltages  needed  to  maintain  the  arc  increases. 
All  this  suggests  that  it  is  best  not  to  run  an  arc  too  long  and  that 
it  might  be  a  good  idea  to  monitor  the  voltage  across  the  arc  to 
detect  warning  signs. 


FIGURE  6.20.  LED  luminous  intensity  for  various  colors  and  dye  compounds.  - 

Efficiency  is  measured  in  lumens/watt  and  refers  to  the  efficiency  with  which  16  The  total  light  will  stay  about  the  same  but  it  will  be  less  concentrated  at  the 
electric  power  is  converted  into  optical  output.  tip  of  the  electrode. 
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FIGURE  6.21 .  What  an  actual,  functioning  LED  micro¬ 
scope  source  looks  like.  A  close-packed  array  with  four 
monochrome  (470,  525,  590,  620  nm)  colors  on  one 
matrix. 


As  each  electron  passing  the  depletion  region  emits  one 
photon,  LEDs  can  be  controlled  by  any  current-stabilized  electri¬ 
cal  source.17  Depending  on  what  is  needed,  the  LED  configuration 
as  well  as  the  control  circuit  can  be  easily  changed.  If  only  a  single 
LED  device  is  used  (e.g.,  a  “white”  LED,  usually  one  that  couples 
a  blue  primary  emission  with  red  and  green  light  from  blue-excited 
phosphors),  only  a  single-channel  current  source  is  needed  and  the 
intensity  is  controlled  by  changing  the  current  flowing  through  the 
LED.  It  is  more  common  to  use  more  complex  LED  structures 
combining  LED  dies  with  different  emission  wavelengths  to  obtain 
either  narrow-band  light  for  multi  fluorescence  or  “white  light”  in 
bright-field  microscopy  (Fig.  6.21).  Such  devices  are  controlled  by 
a  multi-channel  current  source.  By  rapidly  switching  these  currents 
on  and  off,  it  is  possible  to  change  intensity  or  emission  wave¬ 
length  on  a  microsecond  or  even  nanosecond  scale.  This  is  a  very 
important  feature  for  short-time  scale  methods  such  as  fluores¬ 
cence  lifetime  imaging  measurements  (FLIM;  Hermann  et  al., 
2001).  The  switching  on  this  timescale  is  called  pulse  mode. 

Because  the  peak  emission  of  a  given  type  of  a  LED  can  be 
shifted  by  changing  the  current  level,  it  is  often  more  suitable  to 
operate  multi-LEDs  in  the  pulse  mode.  One  sets  the  peak  current 
to  produce  the  desired  output  wavelength  and  then  changes  the 
average  source  brightness  by  varying  the  pulse  width  at  a  fixed 
peak  current.  Although,  compared  to  continuous  mode,  more  total 
light  is  available  in  this  way,  using  a  higher  current  for  more  than 
a  short  pulse  will  lead  to  thermal  damage.  The  “damage  thresh¬ 
old”  current  pulse  width  must  be  evaluated  for  each  LED. 

The  spectral  output  of  the  LED  can  be  controlled  very  pre¬ 
cisely  in  this  way.  The  optical  output  follows  the  current  pulse 
without  significant  delay.  Pulse-modulation  frequencies  up  to 
megahertz  are  possible. 

MEASURING  WHAT  COMES  THROUGH 
THE  ILLUMINATION  SYSTEM 

The  procedures  for  measuring  the  light  throughput  of  any  micro¬ 
scope  with  a  photometer  are  thoroughly  described  in  a  step-by-step 
manner  in  the  book  Photomicrography  (Loveland,  1970).  He  even 
describes  making  a  photometer  for  such  a  purpose,  but  a  useful  sub¬ 
stitute  can  be  made  by  attaching  almost  any  small  photodiode  (or 
even  a  small  “solar  cell”)  to  an  inexpensive  digital  voltmeter  set  on 
a  sensitive  current  range.  Using  this,  one  can  measure  light  in  the 


17  On  the  other  hand,  use  of  a  voltage-stabilized  source  will  almost  certainly 
damage  these  devices.  At  a  fixed  voltage  the  current  increases  with  temper¬ 
ature,  causing  thermal  runaway. 


most  obscure  locations  within  the  microscope.  “Photon  book¬ 
keeping”  based  on  such  measurements  is  the  only  way  to  pinpoint 
those  parts  of  the  light  path  where  preventable  loss  is  occurring. 

Microscopists  not  interested  in  building  a  photometer  can 
obtain  one  of  the  commercial  units.  World  Precision  Instruments 
(Sarasota,  FL)  markets  a  fiber-optic  monochromator  and  photo¬ 
multiplier  subsystem  that  can  be  used  to  examine  light  at  the  inten¬ 
sities  present  at  any  location  in  a  confocal  optical  system.  A  50  |im 
fiber  is  standard  with  this  system  and  other  vendors  are  listed  in 
Table  6.3. 

Young  (1989)  described  the  use  of  a  feedback-controlled  LED 
to  generate  known  amounts  of  light  from  small  (5-50pm)  sources. 
Using  this  system  in  either  the  source  plane  or  the  image  plane,  he 
was  able  to  calibrate  the  input-output  characteristics  of  a  micro¬ 
scope  system  over  four  orders  of  magnitude. 

For  a  measurement  of  radiance  resolved  by  angle  and  wave¬ 
length  simultaneously  one  must  use  a  professional  radiospec¬ 
trometer  such  as  the  CS-1000  made  by  Minolta.  Such  a  device  can 
provide  very  detailed  information  on  the  quality  of  an  illumination 
source  and  the  illumination  path. 

Selective  light  loss  can  occur  anywhere  along  the  optical  path, 
heat  filters,  tilted  interference  filters,  and  dust  windows,  as  well  as 
obvious  lens  elements  [consider  that  every  lens  surface  causes  the 
loss  of  at  least  1  %  of  the  incident  light  despite  anti-reflection  ( AR) 
coatings].  In  the  past,  the  transmission  characteristics  of  objectives 
were  seldom  displayed  in  manuals,  and  even  the  general  charac¬ 
teristics  of  UV  versus  non-UV  lenses  are  still  often  hard  to  obtain 
(some  figures  are  listed  in  Chapters  7,  27,  and  29,  this  volume). 
Popular  photography  magazines  often  feature  the  color  bias  of 
various  camera  lenses,  and  these  show  that  the  color  effect  of  a 
given  AR  coating  is  different  for  the  large  NA  rays  than  for  those 
near  the  optical  axis,  due  to  the  quality  and  the  optical  properties 
of  the  lens  coating.  The  AR  coating  on  dust  windows  may  block 
the  transmission  of  UV  or  IR  illumination.  Epi-illumination 
requires  broad  transmission  in  both  the  illumination  and  the 
viewing  direction. 

The  Bare  Minimum 

Even  if  one  isn’t  inclined  to  be  a  full-time  photon  sleuth,  it  is  wise 
at  least  to  monitor  the  performance  of  the  illumination  system 
under  a  few  commonly  used  standard  conditions.  For  example,  one 
should  monitor  how  much  light  emerges  from  a  favorite  high-NA 
objective  when  it  is  set  up  for  Kohler  illumination  with  a  particu¬ 
lar  filter  cube  and  with  the  field  diaphragm  set  to  just  illuminate 
the  full  field  of  this  objective.  This  can  be  measured  with  a  1  cm2 
photometer  paddle  held  in  front  of  the  objective.  Assuming  that 
one  does  not  want  to  oil-couple  the  sensor  to  the  objective,  make 
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sure  that  the  front  of  it  is  free  from  oil  so  that  the  fraction  of  the 
light  escaping  into  the  air  is  set  only  by  total  internal  reflection 
from  the  flat  front  element  (not  an  ideal  situation  but  at  least  one 
that  is  repeatable).  Then  do  the  same  with  the  most  commonly  used 
low  magnification  lens  and  maybe  a  couple  of  different  filter  cubes. 
Changes  in  these  numbers  will  warn  of  misalignment,  dust,  aging 
arc  bulbs,  damaged  filters,  or  help  one  determine  the  final  resting 
place  of  that  bit  of  paper  that  fell  down  inside. 

An  alternative  to  the  photometer  paddle  is  an  Ulbricht  sphere 
(also  called  an  integrating  sphere).  Using  an  Ulbricht  sphere,  there 
is  no  problem  with  rays  being  reflected  by  the  surface  of  the  detec¬ 


tor  because  the  entrance  of  the  detector  is  just  a  hole.  Using  this 
device,  it  is  easy  to  measure  what  fraction  of  light  reaches  the  spec¬ 
imen.  On  an  upright  microscope,  remove  the  condenser  and  the  xy- 
stage  and  mount  the  Ulbricht  sphere  below  the  objective  with  the 
entrance  hole  at  the  specimen  plane.  It  may  help  to  use  the  halogen 
lamp  with  the  bright-field  contrast  in  reflection  to  align  the  parts 
to  each  other.  Mount  the  lamp  of  interest  at  the  reflection  port  for 
illumination.  All  the  light  passing  the  objective  is  captured  by  the 
sphere.  Table  6.2  shows  the  optical  power  delivered  to  the  speci¬ 
men  plane  measured  in  this  way  for  various  light  sources  and  for 
two  objectives  with  different  fields  of  view. 


TABLE  6.2.  Optical  Power  of  Different  Light  Sources  in  the  Specimen  Plane 


Filterset  Excitation  Wavelength  Examples  for 

HBO103W2 

XB075  LEDs 

HAL  100 
(at  12  V) 

No.  Used  (pass  width)  in  nm  Fluorescence  Dyes 

Optical  Power  [mW] 

Objective  PlanNeoFluar  40X/0.75,  field  of  view  0.625  mm  diameter 

#2 

365  (50) 

DAPI,  Hoechst33342 

30.8 

0.1 

4.4 

Nichia  3  W  365  nm 

0 

#47 

436  (20) 

CFP,  ChromomycinA 

10.5 

3.5 

1.4 

Luxeon  3W  450  nm 

0.2 

#9 

450-490 

GFP,  Fluorescein 

6.4 

12.7 

1.9 

Luxeon  1W  470  nm 

4.8 

Luxeon  3W  470  nm 

10.0 

Luxeon  3W  450  nm 

3.2 

#46 

500  (20) 

Calcium  Green,  YFP 

1.6 

4.4 

0.4 

Luxeon  1 W  505  nm 

1.5 

Luxeon  3  W  505  nm 

1.5 

#14 

510-560 

RhodamineB 

20.2 

12.7 

1.3 

Luxeon  1W  530  nm 

2.4 

Luxeon  3W  530  nm 

0.1 

Luxeon  1 W  505  nm 

7.1 

#20 

546  (12) 

Cy3,  Rhodamine 

11.1 

2.8 

© 

1.4 

#26 

575-625 

Cy5 

125.0 

9.7 

0.5 

Luxeon  1 W  590  nm 

8.3 

Objective  fluar  lOx/O.5,  field  of  view  2.5  mm  diameter 


#2 

365  (50) 

61.4 

3.4 

8.5 

Nichia  3  W  365  nm 

0 

#47 

436  (20) 

43.9 

10.5 

3.4 

Luxeon  3W  450  nm 

1.8 

#9 

450^(90 

27.2 

33.8 

4.8 

Luxeon  1W  470  nm 

11.1 

Luxeon  3W  470  nm 

23.0 

Luxeon  3W  450  nm 

11.1 

#46 

500  (20) 

8.5 

11.7 

1.5 

Luxeon  1 W  505  nm 

4.3 

Luxeon  3  W  505  nm 

4.5 

#14 

510-560 

79.0 

36.7 

2.8 

Luxeon  1W  530  nm 

4.0 

Luxeon  3W  530  nm 

0.9 

Luxeon  1 W  505  nm 

22.6 

#20 

546  (12) 

44.2 

8.3 

© 

4.2 

#26 

575-625 

243.1 

30.1 

2.7 

Luxeon  1 W  590  nm 

30.1 
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The  different  filter  sets  were  chosen  to  show  a  representative 
profile  of  the  optical  power  of  the  light  sources  at  different  wave¬ 
lengths.  The  bandpasses  of  filter  sets  #9  and  #46  lie  between  two 
prominent  lines  of  the  HBO103W2.  The  XB075  is  much  more 
radiant.  The  optical  power  of  the  HAL  100  is  between  one  third 
and  one  half  of  the  continuum  of  the  HBO103W2.  This  is  suffi¬ 
cient  to  excite  the  brightest  dyes.  We  have  made  preliminary  mea¬ 
surements  of  two  examples  for  LED-based  sources.  In  each  case, 
a  single  1 W  Luxeon  emitter  was  used  with  a  single  collector  lens 
with  no  further  alignment.  This  means  that  the  values  for  the  LEDs 
in  Table  6.2  represent  a  minimum  for  the  optical  power  at  the  spec¬ 
imen  plane  using  only  a  single,  1 W  die.  With  a  more  radiant 
emitter,  or  more  individual  dies  and  proper  collecting  optics,  the 
optical  power  at  the  specimen  plane  can  easily  be  increased  by  a 
factor  of  4  to  8.  This  would  put  the  LED  radiance  between  the 
HAL  100  and  the  arc  sources. 

TYPES  OF  CONFOCAL  MICROSCOPES  THAT 
CAN  USE  NON-LASER  LIGHT  SOURCES 

The  notion  that  confocal  microscopes  must  use  laser  illumination 
is  widespread  because  most  commercial  confocal  microscopes  are 
single-beam  instruments  (Leica,  Nikon  Real  Time,  Olympus, 
Zeiss),  and  these  currently  use  only  laser  illumination.  In  fact,  none 
of  these  microscopes  even  make  a  provision  for  the  user  to  connect 
a  non-laser  light  source  for  use  in  the  confocal  mode. 

Nevertheless,  it  is  not  true  that  single-beam  confocal  micro¬ 
scopes  require  laser  light.  Minsky  (the  inventor  of  the  first 
confocal  microscope)  used  a  zirconium  arc  illuminator  in  the  func¬ 
tional  prototype  stage-scanning  microscope  he  built  in  the  1950s 
(Minsky,  1988).  Many  current  commercial  disk-scanning  confocal 
microscopes  come  only  with  non-laser  sources  because  only  such 
sources  can  provide  the  broad  beam  needed  to  simultaneously 
illuminate  the  many  confocal  apertures  in  the  field  of  view. 

Using  a  tandem-scanning  confocal  microscope  with  a  high-NA 
water-immersion  objective,  transparent  ciliate  protozoa  such  as 
paramecium  and  vorticella  can  easily  be  viewed  by  eye  swimming 
in  water  in  the  confocal  BSL  mode.  BSL  images  are  formed  using 
the  light  that  is  scattered  by  the  index  of  refraction  difference 
between  organelles  and  water.  By  carefully  adjusting  the  rotation 
speed  of  the  aperture  disk,  one  can  view  the  rapidly  beating  cilia 
with  stroboscopic  illumination.  A  field  of  beating  cilia  viewed  en 
face  appears  as  dots  (cross-sections  of  the  cilia)  slowly  moving  in 
circles.  Though  easily  viewed  by  eye,  this  motion  is  difficult  to 
capture  electronically  or  photographically  because  the  brain  is  able 
to  extract  meaning  out  of  successive  images  with  the  slight  trail- 
ing-edge  blur  that  renders  electronically  captured  single  images 
meaningless. 

Tandem  Scanning:  Basic  Description 

The  tandem- scanning  mechanism  consists  of  a  symmetrical,  spin¬ 
ning  Nipkow  aperture  disk  at  the  intermediate  image  plane  of  the 
objective.  Thousands  of  apertures  arranged  in  spirals  both  send 
beams  to  be  focused  on  the  object  and  sample  the  light  returning 
to  form  the  intermediate  image.  The  double-sided  optical  system 
developed  by  Petran  uniformly  illuminates  the  excitation  area  of 
the  disk  that  is  to  be  imaged  onto  the  object.  Through  a  series  of 
mirrors  and  beam-splitters,  the  image  returning  from  the  spots  in 
the  specimen  is  focused  onto  the  lower  surface  of  the  opposite  side 
of  the  disk  where  the  in-focus  light  passes  through  a  mirror  image 
conjugate  set  of  holes  (Fig.  10.4,  this  volume).  This  series  of 


mirrors  and  beam-splitters  permits  the  illuminating  and  sensing 
apertures  to  be  distinct,  thereby  preventing  light  reflected  by  the 
solid  part  of  the  disk  on  the  illumination  side  from  reaching  the 
imaging  side  of  the  system. 

The  light-source  optics  must  fulfill  two  functions.  They  must 
illuminate  the  area  of  the  disk  that  will  be  utilized  to  form  the  final 
image  (usually  1  to  2.5  cm  diameter).  In  addition,  this  light  must 
leave  the  disk  with  the  correct  angle  of  divergence  to  fill  the  BFP 
of  the  objective  lens.  On  disks  with  very  small  holes,  diffraction 
at  the  holes  will  usually  ensure  that  the  second  condition  is  satis¬ 
fied,  so  the  problem  becomes  how  to  get  the  maximum  amount  of 
light  incident  on  the  active  area  of  the  disk.  This  is  important 
because,  as  only  1%  to  2%  of  the  disk  is  open,  the  system  is  very 
wasteful  of  light.  Careful  matching  of  a  large-NA  collector  lens 
and  an  optimized  condenser  is  needed  to  ensure  that  sufficient  light 
reaches  the  specimen  to  form  an  image  in  a  reasonable  time. 

Single-Sided  Disk  Scanning:  Basic  Description 

In  a  single-sided,  disk-scanning  optical  system,  the  spinning 
Nipkow  aperture  disk  is  again  located  at  the  intermediate  image 
plane  of  the  objective  but,  because  the  beam-splitter  is  above  the 
disk,  the  same  apertures  now  serve  as  both  sources  and  pinholes 
(Fig.  10.4,  this  volume).  The  aperture  disk  is  tilted  and  covered 
with  “black  chrome”  to  reduce  reflections  of  the  source  from 
reaching  the  eyepiece.  Furthermore,  a  polarizer  placed  in  the 
illumination  path,  a  quarter  waveplate  above  the  objective,  and  an 
analyzer  at  the  eyepiece  form  an  “anti-flex”  system  to  further 
reduce  the  effect  of  disk  reflections.18  Because  it  lacks  any  mirrors 
between  the  disk  and  the  objective,  the  single-sided  system  is 
self-aligning. 

Boyde  and  Petran  (1990)  directly  tested  the  light  budget  in  the 
two  types  of  disk  confocal  systems  and  found  little  difference  in 
light  efficiency,  though  the  tandem  system  appeared  to  have  better 
contrast. 


EXPOSURE  TIME  AND  SOURCE  BRIGHTNESS 

In  conventional  microscopy,  data  for  every  point  in  the  image  is 
collected  in  parallel.  This  leads  to  a  short  capture  time  compared 
to  any  scanning  process.  The  capture  time  is  only  limited  by  the 
time  needed  to  fill  the  pixels  in  the  CCD  camera.  Depending  on 
the  magnification,  the  fluorescent  dye  concentration,  and  the 
quality  of  the  CCD  itself,  recording  times  vary  from  a  few  mil¬ 
liseconds  to  several  seconds.  Living  cell  observations  with  moving 
specimens  are  no  problem.  Figure  6.22  shows  comparison  pictures 
of  cells  taken  with  an  HBO  arc  lamp  and  an  LED-based  source. 
Although  the  lower  brightness  of  present  LEDs  required  a  record¬ 
ing  time  about  four  times  longer  than  that  needed  with  the  HBO 
source,  the  quality  of  the  images  is  comparable. 

In  scanning  microscopy,  the  image  is  formed  by  scanning  a 
point  or  a  group  of  points  over  the  surface  to  be  imaged,  and  this 
scanning  process  takes  a  finite  amount  of  time.  If  a  raster  scan  is 
used,  the  image  is  completed  when  the  raster  is  finished,  and  so 
the  confocal  microscope  is  a  sampling  system  in  both  time  and 
space.  To  view  specimens  that  move  or  change  accurately,  the  scan 
time  must  be  short  compared  to  the  expected  rate  of  change.  This 
requires  not  just  a  fast  scanning  system  but  also  a  light  source 


18  While  at  the  same  time,  reducing  light  throughput  by  about  two  thirds. 
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FIGURE  6.22.  Fluorescent  actin  filaments  imaged  using  HBO  and  LED  sources.  (A)  HBO103W/2,  exposure  600ms,  (B)  Luxeon  Star  LED,  1 W,  exposure  2  s. 
In  the  enlarged  insets,  one  can  see  that  some  of  the  broadband  light  from  the  HBO  gets  past  the  excitation  filter  to  excite  the  red  MitoTracker  Red  dye  as  well 
as  the  Bodipy  FL  phallacidin,  green  dye,  while  the  narrowband  light  from  the  LED  does  not  do  so.  (Specimens  are  fixed  BPAE  cells  stained  on  a  Molecular 
Probes  FluoCells  prepared  #1  slide.  Imaged  with  an  Axioplan  2,  63x  /1.4  oil  objective,  Axiocam  HrC,  Filterset  #9.) 


bright  enough  to  elicit  from  the  specimen  sufficient  signal  to  make 
a  usable  image  during  the  available  scan  time.  In  other  words, 
shorter  scan  times  need  brighter  sources.  In  practice,  it  has  been 
the  inability  of  arcs  and  LED-based  sources  to  match  the  bright¬ 
ness  of  the  laser  that  has  prevented  disk-scanning  instruments  from 
seriously  challenging  laser  instruments  for  viewing  low-intensity 
fluorescent  specimens. 

Because  the  rate  at  which  signal  can  be  derived  from  a  fluo¬ 
rescent  specimen  depends  fundamentally  on  the  rate  at  which  exci¬ 
tation  photons  impinge  on  the  imaged  area,  some  idea  of  the 
relative  merits  of  the  two  approaches  can  be  gained  by  measuring 
this  quantity.  When  comparing  disk-  and  laser-scanning  data  rates, 
it  is  important  to  normalize  for  the  size  of  the  area  illuminated  on 
the  specimen  from  which  data  are  being  recorded  because  the 
former  collects  data  in  parallel  while  the  latter  collects  only  from 
one  point  at  a  time.  Given  comparable  pinhole  sizes  and  optical 
efficiencies,  the  crucial  factor  for  a  disk-scanning  microscope  is  the 
rate  at  which  the  narrow-band,  excitatory  radiation  strikes  the  area 
of  the  specimen  that  can  be  imaged  by  a  high-quality  imaging 
detector,  such  as  a  cooled  CCD.  A512x512  CCD  operating  at  the 
total  magnification  needed  for  proper  Nyquist  sampling  of  0.25  |im 
resolution  data  (i.e.,  0.1  |im  pixels  at  the  specimen;  Chapter  4,  this 
volume),  images  an  area  about  50  x  50  |inr  on  the  specimen. 

In  1990,  the  illumination  systems  of  the  commercial,  double¬ 
sided  disk-scanning  confocal  microscopes  could  concentrate  only 
2  to  3pW  of  narrow-band  light  into  a  50  x  50  pm  area,  while  the 
single-sided  instruments  could  produce  6pW  (personal  communi¬ 
cation,  V.  Cejna,  Technical  Instruments,  San  Jose,  CA).  By  con¬ 
trast,  the  laser  sources  on  the  confocal  laser-scanning  microscope 
(CLSM)  can  easily  deliver  lOOx  more  power  (without  producing 


significant  singlet-state  fluorescence  saturation!)  and  can  conse¬ 
quently  produce  data  from  (and  bleaching  of!)  the  specimen  at  a 
proportionally  higher  rate.  This  improvement  is  only  partially 
offset  by  the  fact  that  the  CCD  detector  is  about  3  to  6  times  more 
quantum  efficient  than  the  photomultiplier  tube  used  in  the  laser 
systems. 

On  the  other  hand,  because  disk-scanning  instruments  use 
many  simultaneous  apertures,  the  absolute  limit  on  data  acquisi¬ 
tion  presented  by  singlet-state  fluorescence  saturation  (Chapters  2, 
16,  and  21,  this  volume)  is  far  less  of  a  limitation  on  these  instru¬ 
ments.  As  a  result,  if  higher  radiance  non-laser  sources  are  devel¬ 
oped  and  low-read-noise,  electron-multiplier  CCDs  (EM-CCDs) 
replace  the  conventional  CCDs  now  commonly  used,  the  disk¬ 
scanning  approach  could  eventually  produce  even  higher  useful 
frame  rates  than  single-beam  laser  instruments  and  do  so  with  the 
same  number  of  photons  from  the  specimen. 

Another  way  to  speed  up  the  data  rate  is  to  use  larger  CCD 
sensors  (10002).  These  would  permit  parallel  detection  of  data 
from  a  larger  area  of  the  specimen  while  still  maintaining  the  same 
pixel  size.  Although  this  strategy  would  increase  the  effective  data 
acquisition  rate  by  an  amount  proportional  to  the  number  of 
sensors  in  the  detector,  it  would  do  so  only  at  the  price  of  viewing 
ever  larger  fields  of  the  specimen.  In  other  words,  while  it  can  be 
useful  to  survey  larger  fields  in  the  same  amount  of  time,  it  is  not 
the  same  thing  as  increasing  the  source  brightness  or  optical  effi¬ 
ciency  to  permit  imaging  a  particular  cell  more  rapidly. 

As  current  disk-scanning  systems  utilizing  better  illumination 
sources  and  improved  optics  claim  power  levels  at  the  specimen 
that  are  5  to  10  times  higher  than  those  measured  in  1990,  the 
balance  may  soon  shift  to  the  disk  scanners. 
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FUTURE  TRENDS 

Viewed  in  one  way,  the  arc  lamps  produce  enough  light  right  now 
to  saturate  or  bleach  common  fluorescent  dyes.  The  weak  point  of 
these  sources  is  the  difficulty  of  rapidly  controlling  operating 
parameters  such  as  emitted  intensity  or  spectral  distribution.  As 
these  sources  have  been  optimized  over  a  period  of  many  years,  a 
quantum  leap  in  performance  is  unlikely.  On  the  other  hand,  arcs 
using  different  gas  mixtures  and  electrode  materials  are  constantly 
being  developed  and  small  improvements  are  likely  to  continue. 
The  same  is  true  for  tungsten-halogen  sources. 

Viewed  in  another  way,  100  years  after  the  first  use  of  Hg  arcs 
in  microscopy,  and  40  years  after  the  advent  of  the  LED, 
microscopy  has  a  new  light  source  with  many  exciting  possibili¬ 
ties.  LEDs  have  all  the  features  that  arc  lamps  lack  and  they  will 
soon  be  efficient  enough  to  be  run  on  a  small  battery.  Although 
their  weak  point  is  still  their  marginal  intensity,  if  one  looks  at  the 
LED  optical  efficiency  line  over  the  last  few  years  (Fig.  6.23),  one 
notices  a  very  interesting  trend.  LED  brightness  is  projected  to 
increase  by  about  a  factor  of  3  in  the  next  5  years.  Efforts  are  under 
way  to  use  different  growth  mechanisms  to  produce  LED  die  crys¬ 
tals  with  a  geometry  that  decreases  the  loss  of  light  through  inter¬ 
nal  reflection.  If  this  effort  is  successful,  LEDs  should  be  able  to 
succeed  in  all  fluorescence  applications. 

Improvements  in  the  coupling  between  the  LEDs  and  the 
microlens  array  and  diffractive  optical  elements  will  increase  the 
efficiency  of  the  collection  optics.  Molded  plastic  technology  is 
becoming  ever  more  flexible  in  being  formed  to  satisfy  the  require¬ 
ments  of  this  application.  The  brightness  of  a  multi-color  source 
mounted  as  a  single  flat  array,  such  as  that  shown  in  Figure  6.21, 
could  be  increased  by  3  or  4x  by  coupling  together  light  from  three 
to  four  planar  sources,  each  emitting  at  a  different  wavelength, 
using  an  arrangement  of  dichroics  and  prisms  similar  to  that  used 
to  separate  R,  G,  and  B  light  in  a  three-chip  color  CCD  sensor. 
This  would  allow  each  LED  color  to  be  emitted  at  every  location 
in  the  BFR  Three  times  as  many  dies  would  increase  total  light 
output  by  300%. 

Because  LEDs  have  no  emission  at  either  1/2  or  2x  the  design 
wavelength,  it  should  be  possible  to  develop  more  inexpensive  and 
efficient  light-source/filter  sets  for  particular  fluorescent  dyes. 

Somewhat  farther  over  the  horizon  is  the  promise  of  organic 
LEDs  (OLEDs).  First  developed  in  the  early  1990s,  OLEDs  are 
built  of  organic  molecules  and  light  is  generated  by  exciting 
the  molecular  orbitals  of  chromophore  groups.  Organic  molecules 
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FIGURE  6.23.  Development  of  the  optical  efficiency  of  LEDs  in  the  last  40 
years. 


sitting  in  a  plastic  lattice  can  be  formed  into  almost  any  shape  one 
can  imagine.  The  brightness  of  these  sources  is  now  similar  to  that 
of  conventional  LEDs  20  years  ago  (Zhou,  2001).  Although  the 
operating  lifetime  of  these  sources  is  still  very  short,  this  technol¬ 
ogy  is  important  as  a  potential  area-display  for  consumer  goods, 
such  as  video  cellular  phones,  so  substantial  improvements  may 
occur. 

Significant  improvements  in  arc  sources  (~10x)  could  be  real¬ 
ized  by  using  a  modified  elliptical  collector  (Luthjens  et  al.,  1990) 
with  one  of  the  newer  Xe/I  sources  and  by  optimizing  the  magni¬ 
fication  of  the  illumination  optics  to  concentrate  more  light  from 
the  brightest  part  of  this  source  into  the  50  pm  field  covered  by  a 
512x512  CCD. 

Another  development  that  should  be  mentioned  is  the  emer¬ 
gence  of  a  number  of  companies  providing  stand-alone  light 
sources  for  use  in  microscopy  (Table  6.3).  Although  initially  these 
tended  to  be  fiber-optic  illuminators  suitable  for  use  with  dis¬ 
secting  microscopes,  more  recently  sources  suitable  for  use  in 
high-performance  epifluorescence  microscopy  have  been  offered. 
Besides  utilizing  optimized  arcs  in  elliptical  collector  mirrors  and 
high-speed  filters  wheels  for  rapidly  shifting  the  output  wave¬ 
length,  they  also  provide  fiber-optic  light  scrambling. 


TABLE  6.3.  Companies  Making  Stand-Alone  Microscopy  Light  Sources 


Company 

Model 

Type  of  Source 

Rapid 

X  Shifting? 

Wavelength 
range,  nm 

Interfaces  to  .  .  . 

URL 

API 

Uniform  light  source 

HBOlOO/2 

Yes 

320-700 

Sold  only  w/system 

http://www.api.com/ 

Dolan-Jenner 

Cold  light  sources 

Arcs,  e.g.,  metal 
halide 

No 

UV-VIS 

Optical  fiber 

http://www.dolanjenner.com 

EXFO 

X-Cite  120PC 

120W  metal  halide 

No 

UV-VIS 

Custom  coupling  optics 

http  ://www.exfolifesciences  .com 

Illumination 

Technologies 

Cold  light  sources 

Halogen 

No 

VIS 

Optical  fiber 

http://www.illuminationtech.com 

Schott 

Cold  light  sources 

Arcs,  e.g.,  metal 
halide 

No 

UV-VIS 

Optical  fiber 

http://www.schott.com 

StockerYale 

Cold  light  sources 

Halogen 

No 

VIS 

Optical  fiber 

http ://  www.  stockery  ale .  com 

Sutter  Inst 

Lambda  DG4 

175WXBO  based 

Yes,  1.2  ms 

300-700 

Optical  fiber  — >  flexible 

http ://  www.  sutter.  com/ 

Till  Photonics 

Polycrome  V 

150W  XBO  based 

Yes,  400nm/ms 

320-680 

Optical  fiber  — »  flexible 

http://www.till-photonics.de 

Technical 

Video  Ltd 

Fiber  optic  light 
scrambler 

HBOlOOW/2 

No 

320-800 

Optical  fiber  — >  All 

http://www.technicalvideo.com 
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FIGURE  6.24.  Spectral  distribution  of  a  200 W  EmArc,  mercury-halide  arc 
source  and  a  75  W  xenon  arc  lamp  {dashed  line)  200  to  2400  nm.  (Spectra 
kindly  provided  by  Oriel/Newport,  Irvine,  CA.) 


In  a  way,  the  really  good  news  is  that  there  is  a  great  demand 
for  high  brightness  light  sources  in  areas  of  the  economy  that  have 
considerably  more  market  power  than  microscopy.  Perusal  of  the 
Internet  will  show  a  wide  variety  of  ingenious  sources  responding 
to  both  the  general  need  to  get  more  light  for  fewer  watts  of  elec¬ 
tric  power  and  the  more  specific  need  to  keep  the  source  size  small. 
The  digital  projectors  used  at  scientific  meetings  are  just  the  early 
scouts  of  a  digital  revolution  about  to  overtake  the  movie  indus¬ 
try.  These  projectors  will  need  bright  sources.  Figure  6.24  shows 
the  performance  of  one  of  the  early  candidates,  a  200  W  enhanced 
metal  arc  source  that  uses  a  mixture  of  gasses  to  produce  an  order 
of  magnitude  increase  in  light  output  in  the  visible,  compared  to  a 
short  arc  75  W  Xe  arc  (USHIO,  Inc.  Cypress,  CA).  Although,  at 
over  US$2000  each,  it  is  unlikely  that  many  microscopists  will  use 
such  sources  immediately,  there  is  every  reason  to  think  that  even¬ 
tually  prices  will  come  down. 
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INTRODUCTION 

No  other  component  of  the  microscope  is  as  instrumental  in  deter¬ 
mining  the  information  content  of  an  image  as  the  objective.  The 
resolved  detail,  the  contrast  at  which  this  detail  is  presented,  the 
depth  through  the  object  from  which  useful  information  can  be 
derived,  and  the  diameter  of  the  useful  field  are  all  limited  by  the 
performance  of  the  objective.  All  other  imaging  components,  such 
as  relay  optics,  Telan  systems,  tube  lenses  and  eyepieces,  or  pro- 
jectives  may  have  some  corrective  function  but  otherwise  serve 
only  to  present  the  image  generated  by  the  objective  to  the  detec¬ 
tor  in  such  a  way  that  most  of  its  information  content  can  be 
recorded  without  degradation. 

While  this  is  true  for  any  conventional  microscope,  it  is  par¬ 
ticularly  true  for  confocal  scanning,  where  the  objective  becomes 
the  condenser  as  well  and  needs  to  combine  a  high  degree  of 
optical  correction  with  good  throughput  and  a  minimum  of  inter¬ 
nal  stray  light  or  photon  noise  generation. 

In  general,  the  demands  on  the  performance  of  the  objective 
for  confocal  scanning  are  identical  to  the  needs  for  demanding 
video  microscopy,  photomicroscopy,  densitometry,  photometry, 
spectrophotometry,  and  morphometry.  However,  this  does  not 
mean  that  confocal  microscopy  will  not  eventually  call  for  special 
new  lenses  in  which  certain  corrections  may  be  sacrificed  to 
enhance  specific  capabilities.  In  biological  applications  involving 
living  cells,  high  photon  efficiency  is  so  important  that  it  may  be 
worth  accepting  a  reduction  in  field  size  and  chromatic  correction 
in  order  to  achieve  the  highest  possible  transmittance  at  a  reason¬ 
able  working  distance  by  using  a  minimum  number  of  lens  ele¬ 
ments.  Another  problem  is  the  loss  of  correction  for  spherical 
aberration  as  an  oil-immersion  lens  is  focused  deep  into  an 
aqueous  specimen.  High  numerical  aperture  (NA)  water- 
immersion  objectives  with  correction  collars  for  coverslip 
thickness  variations,  refractive  index  variations  in  the  medium,  or 
temperature-dependent  index  changes  have  become  the  lenses  of 
choice  for  live-cell  studies.  Because  of  the  difficulty  manually 
rotating  the  correction  collar  while  actually  observing  living  spec¬ 
imens,  correcting  spherical  aberration  may  require  the  addition  of 
either  deformable  mirror  correctors  or  motor-driven  correction 
optics  mounted  later  in  the  optical  path  (see  also  Chapter  20,  this 
volume). 

Because  the  critical  demands  of  light  microscopy  and  confo¬ 
cal  scanning  microscopy  have  increasingly  forced  the  performance 
of  objectives  to  approach  their  theoretical  limits,  a  brief  refresher 
on  aberrations,  design  concepts,  materials,  etc.,  may  be  in  order. 
An  overview  of  optical  aberrations  in  refractive  systems  —  both 


inherent  and  induced  by  improper  use  of  the  microscope  —  and 
the  basic  performance  characteristics  of  the  different  generic  types 
of  objectives  will  be  presented. 

The  basic  design  concepts  of  microscope  optics  —  finite 
versus  infinite  image  distance,  compensating  versus  fully  corrected 
systems  —  need  to  be  understood  to  properly  match  optical 
components  and  their  properties  for  specific  applications.  Optical 
materials,  cements,  and  anti-reflection  coatings  all  influence  the 
performance  of  the  objective.  Immersion  liquids,  the  coverslip,  and 
the  mounting  medium  are  all  part  of  the  optical  train  and  can 
strongly  affect  the  quality  of  the  final  image.  We  will  try  to  put  all 
of  this  into  qualitative  perspective,  particularly  as  it  pertains  to 
confocal  scanning.  Additional  information  on  the  optics  for 
microscopy  can  be  found  in  a  review  paper  by  Inoue  and  Olden- 
bourg  (1995)  and  in  a  review  article  on  objective  lens  design  by 
Shimizu  and  Takenaka  (1994). 

A  detailed,  quantitative  comparison  of  the  performance  of  dif¬ 
ferent  microscope  objectives  must  be  based  on  accepted  criteria 
and  precisely  defined  testing  methods.  Unfortunately,  although  the 
major  microscope  makers  have  developed  their  own  proprietary 
methods,  no  independent,  fully  “objective”  test  procedure  is 
readily  accessible  to  users  to  quantify  all  performance  data  of  an 
objective.  (Juskaitis  describes  a  sophisticated  means  for  inter¬ 
ferometric  testing  of  objectives  in  Chapter  11,  this  volume.) 

How,  then,  should  the  user  of  a  confocal  microscope  judge  the 
performance  of  an  objective?  Measurement  and  analysis  of  the 
real,  not  just  theoretical,  point  spread  function  (PSF)  of  an  objec¬ 
tive,  or  better,  of  the  complete  imaging  system,  is  critical  for  three- 
dimensional  (3D)  deconvolution.  Observations  of  subresolution 
pinholes  in  an  evaporated  silver  or  aluminum  coating  are  adequate 
to  judge  spherical  aberration,  astigmatism,  coma,  and  flatness  in 
transmitted  light  but  do  not  work  well  in  the  epi-mode.  Fluores¬ 
cent  beads  in  the  0. 1  pm  range  are  suitable  replacements,  but  the 
fluorescence  fades.  Diatoms  have  long  been  a  standard  because  of 
their  precise  and  regular  spacing  and  because  they  can  be  viewed 
in  the  backscattered  (BSL,  sometimes  referred  to  as  reflected)  light 
mode  or  after  embedment  in  fluorescent  dye  (Chapter  35,  this 
volume).  For  example,  how  do  we  determine,  at  least  qualitatively, 
how  an  image  is  degraded  —  by  focusing  deep  into  a  specimen  or 
by  pairing  optical  components  that  are  not  matched?  These  are  all 
challenges  that  are  not  yet  fully  resolved.  They  point  to  a  need  for 
detailed  testing  procedures  that  cover  all  aspects  of  the  optics  from 
source  to  detector. 

Still,  with  our  ability  today  to  ray-trace  lenses  for  their  geo¬ 
metrical  optical  performance  and  to  calculate  wave-front  aberra¬ 
tions,  PSF,  and  intensity  ratios  through  the  Airy  disk,  most 
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objectives  now  offered  are  close  to  diffraction  limited,  at  least  in 
the  center  of  the  image  field.  However,  field  size  and  optical  per¬ 
formance  at  the  periphery  of  the  field  are  also  especially  important 
in  beam-scanning  confocal  microscopy. 

Long-term  mechanical,  thermal,  and  chemical  stability  of 
objectives  used  with  lasers  are  a  function  of  manufacturing  toler¬ 
ances  and  materials  chosen.  Submicron  tolerances  for  the  centra- 
tion  and  spacing  of  lens  elements  in  sophisticated,  high-power 
objectives  call  for  careful,  gentle  treatment  by  the  user.  A  minor 
mechanical  shock  may  generate  enough  stress  on  a  lens  element 
to  seriously  reduce  the  objective’s  performance  in  polarized  light, 
in  differential  interference  contrast  (DIC),  or  in  critical  confocal 
scanning. 

ABERRATIONS  OF  REFRACTIVE  SYSTEMS 

The  ideal  “diffraction-limited”  objective  generates  a  3D  PSF  from 
an  infinitely  small  object  point.  A  cross-section  perpendicular  to 
the  optical  axis  through  the  center  of  the  PSF  is  the  Airy  disk,  as 
shown  in  Figure  7.1(A).  The  diameter  </Airy  of  the  first  dark  ring, 
generated  by  destructively  interfering,  diffracted  wavefronts  [Fig. 
7.1(B)]  is 

_  1,2231 
Airy  r)  sina 

where  A  is  the  wavelength  of  light,  T|  is  the  refractive  index  of 
medium  between  the  object  and  the  objective,  and  a  is  the  half¬ 
angle  of  the  collected  rays  from  the  object  point.  The  Rayleigh  (or 
Abbe)  criterion  sets  the  limit  for  the  smallest  resolvable  distance 
d  between  two  points  at  one-half  this  diameter  or, 


_  1.22  A, 

'Aky  ~  2  NA 

or  the  radius  of  the  Airy  disk,  where  T|  sina  is  the  NA.  This  point- 
to-point  resolution  for  a  given  objective  in  turn  determines  the 
magnification  required  to  enable  any  given  detector  to  record  all 
the  resolvable  details.  Taking  the  spacing  of  the  rods  in  the  human 
retina  as  setting  the  limit,  the  total  required  magnification  for  visual 
observation  —  the  so-called  useful  magnification  —  becomes 
about  500  to  1000  times  the  NA  of  the  objective,  while  for  digital 
recording  we  enlarge  the  Airy  disk  to  4  to  5  times  the  pixel  dimen¬ 
sion  of  the  detector  (Nyquist  sampling,  see  Chapter  4,  this  volume). 

Defocusing 

Defocusing  will  change  the  size  and  intensity  distribution  of  the 
unit  image  point  (Fig.  7.2).  Because  defocusing  and  depth  of  field 
are  closely  related,  let  us  take  a  look  at  the  3D  PSF  or  “image 
body”  of  the  “diffraction  limited”  objective  (Chapter  1,  this 
volume). 

Figure  7.3  again  shows  a  cross-section  perpendicular  to  the 
optical  axis  in  the  optimally  focused  image/object  plane,  which  is, 
of  course,  the  intensity  distribution  of  the  Airy  disk,  while  Figure 
7.4  represents  a  section  along  the  optical  axis  and  its  intensity  dis¬ 
tribution  (actually,  the  log  of  the  intensity,  to  make  it  more  visible). 
Defocusing  results  in  alternating  bright  and  dark  spots  along  the 
axis  of  the  Airy  disk  (Fig.  7.2).  The  extension  of  the  central  bright 
body  along  the  axis  is  4A/(NA)2,  but  we  can  detect  a  change  in  the 
image  with  a  defocus  of  only  ±A/(NA)2  (the  Rayleigh/ Abbe  unit 
in  the  z-direction).  We  call  this  the  wave-optical  depth  of  field 
(Figs.  7.3  and  7.4). 


FIGURE  7.1 .  (A)  Airy  disk  and  its  intensity  distri¬ 
bution.  (B)  Generation  diagram  and  profile  of  the 
Airy  disk  or  unit  image. 
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FIGURE  7.2.  Changes  in  intensity  distribution  with  focus  changes. 


Deviations  from  the  “diffraction-limited”  point  image  caused 
by  lens  aberrations  can  be  grouped  into  either  wavelength- 
independent  (monochromatic)  or  chromatic  aberrations. 

Monochromatic  Aberrations 
Spherical  Aberration 

This  axial  aberration  is  generated  by  non-spherical  wavefronts  pro¬ 
duced  by  the  objective  itself  or  by  improper  use  of  the  objective. 


FIGURES  7.3  and  7.4.  Horizontal  (focal  plane)  and  vertical  (optical  axis) 
cross-section  through  unit  image  body. 


in  particular,  failure  to  use  the  correct  coverglass  thickness  or 
maintain  the  designated  tube  length  or  the  presence  of  substances 
between  the  objective  and  the  focus  plane  having  the  wrong  1) 
(Chapter  20,  this  volume ).  Spherical  aberration  has  the  effect  that 
paraxial  rays  have  a  different  focal  length  from  peripheral  rays, 
and  a  blurring  of  the  image  body  produces  an  asymmetrical  inten¬ 
sity  change  when  defocusing  by  ±AZ  (Fig.  7.5  and  Fig.  20.3,  this 
volume).  Spherically  ground  and  polished  lenses  have  a  shorter 
focal  distance  for  peripheral  rays  than  for  paraxial  rays. 

Spherical  aberration  can  be  optimally  corrected  only  for  accu¬ 
rately  specified  object  and  image  distances.  It  can,  therefore,  be 
easily  induced  by  improper  tube  length  caused  by  introduction  of 
optical  elements  into  the  converging  beam  path  of  finitely 
designed  systems  or  by  the  use  of  improper  “windows,”  such  as 
non-standard  coverslips  or  immersion  oil  of  non-specihed  refrac¬ 
tive  index  between  object  and  objective.  Figure  7.6  shows  the 
changes  in  size  and  intensity  distribution  through  the  image  point 
with  increasing  penetration  into  an  object  in  watery  medium  with 
a  planapochromat  63x,  NA  1.4  oil.  The  effect  of  this  induced 
spherical  aberration  on  the  image  point  needs  to  be  considered  and 
either  corrected  for  or  at  least  understood  before  confocal 
microscopy  can  be  applied  optimally  to  3D  reconstruction  (see 
Chapter  24,  this  volume).  More  specifically,  let  us  consider  how 
the  image  quality  of  another  objective  designed  for  diffraction- 
limited  performance  (the  Plan-Neofluar  40x,  NA  1.3  oil)  deterio¬ 
rates  when  focusing  10  pm  into  an  aqueous  medium. 

At  the  water-coverslip  interface,  spherical  aberration  is  gen¬ 
erated,  which  shifts  the  focus  of  peripheral  rays  above  the  optimal 
focus  of  paraxial  rays  [Fig.  7.7(A,B)]. 

Figure  7.8  illustrates  the  change  in  intensity  of  a  fluorescing 
object  detected  with  a  pinhole  equal  to  the  diameter  of  the  Airy 
disk  (1  Airy  Unit)  at  543  nm  as  a  function  of  different  penetration 
depths  or  focus  changes,  indicated  in  both  micrometers  and 
Rayleigh  units  (RE)  into  water.  For  depths  of  0  pm,  5  pm,  10  pm, 
and  20  pm,  the  signal  intensity  at  the  pinhole  detector  drops  dra¬ 
matically  along  with  deteriorating  depth  discrimination.  Minor 
changes  in  refractive  index  are  generated  by  changes  in  salt  con¬ 
centration,  temperature,  and  differences  in  molecular  structure.  All 
these  can  induce  positive  or  negative  spherical  aberration. 

With  increasing  NA,  changes  in  the  thickness  or  the  refractive 
index  of  the  “window”  between  the  object  and  the  objective 
becomes  critical,  particularly  with  “dry”  objectives.  In  the  low- 
power,  low-NA  objective  with  relatively  higher  AAimageside  where 
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FIGURE  7.5.  Nonsymmetrical  change  in  the  intensity  distribu¬ 
tion  with  focus  change  above  (left)  and  below  (right)  best  focus 
in  a  system  limited  by  spherical  aberration. 


NA 

_  objectside 

imageside  Magnification 

small  changes  in  tube  length  quickly  lead  to  inferior  images. 

While  the  spherical  aberration  can  be  corrected  to  less-than- 
perceptible  limits  for  visual  observation  for  all  types  of  objectives, 
this  holds  true  only  if  all  optical  specifications  for  a  given  lens  are 
fulfilled.  For  oil-immersion  lenses  with  high  NA,  this  usually 
means  using  a  coverslip  of  0.17  mm  thickness,  and  Tj  =  1.518  at 
546 nm  and  589 nm  and  an  immersion  oil  with  T|  =  1.5180  +  0.0004 
at  546 nm  or  ij  =  1.515  at  589  nm,  a  condition  complicated  by  the 
fact  that,  in  all  materials,  T|  is  a  function  of  A  and  temperature.  If 
the  exact  properties  of  the  coverslip  and  the  oil  are  specified,  then 
the  manufacturer  can  correct  spherical  aberration  for  several  values 
of  X:  Zeiss  achromats  are  corrected  for  2  As,  neofluars  for  3  A.s,  and 
planapochromats  for  4  As. 


FIGURE  7.6.  Change  in  the  intensity  distribution  with  increasing  penetration 
into  a  watery  medium  with  a  planapochromat.  Penetration  depth  from  0  to 
4  pm.  Note:  In  these  plots,  peak  intensity  has  been  normalized.  In  practice, 
it  should  decrease  dramatically  as  the  base  of  the  intensity  plot  widens. 


With  high  NAdry  or  water-immersion  objectives,  the  thickness 
of  the  coverslip,  standardized  throughout  the  industry  as  0.17  mm 
(#  1.5),  is  particularly  important.  Figure  7.9  shows  the  changes  in 
the  half-width  of  the  intensity  distribution  curve  with  changes  in 
coverslip  thickness.  With  tolerances  of  ±10pm  for  top-quality  cov- 
erslips,  the  half-width  can  change  by  more  than  a  factor  of  2.  With 
increasing  NA  00.5),  particularly  with  dry  and  water-immersion 
lenses,  selection  of  coverslips  for  correct  thickness  is  particularly 
important.  Even  oil-immersion  lenses  such  as  the  planapochromat 


Oil 


Oil 


I 


Focus  with  full 
spherical  correction 


FIGURE  7.7.  Ray  diagram  of  an  NA  =1.3  oil-immersion  objective,  focusing 
into  oil  (left)  versus  focus  10|im  in  water  (right). 
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FIGURE  7.8.  Pinhole  energy  measured  through  a  40x,  NA  1.3  oil  objective;  as  function  of  defocus  and  depth  of  penetration  (Z)  into  water.  As  the  thickness  of 
the  water  layer  increases  from  0  and  50pm,  the  resolution  is  reduced  by  about  3x  and  the  intensity  by  a  similar  amount. 


63x,  NA  1.4  perform  optimally  only  with  a  coverslip  thickness  of 
0.17  mm  (Chapter  8,  this  volume).  Fortunately,  electro-mechanical 
micrometers  capable  of  reading  coverslip  thickness  to  an  accuracy 
of  ±1  pm  are  now  available  at  relatively  low  cost. 

The  spherical  aberration  induced  by  non-specified  coverslip 
thickness  leads  to  loss  of  energy  at  the  pinhole,  reduced  depth  dis¬ 
crimination,  and  an  axial  shift  of  the  best  focus.  This  is  shown  in 
Figure  7.10  for  a  diffraction-limited,  water-immersion  40x,  NA  1.2 
objective. 

These  examples  underline  the  importance  of  maintaining  the 
specified  and  computed  optimal  conditions  for  microscope  objec¬ 
tives  in  order  to  achieve  the  full  benefits  of  confocal  microscopy. 


FWHM 


glass  plate  thickness  [pm] 


FIGURE  7.9.  Changes  in  the  half-width  of  the  intensity  distribution  with 
changing  coverslip  thickness.  Plan-Neofluar  63x,  NA  1.2  water. 


On  high-NA,  dry  objectives  or  on  multi-immersion  objectives, 
eliminating  induced  spherical  aberration  requires  that  the  correc¬ 
tion  collar  be  set  exactly  (+2  pm  of  glass-replaced-by- water  at  NA 
1.2,  see  Figs.  7.9  and  20.3). 

From  the  above,  it  becomes  obvious  that  a  water-immersion 
objective  is  the  best  choice  to  minimize  induced  spherical  aberra¬ 
tion  when  penetrating  aqueous  media.  Unfortunately,  the  collec¬ 
tion  of  3D  data  requires  moving  the  specimen  with  respect  to  the 
lens.  If  this  is  not  to  produce  motion  of  the  specimen,  a  coverslip 
must  be  used.  Experiments  have  shown,  however,  that  precise  cor¬ 
rection  for  coverslip  thickness  or  even  the  use  of  “Cytop”  (a  new 
coverslip  material  with  a  refractive  index  of  1 .34,  which  is  almost 
that  of  water,  developed  by  Olympus)  will  not  suffice.  The  refrac¬ 
tive  differences  in  physiological  media  and  in  biological  materials 
still  require  systems  that  permit  adjustable  correction  for  the  spher¬ 
ical  aberration  induced  by  the  specimen  itself  (see  Figs.  2. 3-2. 5, 
this  volume ). 

With  the  introduction  of  high-NA  water-immersion  objectives 
designed  for  coverglass  use,  some  with  working  distances  of 
0.24  mm,  microscopy  on  live  cells  or  tissue  has  been  greatly  facil¬ 
itated.  Their  correction  collars  not  only  allow  one  to  compensate 
for  variations  in  the  coverglass  thickness  (after  it  has  been  mea¬ 
sured  with  a  micrometer  or  via  the  confocal  software)  but  also 
allows  one  to  compensate  for  refractive  index  changes  due  to  tem¬ 
perature  or  concentration  changes  in  the  medium.  Although  these 
water-immersion  objectives  have  nominally  lower  NAs  than  com¬ 
parable  oil  lenses,  keep  in  mind  that  the  effective  NA  of  the  oil 
objective,  when  looking  through  water,  is  dictated  by  the  refrac¬ 
tive  index  of  the  water  or  of  the  medium  of  lowest  refractive  index 
between  object  and  objective.  For  most  aqueous  specimens  the 
water-immersion  lens  is  clearly  the  preferred  choice. 

Not  all  experiments  permit  optimal  imaging  conditions,  and  the 
question  arises  whether  objectives  with  lower  NA  or  with  a  built-in 
iris  would  not  sometimes  yield  better  results.  Figure  7.11  again  plots 
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FIGURE  7.10.  Pinhole  energy  as  a  function  of  defocus  and  coverslip  thickness  (d). 
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FIGURE  7.1 1 .  Pinhole  energy  as  a  function  of  defocus  and  NA:  40x,  NA  1.3  objective  with  iris  (variable  aperture)  penetrating  10pm  into  water. 
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FIGURE  7.12.  Intensity  distribution  in  unit  image  with  coma 
present:  slight  (left);  serious  (right). 


detected  pinhole  energy  as  a  function  of  focus  change  and  NA  for 
a  40x  oil-immersion  objective  with  variable  NA.  At  the  full  NA  of 
1.3,  penetrating  10pm  into  a  watery  medium  reduces  pinhole 
energy  to  <50%,  while  reducing  the  NA  to  0.92  actually  puts  more 
light  through  the  pinhole  if  one  compensates  for  the  reduction  in 
throughput  by  the  lower  aperture  by  increased  light  input.  By  reduc¬ 
ing  the  effect  of  spherical  aberration,  a  larger  fraction  of  the  energy 
reaches  the  pinhole.  However,  for  NA  <0.5,  the  Airy  disk  becomes 
significantly  larger  than  the  pinhole  and  the  detected  energy  drops 
again  (see  Fig.  7.11).  Here  we  also  need  to  mention  a  very  useful 
device,  called  InFocus,  from  Infinity  PhotoOptical  (Boulder,  CO), 
which,  by  varying  the  effective  tube  length  in  a  way  that  does  not 
change  the  magnification,  permits  external  compensation  of  spher¬ 
ical  aberration  to  some  degree.  The  system  mounts  between  camera 
port  and  the  camera  (or  confocal  scan  head)  and  can  be  motor  driven 
by  a  computer  that  also  moves  the  stage  to  compensate  for  the  small 
focus  changes  that  occur  when  the  corrector  is  adjusted  (“SAC,” 
Intelligent  Imaging  Innovations,  Denver,  CO). 

Coma 

For  object  points  away  from  the  optical  axis  coma,  a  streaking, 
radial  distortion  of  the  image  point  is  generated  (Fig.  7.12).  Ful¬ 
filling  Abbe’s  sine  condition, 

r'  i)'  sin(3'  =  r  r|  sin(3 

where  r  is  the  distance  from  the  optical  axis;  1)  is  the  refractive 
index;  and  (3  is  the  viewing  angle,  all  in  image  space  (')  or  object 
space  (respectively),  eliminates  coma  and  produces  an  aplanatic 
system.  Unfortunately,  all  factors  with  the  potential  to  increase 
spherical  aberration  are  also  especially  critical  to  coma. 

As  coma  is  only  evident  off-axis,  it  is  not  important  in  stage¬ 
scanning  confocal  microscopes.  However,  it  can  be  important  in 
beam-scanning  systems,  or  when  one  scans  off-axis  (even  at  high 
zoom)  to  avoid  on-axis  specular  reflections  sometimes  visible  in 
the  BSL  mode. 

Astigmatism 

For  off-axis  points,  two  orthogonal  cross-sections  through  the 
imaging  wavefront  (tangential  and  sagittal)  can  have  different 
focal  distances  or  radii.  When  this  is  the  case  and  a  perfectly  sym¬ 
metrical  image  point  in  the  center  of  the  field  is  moved  off-axis,  it 


becomes  either  radially  or  tangentially  elongated,  depending  on  the 
focus.  The  intensity  ratio  of  the  unit  image  will  diminish,  and  def¬ 
inition,  detail,  and  contrast  are  lost  with  increasing  distance  from 
the  center. 

Figure  7.13  shows  the  intensity  distribution  through  one 
section  of  an  astigmatic  image  point.  A  four-lobed  Airy  disk  results 
when  the  focus  is  set  at  the  compromise  position  between  the  radial 
and  tangential  extremes  [Fig.  7.13(B)], 

Poor  lens  centration  in  the  objective  or  poor  alignment 
between  the  objective,  the  intermediate  optics,  and  the  eyepiece 
increases  both  astigmatism  and  coma.  Wedge-error  in  poor  quality 
filters  or  dichroics  can  have  the  same  effect. 

Flatness  of  Field 

A  simple  lens  focuses  image  points  from  an  extended  flat  object 
onto  a  spherical  surface  shaped  like  a  shallow  bowl  or  dish.  Central 
and  peripheral  zones  are  not  simultaneously  in  sharp  focus.  Prior 
to  the  advent  of  flat-field  objectives  by  Zeiss  in  the  late  1930s, 
the  “usable”  field  at  the  intermediate  image  plane  was  only  10  to 
12  mm.  With  present-day  flat  field  or  “Plan”  objectives,  ocular 
fields  of  18,  20,  and  25  mm  exhibit  sharp  detail  from  center  to  edge; 
however,  this  improvement  has  been  obtained  at  the  expense  of 
increases  in  complexity  and  cost  and  reduced  transmission.  In  fact, 
it  became  practical  only  after  the  introduction  of  anti-reflection 
coatings. 

In  Figure  7.14,  the  field  curvature  for  an  achromat  and  a  plan- 
apochromat  are  compared.  Also  shown  is  the  astigmatism  of  the  two 
orthogonal  image  spheres.  A  represents  the  depth-of-field  unit 
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It  needs  to  be  mentioned  here  that  the  term  Plan  or  “F”  for  flat 
field  is  no  guarantee  for  a  perfectly  flat  image  with  no  astigma¬ 
tism.  No  standards  have  been  established.  Also,  the  flatness  in  the 
final  image  may  be  affected  by  the  correction  of  the  eyepiece  or 
other  intermediate  optics.  The  diameter  of  the  field  of  view1  over 
which  all  aberrations  have  been  corrected  and  the  field  is  flat  is 
listed  for  a  number  of  Zeiss  objectives  in  Table  7.1. 


1  In  the  intermediate  image  plane. 
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FIGURE  7.1 3.  Intensity  distribution  in  unit  image  with  astig¬ 
matism:  serious  (left);  slight  (right). 


Ill  confocal  scanning  through  thick  materials,  the  dish-shaped 
section  obtained  with  a  non-plan  objective  may  be  of  little  conse¬ 
quence  to  the  biologist,  as  long  as  there  is  no  astigmatism  or  coma 
and  one  remains  mindful  of  the  distortion  this  aberration  produces 
in  3D  data  sets.  This  is  because  the  exact  shape  of  most  biologi¬ 
cal  specimens  is  often  more  likely  to  be  distorted  by  specimen 
preparation  procedures  than  by  optical  problems,  especially 
because,  when  the  normal  512  X  512  image  is  scanned  at  Nyquist 
Zoom  magnification,  it  usually  covers  only  the  small  part  of  the 
field-of-view  nearest  the  axis.  The  off-axis  increase  in  astigmatism, 
coma,  and  field  curvature  is  a  good  reason  to  use  the  "panning” 
controls,  that  shift  the  imaged  area  away  from  the  optical  axis,  with 
caution.  However,  in  materials  science  and  for  many  critical  appli¬ 
cations,  such  as  high-resolution  imaging  in  semiconductor  inspec¬ 
tion.  flatness  of  field  is  essential. 

Distortion 

Nonlinear  magnification  from  center-to-edge  of  field  puts  the 
actual,  off-axis  image  point  either  closer  (barrel)  or  farther  away 
(pincushion)  from  the  axis  than  the  true  image  point.  Barrel  or  pin¬ 
cushion  distortion  results,  and  the  true  geometry  of  an  object  is  no 
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FIGURE  7.14.  Flatness  of  field  and  astigmatism  for  (1)  achromat  and  (2) 
planapochromat  as  a  function  of  position  in  the  intermediate  image  plane. 


longer  maintained  in  the  image.  Although  less  critical  in  biomed¬ 
icine  than  in  materials  science,  distortion  is  reduced  to  <1.5%  of 
the  radial  distance  from  the  axis  in  most  objectives.  It  can  be 
checked  by  imaging  crossed  grating  lines,  such  as  those  found  in 
hemocytometers,  in  the  widefield  mode.  Ideally,  the  lines  should 
be  perfectly  straight  over  the  entire  image  field.  A  similar 
specimen  can  be  useful  to  measure  scan  linearity  in  confocal  laser¬ 
scanning  microscopes,  however,  in  this  case  distortion  from 
nonlinearity  of  the  scan  is  likely  to  be  larger  than  that  produced  by 
the  objective. 

Chromatic  Aberrations 

Wavelength-dependent  aberrations  are  caused  by  ( 1 )  the  fact  that 
the  t|  of  every  optical  glass  varies  with  X,  a  factor  called  disper¬ 
sion;  and  (2)  the  fact  that  X  affects  all  the  dimensions  of  the  unit 
image  point.  This  latter  aberration,  called  diffraction-caused 
spreading,  is  usually  barely  noticeable  in  the  center  of  the  Airy  disk 
but  is  more  noticeable  at  the  edges  of  the  first  fringe,  where  it  influ¬ 
ences  the  resolution.  We  will  discuss  this  later. 

Longitudinal  Chromatic  Aberration 

This  aberration  is  the  result  of  changes  in  lens  focal  length,  (Af), 
with  changing  X.  In  general,  the  image  plane  is  in  sharp  focus  for 
only  one  wavelength  or  for  a  narrow  wave  band.  For  other  wave¬ 
lengths,  the  image  plane  is  slightly  defocused.  Figure  7.15  com- 


TABLE  7.1.  Diameter  of  Corrected  Image  for  Some  Zeiss 
Plan  Objectives 


Flat  Field  of  View 

Objective  Conventional 

Conventional 

Confocal 

Planapo  40x/1.2  water  corr. 

25 

12 

Achroplan  20X/0.5  water 

24 

25 

Achroplan  lOOx/l.O  water 

18 

10 

With  best  focus  on  the  center,  the  “image  disk”  will  not  exceed  the  depth  of  field 
within  the  specified  field;  diameter  in  millimeters. 


Objective  Lenses  for  Confocal  Microscopy  •  Chapter  7  153 


FIGURE  7.15.  Longitudinal  chromatic  correction  of  (1)  achromat  and  (2) 
apochromat. 


pares  the  longitudinal  chromatic  correction  of  an  achromat  with 
that  of  an  apochromat.  In  order  to  judge  the  influence  of  Af  on  the 
image  quality  for  a  given  lens,  the  spectral  emission  of  the  source 
as  well  as  the  spectral  sensitivity  of  the  detector  need  to  be  con¬ 
sidered  because  both  determine  the  effective  brightness  in  the 
center  of  the  disk  (i.e.,  the  energy  that  passes  through  the  pinhole). 

The  peak  sensitivity  for  the  eye  is  at  550  nm,  with  a  spectral 
range  of  -400  to  ~650nm.  Figure  7.16  shows  the  spectral  emis¬ 
sion  of  a  typical  light  source  and  the  spectral  sensitivity  of  the  eye. 
If  we  normalize  the  response  at  550nm  to  100%,  the  sensitivity  of 
the  eye  to  an  image  of  a  sample  illuminated  by  tungsten  light  is 
-10%  at  480nm  and  -30%  at  640nm.  In  confocal  scanning,  the 
emission  spectrum  of  the  source  and  the  spectral  sensitivity  of  the 
detector  need  to  be  considered  along  with  the  emission  peak  and 
bandwidth  of  the  specific  fluorophore. 

Chromatic  aberrations  are  both  produced  and  corrected  by  uti¬ 
lizing  the  different  dispersions  of  the  glasses  used.  Glasses  of 
“normal”  dispersion  have  an  almost  linear  decrease  in  Tj  with 


FIGURE  7.16.  (1)  Spectral  sensitivity  of  the  eye.  (2)  Spectral  emission  of  a 
tungsten  source. 


increasing  X  and  are  used  for  achromats.  Only  two  wavelengths 
have  the  same  focus,  and  the  remaining  “secondary  spectrum”  pro¬ 
duces  greenish  or  purple  fringes  on  images  of  sharp  edges. 

For  objectives  with  better  chromatic  correction,  glasses  of 
“abnormal”  partial  dispersion  are  needed.  Here  the  refractive  index 
changes  more  rapidly  with  wavelength  in  either  the  blue  or  red 
region. 

Abbe  used  the  crystal  fluorite  (CaF2)  to  reduce  the  secondary 
spectrum,  and  more  recently,  glasses  have  become  available  with 
similar  dispersion  properties.  By  replacing  any  single  lens  with  a 
doublet  containing  both  a  positive  and  a  negative  element,  with 
each  element  having  a  different  dispersion,  it  is  possible  to  make 
the  chromatic  effects  cancel  out  at  two  or  more  wavelengths.  This 
has  resulted  in  the  high  degree  of  chromatic  correction  for  an 
apochromat  (Fig.  7.15),  where  up  to  four  or  more  wavelengths  can 
have  the  same  image  location  [see  Fig.  15(A)]. 

With  apochromat  and  semiapochromat  or  fluorite  lenses, 
the  “diffraction-caused  spreading”  of  the  intensity  distribution, 
referred  to  above,  can  also  be  virtually  eliminated,  as  Figure  7.17 
illustrates.  An  achromat  still  has  substantial  intensity  in  the  first 
fringe,  while  the  apochromat  approaches  the  theoretical  resolution 


FIGURE  7.17.  Intensity  distribution  in  the  image  of  a  point  object  illuminated  with  white  light  for  an  achromat,  plan-neofluar,  and  planapochromat. 
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FIGURE  7.18.  Pinhole  energy  as  a  function  of  defocus  and  wavelength:  planapochromat  40x,  NA  1.2  water,  d  =  0.17. 


limit  where  the  longitudinal  chromatic  aberration  is  <A/(NA)2,  the 
wave-optical  depth  of  field. 

Because  the  apochromat  requires  elements  of  abnormal  dis¬ 
persion,  their  characteristics  may  not  be  ideal  for  some  specific 
applications,  such  as  fluorescence  excitation  in  the  near  ultravio¬ 
let  (UV)  or  polarizing  microscopy.  For  this  reason,  a  fluorite  objec¬ 
tive  is  often  more  suitable,  and  Figure  7.17  illustrates  how  close  it 
comes  to  the  performance  of  the  apochromat. 

For  confocal  scanning  fluorescence  images,  it  is  important  that 
the  correction  be  as  similar  as  possible  at  the  excitation  and  emis¬ 
sion  wavelengths  in  order  to  produce  the  best  results  in  terms  of 
the  signal  transmission  and  resolution. 

In  UV-excited  confocal  fluorescence  with  emission  in  the 
visible  range  of  the  spectrum  (e.g.,  351nm  and  514nm),  even 
normal  planapochromats  will  fail  to  assure  precise  parfocality. 
This  will  result  in  energy  loss  at  the  pinhole  unless  either  one  of 
the  new  “blue”  objectives  are  employed2  or  the  UV  laser  is  spe¬ 
cially  adapted  with  its  own  optical  interface  to  compensate  for  the 
objective’s  shortcomings.  To  accomplish  the  latter,  the  beam 
expander  of  the  UV  laser  is  adjusted  to  precisely  parfocalize  the 
excitation  spot  in  the  specimen  with  the  image  of  the  detection 
pinhole. 

Some  of  the  new  water-immersion  objectives  designed  for 
confocal  imaging  also  specifically  address  the  problem  of  UV  exci¬ 
tation,  not  only  by  having  a  high  degree  of  UV  transmission,  but 
also  by  extending  the  corrected  waveband  far  enough  to  assure 
good  confocality  between  UV-exciting  and  visible-emitted  radia- 


2  These  “blue"  objectives  have  been  designed  to  perform  best  in  the  near-UV 
blue  end  of  the  spectrum.  Other  manufacturers  also  now  offer  objectives  opti¬ 
mized  for  best  performance  at  near-IR-red  wavelengths  for  using  in  multi¬ 
photon  excitation  microscopy. 


tion.  Specifically  for  near  UV  excitation,  Nikon  offers  a  CF  Fluor 
40x,  NA  1.2  water  lens  with  high  transmission  down  to  340  nm 
and  a  working  distance  of  220  |im.  Olympus  now  offers  four 
UV-apochromats:  20x,  NA  0.4;  40x,  NA  0.9;  100X,  NA  1.1  with 
working  distances  of  150pm,  130pm,  and  130pm,  respectively. 
All  are  apochromatically  corrected  from  350  nm  to  600  nm  with 
excellent  flatness  over  a  10  mm  field.  The  Zeiss  C-Apochromats 
40X/1.2  water  and  63X/1.2  water  combine  high,  near-UV  trans¬ 
mission  with  apochromatic  color  correction,  flatness  over  a  14  mm 
field,  and  working  distances  of  0.24mm.  Figure  7.18  shows  the 
pinhole  energy  as  a  function  of  focus  and  wavelength  for  such  an 
objective. 

An  interesting  and  imaginative  use  for  longitudinal  chromatic 
aberration  was  described  by  Maly  and  Boyde  (1994)  as  a  means 
of  implementing  real-time,  stereoscopic  confocal  microscopy 
(Boyde  and  Jones,  1995).  They  have  developed  special  objectives 
with  linear  longitudinal  chromatic  dispersion  (LLCD).  When  these 
lenses  are  used  to  detect  the  reflected  light  signal  from  a  disk¬ 
scanning  confocal  microscope  using  “white”  light  illumination, 
discrete  “strata”  of  the  specimen  surface  appear  in  only  those 
colors  for  which  confocal  conditions  exist  (i.e.,  different  strata 
have  different  colors  with  the  blue  layer  being  closest  to  the  objec¬ 
tive  and  the  red  layer  farthest).  These  spectral  colors  can  then  be 
either  recorded  with  a  color  image  sensor  and  converted  directly 
into  height  by  simple  computer  algorithms  or  used  as  the  basis  of 
a  real-time  stereoscopic  imaging  system.  In  the  latter  case,  dis¬ 
persive  viewing  elements  (prisms)  are  used  to  change  the  differ¬ 
ent  colors  into  a  parallax  shift  that  depends  directly  on  the  color 
and  indirectly  on  the  height.  By  mounting  two  such  elements  into 
the  two  sides  of  a  binocular  viewing  head  so  that  they  have  oppo¬ 
site  horizontal  dispersions,  the  viewer  will  perceive  a  striking,  real¬ 
time  stereoscopic  confocal  image.  The  special  LLCD  objectives 
described  range  from  lOx/NAO.3  to  120X/NA  1.0  (oil). 
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FIGURE  7.19.  Lateral  chromatic  aberration  for  a  Planapo  40x,  NA  1.2  water. 


Lateral  Chromatic  Aberration  or  Chromatic 
Magnification  Difference 

Because  magnification  is  proportional  to  1/f,  the  same  change  of  f 
with  X  that  causes  light  of  different  X  to  focus  on  different  planes 
(producing  axial  chromatic  aberration)  also  causes  the  magnifica¬ 
tion  to  change  with  X,  producing  lateral  chromatic  aberration 
(LCA).  If  the  magnification  of  an  objective  is  different  for  differ¬ 
ent  X,  images  of  sharp  edges  in  the  object  will  show  blue  or  red 
fringes.  In  a  non-corrected  system,  the  blue  component  at  436  nm 
may  be  imaged  1.4%  larger  than  the  red  component  at  630  nm. 
Normally,  the  LCA  is  greater  for  objectives  of  short  focal  length 
and  the  magnification  error  can  range  from  1.1%  to  1.9%  of  the 
radial  distance  from  the  optical  axis. 

In  confocal  scanning  fluorescence,  LCA  can  cause  the  excited 
light  to  return  to  a  location  nearer  to,  or  farther  from,  the  axis  than 
the  apparent  location  of  the  source  (see  Fig.  35.14,  this  volume). 
This  results  in  intensity  loss  at  the  pinhole.  Proper  matching  of  all 
components  is  essential  to  assure  full  compensation  of  the  LCA. 
In  multi-channel  fluorescence,  poor  registration  of  blue-,  green-, 
and  red-emitting  markers  can  lead  to  possible  errors  in  mapping 
their  precise  locations.  A  highly  corrected  lens  such  as  the 
planapochromat  40x,  NA  1.2  W  keeps  the  LCA  well  within  0.2% 
over  a  wide  wavelength  range  (Fig.  7.19). 

In  earlier,  conventional  microscope  systems,  all  objectives 
were  calculated  to  produce  a  constant  amount  of  LCA,  which  was 
then  compensated  for  by  the  eyepiece.  This  compensating  system 
is  illustrated  in  Figure  7.20.  The  modern,  infinity-corrected  micro¬ 
scope  objectives  are  either  fully  LCA-corrected  in  the  objective 
itself  (Nikon,  Olympus)  or  use  a  compensating  tube  lens  for  a  full 
correction  of  the  intermediate  image. 

The  Zeiss  ICS  (Infinity  Color-corrected  System)  optics  are 
unusual  among  these  tube-lens  systems  in  that  no  correction  for 
LCA  takes  place  in  the  objective.  The  ICS  tube  lens  is  a  single, 
spherical  lens  which  fully  compensates  for  LCA  (range  = 
1.1% — 1.9%)  in  all  objectives  over  the  entire  25  mm  intermediate 
image  field  (Fig.  7.21).  This  is  possible  because,  although  all  off- 
axis  imaging  bundles  pass  through  the  exit  pupil,  the  position  of 
the  exit  pupil  itself  depends  on  the  focal  length  of  the  objective 
(i.e.,  the  magnification):  the  shorter  the  focal  length,  the  closer  the 
exit  pupil  is  to  the  specimen.  If  we  trace  off-axis  bundles  from  a 
short  focal  length  objective  to  the  tube  lens,  we  find  that  they  reach 
more  peripheral  zones  of  the  tube  lens,  which  compensate  more 
strongly  for  the  higher  LCA  present  in  high-magnification  lenses. 
On  the  other  hand,  bundles  from  long  focal-length  objectives  reach 
only  the  more  axial  zones  that  are  appropriate  for  the  lower  LCA 
compensation  needed  in  low-magnification  objectives.  Relieving 
the  objective  from  LCA  correction  in  this  way  means  fewer  ele¬ 
ments  in  the  objective  and  a  choice  of  more  suitable  glass  types, 
particularly  for  the  Plan-Neofluars,  where  only  one  selected  heavy 
flint  glass  and  no  fluorite  is  used.  This  virtually  eliminates  auto¬ 
fluorescence  in  the  objective,  improves  extinction  ratios  for 
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FIGURE  7.21.  Zeiss’s  ICS  optics  operation.  The  same  tube  lens  produces 
more  chromatic  correction  for  high  magnification  objectives  than  for  low  mag¬ 
nification  objectives  because  the  relative  size  and  position  of  the  exit  pupils, 
and  the  optics  of  the  infinity-focus  objectives  themselves,  causes  rays  to  pass 
through  different  parts  of  the  tube  lens  (see  text  for  more  detail). 
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polarizing  microscopy,  and  assures  high  light  throughput  and  good 
contrast. 

All  off-axis  aberrations  can  adversely  affect  the  performance 
of  beam-scanning  systems.  Potentially  most  critical  is  LCA  in  the 
confocal  fluorescence  mode,  where  the  excited  off-axis  object 
point  may  be  sufficiently  shifted  from  the  pinhole  that  a  substan¬ 
tial  signal  loss  may  occur. 

Example:  A  lOOx  oil  objective  may  scan  a  100  x  100pm  field 
in  the  object  with  500  pixels  per  scan  line.  Assume  that  the  posi¬ 
tion  of  a  0.2  pm  diameter  blue  Airy  disk  of  exciting  light  is  shifted 
1 .9%  of  the  distance  from  the  center  of  the  field  compared  to  its 
corresponding  red  emission  disk.  Pixels  that  are  50  pm  from  the 
center  will  be  displaced  almost  1  pm  or  5  pixel  diameters  away 
from  the  pinhole.  Depending  on  the  size  of  the  pinhole,  the  severe 
loss  of  signal  is  obvious. 

For  confocal  microscopy,  the  LCA  —  transferred  to  the  spec¬ 
imen  field  —  must  be  smaller  than  the  pixel  spacing  on  the  periph¬ 
ery  of  the  scanned  field.  For  a  raster  scan  of  512  x  512,  this  means 
an  LCA  of  less  than  0.19%,  if  we  assume  the  ~1.4x  resolution 
increase  of  confocal  scanning.  The  LCA  for  a  Planapo  40x,  NA 
1.2W  is  shown  in  Figure  7.19.  In  more  general  terms,  it  means 
that  Nikon  CF  objectives  cannot  be  replaced  by  compensating 
objectives  unless  the  rest  of  the  system  optics  are  also  changed. 

With  the  increasing  interest  in  two-photon  excitation,  which 
requires  optimal  chromatic  correction  for  the  incoming  infrared 
(IR)  beam  combined  with  efficient  collection  of  the  emitted 
photons,  more  and  more  lenses  for  that  application  have  become 
available. 


m  =  mechanical  tube  length  =  1 60  mm 
a  =  object-to-image  distance  =  195  mm 
b  =  object  distance  of  objective  =  45  mm 
d  =  intermediate  image  distance 
of  eyepiece  =  1 0  mm 
c  =  working  distance  of  the  objective 
r  =  distance  mounting  shoulder  monocular 
tube  to  intermediate  image 

FIGURE  7.22.  Standard  dimensions  of  DIN  standard  conventional 
microscope. 


Reflecting  Objectives 

At  first  sight  it  might  appear  that  a  logical  solution  to  the  problem 
of  chromatic  aberration  might  be  to  use  reflecting  rather  than 
refracting  objectives.  Reflection  is  insensitive  to  X  and,  like  the 
reflecting  telescope,  a  reflecting  microscope  objective  could  offer 
excellent  chromatic  correction  over  a  wide  spectral  range.  In  fact, 
some  low  and  medium  power  reflecting  objectives  are  routinely 
used  particularly  for  far-IR  microscopy.  However,  high  resolution 
requires  objectives  with  high  NA  and  short  focal  length  or 
even  immersion  systems.  Though  the  diamond-turning  lathe  could 
produce  the  aspherical  surfaces  required  with  the  necessary  preci¬ 
sion,  such  lenses  cannot  now  be  produced  at  reasonable  cost.  Fur¬ 
thermore,  an  inherent  limitation  of  reflecting  objectives  is  the 
elimination  of  paraxial  imaging  bundles  with  the  corresponding 
loss  of  diffraction  orders  and  effective  NA. 

FINITE  VERSUS  INFINITY  OPTICS 

Some  conventinal  microscopes  still  in  use  were  built  around  the 
German  DIN  (now  ISO)  standard,  which  calls  for  the  objective  to 
form  a  real  image  at  a  tube  length  of  160  mm.  The  parfocal  dis¬ 
tance  is  set  at  45  mm  and  the  object-to-image  distance  is  195  mm 
(Fig.  7.22).  This  standard  applies  to  many  transmitted  light  tech¬ 
niques  and  has  served  us  well  for  over  50  years. 

Objectives  with  a  finite  image  distance  form  a  real,  intermedi¬ 
ate  image  directly.  This  intermediate  image  may  either  be  corrected 
(Nikon)  or  it  may  have  residual  LCA  to  be  compensated  for  by  the 
eyepiece.  Infinity-designed  objectives  require  a  tube  lens  to  form 
this  real,  intermediate  image.  While  this  tube  lens  can  also  be 
employed  to  correct  for  residual  aberrations,  the  intrinsic  design 
advantage  of  infinite  image  distance  (Fig.  7.23)  is  its  relative  insen¬ 
sitivity  to  optical  components  (such  as  filters,  analyzers,  compen¬ 
sators,  DIC  prisms,  and  reflectors)  in  the  telescopic  space  between 


FIGURE  7.23.  Advantage  of  using  objectives  with  infinity  correction.  A 
number  of  accessories,  such  as  the  Wollaston  prisms  used  for  DIC  and  the  filters 
for  epi-fiuorescence,  can  be  introduced  between  the  objective  and  the  tube  lens 
without  affecting  the  image  of  the  structure  at  the  front  focal  plane. 
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FIGURE  7.24.  Conventional  optics  using  Telan  lenses.  Requires  more  com¬ 
ponents  that  not  only  reduce  resolution  and  cause  reflection  losses  but  also 
increase  magnification. 


objective  and  tube  lens.  Infinity  or  parallel  beams  are  not  affected 
by  the  thickness  or  Tj  of  such  components  as  long  as  they  are  plane 
parallel.  The  location  of  the  image  point  remains  constant,  both 
axially  and  laterally,  as  does  the  alignment  between  the  objective 
and  the  tube  lens. 

Another  advantage  of  infinity-corrected  optics  is  that  one  can 
focus  by  moving  the  objective  rather  than  having  to  move  the  stage, 
as  is  the  case  in  most  inverted  and  some  upright  microscopes. 

In  the  converging  beam  path  behind  a  160  mm  tube  length 
objective,  similar  elements  would  cause  axial,  and  possibly  lateral, 
shift.  This  not  only  causes  aberrations  but  makes  it  difficult  to  get 
good  image  registration  in  multi-parameter  techniques  (double  or 
triple  fluorescence).  To  avoid  this,  so-called  Telan  systems  are 
added  to  generate  an  “infinity  space”  into  which  the  “flat”  com¬ 
ponents  can  be  introduced.  This  eliminates  the  image  shift  and  dis¬ 


tortion  but  only  at  the  cost  of  two  additional  optical  elements  — 
one  negative  and  one  positive  —  and  with  them  some  additional 
magnification  (usually  1.25x)  and  the  potential  for  more  flare  and 
stray  light  (Fig.  7.24). 

The  design  concepts  followed  by  major  microscope  manufac¬ 
turers  are  summarized  in  Table  7.2. 


WORKING  DISTANCE 

For  reflected  light  objectives,  the  distance  between  the  focus  plane 
and  the  closest  structural  element  of  the  objective  is  called  the 
working  distance  (WD).  For  transmitted  light  lenses,  the  specified 
free  working  distance  is  the  distance  from  the  coverslip  to  the 
closest  structural  element  of  the  lens.  In  a  confocal  microscope, 
the  WD  sets  an  absolute  limit  that  the  focus  plane  can  be  below 
the  top  surface  of  the  coverslip.  As  an  optical  design  parameter, 
WD  can  be  varied  over  a  considerable  range,  but,  as  can  be  seen 
from  Table  7.3,  it  becomes  decidedly  shorter  in  objectives  with 
higher  NA  and  greater  correction. 

In  oil  lenses,  the  first  refracting  surface  struck  by  a  light  ray  is 
the  far  side  of  the  first  element  of  the  objective.  This  surface  is 
often  a  segment  of  a  sphere  that  is  large  enough  to  encompass  the 
full  NA  of  the  design.  (The  lens  half-angle,  a,  is  74°  at  NA  1.4.) 
As  the  focus  plane  moves  away  from  the  glass,  this  sphere  segment 
must  become  larger  with  the  result  that  soon  there  is  no  longer  suf¬ 
ficient  space  in  which  to  install  the  other  elements  needed  for  chro¬ 
matic  and  field  corrections.  These  larger  elements  are  also  more 
expensive  to  produce. 

When  working  with  living  materials,  the  matter  of  WD  can 
have  a  second  aspect  because  in  such  cases,  it  is  often  convenient 
to  be  able  to  change  the  ^-position  of  the  imaged  plane  in  the  object 
without  actually  moving  either  the  position  of  the  front  surface  of 
the  objective  or  the  specimen  itself.  This  can  occur  when  imaging 
the  living  eye  where  actual  contact  between  the  cornea  and  the 
objective  is  needed  to  reduce  the  effect  of  pulse-induced  motion. 
To  solve  this  problem,  Petroll  and  colleagues  (1993)  have  used  an 
objective  with  internal  focusing  elements  to  permit  changes  in 
focus  plane  while  maintaining  mechanical  contact. 

The  working  distance  is  a  critical  issue  for  3D  imaging  of  thick 
samples  or  when  manipulation  or  probing  under  the  objective  is 
required.  However,  we  must  also  keep  in  mind  that  thick  layers  of 
medium  between  object  and  objective  can  easily  create  turbulence 
(e.g.,  when  focusing)  or  inhomogeneities  (such  as  air  bubbles!) 
that  can  seriously  impair  image  quality  or  signal  intensity. 


TABLE  7.2.  Tube  Length  and  Chromatic  Corrections  of 
the  Major  Microscope  Manufacturers 


Manufacturer 

Mode 

Fixed,  Finite  Tube 

Length  (mm) 

Infinite  Tube  Length 

Lens,  f  =  mm 

Chromatic  Correction 

Leica 

Reflected 

_ 

200 

In-tube  lens 

Transmit  (older) 

— 

200 

In-tube  lens 

160 

— 

Leica  correction 

Nikon 

Reflected  (few)  (most) 

— 

210 

None  required 

210 

— 

None  required 

Transmit 

160 

— 

None  required 

Olympus 

Reflected 

— 

180 

In-tube  lens 

Transmit  (some)  (older) 

— 

180 

In-tube  lens 

160 

— 

Olympus  correction 

Zeiss 

Reflected 

— 

160 

Universal  tube  lens 

Transmit  (older) 

— 

160 

Universal  tube  lens 

160 

— 

Zeiss  correction 
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TABLE  7.3.  Working  Distance  of  Some  Zeiss  Objectives 


Objective 

Immersion 

Cover  Glass 

Working  Distance 

Epiplan  50x/0.5 

Air 

No 

7  mm 

Epiplan  100x/0.75 

Air 

No 

0.98  mm 

Epiplan-Neofluar  100x/0.9 

Air 

No  (correction  collar) 

0.28-0.31  mm 

LD  Plan-Neofluar  40x/0.6 

W/Gly 

Yes  (correction  collar) 

2. 5-3. 3  mm 

Planapo  40x/0.95 

Air 

0.13-0.17  (Correction  collar) 

0.16mm 

Achroplan  40X/0.8 

Water/dip 

No 

3.61  mm 

Achroplan  IR  63x/0.9 

Water/dip 

No 

2.2  mm 

Achroplan  100x/1.0 

Water/dip 

No 

1.00  mm 

c-Planapo  40x/1.2 

Water 

Yes 

0.22  mm 

Planapo  100x/1.4 

Oil 

Yes 

0.17  mm 

OPTICAL  MATERIALS 

More  than  200  optical  glasses  are  available  to  the  optical  designer. 
Their  properties,  such  as  refractive  index,  dispersion,  transmis¬ 
sion,  contaminants,  potential  for  auto-fluorescence,  chemical  and 
thermal  resistance,  and  overall  homogeneity  are  usually  carefully 
selected  to  ensure  superb  optical  performance.  However,  this  may 
compromise  other  requirements,  such  as  high  transmission  in  the 
near-UV  range  or  high  extinction  factors  in  polarizing  microscopy. 
Some  new  materials,  such  as  fluorocrown  glass,  approach  the  prop¬ 
erties  of  natural  fluorite  while  avoiding  most  of  its  drawbacks,  such 
as  the  presence  of  organic  contaminants  and  a  crystalline  structure 
that  can  seriously  degrade  performance  in  fluorescence  and  polar¬ 
izing  microscopy.  Fully  apochromatic  correction,  however,  still 
requires  both  natural  fluorite  and  glasses  that  have  low  transmis¬ 
sion  in  near  UV. 

Therefore,  the  semi-apochromat  is  often  the  ideal  compromise. 
It  can  be  a  true  multi-purpose  objective,  combining  excellent  cor¬ 
rection  with  good  contrast,  high  NA,  and  high  spectral  throughput. 

Even  though  their  thickness  is  usually  <10|im,  the  cements 
between  doublet  or  multiple  lens  elements  can  have  spectral 
absorption  properties  that  may  render  an  objective  unusable  for 
specific  applications. 

The  chemical  and  optical  properties  of  optical  glasses  are 
usually  proprietary.  Decementation  between  lenses,  caused  by  heat 
and  different  coefficients  of  expansion,  is  rare.  Lasers  of  <100mW 
energy  in  near-UV  and  visible  range  will  cause  no  damage  if 
their  power  is  evenly  distributed  over  the  entire  entrance  pupil, 
however,  damage  can  occur  if  a  similar  amount  of  light  is  brought 
to  a  focus  on  any  optical  surface  having  an  appreciable  absorption 
at  that  X. 

ANTI-REFLECTION  COATINGS 

As  the  sophistication  of  objectives  increases,  more  elements  are 
required,  and  this  accentuates  the  need  to  eliminate  internal  reflec¬ 
tions  to  produce  higher  transmission,  better  contrast,  and  less  flare, 
particularly  in  incident-  or  reflected-light  applications.  Single¬ 
layer  anti-reflection  (AR)  coatings  dating  back  to  the  1940s  have 
since  been  refined  and  supplemented  by  multi-layer  coatings, 
increasing  the  transmission  in  the  visible  spectral  range  through  an 
air-glass  interface  from  -96%  (not  coated)  to  -99%  (single  layer 
coating)  to  -99.9%  (multi-layer  coating)  (Figs.  7.25  and  7.26). 

Coating  materials  can  be  magnesium  fluoride  and  a  multitude 
of  proprietary  materials  (see  Chapter  3,  this  volume ),  all  of  which 
have  their  own  optical  properties  potentially  affecting  the  trans¬ 


mission  of  the  system  in  given  spectral  regions.  In  general,  the 
interference  characteristics  of  AR  coatings  are  spectrally  limited, 
and  constructive  interference  for  high  transmission  in  the  visible 
range  means  destructive  interference  in  harmonically  related  fre¬ 
quencies  outside  the  transmission  band. 

For  specimens  of  very  low  reflectivity,  the  weak  BSL  signal 
may  be  overwhelmed  by  background  light  produced  by  internal 
reflections  in  the  objective  lens  or  other  optical  surfaces  that  reflect 
light  that  comes  to  a  focus  near  an  image  plane.  In  this  case,  a  so- 
called  Antiflex  system  can  have  a  great  advantage.  In  this 
approach,  we  illuminate  with  polarized  light  and  observe  through 
an  analyzer  at  90°  to  the  polarizer  (Chapter  10,  this  volume). 
Because  stray  light  produced  by  specular  reflection  from  optical 
surfaces  retains  its  direction  of  polarization,  it  can  be  blocked  by 
the  analyzer.  If  a  1/4  wave  plate  is  incorporated  into  the  front 
element  of  the  objective  with  its  vibration  direction  diagonal  to 
both  the  polarizer  and  analyzer  for  a  specific  wavelength  (usually 
550  nm),  this  will  generate  circularly  polarized  light  below  the 
objective.  The  1/4  wave  plate  rotates  the  plane  of  vibration  of  both 
the  illumination  and  the  returning  BSL  by  45°  for  a  total  of  90°, 
so  light  reflected  from  the  object  is  now  oriented  parallel  to  the 
analyzer  and  can  pass  freely  to  the  detector.  This  system  is  used 
for  the  so-called  reflection  contrast  or  reflection  interference  tech¬ 
niques  but  is  also  most  effective  in  confocal  studies  of  weakly  scat¬ 
tering  objects  (Pawley  et  al.,  1993). 

Transmission  of  Microscope  Objectives 

We  have  seen  that  a  number  of  factors  influence  the  spectral 
throughput  of  an  objective:  glasses,  cements,  coatings.  However, 
these  considerations  must  apply  to  not  only  the  objective  but  to  the 
whole  system  if  it  is  to  provide  high  transmission  in  the  desired 
wavelength  range. 


R  [%] 


FIGURE  7.25.  Reflectivity  of  glass  surfaces  covered  with  single-  versus  multi¬ 
layer  coatings. 
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I  I  reflected  light  decreases  image  contrast 


air-glass  surfaces 

FIGURE  7.26.  Reflections  on  surfaces  of  n  =  1.5  (above)  versus  n  =  1.8 
(below).  At  n  =  1.5,  the  eight  elements  with  their  16  surfaces,  each  reflecting 
4%,  result  in  a  throughput  of  only  52%.  At  n  =  1.8,  the  16  uncoated  surfaces 
would  pass  only  26%.  Single-layer  anti-reflection  coating  increases  total  trans¬ 
mission  to  85%,  and  multi-layer  coating  to  94.6%.  This  increase  in  through¬ 
put,  and  corresponding  reduction  in  internal  scatter  and  “noise,”  substantially 
enhances  the  contrast  of  the  image  because  it  both  makes  bright  features 
brighter  and  dark  features  darker. 


A  typical  arrangement  to  measure  the  spectral  transmission  of 
an  objective  is  shown  in  Figure  7.27:  light  source  (tungsten 
halogen  and/or  xenon)  with  monochromator  and  iris  diaphragm  fill 
the  objective’s  back-aperture.  A  parallel  light  beam  equal  in  diam¬ 
eter  to  the  pupil  of  the  objective  strikes  the  rear  of  the  objective 
and  emerges  through  the  front  lens  where  it  is  collected  by  an  inte¬ 
grating  sphere  and  measured  by  a  PMT  with  photometer  read-out. 
The  result  is  compared  to  a  second  measurement  made  with  the 
objective  removed.3 

Table  7.4  shows  the  relative  spectral  transmission  of  several 
representative  objectives  that  have  been  optimized  for  high  trans¬ 
mission  in  the  near-UV.  With  confocal  fluorescence  microscopy 
extending  both  towards  the  near  UV  as  well  as  towards  the  IR,  the 


3  The  high-NA  rays  from  an  oil-immersion  objective  can  only  escape  from  the 
glass  into  the  sphere  if  a  small,  planoconvex  lens  is  attached  to  the  front 
element  with  oil. 


Photometer 


Integrating  Sphere 
with  Detector 

Objective 
Iris  Diaphragm 

Monochromator 

Light  Source 

FIGURE  7.27.  Setup  to  measure 


objective’s  spectral  transmission. 


spectral  transmission  of  a  total  system  needs  to  be  looked  at.  The 
transmission  curves  for  a  number  of  objectives  by  Nikon,  Leica, 
and  Olympus  are  included  in  the  Appendix  at  the  end  of  this 
chapter.  Keep  in  mind  that  these  transmission  curves  are  typical 
only  for  a  given  lens.  Tolerances,  particularly  in  the  AR  coating, 
can  introduce  significant  variations,  particularly  at  the  cut-on  and 
cut-off  ends  of  the  transmission  curve. 

Depending  on  the  application,  objectives  can  be  chosen  with 
high  throughput  in  the  near-UV  or  with  extended  IR  transmission. 
Having  good  transmission  in  both  the  near-UV  and  the  near-IR  is 
not  possible. 

A  number  of  new  objectives,  designed  specifically  for  confo¬ 
cal  microscopy,  are  either  already  available  or  in  preparation.  More 
detailed  information  on  their  performance  and  spectral  throughput 
is  best  obtained  directly  from  the  manufacturers. 


TABLE  7.4.  Relative  Transmission  of  Some  Typical 
Objectives 


Manufactuerer 

Objective 

320 

Wavelength  (nm) 

350  400  500 

600 

Nikon 

Fluor  40X/1.3  oil 

16% 

66% 

80% 

90% 

91% 

Olympus 

UVSLM  40X/0.9  water 

— 

56% 

— 

88% 

— 

UVSLM  lOOx/1.1  water 

— 

60% 

— 

90% 

— 

Zeiss 

Fluar  40X/1.3  oil 

29% 

79% 

88% 

95% 

99% 

Planapo  40x/1.2  water 

20% 

54% 

86% 

89% 

92% 

Achropam  100x/1.0  water 

— 

605 

90% 

94% 

90% 
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CONCLUSION 

Many  modern  microscope  objectives  are  well  suited  for  confocal 
scanning  as  long  as  they  are  used  within  their  design  specifica¬ 
tions.  With  the  ever- wider  use  of  confocal  scanning  microscopy 
for  3D  live-cell  imaging  and  with  new  microscope  techniques 
emerging,  new  and  improved  objectives  continue  to  appear.  Often 
they  are  optimized  for  a  specific  application.  This  trend  will  con¬ 
tinue.  Just  a  few  examples  are  the  new  highest-NA  60x/1.45  and 
100x/1.45  oil-immersion  lenses  or  the  special  objectives  Olympus 
introduced  for  epi-illuminated  total  internal  reflection  fluorescence 
(TIRF)  microscopy  with  an  NA  of  1.45  or  1.65.  It  follows  from 
the  discussion  above  that  the  latter  NA  requires  a  special  immer¬ 
sion  medium  and  rather  costly  high-index  coverglasses. 

In  addition  to  the  call  for  objectives  with  higher  NAs,  longer 
working  distances,  and  maximum  transmission  in  the  visible  and 
UV,  this  may  be  the  place  to  “dream”  about  other  exotic  lens 
features.  We  might  imagine  a  tunable  objective,  a  lens  whose 
chromatic  correction  can  be  “tuned”  to  the  specific  excitation 
and  emission  wavelengths  actually  in  use  to  produce  the  best 
image  quality  with  only  a  few  elements  and  high  transmission.  If 
such  tunability  is  impossible,  perhaps  special  achromatic  lenses 
will  be  designed  for  use  with  specific  fluorophore/laser  com¬ 
binations,  such  as  the  “blue”  objectives  recently  offered  by  Leica. 

An  even  more  exotic  dream  (included  in  the  second  edition  of 
this  volume)  for  a  substantial  increase  in  signal  intensity  realized 
by  using  two,  matched  lenses  above  and  below  the  specimen,  has 
now  been  commercially  realized  in  the  Leica  4Pi  microscope 
(Chapter  30,  this  volume).  On  optically  homogeneous  specimens, 
equipping  the  “far”  objective  with  a  reflector  to  return  the  spot 
into  itself,  might  not  only  increase  photon  efficiency  but  also 
might  permit  combining  reflected  and  transmitted  confocal 
scanning. 

As  mentioned  earlier,  two-photon  excitation,  will  probably 
also  spawn  the  development  of  special  objectives,  not  only  in  terms 
of  their  spectral  characteristics  but  with  spherical  and  chromatic 
correction  optimized  for  the  IR,  such  as  the  recent  “IR”  objectives 
from  Olympus. 

For  the  study  of  live  organisms,  one  needs  water-immersion 
objectives  that  combine  the  long  working  distance  needed  for  deep 


penetration,  with  the  highest  possible  NA  for  better  resolution  and 
improved  photon  collection  efficiency. 

These  are  just  a  few  thoughts  on  possible  future  developments 
in  this  area.  No  doubt,  specialists  in  optical  design  will  quickly 
return  us  to  reality  and  make  us  do  with  less  than  our  dreams. 
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APPENDIX:  LIGHT  TRANSMISSION  SPECIFICATIONS  FOR  A  NUMBER  OF  MODERN  OBJECTIVES 
MADE  BY  DIFFERENT  MANUFACTURERS.4 


Leica  objectives 


Objective 

Mag 

NA 

WD(g) 

T(350)% 

T(546)% 

T(900)% 

Blue  Immersion 

HC  PLAPO  1 0x/0.40  CS 

10 

0.40 

2200 

40 

90 

75 

dry 

APO  L  20X/0.50W  UVI 

20 

0.50 

3500 

60 

91 

75 

water 

HC  PL  APO  20x/0.70,  multi-imm,  collar 

20 

0.70 

260 

40 

87 

71 

•  water/glycerol/oil 

HCX  PLAPO  63x/l. 40-0.60  oil 

63 

1.40 

100 

20 

82 

63 

•  oil 

HCX  PLAPO  63X/1.30  glycerin  collar 

63 

1.30 

280 

18 

87 

64 

glycerol :  water,  80 :20 

HCX  PL  APO  63X/1.2  W  Corr 

63 

1.20 

220 

16 

85 

65 

water 

Blue  =  Blue  variant  available  with  color  correction  optimized  in  blue  range  for  UV  applications  and  GFPs  with  405  or  430  excitation 


The  specifications  printed  in  this  chapter  are  as  supplied  to  the  author  by  the  manufacturers  at  the  time  that  this  book  went  to  press  (May  2005).  The  author 
and  the  editor  have  endeavored  to  insure  that  the  printed  versions  of  these  specifications  accurately  reflect  the  reports  that  they  received  or  that  were  available 
on  the  Internet.  We  make  no  other  claim  and  readers  are  advised  that  improvements  are  to  be  expected  and  will  doubtless  be  announced  on  the  Internet. 
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Nikon  objectives 


Objectives 

N.A. 

Coverglass 

(mm) 

Working 

Distance  (mm) 

Trans  % 

@350nm* 

Trans  % 

@550nm* 

Trans  % 

@900  nm* 

CFI  PLAN  ACHROMAT  lOOx  OIL 

1.25 

0.17 

0.20 

B 

A 

A 

CFI  PLAN  APO  20x 

0.75 

0.17 

1.00 

E 

A 

B 

CFI  PLAN  APO  40x  w/collar. 

0.95 

0.17 

.12-.16 

E 

A 

B 

CFI  PLAN  APO  60x  A  OIL  IR 

1.40 

0.17 

0.21 

E 

A 

B 

CFI  PLAN  APO  60x  WI  w/collar 

1.20 

0.15-0.18 

0.22 

E 

A 

B 

CFI  PLAN  APO  VC  60x  WI 

1.20 

0.13-0.19 

0.27 

E 

A 

B 

CFI  PLAN  FLUOR  20x  mult-imm 

0.75 

0.17 

.033-035 

B 

A 

A 

CFI  PLAN  FLUOR  40x 

0.75 

0.17 

0.72 

A 

A 

A 

CFI  PLAN  FLUOR  lOOx  OIL 

1.30 

0.17 

0.20 

B 

A 

A 

CFI  SUPER  FLUOR  40x  w/collar 

0.90 

0.17 

0.30 

B 

A 

B 

CFI  SUPER  FLUOR  40x  OIL 

1.30 

0.17 

0.22 

B 

A 

A 

CFI  W  FLUOR  60x  dipping 

1.00 

n/a 

2.00 

B 

A 

A 

CFI  PLAN  lOOx  dipping 

1.10 

n/a 

2.50 

E 

A 

A 

CFI  Plan  Apo  TIRF  60xH  w/collar 

1.45 

0.17 

0.13 

E 

A 

B 

CFI  Plan  Apo  TIRF  lOOxH 

1.45 

0.17 

0.13 

E 

A 

B 

Key:  For  transmission  percentages:  A  = 

>71%.  B  =  51-70%,  C 

=  31-50%,  D  = 

16-30%. 
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The  Contrast  Formation  in  Optical  Microscopy 

Ping-Chin  Cheng 


INTRODUCTION 

In  any  form  of  microscopy,  one  needs  not  only  an  imaging  system 
with  enough  resolution  to  delineate  the  fine  details  of  the  speci¬ 
men  but  also  a  suitable  contrast  mechanism  by  which  to  “see”  the 
shape  of  the  structures  of  interest.  Contrast  is  the  difference 
between  the  signal  in  one  pixel  and  that  in  another  that  conveys  to 
the  viewer  information  about  the  shape  of  the  specimen.  It  is  the 
difference  between  a  blank  screen  and  an  image. 

In  photographic  terms,  contrast  is  the  change  in  brightness  of 
a  negative  or  print.  In  other  words,  contrast  is  the  difference  in 
signal  strength  between  various  parts  of  an  image  or  between 
details  of  interest  and  “background”  (see  also  Chapter  4,  this 
volume).  The  contrast  (y)  is  proportional  to  the  intensity  difference 
(A I)  between  two  image  areas,  divided  by  the  average  image 
brightness  /. 


M 


In  optical  microscopy,  contrast  derives  from  differences  in  the 
way  the  various  sub-volumes  of  the  specimen  (voxels)  interact 
with  the  illumination.  This  interaction  may  include  linear  and  non¬ 
linear  absorption,  single-  and  multiple-photon  fluorescence, 
Raman  emission,  fluorescence  spectral  shift,  fluorescence  lifetime, 
refraction,  reflection,  phase  shift,  scattering,  changes  in  polariza¬ 
tion,  harmonic  generation,  etc.  A  contrast  mechanism  can  thus 
be  considered  to  be  a  special  “filter”  function  by  which  specific 
types  of  spatial  or  temporal  signals  are  selected  to  form  a  two- 
dimensional  (2D)  or  three-dimensional  (3D)  image. 

This  chapter  will  provide  an  introduction  to  the  contrast  char¬ 
acteristics  of  those  modalities  that  have  been  well  investigated  in 
confocal  and  nonlinear  microscopy,  including  fluorescence  and 
scattering  contrast.  It  will  also  consider  the  deleterious  influence 
on  the  confocal  and  multi-photon  image  of  the  absorptive,  refrac¬ 
tive,  and  reflective  properties  of  structures  that  are  between  the 
plane-of-focus  and  the  objective  lens,  of  these  topics  are  relevant 
to  the  operation  of  all  epi-illuminated  microscopes  and  some  also 
apply  to  signals  detected  in  transmission,  particularly  as  used  to 
detect  second  harmonic  (SHG)  and  third  harmonic  (THG)  signals. 

In  addition,  this  chapter  will  introduce  ways  in  which  the  con¬ 
trast  present  in  the  raw  data  from  the  microscope  can  be  digitally 
modified  before  being  presented  in  the  final  image.  At  various 
stages  in  this  sequence,  the  signal  may  be  ratioed,  filtered,  and  cor¬ 
rupted  and  the  contrast  reduced  by  the  addition  of  noise.  In  all 
these  areas,  this  chapter  serves  as  an  introduction  to  other  chapters 
in  which  individual  contrast  mechanisms  are  discussed  in  more 
depth. 


Image  contrast  arises  from  the  interaction  of  an  incident  light 
beam  with  the  specimen.  Various  physical  and  digital  “filters”  can 
be  used  to  select  specific  signals.  For  example,  one  can  discrimi¬ 
nate  specific  wavelengths  using  dichroic  beam-splitters  and  barrier 
filters;  the  effective  numerical  aperture  (NA)  of  the  objective  lens 
can  produce  topographic  contrast  from  the  geometric  shape  of 
specimen  surfaces;  polarized  light  can  be  used  to  obtain  contrast 
caused  by  specimen  birefringence;  and  fluorescence  signals  from 
ion-specific  dyes  in  two  different  spectral  channels  can  be  ratioed 
to  detect  the  concentration  of  ions  such  as  |Ca' 1 1  and  [H+]. 

The  contrast  that  forms  a  microscopic  image  is  determined  by 
the  number  of  physical,  chemical,  and  biological  phenomena. 
Contrast  mechanisms  can  be  subdivided  into  (a)  optical  contrast, 
(b)  geometric  contrast,  (c)  biological  and  chemical  contrast,  and 
(d)  synthetic  contrast.  From  the  point  of  view  of  the  specimen,  the 
contrast  mechanism  can  be  intrinsic  or  extrinsic  in  nature. 
Although  each  of  these  contrast  mechanisms  will  be  discussed  sep¬ 
arately,  it  is  not  uncommon  for  more  than  one  to  be  active  at  the 
same  time  and  care  must  be  taken  to  choose  experimental  para¬ 
meters  that  emphasize  the  contrast  that  highlights  the  most  infor¬ 
mative  of  these  interactions. 

The  interaction  of  an  incident  light  beam  with  a  sample  is  a 
complex  event.  Figure  8.1  shows  a  simplified  version  of  such  an 
interaction  as  well  as  some  of  the  effects  produced  by  the  voxels 
above  and  below  the  voxel  being  sampled.  These  interactions  give 
rise  to  the  optical  phenomena  and  the  photochemical  and  bio¬ 
chemical  effects  that  provide  the  bases  of  all  the  contrast. 

When  a  beam  of  light  with  intensity  I0  is  incident  on  a  speci¬ 
men,  a  number  of  physical  phenomena  may  occur.  These  include 
the  scattering  of  light  due  to  Rayleigh,  Mie,  and  Raman  scattering 
( Is  R ,  Is  M,  and  Is  Ram<m)-  Rayleigh  scattering  is  caused  by  interac¬ 
tions  with  very  small  particles  in  the  specimen  (from  the  size  of 
molecules  up  to  -10%  of  the  wavelength)  and  its  strength  (7S  R)  is 
strongly  wavelength  dependent.  It  also  depends  on  direction: 
scattering  at  right  angles  to  the  illumination  is  only  half  the 
forward  intensity  (/j  ,).  Rayleigh  scattering  is  elastic  scattering 
because  the  scattered  photons  have  the  same  energy  as  the 
incident  photons. 

By  contrast,  scattering  in  which  the  scattered  photons  have 
either  a  slightly  higher  or  lower  photon  energy  is  called  Raman 
scattering  (1 s  Raman)-  This  energy  change  usually  involves  either  the 
excitation  of  some  vibrational  mode  of  the  molecule  (giving  the 
scattered  photon  a  lower  energy),  or  the  scattering  of  the  photon 
off  an  excited  vibrational  state  of  a  molecule  (which  adds  its  vibra¬ 
tional  energy  to  the  incident  photon). 

Cell  organelles  and  other  refractive  structures  larger  than  the 
wavelength  of  the  illumination  contribute  to  Mie  scattering  (Is  m)- 
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Mie  scattering  has  a  sharper  and  more  intense  forward  lobe  for 
larger  particles  than  Rayleigh  scattering  (Fig.  8.2),  is  not  wave¬ 
length  dependent  and  is  responsible  for  the  almost  white  appear¬ 
ance  of  adipose  and  brain  tissue.  Such  a  scattering  center  can  act 
as  a  secondary  light  source  within  the  specimen. 

In  addition  to  these  scattering  events,  refractive  index  (RI  or 
T|)  differences  between  various  biological  structures  can  cause  the 
incident  ray  to  deviate  from  its  original  path,  producing  defocus- 
ing  or  sampling  errors  (i.e.,  imaging  the  signal  from  the  “wrong” 
voxel).  Reflection  occurs  at  any  interface  separating  regions  of  dif¬ 
ferent  RI. 

The  incident  light  can  also  be  attenuated  by  absorption.  Photon 
absorption  occurs  when  the  quantum  energy  of  the  photon  matches 
the  energy  gap  between  the  initial  and  final  states  of  some  electron 
in  the  specimen.  If  no  pair  of  energy  states  exists  such  that  the 
photon  energy  can  elevate  an  electron  from  a  lower  to  an  upper 
state,  then  the  matter  is  transparent  to  this  radiation.  In  the  absorp¬ 
tion  process,  some  of  the  absorbed  energy  can  be  re-emitted  in  the 
form  of  a  fluorescent  or  phosphorescent  photon.  Biological  mole¬ 
cules  have  specific  absorption  characteristics  and  the  low  effective 
absorbance  of  most  tissue  provides  a  relatively  transparent 
“window”  in  the  visible  and  near-infrared  range  (NIR).  This 
window  provides  the  working  spectrum  for  all  the  optical 
microscopy  (see  Figs.  21.1  and  40.2,  this  volume).  When  the 


FIGURE  8.2.  Rayleigh  scattering  (left)  and  Mie  scattering  (right).  In  Mie  scat¬ 
tering,  the  forward  lobe  becomes  larger  as  particles  become  larger.  This  is  the 
reason  that  dark-field  illumination  produces  more  signal  in  transmission  than 
in  epi  mode. 


FIGURE  8.1.  Interaction  of  light  with  a  voxel  of  the  specimen.  The  hypo¬ 
thetical  specimen  consists  of  four  layers  with  different  RIs  (n2,  n3,  n4,  n5)  and 
the  surface  of  the  top  layer  is  slanted  with  respect  to  the  optical  axis.  The  illu¬ 
minating  beam  (70)  is  refracted  by  the  surface  of  the  top  layer  (pink)  and  sub¬ 
sequent  layers  with  the  result  that  the  focal  spot  is  not  on  the  original  optical 
axis  (red  line).  Various  signals,  including  fluorescence  (/flu)>  scattering  (7S), 
reflection  (7Refl),  Raman  (7Raman),  second  harmonic  generation  in  both  forward 
and  backward  direction  (7b.Shg  and  7f-shg)>  and  third  harmonic  generation  (7THg) 
may  be  produced  when  conditions  are  favorable.  The  pink  spot  at  the  focus 
indicates  the  volume  where  nonlinear  phenomena  prevail  at  high  illumination 
intensity.  Transmission  intensity,  tTran. 


incident  light  intensity  is  very  high,  nonlinear  optical  phenomena 
such  as  nonlinear  absorption,  multi-photon  fluorescence,  and  har¬ 
monic  generation  of  incident  light  become  prevalent  (Fig.  8.1). 

It  is  well  known  that  different  RIs  can  be  associated  with  dif¬ 
ferent  crystallographic  orientations  of  crystalline  materials.  For 
example,  calcite  crystals  have  indices  of  refraction  for  the  o-  and 
e-rays  of  1.6584  and  1.4864,  respectively.  Mineral  crystals 
showing  two  distinct  indices  of  refraction  are  referred  to  as  bire- 
fringent  materials.  Birefringence  has  to  do  with  anisotropy  in  the 
binding  forces  between  the  atoms  forming  a  crystal.  A  large 
number  of  quasi-crystalline  biological  materials  also  exhibit 
birefringence  (Fig.  8.3). 


SOURCES  OF  CONTRAST 


Absorption  Contrast 

If  the  light-specimen  interaction  is  predominantly  photon  absorp¬ 
tion  (as  it  is  in  the  prepared  tissue  sections,  stained  with  absorb¬ 
ing  dyes),  and  the  specimen  has  a  uniform  thickness  [Fig.  8.4(A)], 
and  if  the  incident  light  is  70,  then  the  transmitted  intensity,  It  and 
I2  through  structures  1  and  2  in  the  voxel  being  sampled  is: 

h  =  hfi** 

and 


h  =  he™ 

where  and  p2  are  the  absorption  coefficients  of  structures  1  and 
2,  respectively,  and  x  is  the  length  of  the  absorption  path  (the  thick¬ 
ness  of  the  voxel  in  this  case).  The  contrast  due  to  absorption  iyabs) 


Li 
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FIGURE  8.3.  Birefringence  of  artificially  reconstituted  collagen  fibers. 
(Upper)  A  retardance  image  of  collagen  fibers.  (Middle)  Orientation  of  the  col¬ 
lagen  fibers  indicated  by  small  arrows.  (Lower)  Orientation  coded  as  color. 
[Images  courtesy  of  Hanry  Yu,  Department  of  Physiology,  National  University 
of  Singapore  and  made  using  a  Polscope  (Cambridge  Research  Inc.,  Cam¬ 
bridge,  MA).] 
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FIGURE  8.4.  Hypothetical  thin  (A)  and  thick  (B)  specimens  composed  of  a 
light-absorbing  matrix  with  an  absorption  coefficient  of  pt  and  containing  small 
structures  with  an  absorption  coefficient  of  p2.  The  transmission  intensities  of 
the  matrix  and  structure  are  Ii,  I2,  and  I3,  as  marked. 


However,  in  real  specimens,  the  structure  of  interest  is  gener¬ 
ally  much  smaller  than  the  thickness  of  the  specimen  and  there¬ 
fore,  it  resembles  the  situation  in  Figure  8.4(B),  the  effective 
absorption  coefficient  pT  for  a  given  light  path  is: 

m 

nr=2n, 

i= 1 


a  term  that  describes  an  absorbing  specimen  composed  of  m  dif¬ 
ferent  subunits,  each  having  an  absorption  coefficient  (p,),  where 
p-,-  is  the  absorption  coefficient  associated  with  the  light  attenua¬ 
tion  along  the  specific  light  path. 

For  a  sample  with  thickness  x,  and  the  effective  absorption 
coefficients  of  an  area  of  interest  (pT1  and  pT2),  the  absorption  con¬ 
trast  (Yafa)  is: 


Y 


abs 


A I 

I 


1  Mtj-*  _ p  Mt2* 


)/JB 


where  I  =  — - =  — — - ~  /„  and  n  is  the  number  of 

1 1  n  B 

pixels  in  the  image.  The  absorption  contrast  (jabs)  between  pixel  1 
and  2  shown  in  Figure  8.4(B)  is: 

Y abs=Io{e~'lTlX  -e~tlT2X)'eliBX 

The  Rose  criterion  (Rose,  1948)  relates  contrast  and  noise  to 
visibility.  It  states  that,  to  be  visible,  the  contrast  between  a  feature 
and  its  surroundings  must  be  5  times  the  noise  in  the  surroundings 
[signal-to-noise  ratio  (S/N)  >5:1], 

>  5 

(p2-Pi)x>ln5 


Apart  from  the  fact  that  the  optical  transfer  function  implies 
that  small  features  will  be  rendered  in  the  image  with  less  contrast 
than  large  ones  and  that  the  Poisson  noise  associated  with  record¬ 
ing  any  image  adds  uncertainty  to  the  measurement  of  contrast, 
large  features  will  also  have  low  contrast  if  the  tissue  is  only  lightly 
stained,  and  small,  intensely  stained  structures  will  have  less  con¬ 
trast  when  visualized  inside  a  thick,  stained  region  of  the  sample. 
Finally,  unstained  biological  tissues  contain  few  molecules  that 
absorb  in  the  visible  and  our  ability  to  detect  small  changes  in 
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FIGURE  8.5.  Nonlinear  absorption  of  the  up- 
converting  dye,  APSS,  and  of  an  ethanol  extract 
of  yellow  petal  of  Canna,  as  a  function  of  excita¬ 
tion  light  intensity. 


Incident  Intensity  (MW/cm2) 


absorbance  by  eye  is  poor.  For  all  these  reasons,  linear  absorbance 
contrast  is  used  mainly  on  material  that  has  been  fixed  and  stained. 

If  one  increases  the  incident  intensity,  the  optical  response  of 
the  specimen  will  eventually  begin  to  show  significant  nonlinear 
components;  in  fact,  the  optical  response  can  be  described  by  the 
following  power  series  [P(f)]: 

p(f)  =  EoZ'1’  m  +  EoX'2’  E(tf  +  e0x(3)  E(tf  +  ... 

=  Pm(t)  +  P^Xt)  +  P{\t)  +  ... 

where  %(I>  is  the  linear  susceptibility,  %(2)  is  the  second-order  non¬ 
linear  susceptibility,  and  %(3>  is  the  third-order  nonlinear  suscepti¬ 
bility.  The  effective  nonlinear  absorption  cross-section  \i{Io)ejf  is  a 
function  of  the  incident  intensity.  The  nonlinear  transmission 
intensity, 

i  trans_nonlmear  L  0° 


assuming  the  concentration  of  the  pigment  is  relatively  low,  and 
that  fluorescence  is  not  a  factor.  Figure  8.5  shows  the  effective 
absorbance,  p(/0)ej-x,  plotted  against  the  incident  illumination  for 
a  solution  of  the  up-converting  dye,  APSS  [Fig.  8.5(A)]  and  a 
methanol  extract  of  yellow  Carina  petals  [Fig.  8.5(B)],  when  illu¬ 
minated  with  780  nm  NIR  light  from  a  Ti:  sapphire  mode-lock  laser 
(85  MHz,  120fs  pulse).  If,  as  in  these  cases,  the  absorber  is  a  flu- 
orophore,  this  nonlinear  absorption  may  produce  fluorescence. 

The  absorption  properties  of  a  specimen  (color  or  opacity)  can 
have  a  negative  impact  on  image  contrast  in  the  backscattered  light 
(BSL)  mode.  For  example,  the  nearly  opaque  carbon  particles  in 
the  polyethylene  cable  material,  shown  in  Figure  8.6,  cause  a  sig¬ 
nificant  loss  in  signal  intensity  by  attenuating  both  the  illuminat¬ 
ing  beam  and  the  BSL  signal. 

Although  deep  coloration  in  biological  specimens,  such  as  that 
caused  by  pigment  granules  and  chloroplasts,  can  also  have  this 


FIGURE  8.6.  (Top)  When  the  objective  is 
focused  into  a  particle-filled  clear  matrix,  the 
scattered  and  reflected  rays  from  the  structure 
of  interest  are  modulated  by  the  particles 
located  between  the  objective  and  focal 
plane.  (Bottom)  Diagram  of  self-shadowing. 
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effect,  the  problem  can  sometimes  be  minimized  by  selecting  a  X 
that  reduces  absorption  to  a  minimum.  For  example,  when  imaging 
plant  cells,  green  illumination  will  be  absorbed  much  less  than  blue 
or  red.  As  many  biological  and  polymeric  specimens  are  very 
transparent  to  IR,  it  is  sometimes  worthwhile  surrendering  some 
resolution  by  using  a  longer  X  in  order  to  penetrate  deeper  into  the 
specimen.  For  example,  NIR  (700-1200  nm)  radiation  can  be  used 
to  study  the  internal  structure  of  insects  because  insect  cuticle  is 
relatively  translucent  in  this  region  of  the  IR  spectrum.  This 
improvement  is  due  to  lower  absorbance  and  also  to  decreased 
Rayleigh  scattering,  which  is  proportional  to  ^»4.  The  other  side  of 
the  equation  is  that  objects  of  interest  must  now  be  somewhat 
larger  to  produce  a  detectable  BSL  signal. 

Modern  confocal  microscopes  are  generally  operated  in  an  epi- 
illuminated  configuration,  both  in  fluorescence  and  BSL  mode. 
Flowever,  it  is  also  possible  to  operate  the  confocal  microscope  in 
the  trans-illumination  configuration  to  image  absorption  contrast 
(An  et  al.,  1990).  The  straightforward  approach  to  designing  a 
transmitted  confocal  microscope  is  to  mount  two  opposed  micro¬ 
scopes  on  a  common  optical  axis  with  both  objective  lenses 
focused  on  a  common  point  in  a  specimen.  One  serves  as  the  light 
source  and  the  other  as  the  detector.  Rastered  images  are  obtained 
by  scanning  the  specimen  with  an  xy  mechanical  stage. 

It  is  clear  that  such  a  design  has  some  major  drawbacks:  (1) 
The  stage  scanning  method  is  relatively  slow  and  not  suitable  for 
most  biological  specimens  because  of  the  stage  vibration  involved. 
(2)  In  a  beam-scanning  system  (either  with  a  single  laser  beam  or 
the  multiple  beams  of  a  disk-scanning  system),  it  is  technically  dif¬ 
ficult  to  synchronize  the  scanning  of  separate  illumination  and 
detection  systems  perfectly.  (3)  Because  of  differences  in  the  RI 
and  optical  path  length  within  the  specimen,  a  small  translation  of 
the  specimen  along  the  optical  axis  (z)  can  result  in  significant 
misalignment  of  the  focal  points  of  the  two  opposed  objectives, 
reducing  signal  levels.  Unfortunately,  3D  sets  of  optical-section 
images  can  only  be  obtained  by  translation  in  z  (Chapter  30,  this 
volume). 

Cheng  and  Lin  (1990)  demonstrated  a  simple  trans-illuminated 
confocal  design  utilizing  folded  optics  [Fig.  8.7(A)].  The  design 
uses  a  second  high-NA,  infinity-corrected  objective  that  is  coaxial 


FIGURE  8.7.  (A)  Diagram  of  a  double-pass,  trans-illumination  folded  optical 
setup  for  trans-illuminated  confocal  light  microscopy.  (B)  A  set  of  trans- 
illuminated  confocal  images  of  a  sea  urchin  embryo  ( S.  purpuratus).  The 
optical  sections  were  obtained  at  4  pm  apart.  The  specimen  was  fixed  in  3 : 1 
(ethanol/acetic  acid),  stained  by  the  Feulgen  method,  dehydrated,  and  cleared 
in  methyl  salicylate. 


and  confocal  with  the  primary  objective  so  that  light  transmitted 
through  the  specimen  is  captured  by  this  second  lens.  A  mirror 
normal  to  the  optical  axis  and  placed  behind  the  second  objective 
reflects  the  transmitted  light  back  to  the  first  objective  along  the 
same  path,  and  a  confocal  configuration  is  maintained  as  long 
as  the  second  objective  remains  confocal  with  the  primary 
objective. 

The  problem  is  that  any  misalignment  between  the  two  objec¬ 
tives  caused  by  moving  an  optically  heterogeneous  specimen  in 
the  x-,  y-,  or  z-direction  will  cause  a  large  reduction  in  signal,  even 
in  the  absence  of  any  absorptive  dye.  To  avoid  this,  we  have  devel¬ 
oped  a  computer-controlled,  precision  piezoelectric/mechanical 
system  that  allows  precise  positioning  of  the  second  objective  lens 
by  monitoring  intensity  changes  in  the  captured  image.  Figure 
8.7(B)  shows  two  series  of  optical  sections  of  a  sea  urchin  embryo 
( Strongylocentrotus  purpuratus )  obtained  in  the  transmitted,  con¬ 
focal  mode  using  a  laser  scanning  system.  This  set  of  images  was 
obtained  by  realigning  the  folding  optics  after  each  focus  change 
(4  pm  total).  Contrast  in  these  images  results  primarily  from 
absorption  by  the  specimen  summed  over  both  passes,  but  there  is 
also  a  small  contribution  from  BSL  and  interference  between  the 
two  beams.  The  effect  of  the  BSL  in  the  image  can  largely  be 
removed  by  subtracting  a  pure  BSL  image  from  the  absorption 
image. 

Because  the  sensitivity  for  detecting  absorptive  tags  is  inher¬ 
ently  low  and  because  any  optical  misalignment  caused  by  refrac- 
tile  organelles  produces  a  signal  reduction  that  can  be  confused 
with  such  absorption,  transmitted  confocal  microscopy  is  unlikely 
to  become  an  important  imaging  modality.  However,  this  optical 
scheme  can  be  used  for  forward-SHG  detection  in  an  epi- 
illumination  setup. 

With  this  basic  transmitted  optical  system,  a  wide  variety  of 
interference  contrast  mechanisms  can  be  implemented  by  adding 
the  appropriate  optical  components  into  the  back-focal  plane  of 
one  or  both  objectives,  or  simply  by  using  mirrors  that  cover  only 
parts  of  these  planes.  Such  images  can,  in  principle,  be  collected 
in  addition  to,  and  independently  of,  any  fluorescent  signals  (see 
Chapters  2,  9,  and  30,  this  volume).  For  example,  Cogswell  (1994) 
demonstrated  confocal  imaging  of  both  phase  and  amplitude 
objects.  To  optimize  the  optical  conditions  of  both  objectives,  the 
specimen  must  be  sandwiched  between  two  coverslips  instead  of 
a  microscope  slide  and  a  coverslip  (Cheng  et  al.,  1994). 

A  bonus  feature  of  this  transmitted  configuration  is  a  small 
improvement  in  the  signal  strength  when  the  microscope  operates 
in  epi-fluorescence  mode.  The  folded  optical  path  provides  double 
excitation  and,  in  addition,  much  of  the  fluorescent  light  captured 
by  the  lower  objective  lens  is  returned  to  the  detector.  Assuming 
that  the  dye  is  not  close  to  singlet  saturation  and  that  the  trans¬ 
mittance  of  the  second  objective  is  high,  the  system  can  increase 
fluorescence  signal  strength  2  to  3x.  Although  this  approach  may 
be  of  some  use  with  disk-scanning  microscopes  where  fluores¬ 
cence  image  intensity  is  relatively  low,  problems  related  to  main¬ 
taining  alignment  between  the  many  beams  used  in  such  a  system 
may  restrict  its  use  to  prepared  specimens  that  are  embedded  in  a 
clearing  medium.  In  addition,  as  the  folded  light  path  corresponds 
to  operating  in  transmission  fluorescence  rather  than  epi- 
fluorescence,  the  barrier  filter  in  front  of  the  detector  must  be  very 
efficient  to  exclude  stray  excitation  light  (see  Chapter  3,  this 
volume). 

As  harmonically  generated  signals  propagate  preferentially  in 
the  forward  direction,  the  trans-detection  configuration  is  the  pre¬ 
ferred  detection  layout.  However,  the  double-pass  method  does 
permit  simple,  descanned  SHG  and  THG  detection  when  beam- 
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scanning.  Sun  and  colleagues  (2005)  have  demonstrated  that 
certain  biological  specimens  also  generate  backward  SHG, 
meaning  that  conventional  epi-detection  can  also  be  used. 

Scattering  and  Reflection  Contrast 

Scattering  includes  Rayleigh,  Mie,  and  Raman  scattering.  While 
the  strongly  wavelength-dependent  Rayleigh  scattering  originates 
from  molecules,  Mie  scattering  primarily  results  from  cellular 
structures  such  as  organelles.  On  the  other  hand,  because  Raman 
scattering  is  a  non-elastic  event,  the  scattered  photon  either  loses 
or  gains  energy  from  the  vibrational  state  of  the  molecule  involved, 
and  consequently  the  scattered  light  carries  information  about  the 
state  of  the  molecule.  When  the  structure  is  much  larger  than  the 
wavelength  of  light  and  presents  a  smooth  RI  interface  or  metal 
surface,  reflection  results.  Depending  on  the  size  and  geometry  of 
the  feature  in  the  specimen  that  causes  it,  the  light  signal  that  pro¬ 
ceeds  from  the  specimen  toward  the  detector  in  an  epi-illuminated, 
widefield,  or  confocal  microscope  is  commonly  either  referred  to 
as  either  BSL  or  reflected  light. 

The  intensity  of  the  Rayleigh-scattered  light  I  varies  inversely 
with  the  4th  power  of  the  wavelength. 

/  =  /  87t4Wa2(i  +  CoS2e) 
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where  N  is  the  number  of  scatters,  a  is  polarizability,  and  R  is  the 
distance  from  the  scattered 

Mie  scattering  is  generated  mainly  by  the  organelles  in  the  cell 
and  is  not  strongly  wavelength  dependent,  but  the  scattering  inten¬ 
sity  is  proportional  to  the  square  of  the  ratio  of  the  RI  of  the  feature 
and  the  RI  of  the  media.  If  scattered  light  is  not  a  desired  signal, 
it  can  be  greatly  reduced  if  the  specimen  is  “cleared”  by  immers¬ 
ing  it  in  an  index-matched  liquid.  This  allows  one  to  image  sig¬ 
nificantly  deeper  into  the  tissue  (see  Chapter  49,  this  volume). 

When  light  is  incident  on  a  smooth  interface  separating  mate¬ 
rials  with  different  RIs,  such  as  the  water/oil  droplet,  the  reflected 
intensity  depends  on  the  incidence  angle  and  the  RI  difference.  In 
geometric  optics,  when  light  travels  from  one  medium  to  another 
of  different  RI,  the  light  separates  into  two  components:  the 
reflected  ray  and  the  refracted  ray.  The  degree  of  reflection  depends 
on  the  gradient  of  RI  between  the  two  media.  The  refracted  ray 
changes  its  propagation  angle  (0,.)  with  respect  to  the  incident  beam 
(0,0  as  described  by  Snell’s  law: 

sin0, 
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where  constant  rj2i  is  the  RI  of  the  second  medium  relative  to  the 
first  medium.  It  can  be  expressed  in  terms  of  the  refractive  indices 
of  ir  and  r|2- 

rj  i  sin  0i  =  r)2sin0r 

If  the  incident  angle  (0j)  is  greater  than  the  critical  angle  (0C), 
where 

sin0c  =  r|21  r|21  <  1 

then  total  reflection  results.  In  reflected-light  confocal  microscopy, 
the  reflectance  (i.e.,  the  ratio  of  illumination  intensity  to  reflected 
intensity)  of  the  specimen  is  one  of  the  key  factors  determining 
signal  strength.  The  reflectance  is  the  ratio  of  the  difference  of  the 
squares  of  the  refractive  indices  of  the  two  media. 
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Therefore,  the  smaller  the  difference  in  RI  between  the  two 
media,  the  lower  the  reflectance.  In  living  biological  specimens, 
the  RIs  of  most  cellular  structures  differ  very  little  from  that  of  the 
surrounding  aqueous  medium,  and  hence,  structures  inside 
biological  specimens  have  much  lower  reflectance  than  most 
materials  sciences  specimens.  For  example,  the  reflectance  of  a 
gold-coated  glass  surface  (a  mirror  or  an  integrated  circuit  chip) 
can  be  >95%  and  that  of  the  metallic  silver  deposited  in  a  Golgi- 
stained  neuron  may  be  -10%,  but  the  typical  reflectance  of  the 
surface  of  a  maize  leaf  (surface  cuticle  interface  with  air)  is 
-0.03%  and  the  reflectance  of  human  bone  imaged  under  a  con¬ 
focal  microscope  and  that  of  living  specimens,  such  as  tissue 
culture  cells,  is  even  lower.  Detection  of  reflection  from  a  curved 
surface  can  produce  image  contrast  related  to  the  curvature  and  the 
NA  of  the  objective  lens  used  (Fig.  8.8).  Despite  the  low 
reflectance,  confocal  images  of  tooth  enamel  prisms  and  Golgi- 
stained  Purkinje  cells  have  been  obtained  (Boyde,  1985).  Watson 
(1989)  demonstrated  real-time  imaging  of  an  internal  section  of  a 
tooth  being  abraded  by  a  dental  burr  by  using  a  tandem-scanning 
confocal  microscope  (TSM)  operated  in  BSL  mode. 

When  viewed  with  a  multi-channel  (or  spectral)  confocal 
microscope,  colloidal-gold  particles  as  small  as  5nm  can  be 
detected  because  they  appear  much  brighter  under  green  laser  illu¬ 
mination  than  under  blue  or  red.  This  is  because  the  molecular  res¬ 
onance  properties  of  gold  particles  cause  them  to  Rayleigh  scatter 
green  light  more  strongly  than  other  visible  wavelengths  (Born  and 
Wolf,  1980).  As  a  result,  it  is  possible  to  ratio  the  images  obtained 
from  the  blue  and  green  channels  to  identify  the  colloidal  gold 
specifically  (Cogswell,  1994).  This  technique  can  be  used  to 
study  the  colloidal  gold-labeled  samples  prior  to  fixation  for  EM 
study. 

When  light  passes  through  matter,  some  of  it  is  scattered  in 
random  directions  with  a  small  energy  gain  or  loss  due  to  Raman 
scattering.  If  the  initial  beam  is  sufficiently  intense  and  very  mono¬ 
chromatic,  a  threshold  can  be  reached1  beyond  which  light  at  the 
Raman  frequencies  is  amplified  and  the  output  generally  exhibits 
the  characteristic  of  stimulated  emission.  Raman  spectroscopy 
is  a  powerful  analytical  method  in  microscopy.  As  early  as  1992, 
Sharonov  and  colleagues  (1992)  used  it  in  confocal  microscopy. 
Image  contrast  in  a  Raman  confocal  image  can  result  either  from 
intensity  variations  at  a  selected  energy  level  or  from  variations  in 
the  Raman  spectra  obtained  from  different  optical  sections.  In  the 
latter  case,  the  basic  image  unit  in  such  an  xyk  image  is  referred 
to  as  “lambxel”  (Cheng  et  al,  1994). 

Because  light  reflecting  from  inclined  surfaces  may  be 
deflected  away  from  the  objective  lens,  BSL  images  of  silver- 
impregnated  nerve  cells  appear  punctuate.  Even  though  the  den¬ 
drites  are  actually  continuous  structures,  the  silver  grains  with 
which  they  are  decorated  are  not.  Therefore,  small  “gaps”  between 
nerve  fibers  seen  in  the  confocal  BSL  images  may  not  indicate  a 
lack  of  continuity  but  only  that  the  reflecting  surface  is  tilted  too 
steeply  for  the  reflected  light  to  reach  the  objective. 

Golgi-stained  Purkinje  cells  show  strong  contrast  between  the 
surface  reflections  from  the  silver-decorated  cells  and  the  weak 
scattering  by  the  surrounding  non-impregnated  tissues.  Golgi 
preparations  are  of  interest  primarily  because  the  method  stains 
only  1%  to  5%  of  the  neurons  in  the  preparation  (Barr  and  Kieman, 
1988:  Scheibel  and  Scheibel,  1970).  This  creates  a  clear  image  of 
the  dendritic  tree  of  a  single  neuron  hundreds  of  micrometers  in 


1  Something  more  likely  to  occur  near  the  focus,  where  intensity  is  highest,  a 
feature  that  gives  the  technique  some  j-resolution. 
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FIGURE  8.8.  The  effect  of  NA  on  the  fraction  of 
the  light  reflected  from  a  sphere  that  is  detected.  A 
projection  of  confocal  images  made  with  reflected 
light  (A)  and  autofluorescence  (B),  of  the  apical 
meristem  of  maize.  (A-C)  were  made  using  an  NA 
=  0.9  dry  objective.  (C)  is  a  combined  image.  (D-F) 
The  same,  using  an  NA  =  0.4  dry  objective  (magni¬ 
fication  adjusted).  Note  that  the  region  of  detectable 
surface  reflection  is  much  smaller  than  that  from 
which  fluorescence  can  be  detected  and  is  lower  with 
the  low  NA  objective.  (G-J)  Diagrams  that  show 
how  the  NA  and  the  height  of  the  focus  plane  affects 
the  fraction  of  a  spherical  surface  from  which 
reflected  light  can  be  collected  by  the  objective  lens 
(as  defined  by  the  red  and  green  rays). 


depth.  If  all  the  neurons  stained,  optical  sectioning  would  be 
impossible  because  the  high  reflectivity  would  prevent  light  from 
penetrating  much  below  the  outer  surface. 

BSL  images  of  living  biological  specimens  present  the  same 
problems.  Image  contrast  degrades  as  the  objective  lens  focuses 
deeper  into  the  tissue.  Interfacial  reflections  from  the  various  cel¬ 
lular  components  increasingly  degrade  the  beam  as  it  passes 
through  more  overlying  cells.  In  addition,  the  deeper  the  micro¬ 
scope  is  focused  into  the  specimen,  the  more  heterogeneous  the 
“medium”  between  the  focal  plane  and  the  objective  becomes, 
and  the  more  that  refraction  produced  by  these  inhomogeneities 
degrades  the  optical  performance  of  the  microscope.  Figure  8.9 
shows  a  BSL  image  of  part  of  an  amoebic  pseudopod.  The  surface 
of  the  pseudopod,  the  mitochondria,  vacuoles,  and  a  number  of 
other  organelles  and  cytoplasmic  crystals  produce  significant  BSL. 
Chen  and  Cheng  (1989)  used  the  BSL  mode  to  image  the  macro¬ 
nuclei  of  Stentor  coeruleus. 

Although  the  low  reflectance  of  many  living  animal  specimens 
makes  is  possible  to  see  some  distance  into  them,  the  low  signal 
level  that  results  complicates  real-time  BSL  imaging  of  moving 
specimens.  However,  in  plant  cells,  the  interface  reflections  may 
be  so  strong  that  they  bend  the  beam  and  produce  “doubled” 
images  in  extended-focus  (projection)  views.  This  deflected  illu¬ 
mination  beam  can  also  excite  fluorophores  to  produce  spurious 
fluorescent  signals  (Fig.  8.10,  arrows). 

When  two  reflective  surfaces  are  in  close  proximity,  interfer¬ 
ence  fringes  can  occur.  Although  these  fringes  may  be  useful  for 
measuring  height,  they  also  complicate  image  interpretation.  Inter¬ 
ference  contrast  can  be  used  to  study  focal  contacts  (regions  of 
close  contact  between  the  cell  membrane  with  supporting  substra¬ 


tum)  in  cultured  animal  cells.  Figure  8.11  shows  the  interference 
fringes  generated  by  the  opposing  surfaces  of  the  plasma  mem¬ 
brane  of  an  amoeba  and  the  cover  glass. 

In  the  materials  sciences,  Shinozald  and  colleagues  (1991) 
used  such  interference  fringes  to  study  the  delamination  of  a  Cu 
thin  film  from  an  underlying  polyimide  substratum  during  a 
mechanical  tensile  test.  This  technique  can  also  be  used  for  detect¬ 
ing  adhesion  of  transparent  materials  to  the  underlying  support 
matrix.  Figure  8.12  shows  the  delamination  of  a  polyethylene  film 
from  glass  in  a  micro-indenter  test.  Local  detachment  of  the  film 
from  the  underlying  glass  substratum  results  in  pronounced  inter¬ 
ference  fringes. 

Figure  8.13  shows  the  surface  of  an  ancient  rice  grain  imaged 
by  BSL  confocal  microscopy.  The  specimen  is  an  artifact  from  an 
archeological  site  in  southern  Taiwan  and  contains  mainly  the 
carbonized  skeleton  of  the  endosperm.  No  trace  of  DNA  was 
detectable  in  the  grain.  The  double  image  of  the  surface  actually 
represents  separate  images  of  the  top  and  bottom  surfaces  of  the 
cell  wall  (Fig.  8.13,  xz-  and  vz-images).  When  imaging  the  inter¬ 
face  of  a  dense  block  in  a  transparent  medium,  as  shown  in  Figure 
8.14,  edge  effects  become  prevalent.  The  vertical  edge  can  be 
imaged  in  great  depth  but  not  the  internal  structures  of  the  dense 
block.  Figure  8.14(B)  shows  both  the  fluorescent  and  BS  images 
of  the  edge  of  a  maize  stem  (cross-section)  mounted  in  water. 

In  disk-scanning  confocal  microscopes  that  utilize  single-sided 
scanning,  image  contrast  is  reduced  if  light  reflected  by  the  disk 
surface  is  inadequately  trapped.  This  increase  in  background  signal 
level  is  particularly  serious  when  the  reflectance  of  the  specimen 
is  low.  On-axis  reflections  of  coherent  light  from  optical  surfaces 
are  a  serious  problem  when  using  a  confocal  microscope  in  the 
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FIGURE  8.9.  Two-channel  confocal  images  of  the 
pseudopod  of  an  amoeba  ( Chaos  carolinense ) 
showing  mitochrondria  (m)  and  vacuoles  in  red  flu¬ 
orescence  and  backscattered  light  in  green.  The 
large,  highly  reflective  structures  are  crystals  (Cy)  in 
the  cytoplasm.  A  central  scattering  “core”  (arrow) 
can  be  found  in  the  spherical  vacuoles  (for  an  expla¬ 
nation,  please  see  Fig.  8.19).  (Images  by  Y.  Deng, 
Z.  Almshergi,  and  P.C.  Cheng,  Departments  of 
Physiology  and  Biological  Sciences,  National  Uni¬ 
versity  of  Singapore.) 


FIGURE  8.10.  (A)  Diagram  of  a  possible  situation  that  can  generate  a  double  image  from  a  fluorescent  organelle.  In  certain  botanical  specimens,  the  cyto¬ 
plasm-cell  membrane  (cell  wall)  interface  acts  as  an  efficient  reflective  mirror,  producing  paired,  mirror  images  of  bright  structures  in  an  optical  slice.  In  some 
cases,  the  position  of  the  imaged  organelle  may  be  very  different  from  the  expected  focal  plane.  (B)  An  extended  focus  (projection)  view  of  an  A.  thaliana  leaf. 
Note  the  double-imaged  GFP  organelles  in  the  epidermal  cells.  The  chloroplast-containing  cells  are  mesophyll  cells,  situated  below.  (Image  *courtesy  of  M.  and 
D’ A  Weis,  Agriculture  Canada  Laboratory,  Summerland,  BC.) 
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FIGURE  8.1 1 .  The  separation  between  the  plasma  membrane  of  an  amoeba  and  the  coverslip  can  be  monitored  by  the  interference  patterns  generated  by  light 
reflected  from  these  two  surfaces.  (A-F)  time  series  showing  movement  of  the  plasma  membrane.  {Green,  reflected/backscattered  light;  red,  Mitotracker 
fluorescence.) 


FIGURE  8.1 2.  Delamination  of  plastic  film  from  its  substrate  can  be  imaged  by  reflective  confocal  microscopy.  (A)  Focused  near  the  surface  of  the  polyethyl¬ 
ene  film,  (B)  and  (C)  focused  near  the  plastic-glass  interface.  The  interference  patterns  provide  a  good  measure  of  the  gap  resulting  from  delamination.  (D) 
Diagram  of  the  experimental  setup.  A  micro-indenter  was  used  to  deform  the  plastic,  causing  delamination.  (Specimen  courtesy  of  D.M.  Shinozaki,  Department 
of  Mechanical  Engineering,  University  of  Western  Ontario,  London,  Canada.) 
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FIGURE  8.13.  The  surface  of  an  ancient  rice  grain  imaged 
by  BSL  confocal  microscopy.  The  specimen  is  an  artifact 
from  a  southern  Taiwan  archeological  site  and  contains 
mainly  the  carbonized  skeleton  of  the  endosperm.  (A)  xz 
view,  (B)  extended  focus  view  (projection  view),  (C)  xy 
optical  section,  (D)  yz  view.  Note  the  double-line  in  the  xz 
and  yz  image.  (E)  Diagrammatic  representation  of  the 
imaging  condition.  (Specimen  kindly  provided  by  Yue-Ie 
Caroline  Hsing,  Institute  of  Plant  and  Microbial  Biology, 
Academia  Sinica,  Taipei,  Taiwan.) 


BSL  mode.  The  result  is  a  bright  spot  or  a  series  of  diffraction 
fringes  in  the  center  of  the  image  (see  Fig.  8.42,  discussed  later). 
Although  this  artifact  can  be  avoided  by  panning  the  scanned  area 
away  from  the  optical  axis,  this  approach  sacrifices  the  field  of 
view  and  degrades  the  image  by  using  off-axis  portions  of  the 
optical  system  in  which  the  image  is  less  well  corrected  (see 
Chapter  11,  this  volume ). 

In  the  single-sided  confocal  microscopes  constructed  by  Kino 
(Xiao  et  al.,  1988)  and  by  us  (Cheng  et  al,  1989),  the  light 
reflected  from  the  surface  of  the  spinning  disk  and  from  optical 
surfaces  is  removed  by  using  a  Glan-Taylor-type  polarizer  to 
achieve  high  polarization  extinction  (10-5)  of  the  illuminating 
laser  beam,  and  adding  a  1/4-wavelength  retardation  plate  to  the 
optical  path  just  above  the  objective  to  convert  the  linearly  polar¬ 
ized  illuminating  beam  to  circularly  polarized  light.  With  an  ana¬ 
lyzer  placed  just  in  front  of  the  detector,  this  arrangement  removes 
most  unwanted  reflections  although  there  is  still  a  small  amount  of 
reflection  from  the  front  surface  of  the  objective  lens. 

Of  greater  concern  is  the  fact  that  on-axis  reflections  from 
optical  elements,  or  from  the  surface  of  the  coverslip  or  the  slide, 
can  interrupt  mode-locked  operation  of  ultrafast  lasers.  This  situ¬ 
ation  can  be  avoided  by  inserting  a  Faraday  rotator  and  1/4-wave 


plate  into  the  illumination  beam  to  prevent  the  reflected  beam  from 
re-entering  the  laser  cavity  (see  also  the  discussion  of  Antiflex 
techniques  in  Figs.  2.7  and  10.5,  this  volume ). 

Phase  Contrast 

Because  the  human  eye  [or  the  charge-coupled  device  (CCD) 
camera]  is  sensitive  only  to  changes  in  amplitude,  phase  variations 
produced  by  the  specimen  must  be  converted  by  the  optical  system 
into  amplitude  variations  for  them  to  become  visible.  For  example, 
phase  contrast  and  differential  interference  contrast  (DIC)  convert 
specimen-induced  phase  shifts  of  the  illuminating  light  into  ampli¬ 
tude  contrast  at  the  final  image  plane.  Most  biological  specimens 
are  3D  phase  objects  consisting  of  many  compartments,  each  with 
a  different  RI.  Apart  from  generating  contrast,  these  structures  also 
distort  and  shift  the  wavefront  of  the  illuminating  beam,  resulting 
in  complex  interference  patterns.  Both  phase  and  DIC  are  fre¬ 
quently  used  in  conjunction  with  confocal  microscopy  to  obtain 
the  general  morphology  of  the  specimen.  Because  DIC  has  good 
z  resolution,  3D  digital  image  restoration  of  DIC  image  stacks  is 
now  commercially  available  (AutoQuant,  Troy,  NY;  see  also 
Chapters  2  and  24,  this  volume). 


172  Chapter  8  •  P.-C.  Cheng 


FIGURE  8.14.  Edge  effect.  (A,  B,  C)  Optical  cross-sections  (xy)  of  the  edge  of  a  maize  stem  at  various  depths.  (D)  xz  section  of  the  image  stack  showing  the 
highlighted  edge.  Green,  BS;  red,  autofluorescence  (FL).  (E)  Diagram  of  the  imaging  situation.  (Note  the  edge  of  the  maize  stem  is  actually  tilted  with  respect 
to  the  optical  axis  although  the  diagram  shows  a  vertical  edge.) 


Fluorescence  Contrast 

When  one  images  emission  signals,  such  as  those  produced  by  flu¬ 
orescence  microscopy,  for  a  given  A /,  one  can  increase  the  con¬ 
trast  by  lowering  /.  In  a  typical  epi-fluorescence  microscopy,  the 
background  intensity  from  scattered  illuminating  light  is  removed 
by  the  dichroic  beam-splitter  and  the  barrier  filter  (see  Chapter  3, 
this  volume).  In  the  absence  of  diffuse  staining,  this  darkfield 
optical  system  produces  a  very  low  I.  This  increases  the  contrast 
between  the  background  and  very  weakly  fluorescing  structures,  a 
feature  that  has  ensured  the  popularity  of  this  technique  for  the  past 
40  years. 

The  factors  affecting  the  detectability  of  fluorescence  from  a 
voxel  are  the  intensity  of  incident  radiation,  the  intensity  of  the 
fluorescence  (itself  a  function  of  the  concentration  of  the  fluo- 
rochrome,  its  absorption  cross-section  and  its  quantum  yield  (QY) 
at  a  given  excitation  wavelength),  self-absorption  within  the  voxel 
and  between  the  voxel  and  objective,  and  the  effective  NA  of  the 
objective.  Additional  background  signal  may  reach  the  detector 
because  of  imperfections  in  the  dichroic  elements  and  overlap 


between  the  absorption  and  emission  spectra  of  the  dyes  involved 
(see  Chapter  36,  this  volume). 

In  multi-photon  fluorescence  microscopy,  the  axial  and  trans- 
verse-contrast  is  the  result  of  the  3D  energy  gradient  at  the  focal 
point,  the  concentration  and  nonlinear  QY  of  fluorophores,  and 
background  signal  level  under  multi-photon  excitation.  Readers 
are  kindly  referred  to  Chapters  16,  28,  and  40  for  a  detailed  dis¬ 
cussion  of  fluorescence  imaging. 

Because  of  its  exquisite  sensitivity  and  specificity,  fluores¬ 
cence  microscopy  is  an  extremely  important  tool  in  the  biological 
sciences.  In  widefield  fluorescence  microscopy,  the  detected  image 
data  includes  signal  from  both  in-focus  and  out-of-focus  sub¬ 
volumes.  This  often  produces  an  image  that  has  excessive  haze  and 
lacks  contrast.  As  this  haze  or  flare  is  proportional  to  the  thickness 
and  stain  density  of  the  specimen,  conventional,  widefield  epi-flu- 
orescence  works  best  when  imaging  thin  specimens  such  as  tissue 
culture  cells.  In  confocal  imaging,  the  signals  from  out-of-focus 
features  are  almost  completely  removed,  resulting  in  an  image  with 
exceptionally  high  contrast  (Fig.  8.15;  Wijaendts  van  Resandt  et 
al,  1984;  White  et  al,  1987,  Sheppard,  1993).  In  addition,  modern 
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FIGURE  8.15.  Series  of  confocal  optical  sections  of  the  first  mitotic  division  of  a  sea  urchin  embryo  in  which  the  microtubules  have  been  labeled  with  a  fluo¬ 
rescent  dye. 


3D  image  processing  and  restoration  can  further  improve  the  S/N 
and  contrast  parameters  of  the  image  (see  Fig.  8.40  in  the  Decon¬ 
volution  section  below). 

Contrast  Related  to  Excitation 
Wavelength  Change 

Biological  specimens  often  naturally  contain  a  large  number  of 
fluorescent  molecules  that  produce  autofluorescence.  Because  each 
fluorescent  species  has  a  unique  absorption  and  emission  spectra 
as  well  as  a  set  QY,  the  total  autofluorescence  spectrum  can  vary 
with  the  excitation  wavelength.  Figure  21.13  shows  the  changes  in 
the  fluorescence  spectra  obtained  from  Canna,  produced  by  using 
different  excitation  wavelengths  from  350nm  to  514nm.  Figure 
8.16  demonstrates  this  effect  on  the  root  tip  of  Arabiclopsis 
thaliana.  Although  the  signal  levels  are  normalized  in  the  515nm 
detection  window,  the  fluorescence  excited  at  477  lim  (yellow 
label)  produces  far  less  signal  longer  than  547  nm  than  does  that 
excited  using  either  458  nm  or  488  nm.  The  false-color  images  in 
Figure  8.16  clearly  demonstrate  the  differences  in  signal  level 
when  the  root  tip  is  excited  by  various  wavelengths. 

When  attempting  to  detect  an  exogenous  fluorophore,  one  can 
often  reduce  the  background  level  by  picking  exactly  the  right 
excitation  wavelength.  Therefore,  to  determine  the  optimal  excita¬ 
tion  and  emission  bandwidths,  one  must  know  the  excitation 
spectrum  of  both  the  background  autofluorescence  and  the 
tagged  fluorescent  probe(s).  Figure  8.17  shows  how  images 


obtained  in  different  emission  bands  can  vary  in  appearance 
and  contrast.  The  emission  spectrum  shown  in  Figure  8.16  indi¬ 
cated  the  presence  of  BSL  and  fluorescence  from  GFP  and  chloro¬ 
phylls  (Chi).  Figure  8.18  is  an  extended-focus  fluorescence  image 
of  an  alga  showing  how  using  different  emission  filters  [green 
bandpass  (G)  and  red  longpass  (R)]  changes  the  appearance  of  a 
chloroplast. 

Negative  Contrast 

Small,  isolated,  non-fluorescent  structures  can  be  imaged  with  flu¬ 
orescence  confocal  microscopy  by  filling  the  surrounding  medium 
with  a  fluorescent  solution.  This  dye-excluding  technique  was 
originally  used  in  the  study  of  polyethylene  crystallites  in  xylene 
and  can  also  be  used  to  outline  small  crystal  deposits  and  the  extra¬ 
cellular  compartment  in  biological  specimens.  Because  dextrans 
are  relatively  inert  and  exhibit  low  toxicity,  they  are  ideal  markers 
to  probe  either  extracellular  or  intracellular  compartments.  In  the 
material  sciences,  dye-filling  techniques  are  frequently  used  to 
provide  contrast  when  studying  micro-cavities  (Harridose  et  al. , 
1991;  Shinozaki  et  al.,  1991;  Cheng  and  Kriete,  1995). 

Negative  contrast  can  also  be  used  in  a  number  of  biological 
applications.  Figure  8.19  shows  that  the  vacuoles  in  cells  of  the 
peony  flower  petal  are  sufficiently  autofluorescent  to  reveal  cyto¬ 
plasmic  strands  traversing  them  as  dark  threads  penetrating  this 
fluorescent  sea.  After  acquisition,  the  fluorescence  image  can  be 
inverted  for  3D  reconstruction. 
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FIGURE  8.16.  GFP  in  the  root  tip  of  A.  thaliana,  showing  image  contrast  variation  in  response  to  different  excitation  wavelengths.  For  each  excitation  wave¬ 
length  ( yellow  labeling )  at  458  nm,  477  nm,  and  488  nm,  a  series  of  spectral  sections  was  obtained  at  lOnm  intervals  using  a  Zeiss  LSM510  Meta  confocal  micro¬ 
scope  with  the  channel  gain  adjusted  for  normalized  spectral  response.  The  GFP  emission  peaks  at  505  to  515nm,  and  shows  little  variation  between  the  three 
excitation  wavelengths.  However,  the  autofluorescence  image  (i.e.,  that  at  wavelengths  >547  nm)  shows  significant  variations  at  the  different  excitation  wave¬ 
lengths.  The  three  color  rows  show  the  same  images  in  false  color  to  emphasize  low-intensity  signals.  (Images  obtained  at  the  2005  International  Course  on  3D 
Microscopy  of  Living  Cells  at  the  University  of  British  Columbia,  Vancouver,  Canada.) 


Figure  8.20(A)  shows  the  fluorescence  spectra  (excited  at  dif¬ 
ferent  wavelengths)  of  a  zebrafish  expressing  green  fluorescent 
protein  (GFP)  and  DsRed  in  its  liver  and  pancreas.  Using  these 
spectra,  we  can  understand  the  contrast  and  relative  signal  strength 
that  will  be  produced  when  particular  excitation  wavelengths  and 
filters  are  used.  Figure  8.20(B)  shows  how  the  ratio  of  GFP  signal 
to  DsRed  signal  varies  as  a  function  of  filter  bandwidth.  When  the 
total  fluorescence  emission  spectrum  represents  the  sum  of  the 
overlapping  spectra  of  a  number  of  different  types  of  fluorescent 
molecule  (a  situation  typical  of  autofluorescence),  the  contrast  may 
be  improved  by  carefully  selecting  the  excitation  and  detection 
wavelengths  and  bandwidths.  The  differences  in  photobleaching 
dynamics  among  different  fluorophores  may  cause  the  contrast 
between  two  channels  to  vary  during  the  course  of  a  time-series 
image  stack.  Figure  8.21(A)  shows  the  photobleaching  spectra  of 
a  zebrafish  expressing  GFP  (512nm)  and  autofluorescence.  In  this 
case,  both  GFP  and  the  autofluorescent  species  in  the  sample 
exhibit  similar  bleaching  dynamics,  resulting  in  a  nearly  constant 
intensity  ratio  (contrast)  over  the  time  course  of  the  study  [Fig. 
8.21(B)].  Figure  8.22  shows  an  extended  focus  view  of  a  zebrafish 
specimen  showing  green  fluorescence  from  the  pancreas  and  red 
fluorescence  from  the  liver.  In  Figure  8.22(A)  one  can  see  that  the 
liver  is  bisected  into  two  lobes  by  the  pigment  cells  [Fig.  8.22(B)]. 
This  self-shadowing  phenomenon  is  commonly  observed  in  con- 
focal  optical  sections  [Fig.  8.22(C,D)]. 


Special  Concerns  in  Ultraviolet  and 
Near-Infrared  Range  Confocal  Microscopy 

A  micrograph  is  a  plot  of  the  intensity  of  some  signal  versus  posi¬ 
tion.  Changes  in  this  signal  represent  contrast.  But  what  if  there  is 
an  error  in  the  position?  If  an  optical  system  focuses  two  wave¬ 
lengths  from  the  same  location  on  the  object  into  two  different 
locations  in  the  image,  then  we  have  created  some  contrast  that 
should  not  exist.  This  can  happen  if  the  optical  system  is  not 
achromatic. 

Although  most  objective  lenses  are  properly  corrected  for 
chromatic  errors  in  the  visible  spectrum,  it  is  wise  to  check  their 
performance  (Fig.  8.23).  Normally,  the  wavelength  difference 
between  the  illumination  and  the  detection  only  becomes  a  serious 
concern  in  epi-fluorescence  confocal  microscopy  when  one  wave¬ 
length  is  outside  the  visible  range.  When  the  excitation  wavelength 
is  in  the  UV,  one  must  use  specially  designed  UV  objective  lenses 
that  are  both  chromatically  corrected  and  also  transparent  for 
both  the  excitation  and  emission  wavelengths  (Wells  et  al.,  1990; 
Cheng  et  al.,  1994).  Similar  conditions  apply  when  detecting  IR 
fluorescence. 

The  situation  is  a  bit  different  in  multi-photon  fluorescence  and 
harmonic  generation  microscopy  where  the  fluorescence  or  har¬ 
monic  signal  intensity  is  proportional  to  the  square  of  the  illumi¬ 
nation  intensity.  Assuming  the  fluorescence  is  not  near  saturation, 
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FIGURE  8.17.  Spectral  confocal  image  of  a  young  seedling  of  A.  thaliana  showing  GFP  and  autofluorescence  from  several  chlorophylls.  The  specimen  was 
excited  at  488  nm.  The  spectrum  was  obtained  from  the  circled  region.  The  489  nm  image  (saturated)  represents  reflections  and  scattering  of  the  specimen  and 
cover  glass.  Based  on  the  spectrum,  the  5 1 1  nm  signal  results  from  GFP  while  the  other  “green”  fluorescence  is  the  autofluorescence  background.  The  image 
contrast  for  the  chlorophyll  autofluorescence  depends  on  the  wavelength  of  the  detecting  band  (650nm-703nm).  (Images  obtained  with  a  Zeiss  510  Meta. 
Specimen  kindly  provided  by  Shu-Hsing  Wu,  Institute  of  Plant  and  Microbial  Biology,  Academia  Sinica,  Taipei,  Taiwan.) 


FIGURE  8.1 8.  Autofluorescent  image  of  the  chloroplast  of  an  alga.  (A)  Two-channel  false-color  image,  (B)  green  channel  (em:  540nm  bandpass),  and  (C)  red 
channel  (em:  long-pass  >600 nm).  Note  the  difference  in  image  contrast  between  (B)  and  (C). 
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FIGURE  8.19.  Two-channel  confocal  image  of  a  peony  petal.  The  strong  autofluorescence  of  compounds  in  the  vacuoles  (red)  allows  one  to  image  the  fine 
(dark)  cytoplasmic  strands  that  traverse  them,  in  negative  contrast. 


A  Wavelength  (nm)  B  Excitation  Wavelength  (nm) 


FIGURE  8.20.  Emission  spectra  of  GFP  and  DsRed  in  zebrafish  (pancreas  and  liver,  respectively)  when  excited  at  different  wavelengths  (ranging  from  400 nm 
to  520  nm).  Two  hypotheical  bandpass  filters  (green  and  pink  bands)  were  used  to  separate  the  emission  for  imaging.  In  (B),  the  contrast  ratio  of  these  two  chan¬ 
nels  are  compared  as  a  function  of  filter  bandwidth. 
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FIGURE  8.21.  (A)  Emission  spectra  of  GFP  (pancreas)  and  autofluorescence  in  zebrafish  when  excited  using  wavelengths  from  400nm  to  520nm.  (B)  Con¬ 
trast  ratios  of  the  two  channels  when  using  filters  having  different  passbands. 
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FIGURE  8.22.  (A)  Two-channel  confocal  projec¬ 
tion  images  of  zebrafish  embryos,  with  a  gray,  non- 
confocal  transmission  image  overlay,  showing 
pancreas  (GFP).  liver  (DsRed),  and  pigment  cell 
(arrow).  (B)  The  fluorescence  images  only.  Note  the 
liver  is  bisected  into  two  lobes  by  the  high  absorp¬ 
tion  of  the  overlying  pigment  cells.  (C)  Embryo  at  a 
later  stage,  from  above  (D)  imaging  from  the  side 
can  reduce  the  masking  effect  of  the  dorsally  located 
pigment  cells. 


nonlinear  signal  generation  is  limited  to  the  vicinity  of  the  focal 
spot,  and  non-decanned  detection  can  be  used.  In  this  case,  chro¬ 
matic  correction  from  the  excitation  to  the  emission  wavelength  is 
superfluous  because  signal  is  only  emitted  from  the  focal  region 
and  therefore  need  not  be  focused.  However,  if  one  wishes  to 
detect  simultaneously  several  flurophores  with  large  differences  in 
QY,  the  fluorescence  emission  volume  can  be  significantly  differ¬ 
ent  for  each  fluorophore. 

If  one  uses  a  beam-scanning  confocal  unit  with  a  detection 
pinhole  to  limit  the  size  of  the  detection  sub-volume,  one  must  use 
an  objective  corrected  for  both  the  excitation  and  emission  wave¬ 
lengths  but  one  can  adjust  the  size  and  alignment  of  the  pinholes 
for  each  detection  channel  to  ensure  that  all  signals  are  obtained 
from  the  same  sub-volume.  Such  adjustments  are  particularly 
important  in  co-localization  studies.  Because  the  refractive  index 
of  the  specimen  (or  medium)  varies  with  wavelength,  operation 
outside  the  waveband  for  which  the  objective  is  corrected  can 
produce  RI  mismatches  that  can  introduce  spherical  as  well  as 
chromatic  aberration  (Fig.  8.23). 

Total  Internal  Reflection  Contrast 

The  distance  separating  the  flat  surface  of  a  glass  plate  (such  as  a 
coverslip)  and  a  cellular  structure  having  an  RI  higher  than  that  of 
water  and  can  be  measured  using  total  internal  reflection 
microscopy  (TIRM).  Changes  in  this  distance  as  small  as  1  nm  can 
be  detected  (Prieve  and  Walz,  1993).  When  a  structure  settles  near 
a  glass/water  interface  at  which  total  internal  reflection  is  occur¬ 


ring,  some  of  the  evanescent  wave  will  be  scattered  by  the 
structure,  as  diagrammed  in  Figure  8.24(A).  This  situation  is 
called  “frustrated”  total  internal  reflection.  Because  of  the  non- 
uniform  illumination  of  the  water  by  the  evanescent  wave,  the 
amount  of  light  scattered  by  the  structure  is  exquisitely  sensitive 
to  its  proximity  to  the  interface.  This  has  been  exploited  to  delin¬ 
eate  focal  contacts  between  cells  and  their  substrates  (Paddock, 
1989). 

When  a  scattering  object  is  illuminated  by  an  evanescent  wave, 
the  intensity  of  scattered  light  is  a  measure  of  the  separation  dis¬ 
tance.  The  penetration  depth  (P_1)  of  the  evanescent  wave  is: 

P  =  sin0,)2  - «| 

where  X  is  the  wavelength  of  the  incident  light,  Tp  and  Tj2  are  the 
RIs  of  the  coverslip  and  the  fluid  containing  the  particle,  and  9,  is 
the  angle  of  incidence  of  the  illumination  in  the  plate.  The  inten¬ 
sity  of  the  light  scattered  by  the  cell,  /,  decays  with  the  intensity 
of  the  evanescent  wave.  The  scattered  intensity  I(x)  varies  with  the 
separation,  x,  as: 

I(x)  »  V* 

Typically  P_1  is  about  lOOnm  when  a  glass  coverslip  is  used 
as  the  “plate”  and  water  is  used  for  the  bathing  medium  of  a 
cellular  “particle,”  having  an  RI  of  from  1.4  to  1.5  (proteins  and 
lipids).  Because  of  the  exponential  relationship,  a  very  small 
change  in  x  produces  a  measurable  change  in  intensity.  In  a  typical 
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FIGURE  8.23.  Chromatic  aberration.  (A)  Optical  sections  of  a  latex  bead  with  its  shell  labeled  with  fluorescent  dyes  (Tetra-spec,  Molecular  Probes).  The  image 
was  obtained  with  two  excitation  wavelengths  and  recorded  in  two  fluorescent  channels.  The  separation  of  the  “rings”  in  the  images  made  at  the  three  excita¬ 
tion  wavelengths  demonstrates  that  chromatic  aberration  is  present.  Note  that,  at  the  northern  hemisphere,  the  red  ring  is  outside  the  green  ring,  but  that  the  order 
changes  in  the  southern  hemisphere.  Although  there  is  also  significant  chromatic  separation  at  the  equator,  it  is  difficult  to  visualize  in  a  single  optical  section 
because  the  aberrated  rings  are  all  about  the  same  size.  (B)  Diagram  of  the  imaging  situation  for  (A).  If  only  one  excitation  wavelength  is  used  (C).  Chromatic 
aberration  primarily  produces  only  misfocus  at  the  detecting  pinhole,  resulting  in  signal  loss  of  the  “out-of-focus”  rays.  (D)  The  resulting  images  show  less 
co-localization  error.  (Images  obtained  on  a  Leica  confocal  microscope  with  objective  lens  corrector  collar  intentionally  set  wrongly.  M.  Weis  and  PC.  Cheng, 
Agriculture  Canada,  Summerland,  British  Columbia,  Canada,  and  State  University  of  New  York  at  Buffalo.) 


FIGURE  8.24.  Total  internal  reflection  microscopy  (TIRM).  (A)  An  evanescent  wave  generated  at  the  glass/water  surface  can  be  scattered  by  particles  near 
the  surface.  The  scattering  intensity  7d  is  a  function  of  the  distance  between  the  surface  and  the  particle.  (B)  TIRM  image  of  a  tissue-culture  cell  adhered  to  a 
coverslip. 
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biological  imaging  setup,  Tp  =  1.5  (glass  slide),  and  r\2  =  1.33  for 
the  aqueous  solution  bathing  the  cell,  a  1%  change  in  signal  inten¬ 
sity  represents  a  1  nm  change  in  distance  ( x ).  It  is  important  to  note 
that  the  equation  for  I(x)  holds  for  all  separations  including  the 
contact  situation  (x  =  0).  Figure  8.24(B)  shows  a  TIRM  image  of 
a  tissue-culture  cell  attached  to  a  glass  coverslip  surface.  The 
image  intensity  is  a  function  of  the  separation  between  the  cell 
membrane  and  the  coverglass.  Although  interpreting  TIRM  images 
obtained  from  3D  scattering  objects  is  often  difficult  (Van  Labeke 
et  al.,  1995;  Carminati  and  Greffet,  1995a,b),  recently,  Carney 
and  Schotland  (2001)  derived  a  theoretical  foundation  for  three- 
dimensional  TIRM. 

Harmonic  Generation  Contrast 

Recently,  a  number  of  new  imaging  modalities  based  on  nonlinear 
optical  phenomena  have  been  developed  (Cheng  et  al.,  2001,  2002, 
2003;  Chu  et  al.,  2001,  2003;  Cheng  et  al.,  2002).  From  the  power 
series  equation  above,  one  can  see  that  the  second-  and  third-order 
nonlinear  susceptibility,  /(2)  and  %(3)  (respectively)  determine  the 
contrast  formation  mechanism  in  nonlinear  microscopy.  Second 
harmonic  generation  can  be  considered  to  be  an  interaction  involv¬ 
ing  the  exchange  of  photons  between  the  various  frequency  com¬ 
ponents  of  the  electromagnetic  field.  In  SHG,  two  photons  of 
frequency  CO  are  destroyed  and,  simultaneously,  one  photon  of  fre¬ 
quency  2cp  (exactly)  is  created  in  a  single,  quantum  mechanical 
process.  As  SHG  is  sensitive  to  the  relative  orientation  between 
the  polarization  of  the  incident  light  and  the  symmetry  condition 


of  the  material,  the  SHG  signal  can  be  used  to  provide  informa¬ 
tion  about  crystal  orientation  and  perfection,  molecular  structure, 
and  regions  where  the  centra- symmetry  breaks  down,  such  as  sur¬ 
faces  and  optical  interfaces.  It  can  also  be  induced  either  by  struc¬ 
tures  that  are  organized  on  the  sub-micron  level  or  by  a  large,  local 
residual  electric  field  (e.g.,  a  membrane  potential). 

Third  harmonic  generation  involves  three  photons  of  fre¬ 
quency  (D  that  are  destroyed  to  create  one  photon  of  frequency  3(0. 
A  nonlinear  contribution  to  the  polarization  at  the  frequency  of  the 
incident  field  leads  to  a  nonlinear  contribution  to  the  RI  experi¬ 
enced  by  a  wave  at  frequency  (0.  This  is  known  as  the  Kerr  effect. 

Barad  demonstrated  that  imaging  with  the  third  harmonic  is 
possible  and  is  especially  suitable  for  imaging  structures  in  trans¬ 
parent  specimens  with  low  intrinsic  contrast.  It  is  sensitive  to 
changes  in  the  nonlinear  optical  properties  of  the  specimen  such 
as  interfaces  between  media  of  differing  RI  (Barad  et  al,  1997). 

Because  harmonic  generation  and  the  Kerr  effect  involve  only 
virtual  state  transitions  that  are  related  to  the  imaginary  part  of  the 
nonlinear  susceptibilities,  no  energy  is  deposited  in  the  specimen 
and  no  damage  can  be  produced.  However,  interactions  involving 
real-level  transitions,  such  as  multi-photon  absorption  (that  may 
lead  to  the  generation  of  fluorescence,  2PF)  do  involve  energy 
deposition  and  can  induce  photochemical  responses  such  as  pho¬ 
todamage  or  photo-induced  polymerization.  It  is  important  to  point 
out  that  both  nonlinear  absorption  (fluorescence)  and  harmonic 
generation  can  occur  simultaneously  in  the  same  specimen.  Figure 
8.25  shows  a  section  of  a  maize  leaf  just  below  the  epidermal  cell 
using  different  excitation  polarizations  (THG,  blue;  SHG,  green; 


FIGURE  8.25.  Nonlinear  optical  section  of  a  maize  leaf  just  below  the  epidermal  cells  at  different  illumination  polarization  (330°-90°).  Note  that,  although 
there  are  few  changes  in  two-photon  fluorescence  (red),  there  are  significant  changes  in  the  THG  signal  intensity  (blue).  The  optical  section  was  obtained  just 
tangential  to  the  lower  surface  of  silica  cells,  therefore,  only  high  THG  is  observed  from  the  interface.  Very  weak  SHG  (green)  is  detected  from  the  silica  cell 
at  this  optical  section. 
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2PF,  red).  For  a  detailed  discussion  of  these  topics,  the  reader  is 
kindly  referred  to  Chapters  38,  39,  and  40. 


GEOMETRIC  CONTRAST 
z-Contrast  in  Confocal  Microscopy 

One  of  the  basic  properties  of  the  confocal  microscope  is  its  ability 
to  use  a  confocal  aperture  to  reject  out-of-focus  signal.  Therefore, 
confocal  images  are  acquired  from  a  narrow  slice  of  the  specimen. 
This  optical  sectioning  capability  gives  the  basic  optical  contrast 
mechanism  of  the  confocal  microscope  and  is  commonly  referred 
to  as  the  ’-contrast  (axial  contrast). 

One  can  use  this  contrast  mechanism  when  examining  inte¬ 
grated  circuits  with  reflected  light  confocal  microscopy  (Fig.  8.26). 
Although  some  image  contrast  results  from  differences  in  surface 
reflectivity  and/or  roughness,  most  of  the  contrast  between  adja¬ 
cent  circuit  structures  is  due  to  the  variation  in  the  height  of  the 
surface.  The  surface  topography  of  such  specimens  can  be  obtained 
by  plotting  the  maximum  intensity  profile  from  a  confocal  z-stack. 

The  tilt  of  structural  surfaces  within  a  specimen  also  affects  the 
reflection  signal  strength  (Fig.  8.27).  If  the  surface  tilt  is  so  high 
that  no  light  reflects  directly  into  the  objective,  then  this  part  of  the 
surface  appears  dark  (or  absent)  in  the  image  while  a  surface  per¬ 
pendicular  to  the  optical  axis  will  give  a  maximum  signal.  This  type 
of  image  contrast  is  highly  dependent  on  the  NA  of  the  objective: 
high-NA  objectives  reduce  the  amount  of  “surface  angle”  contrast 
(Fig.  8.8).  On  the  other  hand,  such  contrast  can  be  increased  if  the 
detector  is  placed  not  directly  behind  the  pinhole  but  in  the  dif¬ 
fraction  plane  of  an  auxiliary  lens  focused  on  the  pinhole  plane.  A 
four-segment  detector  in  this  position  gives  independent  signals 
proportional  to  the  four  possible  different  surface  tilt  directions 
(Fig.  8.28).  Figure  8.29  shows  a  series  of  optical  sections  (reflec¬ 
tive,  RL;  backscattered,  BS;  fluorescence,  FL)  of  a  fluorescent  latex 
bead  in  water.  The  surface  curvature  determined  the  signal  strength 
of  the  reflective  signal.  The  center  reflective  core  (arrow)  in  the 
bead  is  an  artifact  due  to  a  lensing  effect.  When  imaging  a  speci¬ 


men  consisting  high  absorbance  features,  such  as  a  cross-section 
through  the  vasculature  bundle  of  plant  stem,  the  “effective”  NA 
of  the  objective  lens  is  limited  by  the  fact  that  the  surrounding  sec¬ 
ondary  wall  obscures  the  high-NA  rays,  resulting  in  a  “pipe  effect” 
(Fig.  8.30).  As  a  result,  the  transverse  and  axial  resolution  decreases 
as  one  images  deeper  into  this  part  of  the  tissue. 

The  field  curvature  of  the  objective  lens  can  create  artificial  z- 
contrast  when  imaging  a  flat  specimen  because  the  spherical  focus 
plane  intersects  only  a  ring-like  area  of  the  flat  surface  (Cheng  and 
Kriete,  1994). 

Total  Internal  Refraction  Fluorescence  Contrast 

A  light  beam,  incident  on  an  interface  separating  regions  of  dif¬ 
ferent  RI,  will  be  totally  reflected  if  the  incident  angle  exceeds  a 
critical  angle,  9C.  While  “all”  the  light  is  reflected  in  the  classic 
sense,  quantum  mechanics  requires  that  a  portion  of  the  radiation 
penetrates  into  the  distal  phase.  This  radiation  is  called  the  evanes¬ 
cent  wave  and  its  strength  decays  exponentially. 

Therefore,  if  a  fluorescent  structure  is  located  in  the  second 
medium  near  the  interface,  a  fluorescence  signal  can  be  excited 
and  detected  from  it.  The  signal  level  is  determined  by  the  expo¬ 
nential  decay  function  of  the  evanescent  wave  and  by  the  QY  and 
concentration  of  the  fluorophore.  This  process  creates  a  very  thin 
optical  section  near  the  RI  interface.  The  fluorescence  intensity 
IF  is: 

Ifix)  =  QY[con\I0e-(‘x 

where  QY  is  the  quantum  yield  of  a  fluorophore,  [con]  is  its  con¬ 
centration,  I0  is  the  incident  light  intensity,  and  x  is  the  separation 
distance  between  the  dye  molecule  and  the  RI  interface. 

Figure  8.30  shows  that  total  internal  refraction  fluorescence 
contrast  (TIRF)  can  be  used  for  measuring  the  |Ca  |  in  the  sub- 
cellular  cleft  using  the  Fluo-3/Fura  Red  ratiometric  system  [Fig. 
8.31(A)].  The  calcium  image  in  Figure  8.31(B)  is  superimposed 
on  a  TIRM  image  that  shows  the  contact  points  of  the  cell  from 
Figure  8.24.  Figure  8.32  shows  the  cortical  microtubule  assembly 
near  to  the  coverslip  in  a  Vero  cell.  Figure  8.33  shows  the 


FIGURE  8.26.  (A-D)  Reflected-light  confocal  images  of  an  integrated  circuit  at  different  focus  levels.  The  image  contrast  is  primarily  the  result  of  variations 
in  the  signal  being  focused  through  the  pinhole  between  in-focus  and  out-of-focus  structures.  (E)  Diagram  representing  confocal  imaging  of  a  stepped  structure 
in  which  the  steps  are  thicker  than  the  thickness  of  the  imaging  plane  ( shaded  bar). 
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FIGURE  8.27.  Reflected-light  image  of  a  glass  bead,  immersed  in  water 
and  placed  just  under  a  cover  glass.  The  bead  was  placed  in  the  center  of 
the  microscope  field,  almost  on  the  optical  axis.  (A)  xz  view  shows  the 
intensity  “ripples”  on  the  reflection  signal  from  the  coverslip  surface 
caused  by  interference  between  the  light  reflected  by  the  surface  of  the 
coverslip  and  that  from  the  bead.  (B)  False  color  representation  of  (A); 
note  the  intensity  fall-off  from  the  “north  pole”  towards  equator.  (C)  Same 
data  with  a  higher  detector  gain  setting  to  amplify  weakly  scattering 
signals  such  as  the  axial  reflection  (ax)  from  the  microscope  optics  that 
is  consequently  visible  at  all  z-positions.  The  surface  of  the  sphere  can 
be  visualized  down  to  ~30  degree  north  of  the  equator.  (D)  Diagram 
representing  the  relevant  reflections. 
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FIGURE  8.28.  Surface  orientation  detection  using  reflected  light  imaging.  (A)  Color-coded  surface  orientation  image  of  the  surface  features  on  a  Singporian 
coin  {green  and  red  arrows  show  the  orientation  of  surfaces).  (B)  and  (C)  are  the  individual  images  of  the  two  surface  orientations  shown  in  (A).  (D)  Color- 
coded  surface  orientation  image  of  a  milled  plastic  surface  showing  cutting  marks.  (E,  F)  Corresponding  orientation  images  shown  in  (D).  (G-J)  Color-coded 
surface  orientation  image  of  a  glass  bead.  (K)  Diagram  of  the  microscope  setup.  Note  how  the  use  of  a  “fan  aperture”  obstructs  signals  from  specific  orienta¬ 
tions.  These  effects  were  produced  by  introducing  a  two-position  blade  to  obscure  part  of  the  “fan”  of  light  behind  the  pinhole. 
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FIGURE  8.29.  Confocal  serial  optical 
sections  (A-E)  of  an  8  pm  latex  bead  in 
water  in  fluorescence  (F-J)  and  BSL 
(K-O)  modes.  (A-E)  are  combined 
images  of  fluorescence  and  BSL  images. 
Note  the  axial  reflection  from  the  center 
of  the  bead  and  the  “core”  (arrow).  The 
latex  bead  contains  numerous  “defects” 
that  produce  significant  backscattered 
signal  at  various  positions.  (P,  Q)  Low 
magification  optical  sections  of  the  beads 
showing  the  scatter  “core”  artifacts.  (R) 
xz  section  of  the  bead  shown  in  (A-E); 
(S)  Diagrammatic  representation  of  the 
imaging  situation. 


FIGURE  8.30.  The  “light  piping”  effect.  When  imaging  features  inside  a  highly  absorbing  tube-like  structure 
(such  as  a  vascular  bundle  in  plants),  the  effective  NA  (or  detection  solid  angle)  decreases  as  the  imaged  plane 
moves  deeper  into  the  tissue  (from  0  to  (j)).  This  not  only  lowers  the  transverse  and  axial  resolution  of  the 
microscope,  but  also  decreases  the  signal  intensity. 
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FIGURE  8.31 .  (A)  The  Fluo-3  and  Fura  Red  indicator  dye  system  allows  ratiometric  (Ca-^)  determination  using  488 nm  excitation  and  detection  at  520nm  and 
660  nm.  (B)  Fluo-3/Fura  Red  used  in  TIRF  to  determine  the  (Ca**)  in  the  extracellular  space  between  the  cell  membrane  and  the  substrate. 


FIGURE  8.32.  (A)  TIRF  micrograph  of  Vero  cells  transfected  with  EB3-GFP,  a  construct  that  marks  microtubule  plus  ends,  obtained  with  an  Olympus  TIRF 
microscope  using  an  60x/NA  1.42  oil-immersion  objective  and  488  nm  excitation.  (B)  A  fast  Fourier  transform  (FFT)  of  the  TIRF  image;  (C)  false-color  LUT 
of  (B);  (D)  an  intensity  plot  made  horizontally  across  the  diffraction  pattern.  (Images  obtained  at  the  2005  International  Course  on  3D  Microscopy  of  Living 
Cells  at  the  University  of  British  Columbia,  Vancouver,  Canada.) 
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FIGURE  8.33.  Comparison  of  confocal  and  TIRF  Images  of  C2C12 
cells  transfected  with  GFP-MusculoTIRM,  a  novel  TIRM  family 
member  specifically  expressed  only  in  skeletal  and  cardiac  muscle  cells. 
(A)  Confocal  image,  collected  with  an  Olympus  Fluoview  1000  (60x  NA 
1.2/w  objective)  and  deconvolved  using  AutoQuant  AutoDeburr  soft¬ 
ware.  (B)  TIRF  image  collected  with  Olympus  TIRF  system  (60x  NA 
1.45/oil  objective).  Note  the  filamentous  distribution  near  the  plasma 
membrane  is  much  clearer,  and  the  trafficking  of  small  particles  can  be 
observed  better  in  the  TIRF  images.  (Image  courtesy  of  Zui  Pan,  Depart¬ 
ment  of  Physiology  and  Biophysics,  UMDNJ-RWJMS,  Piscataway,  NJ. 
Image  was  taken  at  the  10th  International  Course  on  3D  Microscopy  of 
Living  Cells  held  at  University  of  British  Columbia,  BC,  Canada,  2005.) 


comparison  of  confocal  [Fig.  8.33(A)]  and  TIRF  [Fig.  8.33(B)]. 
Images  of  C2C12  cells  transfected  with  GFP-MusculoTIRM 
(a  novel  TIRM  family  member  only  specifically  expressed  in 
skeletal  and  cardiac  muscle  cells). 

In  TIRF  imaging,  whenever  a  high-RI  cellular  structure  (e.g., 
an  oil  droplet)  is  in  intimate  contact  with  the  coverglass,  the  inci¬ 
dent  ray  will  propagate  into  the  droplet  and  become  far-held  illu¬ 
mination.  “Leaked”  light  produced  in  this  way  can  increase  the 
background  signal  dramatically.  In  addition,  TIRF  excitation  can 
occur  not  only  at  the  defined  glass-water  interface,  but  also  in  a 
conventional  epi-fluorescent  microscopy  wherever  structural  inter¬ 
faces  occur  that  have  the  proper  RI  and  orientation  to  satisfy  the 
TIRF  condition.  Consequently,  most  recorded  fluorescence  images 
are  “contaminated"  by  a  small  but  variable  TIRF  component. 

Fluorescence  Resonant  Energy  Transfer 

Fluorescence  resonance  energy  transfer  (FRET)  occurs  when 
energy  passes  from  one  excited  molecule  (the  donor)  to  a  second 
molecule  (the  acceptor)  because  the  emission  spectra  of  the  former 
overlaps  the  excitation  spectra  of  the  latter.  However,  the  efficiency 
of  this  transfer  is  only  significant  if  the  two  molecules  are  very 
close  to  each  other  (l-10nm).  As  the  efficiency  of  this  energy 
transfer  (E)  varies  inversely  with  the  sixth  power  of  the  distance 
(r)  between  the  donor  and  the  acceptor,  the  ratio  of  the  fluores¬ 
cence  intensity  of  the  donor  compared  to  that  of  the  receptor  is  a 
measure  of  distance  between  them.  Therefore,  FRET  can  be  a  sen¬ 
sitive  nonlinear  measurement  of  inter- fluorophore  distance,  r,  or 
of  the  rate  of  energy  transfer, 

E  =  R60/(R60  +  r6) 

where  E  is  the  energy  transfer  efficiency  and  r  is  the  distance 
between  donor  and  acceptor.  This  leads  to 

E  =  l-  (IDA/ID) 


where  ID  and  IDA  is  the  intensity  of  the  signal  from  the  donor  in  the 
absence  of,  and  the  presence  of,  the  acceptor,  respectively. 

The  Forster  distance  R0  is  the  distance,  r,  at  which  half  the 
excitation  energy  of  the  donor  is  transferred  to  the  acceptor  while 
the  other  half  is  dissipated  by  all  other  processes,  including  light 
emission. 

r=R0{(\/E)-  I  }l/6 
R0  =  0.2  I  I  j  KhEAQr>J(X) } 1/6 

where  QD  is  the  quantum  yield  for  the  donor,  K2  is  a  factor  describ¬ 
ing  the  relative  dipole  orientation  between  the  two  molecules,  and 
J(X)  is  the  overlap  integral  that  accounts  for  the  degree  of  spectral 
overlap  between  the  donor  emission  and  the  acceptor  absorption 

[fD0fcAQWdk 
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where  fF(X)  is  the  corrected  fluorescence  intensity  of  the  donor 
wavelength  in  the  range  X  and  X  +  dA,,  with  the  total  intensity  nor¬ 
malized  to  unity,  eA(X)  is  the  extinction  coefficient  of  the  acceptor 
at  X,  and  is  measured  in  units  of  M^'cnL1  (see  also  Chapter  45,  this 
volume). 

Figure  8.34  shows  an  intensity-based  2p-FRET  data  analysis 
that  was  used  to  localize  the  dimerization  of  C/EBPa  proteins  in 
the  nuclei  of  GHFT1-5  cells.  Figure  8.35  shows  tissue  FRET  in 
traumatic  axonal  injury  using  BAD/Alexa  488  as  the  donor  and 
Bcl-xL/Alexa  555  as  the  acceptor.  The  image  demonstrates  energy 
transfer  consistent  with  BAD-Bcl-xL  heterodimerization. 

When  the  acceptor  is  bleached,  the  lifetime  of  the  donor  mol¬ 
ecule  increases  because  there  are  now  fewer  paths  for  excited 
donor  molecules  to  lose  their  energy.  Figure  8.36  combines  FRET 
and  fluorescence  lifetime  microscopy  (FLIM,  see  below  and  also 
Chapter  27,  this  volume)  to  localize  interacting  protein  molecules. 
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FIGURE  8.34.  Intensity-based,  quantitative  2p-FRET  data  analysis  to  localize  the  association  of  C/EBPa  proteins  in  the  nuclei  of  GHFT1-5  cells.  Seven  images 
were  acquired  using  a  Bio-Rad  Radiance  2100  confocal/multiphoton  microscopy  as  the  data  input  for  “precision-FRET”  (PFRET,  http://www.circusoft.com) 
imaging.  The  color  bars  clearly  demonstrate  the  presence  of  acceptor  and  donor  spectral  bleedthrough  (SBT)  requiring  correction.  The  top  panel  shows  the  uncor¬ 
rected  FRET,  E%,  and  distance  images  and  their  respective  color  bars.  The  bottom  panel  shows  the  two-dimensional  distribution  of  corrected  FRET,  E%,  and 
the  distance  (r)  images.  For  the  protein  complex  marked  with  an  arrow,  the  energy  transfer  efficiency,  E,  is  64%  before  correction  and  50%  after  correction  for 
Donor,  and  Acceptor  spectral  bleedthrough.  The  color  image  helps  to  identify  the  2D  distribution  of  distance  between  interacting  protein  molecules.  (Images 
courtesy  of  A.  Periasamy,  University  of  Virginia,  Charlottesville,  Virginia.) 
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FIGURE  8.35.  Tissue  FRET  in  traumatic  axonal  injury.  Six  hours  postinjury,  the  tissue  was  labeled  with  BAD/Alexa  488  (donor)  and  Bcl-xL/Alexa  555  (accep¬ 
tor).  These  results  show  energy  transfer  consistent  with  B  AD-Bcl-xL  heterodimerization.  The  same  tissue  was  used  for  both  confocal-FRET  and  2p-FRET  imaging 
using  a  Bio-Rad  Radiance  2100  confocal/multi-photon  microscope.  Although  confocal  “precision-FRET”  (PFRET,  http://www.circusoft.com)  imaging  was  con¬ 
fined  to  the  outer  40  pm  of  the  specimen  by  low  signal  levels,  2p  (or  multi-photon)  PFRET  images  provided  useful  data  up  to  200  pm  into  the  tissue.  This  dif¬ 
ference  can  be  attributed  to  higher  concentrations  of  the  fluorophore  deep  inside  the  tissue  and  also  to  the  fact  that  visible  FRET  signals  scatter  in  the  tissue  and 
can  only  be  detected  with  a  non-descanned  detector.  Bar,  10pm;  20x  MIMM  NA  0.75;  Confocal-exD  488nm,  emD  528/30,  exA  543,  emA  590/70;  2-photon- 
exD  790,  exA  730.  (Images  courtesy  of  A.  Periasamy,  University  of  Virginia,  Charlottesville,  Virginia.) 
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FIGURE  8.36.  FRET-FLIM  microscopy  to  localize  interacting  protein  molecules.  Images  of  cells  expressing  a  FRET  test  molecule  (eCFP  coupled  directly  to 
eYFP  through  a  15  amino  acid  linker:  eCFP-15aa-eYFP)  were  acquired  and  analyzed  using  time-correlated  single-photon  counting  FRET-FLIM  microscopy 
(TPSPC,  Becker-Hickl,  Berlin,  Germany).  The  mean  lifetime  of  the  selected  ROI  was  1.87  ns  in  the  presence  of  acceptor  (A1  and  A2)  but  the  lifetime  returned 
to  2.52  ns,  the  natural  lifetime  of  eCFP  (D1  and  D2)  after  photobleaching  of  the  acceptor  molecule  with  514nm  light,  demonstrating  that  FRET  had  been  occur¬ 
ring.  As  is  illustrated  in  the  2D  (A2-D2)  and  3D  (A3-D3)  lifetime  distribution,  the  average  lifetime  of  the  donor  molecule  steadily  increases  as  more  acceptor 
molecules  are  bleached.  (Images  courtesy  of  A.  Periasamy,  University  of  Virginia,  Charlottesville,  Virginia.) 
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FIGURE  8.37.  Fluorescence  recovery  after  photobleaching  (FRAP).  Photobleaching  of  the  fluorescent  protein  Kaede  causes  it  to  emit  in  the  red,  rather  than 
the  green.  After  bleaching  between  the  first  two  images,  the  red  fluorophores  are  slowly  replaced  by  green  ones. 


Images  of  cells  expressing  a  FRET  test  molecule  (eCFP  coupled 
directly  to  eYFP  through  a  15  amino  acid  linker:  eCFP-15aa- 
eYFP)  were  acquired  and  analyzed  using  TCSPC  (Becker-Hickl, 
Berlin,  Germany)  FRET-FLIM  microscopy.  The  fact  that  the  mean 
lifetime  of  the  selected  region  of  interest  (ROI)  was  1.87  ns  in  the 
presence  of  acceptor  (A1  and  A2),  but  returned  to  2.52  ns,  the 
“natural”  lifetime  of  eCFP  after  the  acceptor  molecule  was  photo- 
bleached  using  514nm  light  (D1  and  D2)  shows  that  FRET  had 
been  occurring.  This  increase  is  illustrated  in  2D  (A2-D2)  and  3D 
(A3-D3)  lifetime  distributions  (Chen  et  al.,  2003). 

Fluorescence  Recovery  After  Photobleaching 
(FRAP  and  FLIP) 

Fluorescence  recovery  after  photobleaching  (FRAP)  has  been  used 
in  biological  research  for  more  than  three  decades  following  the 
pioneering  work  of  Webb  (1976),  who  used  it  to  study  the  lateral 
movement  of  Ach  receptors  in  cell  membranes.  In  this  technique, 
a  small  region  of  interest  is  selectively  photobleached  with  a  high- 
intensity  laser,  and  one  then  uses  a  low-intensity  beam  to  monitor 
the  recovery  of  the  fluorescence  intensity  as  unbleached  molecules 
move  back  into  the  bleached  region.  The  movement  of  the  mole¬ 
cules  can  result  from  simple  diffusion,  from  binding/dissociation, 
and/or  from  active  transport,  etc. 

Analysis  of  the  fluorescence  recovery  can  be  used  to  determine 
kinetic  parameters  of  tagged  proteins  or  other  molecules,  includ¬ 
ing  their  diffusion  coefficients,  pool  fraction,  transport  rates,  or 
binding/dissociation  from  other  molecules. 

In  a  typical  FRAP  experiment,  the  fluorescence  in  the  region 
of  interest  is  measured  just  before,  ideally  also  during,  and  also 
after  photobleaching.  Figure  8.37  shows  a  pair  of  cells  expressing 
the  fluorescent  protein  Kaede,  which  changes  its  emission  from 
green  to  red  in  response  to  photodamage.  The  sequence  of  eight 
images  allows  one  to  see  the  upper  cell  be  “bleached”  and  then  to 
follow  the  replacement  of  the  damaged,  “red”  Kaede  with  undam¬ 
aged  green-emitting  molecules. 

From  such  images  one  may  plot  a  recovery  curve.  Depending 
on  the  complexity  of  the  interactions  of  the  molecule  of  interest, 
the  curve  may  have  different  shapes.  From  a  simple,  single  expo¬ 
nential  curve  one  can  calculate  the  mobile  fraction,  R: 


rJAzM 

The  tu 2  is  the  time  required  for  the  fluorescence  to  recover  half 
the  way  between  the  fluorescence  level  after  bleaching  (F0)  and 
that  at  the  plateau  level  (FJ.  Although  measuring  the  tm  allows 
one  to  compare  diffusion  rates  under  different  conditions,  one  must 
correct  for  the  fact  that  the  data-acquisition  scans  used  to  monitor 
fluorescence  recovery  may  also  produce  some  bleaching. 

In  the  image,  the  difference  between  the  bleached  area  and  the 
surrounding  un-bleached  area  provides  bleaching  contrast  and  this 
can  be  seen  by  simple  image  subtraction. 

Following  similar  reasoning,  one  can  monitor  the  continuity  of 
cellular  compartments  with  a  technique  called  fluorescence  loss  in 
photobleaching  (FLIP).  In  FFIP,  a  small  region  within  a  fluores¬ 
cent  cell  is  continuously  photobleached  while  one  images  the 
whole  cell.  Over  time,  any  cellular  compartments  connected  to  the 
area  being  bleached  will  gradually  lose  fluorescence  as  mobile 
molecules  move  into  the  bleached  area,  while  the  fluorescence  in 
unconnected  regions  will  be  unaffected.  In  addition,  FLIP  can  be 
used  to  assess  whether  the  tagged  molecules  move  freely  between 
particular  cellular  compartments  or  if  they  undergo  interactions 
that  impede  their  motion.  FLIP  can  also  be  used  to  reveal  faint  flu¬ 
orescence  in  unconnected  compartments  that  is  obscured  by  bright 
fluorescence  in  the  component  being  bleached. 

A  typical  method  for  doing  FRAP  with  a  single-beam  confo- 
cal  or  multi-photon  microscope  is  to  scan  a  small  region  of 
interest  using  a  high-intensity  illumination,2  then  use  a  low 
magnification  setting  and  low  beam  power  to  scan  a  larger  area. 
Some  modern  confocal  microscopes,  such  as  the  Olympus  Fluo- 
viewlOOO,  are  equipped  with  dual  scanners:  one  for  bleaching 
(and  photoactivation),  and  the  other  for  imaging.  The  FRAP  data 
are  typically  either  presented  in  xyt  format  or  condensed  into  an 
intensity  versus  time  plot  (Reits  and  Neefjes,  2001;  Patterson  and 
Lippincott-Schwartz,  2002). 


2  Be  careful  not  to  use  such  a  high  power  density  that  nonlinear  damage  mech¬ 
anisms  bias  the  outcome.  See  Chapters  38  and  49,  this  volume,  for  details. 
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STRUCTURAL  CONTRAST 
Harmonic  Generation  Contrast 

At  high  illumination  intensity,  biological  specimens  exhibit  a 
number  of  nonlinear  optical  responses;  these  include  the  emission 
of  multi-photon  fluorescence,  nonlinear  absorption  that  can  lead  to 
photodamage,  and  other  photochemical  responses  (see  Chapters  21 
and  28,  this  volume).  In  addition,  a  number  of  biological  structures, 
such  as  collagen  fibers  and  plant  cell  walls,  exhibit  bio¬ 
photonic  properties  capable  of  generating  a  SHG  signal.  Fre¬ 
quently,  structures  of  this  type  are  also  birefringent. 

The  SHG  signal  provides  information  about  features  of  the 
object  that  are  composed  of  nano-scale  crystalline  structures  that 
destroy  optical  centro-symmetry.  On  the  other  hand,  optical  inter¬ 
faces  in  a  cell  or  tissue  provide  the  conditions  needed  to  generate 
THG.  Therefore,  the  THG  signal  represents  optical  interfaces  such 
as  those  found  at  the  cell  membrane  or  at  the  surfaces  of  organelles. 
For  a  detailed  discussion  on  harmonic  signal  generation,  readers 
are  kindly  referred  to  Chapter  40. 

Birefringence  Contrast 

A  number  of  biological  structures,  including  many  of  those  impor¬ 
tant  to  cell  motility,  organization,  and  division,  are  birefringent. 
For  example,  microtubules,  thin  filaments,  and  myofibrils  are  all 
birefringent  and  can  therefore  be  studied  by  using  polarization 
microscopy  (Fig.  8.38;  also  see  Fig.  8.3,  a  set  of  polarization 
images  of  the  collagen  fiber  orientation  in  an  artificial  3D  substrate 
used  in  tissue  engineering).  Conventional  polarization  microscopy 
uses  two  crossed  polarizers  above  and  below  the  specimen  to 
achieved  background  extinction  and  make  birefringent  material 
appear  as  shades  of  gray  on  a  black  background.  However,  because 
obtaining  a  full  understanding  of  the  birefringence  properties  of 
the  specimen  using  traditional  polarization  microscopy  requires 
that  the  specimen  be  mounted  on  a  centerable  rotation  stage,  the 
technique  was  seldom  used  in  biology.  More  recently,  develop¬ 
ments  in  liquid  crystal  technology  have  made  it  possible  to  imple¬ 
ment  dynamic  polarization  microscopy  without  mechanically 
rotating  the  specimen  (Shribak  and  Oldenbourg,  2003).  Liquid- 


crystal  components  permit  one  to  change  the  orientation  of  polar¬ 
izers,  compensators,  and  analyzers  rapidly  and  electronically,  and 
to  record  three  images,  each  made  with  the  polarizing  components 
in  a  different  orientation,  and  then  display  a  processed  image  in 
which  the  brightness  of  each  pixel  is  proportional  to  the  local 
retardance  while  its  color  codes  the  orientation  of  the  structure 
(Oldenbourg,  2004).  Figure  8.39  demonstrates  variations  in  the 
retardance  and  orientation  of  the  cell  wall  in  plant  tissues. 

This  technology  has  been  used  extensively  in  the  study  of  the 
microtubule  dynamics  that  drive  cell  movement.  In  addition,  the 
technique  has  been  used  to  identify  the  position  and  orientation  of 
the  first-division,  mitotic  apparatus  in  in  vitro  fertilization  (IVF) 
procedures  involving  both  human  and  endangered  species  (see 
Figs.  40.19  and  40.20,  this  volume). 

Both  the  retardance  and  orientation  of  the  birefringent  struc¬ 
ture  can  be  calculated  by  simple  algorithms  from  a  set  of  three 
images  obtained  under  different  polarization  conditions. 

DERIVED  CONTRAST  (SYNTHETIC  CONTRAST) 

A  combination  of  intrinsic  and  extrinsic  factors  can  generate  “syn¬ 
thetic”  contrast.  For  example,  images  can  show  structural  or  spec¬ 
tral  variations  within  a  cell  as  a  function  of  time,  or  indicate 
membrane  potential  or  pH  (ratio  imaging).  If  fluorochrome- 
conjugated  antibodies  or  fluorescent  dyes  are  used  to  chemically 
tag  specific  molecules,  these  will  fluoresce  when  the  specimen  is 
excited  by  a  suitable  light  source,  and  create  extrinsic  contrast 
because  the  contrast  depends  on  the  binding  specificity  of  fluores¬ 
cent  probes.  This  probe  specificity  is  frequently  used  in  conjunc¬ 
tion  with  pulse-chase  experiments  to  elucidate  time-dependent 
events  (in  fact,  such  pulse-chase  experiments  can  be  considered 
another  contrast  mechanism). 

To  obtain  a  good  confocal  image,  one  must  selectively  combine 
extrinsic  and  intrinsic  factors  to  emphasize  the  “image”  signal  and 
decrease  the  “background”  signal.  This  can  be  done  by  improving 
the  performance  of  the  optics  and  by  using  special  optical  contrast 
methods.  However,  the  prime  factor  determining  the  usefulness  of 
an  image  is  almost  always  the  preparation  of  the  specimen  (Chapter 
18,  this  volume).  Because  commercially  available,  multi-channel 


FIGURE  8.38.  Polarization  micrograph  showing  the  orien¬ 
tation  of  cytoskeletal  structures  in  a  tissue-culture  cell. 
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FIGURE  8.39.  Birefringence  images  of  developing  microspores  in  Cymbopetalum  baillonii  (A,  B)  and  in  the  cell  wall  of  tobacco  suspension-culture  cells  (C. 
D).  (A)  and  (C)  show  retardance,  (B)  and  (D)  are  color-coded  for  orientation.  (Images  courtesy  of  Chih-Hua  Tsou  and  Hsing  Yue-Ie  Caroline  of  the  Institute  of 
Plant  and  Microbial  Biology,  Academia  Sinica,  Taipei,  Taiwan.) 


confocal  microscopes  can  form  images  based  on  the  detection  of 
BSL  and  epi-fluorescence  signals,  we  will  discuss  the  various 
factors  that  affect  the  contrast  of  these  images  in  more  detail. 

Ratiometric 

In  most  fluorescence  microscopy  experiments,  the  intensity 
recorded  in  any  pixel  is  proportional  to  the  absorption  cross- 
section,  QY  and  concentration  of  the  dye,  and  the  size  of  the  volume 
in  the  specimen  recorded  by  this  measurement  (i.e.,  its  x-,  y-, 
z-resolution).  These  techniques  depend  on  the  fact  that  the  spectral 
characteristics  (absorption  cross-section,  QY),  of  some  fluo- 
rophores  (or  combination  of  fluorophores)  may  change  in  response 
to  the  ionic  or  pH  environment.  However,  these  measurements 
can  only  be  qualitative  unless  one  can  normalize  for  the  concen¬ 
tration  of  the  dye.  As  the  concentration  is  difficult  to  control,  it  is 
often  more  convenient  to  normalize  this  number  by  making  a  ratio 
of  two  different  measurements  of  the  same  molecules.  Although  the 
classic  example  involves  the  CaH  indicator  dyes,  Indo- 1  or  Fura- 
2,  other  types  of  ratiometric  imaging  are  also  used  to  detecting  ionic 
or  pH  changes  in  cells  (see  Chapters  16  and  42,  this  volume ).  By 
using  a  standard  titration  curve  and  ratioing  the  intensity  of  images 
obtained  at  two  wavelengths,  one  can  assign  false  colors  in  the 
resulting  image  to  represent  the  actual  ion  concentrations  or  pH 
values.  Figure  8.31  shows  a  typical  fluorophore  combination 
(Fluo3  and  Fluro-red)  used  for  detecting  the  Ca' *  concentration  (see 
also  Fig.  21.33,  this  volume).  Extensions  of  this  technique  can  be 


applied  to  other  situations;  for  example,  in  a  photodamage  experi¬ 
ment,  images  obtained  at  different  time  points  can  be  ratioed  to 
provide  a  contrast  that  shows  the  photodamage  dynamics  of  various 
cellular  compartments. 

Deconvolution 

In  a  widefield  optical  microscope,  energy  emitted  by  a  point  source 
is  captured  by  the  objective  lens  and  imaged  as  a  blurred  set  of 
diffraction  fringes  known  as  the  point  spread  function  (PSF).  Thus, 
any  image  obtained  from  a  microscope,  o(x,  y,  z),  is  actually  the 
convolution  of  the  function  describing  the  actual  light  intensity 
distribution  inside  the  original  specimen  in  3D  i(x,  y,  z )  with  the 
PSF  p(x,  y,  z), 

o(x,  y,  z)  =  i(x,  y,  z)  ®  p(x,  y,  z) 

Therefore,  the  PSF  is  the  function  that  redistributes  the  origi¬ 
nal  energy  distribution  from  the  specimen  into  the  imaging  space 
(Chapter  21,  this  volume).  In  the  absence  of  noise,  the  mathemat¬ 
ical  operation  of  deconvolution  reverses  this  process,  creating  a 
new  3D  image  data  set  that  more  closely  approximates  the  struc¬ 
tures  of  the  specimen.  It  can  increase  the  image  contrast,  reduce 
the  effect  of  Poisson  noise,  and  also  decrease  the  effect  of  out-of¬ 
focus  signals  (Holmes  and  Cheng,  2005).  Although  in  general,  suc¬ 
cessful  use  of  iterative  deconvolution  to  improve  3D  microscopical 
data  requires  that  one  know  the  PSF,  so-called  “blind”  techniques 
can  also  be  successful,  as  is  explained  in  Chapter  24.  However, 
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FIGURE  8.40.  (A)  Three-channel  confocal  microscopy  of  a  5-day-old,  stable,  transgenic  seedling  of  Arabidopsis  thaliana,  simultaneously  expressing  four  dif¬ 
ferent  recombinant  proteins,  ECFP-Gal4,  EGFP  BDCaM53,  EYFP-CoxIV,  and  DsRed2-RecA.  ECFP-Gal4  accumulates  in  the  nucleus  and  EGFP  BDCaM53  in 
the  plasma  membrane,  EYFP-CoxIV  accumulates  in  mitochondria  and  DsRed2-RecA  in  plastids.  ECFP  and  EGFP  can  be  excited  and  collected  in  a  single  channel 
because,  as  they  are  expressed  in  two  distinct  structures  —  namely,  nucleus  and  plasma  membrane  —  they  are  unlikely  to  be  confused.  (B)  The  same  data,  after 
blind  deconvolution.  (Images  kindly  provided  by  Naohiro  Kato,  Department  of  Biological  Sciences,  Louisiana  State  University,  Baton  Rouge,  Louisiana,  and 
obtained  at  the  2005  International  Course  on  3D  Microscopy  of  Living  Cells  held  at  University  of  British  Columbia,  Vancouver,  Canada.) 


iterative  techniques  require  that  one  be  able  to  describe  image 
quality  in  a  mathematical  and  measurable  fashion  so  that  the 
results  from  one  iteration  can  be  compared  with  the  previous  one. 
For  a  detailed  discussion  on  deconvolution  operation,  readers  are 
kindly  referred  to  Chapters  22,  23,  24,  and  25. 

The  practical  effect  of  deconvolution  on  contrast  is  that,  on  a 
basic  level,  deconvolution  of  confocal  data  acts  as  a  nonlinear 
filter,  something  that  reduces  image  contrast  somewhat  but  also 
adaptively  averages  the  signal  over  a  large  number  of  voxels, 
effectively  eliminating  the  “single-pixel”  noise  features  produced 
by  Poisson  noise  (Fig.  8.40). 

Movement  Contrast  (Subtraction  of 
Previous  Image) 

Organelle  movements  within  the  cytoplasm  or  any  other  move¬ 
ments  of  the  object  can  be  emphasized  and  tracked  by  sequentially 
subtracting  from  each  other  images  in  a  time  series.  For  instance, 
subtracting  an  image  of  a  culture  cell  obtained  at  time  A  from  one 
obtained  at  time  B  can  reveal  small  movements  of  organelles 
and/or  the  cytoskeleton;  on  the  other  hand,  converting  three  con¬ 
secutive  time-sequenced  images  into  RGB  format  can  also  effec¬ 
tively  display  structural  movement  (Fig.  8.41).  This  display 
technique  emphasizes  those  structures  that  have  moved  between 
the  two  sampling  times.  Clearly,  in  order  to  obtain  useful  and  reli¬ 
able  movement  contrast,  it  is  important  that  the  sampling  interval 


is  small  enough  to  faithfully  record  the  movement.  Otherwise,  sub¬ 
tracting  the  resulting  images  can  produce  only  nonsense. 

Image  movement  due  to  specimen  drift  or  a  change  in  the 
water  level  of  a  water-immersion  setup  is  a  common  problem.  It 
is  important  to  keep  the  specimen  immobile  while  not  exerting 
external  forces  that  can  deform  it.  Figure  8.42  shows  a  number  of 
methods  for  holding  the  specimen  and  a  simple  apparatus  to  main¬ 
tain  immersion  water  levels  for  a  prolonged  period. 

Spectral  Unmixing  and  Color  Reassignment 

Using  spectral  unmixing  techniques,  one  can  characterize  a  pixel 
or  an  image  region  based  on  its  spectral  signature.  Although  such 
pixels  may  appear  similar  in  RGB  color  to  the  naked  eye  [Fig. 
8.43(A)],  they  may  in  fact  have  very  different  spectral  signatures 
that  only  become  visible  when  each  one  is  assigned  to  a  specific 
color.  For  example.  Figure  8.43(B)  shows  a  RGB  image  of  a  his¬ 
tological  section  of  mouse  intestine,  stained  with  Syto  Blue  [Fig. 
8.43(C)],  Alexa  Fluor  488  [Fig.  8.43(D)],  Alexa  Fluor  594  [Fig. 
8.43(E)],  and  Alexa  Fluor  647  [Fig.  8.43(F)],  and  also  containing 
a  significant  amount  of  autofluorescence  [Fig.  8.43(G)].  One  can 
use  spectral  unmixing  to  separate  the  signals  from  the  four  fluo¬ 
rescent  probes  and  the  autofluorescence,  and  then  generate  a 
synthetic  false-color  image  based  on  these  four  spectral  categories. 
For  other  examples  of  spectral  unmixing,  see  Figures  36.17  and 
21.20. 
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FIGURE  8.41.  A  time-lapse  series  of  confocal  optical  sections  was  made  using  backscattered  light  to  show  movement  of  an  amoeboid  pseudopod.  (A,  B,  C) 
and  (E,  F,  G)  are  two  time  sequences  of  the  pseudopod  showing  highly  reflective  crystals  {arrow)  and  the  plasma  membrane.  Note  the  scatter  signal  from  the 
vacuole  membrane.  (D  and  H)  are  RGB  merged  images  of  (A,  B,  C)  and  (E,  F,  G),  respectively,  showing  the  organelle  movement.  The  time  interval  is  approx¬ 
imately  0.5  s. 
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FIGURE  8.42.  Various  specimen  chambers  to  handle  small  specimens.  (A)  and  (B)  chamber  constructed  using  a  slide  with  a  center  hole  (C),  the  bottom  of 
which  is  covered  with  a  cover  glass;  the  chamber  can  either  be  filled  with  liquid  (A)  or  one  can  use  a  drop  of  medium  (B)  to  create  reflective  meniscus,  although 
this  may  introduce  unwanted  reflections.  However,  if  an  upper  cover  glass  is  used,  the  liquid  meniscus  problem  is  resolved  (D).  To  prevent  the  immersion  water 
from  drying  out  (when  using  a  water  immersion  objective),  a  water  reservoir  with  overflow  is  used.  By  careful  adjustment  of  the  position  of  the  overflow  outlet, 
one  can  maintain  the  water  level  at  the  objective.  A  non-lubricated  condom  with  its  tip  cut  off  is  used  to  dress  the  objective  lens  to  prevent  water  damage.  (E) 
A  specially  made  specimen  chamber  to  handle  small  tissues  (Cheng  et  ah,  1999).  (F)  Detailed  diagram  of  the  chamber  shown  in  (E).  The  chamber  consists  of 
two  compartments,  the  upper  chamber  connected  to  a  water  reservoir,  maintaining  a  specific  pressure  on  the  red  membrane  (typically  10  pm  Mylar),  and  this 
film  then  presses  the  small  tissue  fragment  against  the  lower  cover  glass. 
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FIGURE  8.43.  Mouse  intestine  specimen,  labeled  with  Sytox  Blue  and  Alexa  Fluor  488,  594,  and  647.  (A)  Conventional  three-color  image  (B)  composite  of 
four  component  images  extracted  from  the  same  data  by  “spectral  unmixing.”  (C)  Sytox  Blue,  (D)  Alexa  Fluor  488,  (E)  Alexa  Fluor  594,  (F)  Alexa  Fluor  647, 
and  (G)  autofluorescence.  (The  images  were  obtained  using  the  Nuance  imaging  system  of  Cambridge  Research  Inc.  (Cambridge,  MA).  Image  courtesy  of  CRI.) 


Effects  of  the  Specimen:  Spherical  Aberration 
and  Optical  Heterogeneity 

Although  substantially  transparent,  biological  specimens  are 
seldom  optically  ideal.  More  often  they  are  heterogeneous  in  struc¬ 
ture,  absorption,  RI,  and  thickness.  Because  the  specimen  is  part  of 
the  optical  system,  this  heterogeneity  can  be  particularly  serious  in 
a  confocal  microscope  where  each  point  is,  in  effect,  imaged  twice. 

We  can  divide  these  effects  into  two  categories:  those  that  arise 
because  the  average  RI  of  most  biological  specimens  is  not  that 
for  which  the  objective  was  designed  and  those  caused  by  the  fact 
the  cells  actually  do  not  have  one  RI  but  many  (i.e.,  they  are  opti¬ 
cally  lumpy  and  the  lumps  often  have  different  RIs). 

The  first  problem  leads  to  spherical  aberration,  a  topic  that  is 
discussed  in  many  other  chapters,  particularly  Chapter  20. 
Although,  in  general,  spherical  aberration  can  be  avoided  by 
scrupulous  adherence  to  the  recommendations  of  the  manufacturer 
with  regard  to  the  RI  and  thickness  of  all  layers  between  the  objec¬ 
tive  and  the  focus  plane,  this  is  not  always  easy  to  do.  Water 
objectives  assume  that  the  specimen  is  made  of  water  and,  as  this 
is  not  true  of  living  cells,3  some  degree  of  spherical  aberration  is 
usually  present  in  images  of  living  specimens.  Indeed,  spherical 
aberration  is  usually  the  main  reason  why  the  signal  level  drops  as 
the  focus  plane  moves  into  the  specimen  (Fig.  8.44). 

Figure  8.45  clarifies  the  connection  between  reduced  resolu¬ 
tion  and  lower  signal  level.  A  dry  objective  was  used  to  bleach 
planes  into  a  piece  of  fluorescent  plastic.  Because  the  objective 
was  designed  to  provide  full  spherical  correction  only  on  the  far 
side  of  a  170  pm  coverslip,  the  best  resolution  (and  highest  light 
intensity)  is  found  when  the  focus  plane  is  some  distance  into  the 
plastic.  This  figure  confirms  that  the  zone  with  the  sharpest  images 
[as  seen  in  Fig.  8.45(B)]  coincides  with  that  having  the  highest 
signal  [Fig.  8.45(C)].  Figure  8.46  drives  the  point  home:  xy  images 


3  This  is,  after  all,  why  cells  produce  phase  contrast. 


made  of  the  same  specimen,  after  it  had  been  cut  and  polished  to 
show  the  planes  in  edge  view,  show  that  the  planes  bleached  in  the 
“good”  zone  were  indeed  thinner  than  their  neighbors  and  have  the 
correct  spacing.  It  is  important  to  point  out  that  the  RI  mismatch 
of  the  plastic  specimen  and  the  immersion  medium  (1.52)  results 
in  a  change  in  the  effective  focal  length  that  varies  with  focus  plane 
position.  This  causes  the  spacing  between  the  bleached  planes  to 
vary  as  a  function  of  the  depth. 

On  the  horizon  is  a  better  solution:  a  stand-alone,  motorized, 
computer-controlled  spherical-aberration  corrector  is  now  avail¬ 
able  (Intelligent  Imaging,  Denver,  CO)  and  one  can  imagine  the 
day  when  a  device  with  similar  capabilities  may  become  a  stan¬ 
dard  microscope  accessory. 

However,  such  a  device  cannot  correct  for  the  optical  prob¬ 
lems  caused  by  the  optical  heterogeneity  of  the  specimen  itself  (see 
Figs.  2. 3-2. 5).  As  with  the  BSL  imaging,  reflection,  refraction,  and 
scattering  of  the  illuminating  beam  by  structures  situated  between 
the  focal  plane  and  objective  lens  produce  a  spot  that  is  larger  than 
it  would  have  been  if  the  optics  were  only  limited  by  diffraction 
and  this  reduces  the  intensity  of  the  exciting  radiation  reaching  the 
fluorescent  dye  in  each  voxel  being  sampled.4  This  loss  of  excita¬ 
tion  reduces  the  fluorescence  yield  from  the  focal  spot  in  the  spec¬ 
imen.  Furthermore,  the  emitted  fluorescent  light  will  be  attenuated 
by  the  same  factors  on  its  return  to  the  detector  pinhole.  Whether 
these  obstructing  structures  are  the  upper  portion  of  the  structure 
under  study  or  something  of  little  interest,  this  self-shadowing  can 
significantly  reduce  the  image  contrast  by  lowering  the  signal 
strength.  It  is  particularly  pronounced  when  observing  densely 
packed  specimens.  For  example.  Figure  8.47  shows  a  confocal 
autofluorescence  image  of  the  chloroplast  of  Closterium.  This  algal 


4  In  fact,  the  BSL  signal  level  is  a  good  measure  of  how  much  the  ideal  optical 
situation  has  been  disturbed.  It  is  not  unexpected  that  tissues  with 
particularly  good  transparency,  such  as  the  cornea,  also  give  very  low  BSL 
signals. 
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FIGURE  8.44.  (A)  Two-channel  confocal  image  of  epidermal  cells  and  mesophyll  cells  of  A.  thaliana.  The  epidermal  cell  is  GFP  labeled.  Note  the  intensity 
degradation  in  the  deeper  part  of  the  tissue  evident  in  the  xz  and  yz  sections.  (B)  Segmentation  and  a-blending  can  improve  the  visualization  of  the  epidermal 
cell  where  it  is  in  contact  with  the  mesophyll  cells  (using  Imaris  software).  Both  xy  projections  have  had  top  surface  removed.  (Image  courtesy  of  Michael  Weis, 
Yu  Xiang,  and  D’ Ann  Rochon,  Agriculture  and  Agri-Food  Canada,  PARC,  Summerland,  BC,  Canada.) 


FIGURE  8.45.  Two-photon  bleach  planes  were  written  with  equal  spacing  and  equal  intensity  into  a  piece  of  fluorescent  plastic  (Chroma  Scientific, 
Brattleboro,  VT).  After  high-intensity  writing,  the  block  was  re-scanned  at  a  lower  intensity  to  obtain  an  image  of  the  photobleached  pattern  (A).  Due  to  a  mis¬ 
match  between  the  RI  of  the  plastic  and  the  design  condtions  of  the  objective,  the  signal  level  and  sharpness  of  the  image  varies  significantly  as  a  function  of 
depth  into  the  specimen.  The  best  response  is  in  the  upper  mid-range  of  the  image  labelled  “good.”  At  this  depth,  the  spherical  aberration  caused  by  using  this 
“dry”  lens  without  a  coverslip  is  almost  compensated  for  by  the  thickness  of  the  overlying  plastic.  As  a  result,  this  zone  gives  the  best  z-resolution  (i.e.,  the 
sharpest  photobleached  planes)  and  also  the  highest  signal  intensity  (B).  (C)  is  a  false-color  version  of  (B).  Although  the  spacing  between  the  bleached  planes 
seems  to  remain  constant,  this  is  only  because,  as  both  writing  and  imaging  were  done  by  the  same  setup,  errors  caused  by  SA-induced  focal-length  change 
cancel  out  (see  Figs.  8.40  and  8.45). 
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FIGURE  8.46.  Two-photon  bleach  planes  (equal  z-spacing  but 
with  four  different  photobleaching  intensities)  were  written  into 
a  fluorescent  plastic  block.  After  writing,  the  block  was  turned 
90°  and  the  surface  trimmed  with  the  glass  knife  on  an  ultra¬ 
microtome  to  expose  the  edges  of  the  bleached  planes.  In  a  flu¬ 
orescent  image  of  this  cut  surface,  one  can  see  that  the  spacing 
of  the  bleached  planes  is  not  constant.  At  different  depths  the 
spacing  varies  because  the  focal  length  of  the  objective  lens 
changes  as  the  RI  of  the  material  between  it  and  the  focus  plane 
changes.  Compare  the  peak-to-peak  distances  between  a-b  and 
a-c,  red  bars).  The  FWHM  of  the  bleached  planes  (i.e.,  the  z- 
resolution)  also  varies  {green  bars). 


FIGURE  8.47.  Self-shadowing  phenomenon  in  epi-fluorescent  mode.  As  the 
specimen  attenuates  both  the  excitation  and  the  detected  fluorescence,  inten¬ 
sity  is  lower  for  structures  located  in  deeper  parts  of  the  specimen.  This  phe¬ 
nomenon  is  particularly  pronounced  when  the  specimen  is  very  densely  stained 
or  pigmented.  (A,  B,  C)  Epi-fluorescent  confocal  images  of  a  living  alga, 
Closterium.  Three  images  were  obtained  under  the  same  conditions.  (A) 
Optical  section  showing  chlorophyll  autofluorescence  near  the  top  of  the  spec¬ 
imen,  (B)  from  the  middle  of  the  specimen  (35  pm  from  the  top  of  the  surface) 
and  (C)  from  the  bottom  of  the  specimen  (70  pm  from  the  top  of  the  surface). 
Note  the  reduction  of  the  fluorescent  signal  (particularly  in  the  center  of  the 
specimen  where  there  is  maximum  shading). 
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FIGURE  8.48.  (A)  A  stereo-pair  showing  an  epi-fluorescent  confocal  image  of  an  algal  cell  (Spirogyra)  with  a  loosely  packed,  autofluorescent  spiral  chloro- 
plast.  (B)  Epi-fluorescent  confocal  image  obtained  from  the  top  surface  of  the  autofluorescent  algal  chloroplast.  (C,  D.  E)  Optical  sections  at  various  depths  (20 
pm,  40pm,  and  60pm,  respectively).  Because  of  the  loose  packing  of  the  chloroplasts,  self-shadowing  is  much  less  than  in  Figure  8.41. 


cell  contains  two  chloroplasts  with  fin-like  laminar  projections  that 
shield  the  lower  portion  of  the  chloroplast  from  excitation  light 
[Fig.  8.46(C)].  Therefore,  the  images  of  the  lower  optical  sections 
show  a  significant  reduction  in  signal  strength.  In  contrast.  Figure 
8.48  shows  a  similar  auto-fluorescence  image  of  Spirogyra  where, 
because  the  spiral  arrangement  of  the  chloroplast  minimizes  self¬ 
shadowing,  there  is  no  significant  drop  in  signal  strength  from  the 
lower  sections. 

This  self-shadowing  phenomenon  becomes  particularly  notice¬ 
able  when  imaging  a  spherical  structure  such  as  a  pollen  grain. 
Figure  8.49  is  a  demonstration  of  self-shadowing  in  a  fluorescent 
latex  bead.  Note  that  both  fluorescent  and  BS  signals  diminish 
toward  the  lower  part  of  the  bead.  Figure  8.50  shows  a  mesophyll 
protoplast  of  A.  thaliana;  note  the  lower  part  of  the  cell  cannot  be 
imaged  successively.  Self-shadowing  commonly  occurs  when 
imaging  fixed  plant  tissue,  as  these  frequently  appear  brown  as  a 
result  of  natural  tannins  or  from  the  oxidized  products  of  fixation. 
The  absorption  producing  the  coloration  can  significantly  reduce 
the  signal  intensity  and  limit  the  effective  optical  section  depth.  It 
can  often  be  removed  by  treating  the  tissue  for  10  to  20  h  in  Stock- 
well's  solution  (Johansen,  1940). 


Figure  8.51  shows  that  what  should  be  a  constant  signal  from  a 
uniform  fluorescent  plastic  is  distorted  by  the  presence  of  overlying 
material,  in  this  case  either  a  fluorescent  4  pm  latex  bead  or  a  cell. 
Figure  8.52  shows  that  even  the  position  at  which  a  feature  is 
recorded  can  be  modified  by  the  presence  of  such  material  (compare 
with  and  without  a  coverslip).  In  two-photon  fluorescent  micro¬ 
scopy,  transparent  objects  that  are  embedded  in  the  matrix  can  have 
similar  effects  (Fig.  8.53),  but  generally  speaking,  vertical  shadow¬ 
ing  is  more  severe  with  lenses  of  low  NA  than  those  of  higher  NA 
(Fig.  8.54)  (Cheng  and  Cheng,  2001;  Cheng  etal.,  2002). 

As  biological  materials  such  as  proteins  have  a  significantly 
higher  absorption  coefficient  in  the  UV  region  than  in  visible  light, 
a  visually  transparent  specimen  may  pose  a  serious  self¬ 
shadowing  problem  when  imaged  with  a  UV  confocal  microscope 
(Fig.  8.21  and  Chapter  27,  this  volume ).  In  contrast  to  tissue  sec¬ 
tions,  it  is  important  to  note  that,  because  of  the  absorbance  and 
deflections  produced  by  structures  located  between  the  lens  and 
the  focus  plane,  the  optical  sections  obtained  from  confocal  or 
nonlinear  microscopy  should  not  be  considered  to  be  uniformly 
sampled.  It  is  important  to  remember  this  when  the  images  are 
statistically  analyzed. 


FIGURE  8.49.  Fluorescence  and  BSL  images  of  a  latex  bead.  (A)  Optical  sections  from  the  north  hemisphere  to  the  south  hemisphere  (in  an  upright  micro¬ 
scope).  Note  the  fluorescence  intensity  decreases  as  the  section  moves  towards  the  southern  hemisphere.  The  fluorescence  intensity  from  the  center  of  the  bead 
also  diminishes  due  to  self-absorption  of  both  excitation  and  emission  light.  (B)  BSL  image  of  the  same  bead  shown  in  (A).  The  bead  contains  numerous  “defects” 
that  act  as  strong  scattering  centers.  The  color  lookup  table  (LUT)  for  (A)  and  (B)  are  set  as  blue  =  0  and  255  =  red;  this  LUT  allows  one  to  see  the  base-line 
and  saturation  intensity  regions  in  a  printed  document.  (C)  Diagrammatic  representation  of  the  geometry  of  the  optical  section  series.  (D)  Two-channel  optical 
sections  of  selective  views  of  (A)  and  corresponding  views  in  (B):  green,  BS  image;  red,  fluorescent  image. 


FIGURE  8.50.  Confocal  (xy)  and  xz,  yz  optical  sections  of  a  mesophyll  pro¬ 
toplasts  (A.  thaliana )  with  pHBT95s::A/7LP5-GFP  construct.  Note  that  only 
half  of  the  cell  can  be  imaged  as  the  self-shadowing  is  serious  in  this  chloro- 
plast-packed  plant  cell.  (Image  courtesy  of  Chih-Fu  Yang  and  Jei-Fu  Shaw, 
Institute  of  Plant  and  Microbial  Biology,  Academia  Sinica,  Taipei,  Taiwan.) 
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FIGURE  8.51.  The  effect  of  a  latex  bead  and  a  cultured 
cell  on  the  subsequent  optical  sections.  A  4|im  latex  bead 
casts  a  shadow  on  the  surface  of  a  fluorescent  slide  (A),  as 
does  a  cultured  CHO  cell  (B).  The  inserts  diagram  the 
imaging  situation  in  (A)  and  (B),  respectively.  (D)  and  (E) 
are  false  color  versions  of  (A)  and  (B)  that  emphasize  how 
surface  structures  can  distort  the  image  of  the  flat,  plastic 
surface  and  also  reduce  the  signal  levels  from  locations 
below  them. 


FIGURE  8.52.  The  effect  of  a  coverslip  on  the 
focus  position.  (A)  Diagram  showing  the  imaging 
setup,  (B)  the  upper  and  lower  surfaces  of  the 
coverslip  and  the  surface  of  the  fluorescent  plastic 
shows  a  strong  reflective  signal  (green)  and  the 
fluorescent  slide  shows  intense  fluorescence 
(orange).  Note  the  shift  in  focal  position  between 
the  coverslip-covered  and  non-covered  regions. 
The  edge  of  the  coverslip  also  casts  a  strong 
shadow  into  the  plastic  slide. 
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FIGURE  8.53.  Self-shadowing  due  to  RI  mismatch.  Minute  pieces  of  transparent  Mylar  flakes  suspended  in  a  water-based  FITC  gel  (FITC  added  to  “Glitter,” 
a  type  of  cosmetic).  (A-C)  xz  views  of  the  fluorescent  gel  with  Mylar  flakes.  Note  the  intensitity  degradation  under  the  Mylar  flake,  in  particular  near  the  edges. 
The  white  spots  in  the  Mylar  flake  are  caused  by  optical  breakdown  due  to  the  high  illumination  intensity  used  (see  also  Chapter  38,  this  volume).  All  images 
are  two-photon  fluorescent  images  excited  at  780  nm.  (D)  Diagram  of  the  imaging  situation. 


Mounting  Medium  Selection 

There  is  a  general  misconception  that  the  confocal  microscope  is 
capable  of  obtaining  optical  sections  as  deep  into  a  tissue  as  the 
working  distance  of  the  objective  permits.  This  is  seldom  true,  par¬ 
ticularly  if  an  oil-immersion  lens  is  used  to  image  living  tissue 
immersed  in  water  (Chapter  20,  this  volume,  discusses  this  in 
depth).  Figure  8.55  shows  the  degree  of  image  degradation  that  can 
occur  between  optical  sections  obtained  from  the  top  and  bottom 
of  a  glass  bead.  The  top  section  is  just  beneath  the  coverslip  and 
the  bottom  section  is  approximately  70pm  below  the  coverslip. 


This  sort  of  degradation  is  not  only  caused  by  specimens  as 
atypical  as  glass  beads.  Figure  8.56  shows  that  image  degrada¬ 
tion  can  occur  between  optical  sections  obtained  from  the  top 
and  bottom  of  a  sea  urchin  embryo.  The  top  section  is  just  beneath 
the  coverslip  and  the  bottom  section  is  approximately  70  pm 
below  the  coverslip.  Although  this  embryo  had  been  cleared 
in  glycerol  and  is  very  transparent,  the  loss  of  image  definition 
is  obvious  in  the  lower  section.  For  a  detailed  discussion  of  the 
media  commonly  used  to  mount  plant  tissue,  see  Cheng  and 
colleagues  (1994)  and  Chapter  21,  and  for  animal  tissue,  see 
Chapter  17. 


Objective 


Focal  plane 


Objective 


Scan 


FIGURE  8.54.  The  effect  of  the  NA  of  the  objective  lens  on  self-shadowing.  (A)  A  small  object  ( red )  is  included  in  the  shadow  ( magenta )  of  the  green  object 
when  a  low-NA  objective  lens  is  used  (scanning  from  light  blue  position  to  the  dark  blue  position).  The  optical  section  is  indicated  by  the  yellow  bar.  When  the 
objective  is  focused  on  the  surface  of  the  red  object  (by  the  yellow  lens),  the  detecting  solid-angle  of  the  objective  is  not  large  enough  to  detect  the  fluorescence 
emitted.  However,  (B)  the  red  object  can  be  detected  if  a  high  NA  objective  is  used  (the  yellow  rays  define  the  detectable  cone  of  light). 
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FIGURE  8.55.  (A)  Diagram  showing  the  effects  of  a  spherical  structure  situated  between  the  objective  lens  and  focal  plane.  (Left)  The  objective  is  focused 
onto  a  cover  glass  surface.  (Middle)  The  focus  plane  of  the  objective  is  deviated  if  a  spherical  structure  with  an  RI  (t])  different  from  that  of  the  surrounding 
medium  (Tp)  is  introduced  into  the  optical  path.  The  focus  plane  moves  upward  when  T|  >  Tp  {green  rays )  or  downward  when  T|_  <  Tp  {red.  rays).  (Right)  When 
the  refractive  index  of  the  spherical  structure  is  the  same  as  the  surrounding  medium  (Tp),  the  focal  plane  of  the  objective  remains  on  the  mirror  surface.  (C)  xz 
section  showing  the  effect  of  the  glass  bead  on  the  image  of  the  lower  cover  glass  surfaces.  As  an  air  objective  lens  corrected  for  170  pm  cover  glass  thickness 
was  used,  the  image  just  below  the  top  cover  glass  has  the  best  resolution.  (C-E)  Series  of  confocal  images  of  the  surface  of  a  cover  glass  on  which  a  glass 
sphere  (r)  =  1.53)  was  suspended  in  water  (r|  =  1.33)  with  a  trace  of  red  fluorescent  dye.  [Green,  reflective  (RF);  red,  fluorescence  (FL).]  If  the  glass  sphere  is 
suspended  in  immersion  oil  (T|  =  1.53),  the  RIs  of  the  glass  bead  and  “mounting  medium”  now  match,  the  glass  surface  under  the  glass  bead  remains  in  focus 
with  the  surrounding  glass  surface. 
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FIGURE  8.56.  Two  stereo-pairs  showing  the  top  half  (A)  and  bottom  half  (B)  of  a  sea  urchin  embryo  (Strongylocentrotus  purpuratus).  Note  the  deterioration 
of  image  definition  in  the  lower  half.  The  specimen  was  Feulgen  stained,  cleared  in  glycerol,  and  scanned  for  16  s  on  each  of  15  sections  using  a  Bio-Rad  MRC 
500  operating  at  514nm,  full  power  with  a  Nikon  40x,  NA  1.3  Fluor  lens.  The  S.  purpuratus  embryo  [80  pm  diameter  is  less  transparent  than  the  species  Lytech- 
inus  variegatus  shown  in  Fig.  8.27(C)].  An  optical  section  obtained  from  the  top  of  the  embryo  (10|im  deep).  (D)  An  optical  section  obtained  from  70jim  deep. 
(Specimen  courtesy  of  Dr.  R.  Summers,  Department  of  Anatomical  Sciences,  State  University  of  New  York  at  Buffalo.) 
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ARTIFICIAL  CONTRAST 

Contrast  Resulting  from  Instrument  Vibration 
and  Ambient  Lighting 

Vibrations  resulting  from  the  mechanical  scanning  systems  of  con- 
focal  microscopes  and/or  vibrations  from  external  sources  can 
have  a  serious  negative  effect  on  image  contrast  because  they  can 
alter  the  position  of  the  specimen  relative  to  the  focal  plane  of  the 
objective  lens  (Overington,  1976). 

Interference  from  room  light  can  also  have  a  drastic  effect  on 
the  recorded  image.  Room  light  scattered  by  the  specimen  and 
entering  the  detecting  system  can  increase  the  DC  background 
level  as  well  as  make  moire  patterns  between  the  AC  component 
from  the  fluorescent  lights  and  the  pixel  clock  of  the  digitizer. 
Figure  8.57  shows  the  effect  of  AC  components  resulting  from  flu¬ 
orescent  room  light.  It  is  important  to  note  that,  especially  when 
using  an  inverted  microscope  for  either  a  widefield  or  multi-photon 
fluorescence  with  non-descanned  detection,  the  specimen  should 
be  covered  with  a  dark  cup  and  the  room  should  be  kept  in  almost 
total  darkness. 

Vibration-induced  variations  in  the  relative  positions  of  the 
pinholes  can  cause  similar  artifacts,  degrading  the  effective  reso¬ 
lution  of  the  confocal  imaging  system.  Figure  8.53  shows  the 
effects  of  mechanical  vibration  on  a  confocal  image  of  a  dirty  first- 
surface  mirror.  The  image  shows  vibration  contrast  due  to  poor 
mechanical  isolation  from  the  building  vibration  (Fig.  8.58).  An 
actively  isolated  optical  bench  can  significantly  improve  the  situ¬ 
ation.  Instability  in  the  pointing  direction  of  the  laser  (Chapter  5, 
this  volume)  can  also  contribute  artificial  contrast  to  the  image,  and 
laser  intensity  variations  can  introduce  complex  moire  patterns  in 
signals  recorded  in  a  single-beam  system. 


FIGURE  8.57.  AC  components  in  a  confocal  image  resulting  from  fluorescent 
lights  in  the  laboratory.  Insert:  Fourier  power  spectrum  of  the  image  showing 
spots  related  to  the  AC  frequency. 


FIGURE  8.58.  The  effect  of  mechanical  vibration  on  the  confocal  reflection 
image  of  a  cover  glass  surface. 


Contrast  Resulting  from  Interference  of  Cover 
Glass  Surfaces 

Interference  can  also  introduce  spurious  contrast  into  the  recorded 
data.  Figure  8.59  shows  three  confocal  images  (and  their  Fourier 
transforms)  recorded  from  a  focus  plane  located  near  the  surface 
of  a  coverslip  on  which  an  A.  thaliana  seedling  was  resting.  Pat¬ 
terns  generated  by  the  interference  of  light  reflected  from  the  two 
surfaces  of  the  cover  glass  are  clearly  visible  in  the  reflection 
image  (489  nm)  collected  in  a  channel  near  the  excitation  wave¬ 
length  (488 nm)  [Fig.  8.59(A,D)].  However,  this  reflective  light 
can  still  be  detected  in  the  500  nm  channel  of  the  Zeiss  510  Meta 
system  as  shown  in  the  Fourier  transform  of  the  fluorescence 
image  [Fig.  8.59(B,E)];  since  the  cover  glass  surface  reflection  is 
much  more  intense  than  the  fluorescence  of  the  specimen,  this 
signal  is  actually  the  scattered  light  leaking  through  the  microscope 
monochrometer  system.  The  spots  are  not  visible  in  the  image  col¬ 
lected  at  703  nm  because  the  detecting  channel  is  far  away  from 
the  489 nm  [Fig.  8.59(C,F)].  Using  the  Fourier  transform  in  this 
way  provides  a  very  sensitive  test  for  the  leakage  of  excitation  light 
into  the  detector  channels.  One  focuses  the  microscope  on  the 
surface  of  a  cover  glass  to  obtain  an  interference  pattern  and 
records  the  image  in  all  the  spectral  channels  in  a  normal  way. 
Then  examine  the  power  spectrum  of  each  image  to  determine  if 
specific  diffraction  spots  corresponding  to  the  frequency  of  the 
interference  pattern  are  present. 

Background  Level  and  Ghost  Images  from  the 
Transmission  Illuminator 

Because  the  confocal  microscope  has  high  resolution  along  the  z- 
axis,  it  can  be  very  difficult  to  locate  areas  of  interest  if  one 
searches  in  the  confocal  mode.  Therefore,  initial  searching  is  gen¬ 
erally  done  using  conventional  bright-field  or  widefield  epi- 
fluorescence  microscopy.  Once  an  area  of  interest  has  been  iden¬ 
tified,  one  switches  to  confocal  mode  by  redirecting  the  imaging 
path  to  the  confocal  scanning/detecting  unit  and  by  blocking  the 
optical  path  from  the  trans-/epi-illuminator.  However,  if  one 
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FIGURE  8.59.  A  young  seeding  of  A.  thaliana  was  placed  in  water  under  a  cover  glass  and  imaged  at  488nm  using  a  Zeiss  510  META  confocal  microscope. 
(A)  489  nm  channel  shows  strong  backscattered  light  from  the  specimen  and  the  surface  of  cover  glass.  Interference  fringes  are  clearly  visible.  (B)  500  nm  channel 
shows  low  intensity  of  autofluorescence.  (C)  703  nm  channel  shows  autofluorescence  of  chlorophylls.  (D)  Fast  Fourier  transform  (FFT)  of  (A)  shows  the  dif¬ 
fraction  spots  resulting  from  the  interference  fringes.  (E)  Even  though  there  are  no  visible  interference  fringes  in  (B),  its  FFT  indicates  a  significant  amount  of 
488  nm  illumination  contamination  in  the  image.  (F)  FFT  of  (C).  (Specimen  courtesy  of  Shu-Hsing  Wu,  Institute  of  Plant  and  Microbial  Biology,  Academia 
Sinica,  Taipei.  Taiwan.) 


forgets  to  block  the  light  from  the  conventional  illuminator,  some 
of  it  will  reach  the  detector,  producing  excessive  background. 
Depending  on  the  spectral  output  of  the  illuminator,  different 
fluorescence  channels  will  record  particular  background  levels.  In 
an  extreme  case,  when  the  specimen  is  relatively  opaque,  a  ghost 
image  may  form. 

Figure  8.60  shows  a  pollen  grain  of  maize  stained  with 
periodic-acid  Schiff  (PAS)  reagent  for  starch.  The  trans-illumina¬ 
tor  was  turned  on  at  low  intensity  during  the  confocal  scanning. 
The  substage  condenser  was  purposely  misaligned  off  the  optical 
axis  to  emphasize  the  ghost  image  effect.  Note  that  the  confocal 
fluorescence  image  of  the  pollen  grain  shows  only  the  periphery 
of  the  pollen  grain  as  a  result  of  self-shadowing.  However,  the  non- 
confocal  transmitted-light  ghost  image  (arrows)  simply  shows  the 
shadow  of  the  pollen  grain.  Therefore,  it  is  important  to  turn  off 
the  trans-illuminator/epi-illuminator  of  the  microscope  during 
confocal  observation  as  even  a  low  glow  can  add  significantly  to 
the  background  level  and  produce  complex  image  features  that 
confuse  interpretation. 

Contrast  Resulting  from  Differences  in 
Photobleaching  Dynamics 

Because  different  fluorophores  in  a  multi-labeled  specimen  often 
photobleach  at  different  rates,  photobleaching  that  occurs  while 
recording  a  series  of  optical  sections  can  cause  the  first  section  to 
have  a  different  spectral  balance  from  the  last.  This  complicates 


FIGURE  8.60.  An  epi-fluorescence  confocal  image  of  a  pollen  grain  of  maize 
stained  with  PAS.  Due  to  the  abundance  of  starch  granules,  the  pollen  was 
deeply  stained.  During  the  image  acquisition,  the  substage  illuminator  was  pur¬ 
posely  turned  on  and  misaligned  to  demonstrate  the  effect  of  a  “ghost”  image. 
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FIGURE  8.61.  Photobleaching  in  the  mesophyll  protoplasts  of  A.  thaliana  by  780nm  NIR  (A).  Note  that  the  green  autofluorescence  (B)  bleaches  more  slowly 
than  does  the  red  autofluorescence  (C)  caused  by  chlorophyll. 


any  effort  to  correct  for  bleaching  or  for  the  signal  loss  that  occurs 
as  the  '-stack  penetrates  farther  into  the  specimen.  Figure  8.61  is 
a  dual-channel  recording  of  mesophyll  protoplasts  of  A.  thaliana. 
It  shows  that  the  red  fluorescence  (chlorophylls)  and  the  green 
autofluorescence  photobleach  at  very  different  rates  and,  therefore, 
that  bleach  rate  must  be  considered  as  a  contrast  parameter  when 
imaging  such  specimens. 

Effect  of  Spectral  Leakage  and  Signal  Imbalance 
Between  Different  Channels 

The  spectral  resolution  of  the  detection  system  is  an  important 
factor  controlling  image  contrast  in  fluorescence  microscopy.  Poor 
spectral  resolution,  caused  either  by  leakage  and  poor  cut-off 
sharpness  in  the  filter  system  (see  Chapter  3,  this  volume),  the  spec¬ 
tral  response  of  detector(s),  or  variations  in  the  transmittance  or 
properties  of  the  optics  can  have  a  pronounced  impact  on  image 
contrast  and  S/N. 

A  significant  imbalance  in  image  intensity  among  different 
detection  channels  in  a  multi-channel  setup  can  have  a  serious 


impact  on  the  spectral  purity  of  the  final  image.  If,  in  a  typical 
specimen,  labeled  with  fluoroscein-isothiocyanate  and  rhodamine 
(FITC-Rh),  the  concentration  of  FITC  is  significantly  higher  than 
that  of  Rh,  then  given  equal  excitation,  the  FITC  will  emit  much 
more  fluorescence  than  the  Rh.  Even  0.1%  leakage  of  stray  FITC 
light  through  the  dichroic  mirror  and  barrier  filters  into  the  Rh 
channel  can  contribute  significantly  to  the  detected  signal.  Clearly, 
any  imbalance  noted  between  two  fluorescence  channels  may  be 
due  not  to  a  difference  in  the  concentration  of  the  two  fluorescent 
dyes  but  to  an  imbalance  in  their  fluorescence  efficiency  with 
respect  to  the  spectral  bandwidth  and  blocking  efficiency  of  the 
excitation  and  emission  filters  used.  For  example,  attempts  to  view 
a  specimen  labeled  with  balanced  amounts  of  FITC  and  Rh  but 
excited  using  only  the  488-nm  line  of  an  argon-ion  laser  will  result 
in  a  very  poor  signal  in  the  Rh  channel.  The  use  of  dual  excitation 
wavelengths  (488  and  568  nm)  from  a  krypton/argon  laser  can 
reduce  this  problem. 

A  simple  plot  of  the  pixel-by-pixel  correlation  between  the 
intensity  values  detected  in  each  channel  can  reveal  possible  cross¬ 
talk.  Figure  8.62  shows  such  a  plot  between  two  fluorescence  chan¬ 
nels  recorded  from  a  FITC-labeled  specimen,  excited  at  488  nm. 


FIGURE  8.62.  Spill-over  from  the  FITC  channel  (green)  to  the 
rhodamine  channel  (red)  for  different  low-pass  filters. 
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The  two  channels  were  separated  using  a  570  nm  dichroic  beam¬ 
splitter;  a  535  nm  bandpass  filter  was  used  for  channel  1  and 
various  long-pass  filters  ranging  from  570  to  610nm  were  used  for 
channel  2.  Note  that  the  correlation  line  decreases  in  slope  as  the 
cut-on  wavelength  of  the  barrier  filter  increases  and  the  amount  of 
excitation  light  reaching  the  detector  is  reduced. 

NEW  CONTRASTS:  FLUORESCENCE  LIFETIME 
AND  COHERENT  ANTISTOKES  RAMAN 
SPECTROSCOPY 

Since  the  second  edition  of  the  volume,  the  use  of  confocal  con¬ 
trast  based  on  the  fluorescence  lifetime  (FLIM)  has  become  much 
more  common.  While  this  is  partially  because  the  equipment 
needed  to  record  it  is  now  commercially  available,  to  an  even 
greater  extent  it  is  because  it  seems  that  FLIM  may  be  the  best 
way  to  obtain  a  quantitative  measure  of  FRET  in  living  cells  (Fig. 
8.63).  Chapter  27  explores  this  exciting  new  modality  in  more 
depth. 

Although  Raman  microscopy  had  been  available  for  some 
time,  the  chemical  specificity  it  offers  was  only  available  at  the 
cost  of  having  to  use  relatively  high  illumination  power  while 
having  to  accept  sensitivity  so  low  that  it  was  useless  for  detect- 


FIGURE  8.63.  The  FLIM  images  are  color  coded  and  overlaid  on  an  inten¬ 
sity  image.  Blue  represents  short  lifetime  and  red  represents  long  lifetimes.  The 
image  was  taken  with  an  Olympus  FV1000  with  fiber  output  coupling  to  a  Pico- 
Quant  FLIM  system  and  a  405  nm  pulsed  laser  was  used  for  illumination.  The 
sample  is  a  living  cell  expressing  cyan  fluorescent  protein  (CFP)  and  yellow 
fluorescent  protein  (YFP).  In  the  image,  there  are  four  cells.  Two  of  them 
demonstrate  FRET,  as  is  shown  by  the  blue  color.  (Image  provided  by  Yiwei 
Jia  of  Olympus  and  Uwe  Ortmann  of  PicoQuant;  the  image  was  taken  at  the 
University  of  Virginia  FRET  microscopy  course.) 


FIGURE  8.64.  CARS  Image  of  lecithin  myelin  figures  using  modified 
Olympus  FV300  confocal  system,  cop-cos  is  tuned  to  the  CH2  symmetric  stretch 
vibration  at  2,838  cm-1.  This  image  shows  that  the  myelin  core  is  not  composed 
of  pure  water  but  is  enriched  in  lecithin.  (Provided  by  Ji-Xin  Cheng,  Weldon 
School  of  Biomedical  Engineering,  Purdue  University.) 


mg  most  biological  molecules  at  the  concentrations  likely  to  be 
found  in  biological  specimens.  More  recently,  developments  in 
laser  technology  have  made  coherent  antistokes  raman  spec¬ 
troscopy  (CARS)  a  realistic  contrast  mode  for  3D  microscopy  (Fig. 
8.64).  This  new  technique  has  better  chemical  sensitivity  than 
normal  Raman  microscopy  and  requires  less  laser  power  to  strike 
the  specimen.  Chapter  33  provides  more  details. 

Finally,  although  until  recently,  one  usually  viewed  the  elec¬ 
trical  state  of  excitable  cells  indirectly  by  sensing  Ca++,  there  have 
long  been  membrane-incorporated  or  membrane-translocating  flu¬ 
orescent  dyes  that  change  their  fluorescent  output  with  membrane 
potential  (Zochowski  et  al.,  2000).  Recently,  the  combination  of  a 
disk-scanning  microscope  with  an  electron-multiplying  CCD 
camera  has  provided  the  researcher  with  a  method  of  detecting  the 
electrical  state  directly  and  on  a  time  scale  almost  fast  enough  to 
measure  transients  that  are  physiologically  significant  (Fig.  8.65). 

SUMMARY 

Image  contrast  mechanisms  in  confocal  microscopy  depend  on  a 
number  of  variables,  many  of  which  have  been  presented  in  this 
chapter.  The  acquisition  parameters  of  the  imaging  system  should 
be  adjusted  to  emphasize  to  the  signals  carrying  the  information 
required  and  these  adjustments  must  also  take  into  account  the 
characteristics  of  the  specimen,  and  the  image  processing  method 
to  be  used.  To  observe  detailed  structures  with  the  appropriate  con¬ 
trast  requires  careful  preparation  of  the  specimen,  and  this 
depends  in  turn  on  an  understanding  of  its  chemical  and  physical 
properties. 

In  order  to  improve  the  visibility  of  structures  in  3D  while  min¬ 
imizing  unwanted  artifacts,  the  parameters  needed  for  proper 
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FIGURE  8.65.  Cultured  rat  hippocampal  neurons  loaded  with  DiSBAC2(3),  a  voltage  sensitive,  membrane-translocating  dye,  and  imaged  with  an  Andor  Rev¬ 
olution  488.  Twenty  100ms  exposures  on  the  iXon  887BV  EM-CCD  were  obtained  for  each  z-stack,  using  an  EM  Gain  of  200.  The  last  frame  was  taken  almost 
simultaneously  with  the  addition  of  lOOmM  KC1,  which  depolarizes  the  cells  and  shows  a  near  instantaneous  rise  in  the  DiSBAC2(3)  signal.  The  change  in  signal 
is  ~45%.  (Image  provided  by  Iain  Johnson,  Molecular  Probes/Invitrogen,  Eugene,  Oregon,  and  Mark  Browne,  Andor,  South  Windsor,  Connecticut.) 


volume  rendering  must  be  properly  selected  (particularly,  Nyquist 
sampling  in  x,  y,  z,  and  t).  Improving  the  quality  of  the  confocal 
data  should  always  be  the  first  goal.  This  includes  maximizing  the 
dynamic  range  of  the  recording  system,  the  contrast  between 
the  structure  of  interest,  and  the  background  and  the  S/N  of  the 
recorded  data. 
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The  Intermediate  Optical  System  of  Laser-Scanning 
Confocal  Microscopes 

Ernst  H.K.  Stelzer 


INTRODUCTION 

This  text  explains  some  of  the  basics  of  intermediate  optical 
systems  in  confocal  microscopes.  Important  microscope  compo¬ 
nents  are  discussed  in  more  detail  in  other  chapters:  lasers  (Chapter 
5)  and  other  light  sources  (Chapter  6),  objective  lenses  (Chapter 
7),  special  optical  elements  (Chapter  3),  fibers  (Chapter  26),  and 
detectors  (Chapter  12).  Although  this  chapter  concentrates  on 
laser-based  single-spot  scanning  instruments  while  other  types  of 
instruments  are  covered  elsewhere  (Chapters  10  and  29),  remarks 
on  different  approaches  will  guide  the  interested  reader. 

DESIGN  PRINCIPLES 
Overview 

The  basic  optical  layout  of  a  confocal  fluorescence  microscope  is 
found  in  Figure  9.1.  A  laser  beam  is  focused  into  a  pinhole,  which 
acts  as  a  point-like  light  source.  The  spatially  filtered  light  is 
deflected  by  a  dichroic  mirror  and  focused  by  an  objective  lens  into 
the  fluorescent  specimen.  The  fluorescence  light  is  emitted  in  all 
directions;  a  fraction  of  the  fluorescence  emission  is  collected 
by  the  objective  lens  and  focused  into  a  pinhole  in  front  of  a 
detector. 

The  lateral  discrimination  capability  of  the  confocal  fluores¬ 
cence  microscope  occurs  because  light  that  is  emitted  in  the  focal 
plane  at  a  distance  Ax  from  the  focus  will  miss  the  pinhole  in  front 
of  the  detector  by  a  distance  MAx.  The  main  advantage  of  a  con¬ 
focal  microscope  is,  however,  its  axial  discrimination  capability.  If 
the  sample  is  flat,  this  property  is  not  required,  but  in  a  thick 
sample,  fluorophores  located  in  front  and  behind  the  focal  plane 
(A z  0)  will  also  be  excited  by  the  incoming  laser  beam.  However, 
their  fluorescence  emission  will  be  focused  either  in  front  of  or 
behind  the  image  plane  that  includes  the  detection  pinhole  (A z  ^ 
0).  Hence  their  images  are  expanded  at  the  pinhole  plane  and  only 
a  fraction  of  the  fluorescence  light  passes  the  pinhole  to  reach  the 
detector.  If  one  removes  the  pinhole,  the  axial  discrimination  is  lost 
and  such  a  laser-scanning  system  has  the  properties  of  a  conven¬ 
tional  fluorescence  microscope. 

A  slightly  different  way  to  understand  Figure  9.1  is  to  realize 
that  the  objective  lens  and  the  two  tube  lenses  form  images  of  both 
the  illumination  pinhole  and  the  detection  pinhole  in  the  object. 
Ideally,  both  pinholes  are  located  in  conjugate  image  planes  and 
on  the  optical  axis.  Therefore,  their  images  will  overlap  perfectly 
in  the  object  and  define  the  volume  element  that  is  observable.  This 
description  emphasizes  symmetry  considerations. 


Because  a  confocal  microscope  observes  only  one  spot  in  the 
object  at  a  time  and  the  main  interest  is  to  form  a  two-  or  three- 
dimensional  image,  the  sampling  light  spot  must  be  either  scanned 
through  the  object  (beam  scanner)  or  the  object  must  be  moved 
through  the  light  spot  (object  scanner). 

In  most  confocal  microscopes  the  scan  unit  is  attached  to  a 
conventional  microscope,  and  the  stage  that  positions  the  sample 
and  the  objective  lens  are  part  of  the  conventional  instrument.  This 
chapter  discusses  the  other  opto-mechanical  elements,  that  is,  the 
illumination  and  detection  units  and  the  confocal  scanner.  Essen¬ 
tially  it  provides  a  detailed  description  of  how  to  build  a  beam 
scanning  confocal  fluorescence  microscope. 

Telecentricity 

Figures  9. 1  and  9.2  show  the  basic  geometrical  optics  of  a  confocal 
microscope.  Figure  9.1  outlines  the  major  elements  of  the  confocal 
microscope  in  terms  of  the  objective  lens  and  its  image  and  object 
planes  (a  complete  description  is  found  in  Chapter  7,  this  volume). 
In  practice,  the  user  of  the  microscope  needs  to  know  only  the  mag¬ 
nification  and  the  numerical  aperture  (NA)  of  the  lens.  Most  of  the 
other  properties  (e.g.,  the  diameter  of  the  entrance  aperture)  can  be 
easily  calculated  (see  Tables  9.1  and  9.2).  The  sketch  in  Figure  9.1 
show  an  ideal  setup  as  it  is  well  known  from  many  papers  on  con¬ 
focal  microscopy  (Bacallao  and  Stelzer,  1989).  The  microscope 
objective  is,  however,  a  multi-lens  system,  and  the  locations  of  its 
principal  planes  are  usually  provided  by  the  manufacturer.  The 
image  and  object  distances  must  remain  fixed  if  the  full  correction 
of  the  objective  is  to  be  exploited.  This  fixes  the  positions  of  the 
image  and  the  object  planes  relative  to  the  position  of  the  objective 
(Table  9.1). 

Ordinary  photographic  cameras  use  a  single  lens  to  form  an 
image.  This  has  the  property  that  the  magnification  varies  with  the 
distance  of  the  object  from  the  lens.  As  a  result,  in  pictures  of  high 
buildings  taken  from  ground  level,  the  edges  are  tilted  towards  the 
image’s  top  center.  This  phenomenon  is  also  known  as  the  con¬ 
verging  lines  effect.  A  single  lens  is,  therefore,  of  limited  use  when 
one  intends  to  perform  measurements  in  objects  that  have  depth. 
A  system  that  uses  two  lenses  spaced  the  sum  of  their  focal  lengths 
apart  has  a  magnification  that  depends  only  on  the  ratio  of  their 
respective  focal  lengths.  Such  systems  are  called  telecentric. 

All  microscope  objective  lenses  are  corrected  telecentrically 
(Fig.  9.2).  A  telecentric  system  is  arranged  like  a  Keplerian  tele¬ 
scope  with  a  stop  in  the  common  focal  plane  of  the  two  lenses. 
The  entrance  and  exit  pupils  are  both  at  infinity  in  object  and  image 
space,  respectively.  Telecentric  systems  are  space  invariant  and 
linear.  Space  invariance  means  that  the  lateral  and  longitudinal 
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FIGURE  9.1.  Principles  of  a  confocal  fluorescence  microscope.  The  laser 
light  of  the  excitation  beam  enters  the  instrument  from  the  right-hand  side.  It 
is  expanded  by  a  telescope  with  the  illumination  pinhole  in  its  center.  A  dichroic 
mirror  deflects  the  laser  beam  towards  the  objective  lens.  The  laser  light  excites 
the  fluorophores  in  the  focal  plane.  Some  of  the  fluorescence  emission  is  col¬ 
lected  by  the  objective  lens,  passes  the  dichroic  mirror,  and  is  focused  into  the 
detection  pinhole  in  front  of  a  light  detector. 


magnifications  of  the  optical  system  are  constant  throughout  the 
whole  space,  and  the  shape  of  the  point  spread  function  (PSF)  is 
independent  of  the  absolute  location  of  the  point  source.  When 
observing  two  points  near  the  focal  plane  of  a  microscope  as  the 
focal  position  is  changed,  the  points  become  blurred,  but  their 
center-center  distance  does  not  change.  Because  of  this  particular 
property  when  defocusing,  telecentricity  is  used  in  all  optical  mea¬ 
surement  devices. 

The  most  important  feature  of  a  telecentric  system  is  that  the 
magnification  factor  determines  all  the  properties  of  the  system. 
All  beams  pass  the  telecentric  plane  with  an  angle  that  is  charac¬ 
teristic  of  the  position  of  their  focal  point  in  the  object  plane.  This 
property  is  very  important,  and  the  conjugate  telecentric  planes, 
like  the  conjugate  image  planes,  are  often  shown  in  overviews  of 
the  optical  paths  of  conventional  microscopes.  As  we  will  notice 
later,  it  is  important  to  know  the  positions  of  these  planes  if  one 
wishes  to  extend  optical  paths. 

The  Scanning  System 

Because  the  position  of  the  sampling  light  spot  depends  on  the 
angle  of  the  beam  in  the  telecentric  plane  (Fig.  9.2,  Table  9.1),  the 
method  of  choice  to  scan  the  spot  through  the  object  plane  is  to 
change  this  angle.  From  the  description  above,  it  should  be  obvious 
that  this  can  be  achieved  by  placing  a  scanning  mirror  such  that 
its  pivot  point  (the  center  of  the  tilting  movement)  is  in  the  center 
of  a  conjugate  telecentric  plane  of  the  microscope  objective.  The 
stationary  beam  falling  on  this  scanning  mirror  is  reflected  and, 
proportional  to  the  tangent  of  the  angle  of  the  mirror,  the  spot 
moves  to  different  positions  in  the  object  plane. 


Ax 

FIGURE  9.2.  Simplified  diagram  of  the  components  of  a  confocal  fluores¬ 
cence  microscope.  The  light  enters  the  instrument  from  the  right-hand  side.  A 
filtering  device  selects  at  least  one  wavelength  for  the  excitation  of  the  fluo¬ 
rophores  in  the  sample  and  also  attenuates  the  light  intensity  to  the  required 
level.  A  beam  expander  adjusts  the  beam  diameter  to  fit  the  diameter  of  the 
back-focal  plane  of  the  microscopes  objective  lens  (Fig.  9.5).  A  dichroic  mirror 
deflects  the  excitation  light  towards  the  scanner  pair.  Ideally  both  are  located 
in  conjugate  telecentric  planes  of  the  microscope  objective  lens.  By  tilting  the 
y- scanner  by  an  angle  P',  a  laser  light  spot  is  moved  by  Ax'  in  the  primary  image 
plane  of  the  microscope.  The  tube  lens  and  the  microscope  objective  lens  form 
an  image  of  the  light  spot  in  the  object  plane.  The  detected  fluorescence  light 
follows  the  same  path  backward  and  is  descanned  by  the  y-scanner.  It  passes 
the  dichroic  mirror  and  enters  the  pinhole.  Filters  in  front  of  the  detector  pass 
at  least  one  band  from  the  sample  response.  The  optical  path  is  defined  by  irises 
that  are  also  used  to  align  the  illumination  and  the  detection  paths.  The  two 
irises  around  the  beam  expander,  which  also  remove  any  excess  light  that  will 
not  enter  the  sample  anyway,  and  others  (not  shown)  reduce  stray  light.  Ideally 
they  are  located  in  telecentric  planes,  that  is,  always  close  by  the  focal  plane 
of  a  lens./j  are  the  focal  lengths  of  the  lenses  (see  Table  9.1). 


Any  conjugate  telecentric  plane  is  an  image  of  the  telecentric 
plane  of  the  microscope  objective.  An  intermediate  optical  system 
forms  an  image  of  the  scan  mirror  in  the  entrance  aperture  of  the 
microscope  objective.  Not  every  arrangement  is  valid,  however. 
As  pointed  out  above,  the  microscope  objective  is  designed  to  be 
used  with  fixed  object  and  image  distances.  Otherwise  it  will  not 
function  correctly,  in  particular,  spherical  and  chromatic  aberration 
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TABLE  9.1.  Mathematical  Formulas 


No. 

Symbol/Equation 

Description 

Typical  Value 

/. 

Focal  length  of  microscope  objective  lens 

1.9  mm 

h 

Focal  length  of  tube  lens 

160  mm 

h 

Focal  length  of  scan  lens 

50  mm 

/« 

Focal  length  of  pinhole  lens 

50  mm 

Magnification  of  microscope  objective  lens 

63 

NA 

Numerical  aperture  of  microscope  objective  lens 

1.2 

NA'  =  NAIM 

Numerical  aperture  of  tube  lens  and  scan  lens 

0.02 

n 

Refractive  index  of  immersion  medium 

1.33 

X 

Wavelength  of  light 

xa 

Wavelength  of  excitation  light  (illumination  path) 

0.488  nm 

xm 

Wavelength  of  emission  light  (detection  path) 

0.520  |im 

PWafli-P 

Tilt  angle  of  y-scanner 

4° 

P.v 

Tilt  angle  along  y-axis  in  telecentric  plane 

11° 

SxX  Sy 

Size  of  first  scan  mirror 

2  mm  X  3  mm 

a  =  sir r'NA/n 

Angular  aperture 

64° 

A.--  ^ 

Lateral  resolution 

0.22  |jm 

3  -  2cosa  -  cos2a 

Arimg  =  MAr 

Lateral  resolution  in  primary  image  plane 

14|jm 

< 

ii 

tv 

< 

Axial  resolution 

0.65  |im 

n{  1  -  cosa) 

Arimg  =  M2-Az 

Axial  resolution  around  primary  image  plane 

2.6  mm 

Ax  =fi  ■  tan|3v 

Field  of  view  in  object  plane 

0.4  mm 

Ax'  =  Ax-M 

Image  size  in  primary  image  plane 

24  mm 

(j)BFP  =  320  mm  ■ 

Diameter  of  back-focal  plane  of  microscope  objective  lens 

6.1  mm 

fe,P*=A'4'f  =  Ar'M'7L 

J\  J 3  h 

Diameter  of  illumination  pinhole 

23  nm 

(b. ,  ,=Ar-——  =  Ar-M-  — 

Tdet,p«  r  r  r 

Diameter  of  pinhole  in  front  of  detector 

30  nm 

A  h  f3 


will  no  longer  be  properly  corrected.  If  the  (incident)  light  beam 
is  collimated,  the  intermediate  optical  system  must  have  its  focal 
plane  coincide  with  the  image  plane  of  the  microscope  objective 
(Fig.  9.2). 

The  imaging  outlined  so  far  can  also  be  described  as  follows. 
Every  point  in  the  object  plane  has  a  conjugate  point  in  the  image 
plane.  As  the  light  spot  is  moved  in  the  image  plane,  the  conjugate 
light  spot  is  moved  in  the  object  plane.  To  scan  the  object  plane, 
light  spots  must  be  produced  in  the  image  plane.  If  the  beam  falling 
into  the  intermediate  optical  system  is  collimated,  a  spherically 


corrected  beam  will  be  achieved  only  if  the  focal  plane  of  the  lens 
and  the  image  plane  of  the  microscope  objective  coincide.  The 
position  of  the  light  spot  in  the  image  plane  then  depends  on  the 
scan  angle. 

A  single  scan  mirror  will  be  fixed  on  a  rotation  axis  and  will, 
therefore,  only  be  able  to  provide  a  motion  of  the  focal  point  in 
one  direction.  However,  this  concept  can  be  easily  expanded  to 
include  more  than  one  mirror  and  hence  to  scan  along  two  per¬ 
pendicular  axes.  As  pointed  out,  the  mirrors  should  be  placed  in 
conjugate  telecentric  planes.  Any  number  of  these  planes  may. 


TABLE  9.2.  Resolution  of  Different  Objective  Lenses  in  Conventional  and  Confocal  Fluorescence  Microscopy3 


Lensc 

Resolutionb 

in  Object  Plane 

Parameters  in  Image  Plane 

BFP 

Lateral 

(nm) 

Lateral 

Conf. 

(pm) 

Axial 

(nm) 

Axial 

Conf. 

(nm) 

Lateral 

Resolution*1 

(pm) 

Line 

Pairs/mm 

Line 

Pairs/Image6 

Diameter 

(mm) 

100  X  Oil/1.4 

0.19 

0.14 

0.54 

0.38 

43.6 

46 

920 

4.5 

63  x  Oil/1.4 

0.19 

0.14 

0.54 

0.38 

27.5 

73 

1460 

7.1 

63  x  W*/1.2 

0.23 

0.16 

0.66 

0.47 

32 

62 

1250 

6.1 

40  x  W*/1.2 

0.23 

0.16 

0.66 

0.47 

20.3 

98 

1970 

9.6 

100  x  W/1.0 

0.28 

0.20 

1.10 

0.78 

61 

33 

660 

3.2 

63  x  W/0.9 

0.31 

0.22 

1.43 

1.01 

42.7 

47 

940 

4.6 

40  x  W/0.8 

0.35 

0.25 

1.87 

1.32 

30.5 

66 

1310 

6.4 

20  x  /0.75 

0.37 

0.26 

1.48 

1.04 

16.3 

123 

2460 

12 

20  x  /0.6 

0.47 

0.33 

2.50 

1.77 

20.3 

98 

1970 

9.6 

aFor  a  wavelength  of  500  nm.  Refer  to  Table  9.1  for  precise  definitions  and  formulas. 
bThe  resolution  is  the  full  width  at  half  the  maximum  (FWHM). 

c Oil  refers  to  oil-immersion  lens;  W  to  water-immersion  dipping  lens;  W*  to  a  water-immersion  lens  corrected  for  usage  with  a  cover  glass. 
dThe  resolution  is  given  by  2 M  0.61  A7NA  (see  Table  9.1). 
e  Assuming  an  intermediate  image  diameter  of  approximately  20  mm. 
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however,  be  easily  generated  by  adding  further  optical  elements. 
The  image  of  a  second  mirror  may  then  be  formed  on  the  first 
mirror.  If  the  two  mirrors  move  the  beam  in  orthogonal  directions, 
the  object  is  scanned  along  orthogonal  x  and  y  axes. 

What  happens  with  the  light  emitted  from  the  spot  illuminated 
by  the  laser  beam?  As  explained  above,  the  images  of  the  source 
and  the  detector  pinholes  overlap  in  a  confocal  microscope.  The 
position  of  the  illuminating  light  spot  in  the  image  plane  is,  there¬ 
fore,  linearly  related  to  the  position  of  the  image  of  the  light  spot 
formed  in  the  sample.  Hence,  the  pinhole  in  front  of  the  detector 
must  be  in  a  plane  conjugate  to  all  the  image  planes  of  the  micro¬ 
scope  (Fig.  9.2).  In  a  scanning  system,  this  means  that  we  need  an 
optical  arrangement  that  will  form  an  image  of  the  image  plane. 
The  scan  system  will  move  this  image  across  the  pinhole,  and  the 
light  from  only  one  point  in  the  object  will  enter  the  detector.  If 
everything  is  set  up  correctly,  this  point  will  also  be  the  point  in 
the  object  that  is  illuminated  by  the  laser.  In  most  cases,  the  pinhole 
will  be  placed  on  the  optical  axis  and  in  the  focal  plane  of  a  lens. 
This  detection  lens  and  the  intermediate  optical  system  discussed 
above  form  a  conjugate  image  plane. 

The  Back-Focal  Planes 

To  work  in  the  diffraction  limit,  the  back-focal  plane  (BFP,  also 
known  as  the  entrance  aperture  or  Fourier  plane  and  defined  by  the 
stop  in  the  telecentric  plane)  must  be  uniformly  filled  by  a  planar 
wave.  Failing  to  do  so  results  in  a  smaller  effective  NA  and  hence 
in  a  broader  spot  and  in  poorer  resolution.  As  pointed  out  above, 
the  BFP  diameter  of  a  lens  is  closely  related  to  its  resolution  and 
its  magnification  (Table  9.2).  The  higher  the  resolution  and  the 
lower  the  magnification,  the  larger  is  the  BFP  diameter.  This  means 
that  the  diameter  of  the  illumination  pinhole  must  be  matched  to 
a  specific  lens,  and  the  diameters  of  all  optical  elements  through¬ 
out  the  illumination  and  detection  paths  must  be  large  enough  to 
transmit  a  beam  of  light  this  large  without  truncation.  A  micro¬ 
scope  system  with  interchangeable  objectives  must  either  be 
adjustable  or  at  least  be  usable  with  the  lens  having  the  largest  BFP 
diameter.  All  other  lenses  will  then  be  over-illuminated,  which  pro¬ 
duces  some  stray  reflections  and  involves  some  loss  of  illumina¬ 
tion  intensity.  The  best  idea  is  actually  to  optimize  the  diameter  for 
a  set  of  lenses  that  have  fairly  similar  M/NA  ratios.  Any  design 
requires  a  decision  regarding  the  specific  lenses  likely  to  be  used 
and  instrument  performance  in  terms  of  illumination  and  detection 
efficiency.  When  working  with  lasers  having  a  Gaussian  beam 
profile,  it  is  common  to  over-illuminate  the  BFP  by  a  factor  of  2, 
that  is,  the  1/e2  diameter  of  the  Gaussian  beam  should  be  twice  as 
large  as  the  BFP  diameter.  This  results  in  the  beam  transmitted 
having  only  about  70%  of  the  light  in  the  untruncated  beam. 
Finally,  the  location  of  the  BFPs  on  the  optical  axis  varies  by  a 
few  millimeters.  Although  in  principle  this  causes  vignetting,  the 
effects  can  probably  be  neglected  for  all  practical  purposes  in 
biology. 

PRACTICAL  REQUIREMENTS 

Diffraction  Limit 
Illumination 

The  goal  in  confocal  microscopy  illumination  is  to  make  the  light 
spot  as  small  as  possible.  Its  size  should  be  determined  only  by  the 
wavelength  of  the  beam  of  laser  light  emerging  from  the  single¬ 
mode  fiber  and  the  NA  of  the  objective  lens;  that  is,  the  system 


should  be  diffraction  limited.  According  to  Goodman  (1968,  p. 
103),  “An  imaging  system  is  said  to  be  diffraction-limited  if  a 
diverging  spherical  wave,  emanating  from  any  point-source  object, 
is  converted  by  the  system  into  a  new  wave,  again  spherical,  that 
converges  toward  an  ideal  point  in  the  image  plane.”  The  system, 
in  our  case,  consists  basically  of  the  microscope  objective  lens  and 
the  tube  lens.  Because  we  have  real  elements,  this  requirement  is 
never  fulfilled.  Every  optical  element  will  cause  some  degree  of 
aberration  that  increases  the  spot  size.  The  actual  spot  size  Ar 
(Table  9.1)  can  be  measured  in  the  image  plane  of  the  microscope 
objective. 

While  Ar  is  approximately  the  full-width  at  half-maximum, 
2Ar  is  approximately  the  diameter  of  the  Airy  disk  in  the  image 
plane.  In  general,  it  should  not  be  larger  than  the  spot  in  the  object 
plane  times  the  magnification  of  the  objective  lens  (e.g.,  an  M  = 
63;  NA  =1.2  lens  at  A,  =  488 nm  would  have  a  Ar  in  the  interme¬ 
diate  image  plane  of  about  14  pm).  Another  important  requirement 
to  achieve  the  diffraction  limit  is  that  the  aperture  of  the  objective 
lens  is  uniformly  filled  with  a  planar  light  wave  (if  the  objective 
lens  is  corrected  for  a  finite  distance,  a  spherical  wave  must  fill  its 
aperture). 

Every  optical  element  degrades  the  wavefront  somewhat  (a 
good  description  of  aberrations  introduced  by  optical  elements  is 
given  in  the  Melles  Griot  catalogue  (Chapter  1,  1999);  the  per¬ 
formance  of  real  lenses  is  found  in  Melles  Griot,  Chapters  6  and 
11,  this  volume).  One  goal  of  the  optical  system  is  to  generate  a 
light  spot  in  the  image  plane  that  is  smaller  than  actually  required. 
This  is  equivalent  to  overfilling  the  microscope  entrance  aperture. 
Efficiency  in  the  illumination  path  is  often  not  of  great  importance, 
as  lost  light  can  be  easily  made  up  because  most  of  the  appropri¬ 
ate  lasers  have  more  power  than  is  needed  for  fluorescence  con¬ 
focal  microscopy.  However,  as  applications  such  as  fluorescence 
recovery  after  photobleaching  (FRAP;  Axelrod  et  al.,  1976)  and 
other  related  techniques  usually  try  to  expose  the  sample  to  as 
much  energy  as  possible  within  a  short  period  of  time,  efficient 
light  delivery  has  become  a  much  more  important  issue.1  Multi¬ 
photon  microscopy  (Denk  etcil.,  1990;  Stelzer  etai,  1994;  see  also 
Chapter  28,  this  volume)  is  another  field  that  requires  an  efficient 
illumination  light  path  but  has  the  additional  requirement  that 
group  velocity  dispersions  should  be  under  excellent  control. 

Detection 

It  is  as  important  to  achieve  the  diffraction  limit  in  the  detection 
path  as  it  is  in  the  illumination  path.  A  pinhole  that  is  somewhat 
larger  than  the  ideal  case  produces  only  a  modest  reduction  in  the 
out-of-focus  discrimination  of  the  confocal  setup  (Wilson  and 
Carlini,  1987;  see  also  Chapter  22,  this  volume).  Increasing  the 
pinhole  diameter  is  complementary  to  over-illuminating  the 
entrance  aperture  of  the  microscope  objective  and  can  be  used  to 
correct  the  same  errors  mentioned  above.  If,  however,  the  optical 
arrangement  of  the  detection  path  is  close  to  the  ideal  case,  nothing 
can  be  gained  in  terms  of  detecting  light  from  the  focus  plane  by 
increasing  the  pinhole  size  beyond  1  or  2  Airy  units.  A  particularly 
good  test  is  that  the  intensity  of  the  signal  from  a  single  point-like 
object  (e.g.,  a  100-nm  fluorescent  latex  bead  or  a  non-resolvable 
concentration  of  some  dye)  should  not  increase  by  more  than  a 
factor  of  3  as  the  pinhole  diameter  is  increased.  When  one  observes 
the  image  of  a  single  point  object  that  is  in  focus,  a  well-corrected 


'In  response,  Olympus  recently  introduced  a  microscope  incorporating  an 
entirely  separate  intermediate  optical  system  for  use  in  FRAP  and  uncaging 
experiments. 
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confocal  arrangement  will  collect  at  most  (i.e.,  at  the  highest  NA) 
30%  of  the  available  light  produced  by  the  object  and  focus  it  at 
the  pinhole. 

Detection  efficiency  is  always  crucial  in  fluorescence 
microscopy.  The  basic  rule  is  to  have  as  few  optical  elements  as 
possible  in  the  detection  path.  Sticking  as  closely  as  possible  to 
this  rule  helps  avoid  losses  due  to  reflection  or  absorption  and  pre¬ 
vents  unwanted  degradation  of  the  wave  front.  It  is  a  particularly 
bad  idea  to  place  annular  apertures  into  the  detection  path  that 
obscure  large  fractions  of  the  fluorescence  light  as  is  sometimes 
suggested  to  improve  z-resolution  (Martinez-Corral  et  ah,  2002). 
The  light  loss  caused  by  such  devices  reduces  detectability  limits 
by  increasing  the  effect  of  Poisson  noise. 

Because  only  the  light  emitted  from  the  focal  plane  in  the 
sample  is  collimated  and  properly  focused  into  the  detector 
pinhole,  most  stray  light  is  excluded  by  the  pinhole  and  is  usually 
not  a  serious  problem.  However,  any  laser  light  that  passes  the 
dichroic  beam-splitter  should  be  captured  in  a  simple  beam  stop 
to  prevent  it  being  returned  and  reflected  up  the  axis  from  the  back 
side  of  the  dichroic.  Exposed  surfaces  that  could  reflect  light  (espe¬ 
cially  laser  light)  should  be  covered  with  black  paint,  self-sticking 
velvet,  or  black  anodized  aluminum.  The  intensities  are  usually  too 
low  to  cause  heating. 


Geometric  Distortion 

As  geometric  distortion  is  introduced  by  any  inaccuracies  in  the 
scanning  process,  it  is  important  to  keep  these  as  small  as  possi¬ 
ble.  Distortions  can  arise  from  deficiencies  in  either  the  mirror 
positioning  system  or  the  intermediate  optics.  The  movement  of 
the  scanning  mirror  is  known  only  approximately,  and  it  was  very 
common  to  run  the  horizontal  galvanometers  in  a  sine  function  that 
is  linear  to  ±  4%  only  over  a  small  part  of  the  sine  waveform  (= 
±  30%,  see  Chapter  3,  this  volume).  The  scanning  system  and  the 
way  it  is  used  obviously  define  the  geometric  distortion  to  a  very 
large  degree.  These  factors  also  influence  the  efficiency  of  the  illu¬ 
mination  and  the  average  energy  penetrating  a  unit  area.  Much 
effort  has  been  made  to  drive  the  scanners  accurately,  for  example, 
by  calculating  sums  of  harmonics,  changing  the  acceleration  in  a 
systematic  manner,  etc.,  in  order  to  achieve  a  reasonably  high  duty 
cycle.  The  advent  of  DSP-controlled,  lightweight  ultra-precise 
galvanometers  has  made  this  task  very  easy.  An  essentially  undis¬ 
torted  speed  of  more  than  1  kHz  even  at  large  scanning  angles  is 
now  readily  available  (see  Chapter  3,  this  volume).  Finally,  great 
care  must  be  taken  when  the  intermediate  lens  is  selected.  The 
optimal  solution  is  an  f-theta  lens. 


EVALUATION  OF  THE  ILLUMINATION 
AND  DETECTION  SYSTEMS 


Influence  of  Optical  Elements 

All  optical  elements  in  the  confocal  microscope  influence  the 
characteristics  of  the  light  beam: 


its  energy  spectrum, 
its  polarization, 
its  modal  distribution, 
its  waist  diameter, 
its  divergence, 
its  direction, 

its  position  relative  to  the  optical  axis. 


In  a  fully  corrected  optical  imaging  system,  all  these  parame¬ 
ters  are  considered,  but  in  confocal  microscopy  this  is  usually  not 
necessary.  After  all,  a  confocal  microscope  does  not  form  an 
image,  and  the  wavelength  used  at  any  time  is  only  a  small  frac¬ 
tion  of  the  total  optical  bandwidth. 

Several  rules  govern  the  properties  of  the  incoming  beam  (i.e., 
the  beam  entering  the  entrance  aperture  pupil  of  the  objective  lens), 
and  another  set  of  rules  determines  the  properties  of  the  beam 
entering  the  detector.  The  optical  elements  in  either  the  illumina¬ 
tion  or  the  detection  path  are  used  to  shape  the  beam,  that  is,  to 
ensure  that  the  beam  will  have  the  desired  properties.  The  actual 
problem  is  that  the  optical  elements  are  not  perfect  and  that  they 
influence  to  a  varying  degree  more  than  one  of  the  properties  of 
the  beam. 


Errors 

Beam  Shift 

The  beam  is  shifted  every  time  it  traverses  a  flat  optical  compo¬ 
nent  that  is  tilted  relative  to  the  optical  axis  or  a  lens  that  is  not 
correctly  centered.  If  the  optical  layout  is  designed  for  collimated 
beams,  shifts  can  usually  be  compensated  for  in  the  detection  path, 
but  any  shift  in  the  illumination  path  can  cause  vignetting.  On  the 
other  hand,  in  laser  arrangements  with  collimated  beams,  flat  sur¬ 
faces  are  often  purposely  tilted  by  0.5  to  1°  to  avoid  interference 
effects  caused  by  specular  reflections.  However,  if  these  tilts  are 
too  large,  refraction  and  dispersion  can  cause  a  visible  wavelength- 
dependent  shift  of  the  beam.  If  the  tilt  is  in  the  range  of  1°,  the 
shift  is  about  6 pm/mm  of  glass  while,  in  the  tilt  range  of  45°  (as 
for  a  beam-splitter),  the  shift  is  about  330|im/mm. 

Angular  Deflection 

In  the  confocal  microscope,  any  angular  deflection  of  the  beam  is 
very  critical  and  practically  not  correctable.  Deflections  are  caused 
by  flat  optical  components  that  have  surfaces  that  are  not  parallel 
(i.e.,  they  have  wedge  errors)  or  by  lenses  that  are  tilted  relative 
to  the  optical  axis.  As  pointed  out  above,  in  telecentric  systems  the 
angular  deflection  determines  the  position  of  the  light  spot  in  the 
object  plane.  Every  deflection  of  the  beam  except  that  produced 
by  the  scanning  system  causes  a  shift  of  the  observed  area.  The 
typical  planarity  achieved  on  a  flat  component  is  <1  second  of  arc. 
If  the  thickness  varies  1  pm  over  a  length  of  25  mm,  and  the  col¬ 
limated  beam  is  focused  with  a  90  mm  lens,  the  axial  shift  at  the 
focal  plane  is  2  pm.  The  amount  of  shift  that  is  critical  depends  on 
the  diameter  of  the  pinhole,  and  an  estimate  of  the  tolerable  wedge 
error  depends  to  a  large  extent  on  the  magnification  of  the  objec¬ 
tive  lens.  The  dichroic  mirror  is  in  this  respect  the  most  critical 
part  in  a  confocal  fluorescence  microscope.  Even  slight  changes  in 
its  position  cause  the  image  of  the  source  and  detector  pinholes  not 
to  overlap  anymore.  This  is  particularly  important  as  most  fluo¬ 
rescence  microscopes  operate  with  several  excitation  lines,  and 
this  means  that,  unless  a  multiband  dichroic  or  an  acousto-optical 
beam-splitter  (AOBS,  see  Chapter  3,  this  volume)  is  used,  the  com¬ 
ponent  must  be  frequently  changed,  a  process  that  must  not  affect 
the  angle  of  the  beam.  In  addition,  all  the  different  dichroic  mirrors 
must  have  similar  surface  characteristics  (Fig.  9.2). 

Polarization 

The  state  of  polarization  does  not  seem  to  be  very  critical. 
However,  a  number  of  optical  elements,  especially  the  filters  and 
beam-splitters  with  a  dielectric  coating,  have  a  reflection  coeffi¬ 
cient  that  is  polarization  dependent  as  are  the  imaging  character¬ 
istics  of  microscope  objectives.  Changing  a  dichroic  mirror  can 
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therefore  have  surprising  effects  on  the  image  and  on  the  amount 
of  light  that  penetrates  the  sample.  In  fluorescence  microscopy,  it 
seems  best  to  work  with  light  that  is  either  circularly  polarized  or 
phase  randomized  (Table  9.2). 

Intensity  Loss 

Every  element  in  the  detection  path  causes  some  intensity  loss.  If 
the  elements  are  coated  (broadband,  anti-reflection  coating  in 
transmission;  broadband,  high-reflection  coating  for  mirrors),  the 
energy  losses  can  be  as  low  as  0.5%  per  surface  for  wavelengths 
within  the  bandwidth. 

The  various  filters  cause  the  largest  intensity  losses.  Filters 
come  in  three  general  types  (Chapter  3,  this  volume):  those  made 
from  absorbing/colored  glass,  interference  filters  made  from  trans¬ 
parent  optical  flats  coated  with  several  dielectric  layers,  and 
hybrids  made  from  absorbing  optical  flats  coated  with  dielectric 
layers.  Dichroic  filters  are  always  of  the  interference  type.  An 
interesting  alternative  is  to  take  advantage  of  the  polarization 
dependence  of  the  reflectivity  of  flat  surfaces  and  to  use  an 
uncoated  optical  flat  instead  of  a  dichroic.  At  an  incidence  angle 
of  45°,  an  s-polarized  beam  can  be  reflected  by  40%  while  the  ran¬ 
domly  polarized  fluorescence  emission  may  be  transmitted  up  to 
80%.  A  single  reflective  glass  plate  with  an  anti-reflection  coated 
back  will  provide  an  excellent  performance  across  a  relatively 
large  bandwidth  with  no  need  for  substitutions  that  might  produce 
misalignment.  The  Achrogate  beam-splitter/scan-mirror  found  in 
the  Zeiss  LSM5  line  scanner  is  a  modification  of  this  idea  in  which 
use  is  made  of  the  fact  that  the  diffraction  pattern  of  a  line  is 
another  line.  Therefore,  while  a  small  silvered  line  in  the  center  of 
the  beam-splitter  is  sufficient  to  reflect  the  excitation  line  of  any 
wavelength  towards  the  objective,  most  of  the  returning  signal 
bypasses  it  towards  the  detectors. 

The  narrow-band  interference  filters  used  to  select  a  laser  line 
transmit  between  50%  and  90%,  depending  on  the  batch,  the  man¬ 
ufacturing  process,  and  the  design  bandwidth.  Dichroic  filters 
reflect  80%  to  90%  of  a  single  laser  line  and  should  transmit  more 
than  80%  of  the  light  12  to  20nm  above  the  maximum  reflection. 
Long-pass  color  filters  transmit  more  than  90%  of  the  light  12  to 
20  nm  above  the  50%  cut-off  wavelength.  Special  dichroics  can  be 
designed  that  have  a  particularly  high  reflectance  for  the  excitation 
laser  line.  In  these  cases  the  50%  cut-off  can  be  pushed  further  and 
a  difference  of  less  than  15nm  can  be  achieved  (see  also  Chapter 
3,  this  volume). 

Specifications  of  the  most  suitable  filter  depend  on  the  excita¬ 
tion  and  emission  spectra  of  the  fluorophore  and  therefore  on  the 
laser  line  used.  Thus,  compromises  can  hardly  be  avoided,  and 
carefully  adapting  the  filters  for  any  experiment  involving  more 
than  one  dye  is  worth  both  effort  and  time.  For  single-channel 
operation,  the  goal  is  to  use  a  long-pass  filter  or  a  broadband  in¬ 
terference  filter  that  comes  close  to  the  emission  maxima  in  the 
detection  channels.  If  necessary,  a  change  of  the  laser  wavelength 
(Kr/Ar-ion  lasers  offer  several  lines  between  458  nm  and  647  nm) 
or  a  different  dye  should  be  considered.  Note  that  dyes  often 
change  their  spectral  properties  when  coupled  to  antibodies  or 
incorporated  into  the  target  cell  (Chapters  16  and  17,  this  volume) 
and  filters  can  age  from  prolonged  exposure  to  intense  light,  par¬ 
ticularly  that  from  an  Hg-arc  lamp  without  an  ultraviolet  (UV) 
filter. 

Evaluation  of  Optical  Arrangements 

This  section  discusses  and  evaluates  different  optical  arrangements 
for  confocal  fluorescence  microscopes,  shows  a  drawing  of  the 
compact  confocal  camera  built  at  EMBL,  and  shows  the  spectro- 


FIGURE  9.3.  Schematic  drawing  of  the  compact  confocal  camera  built  at 
EMBL.  Several  lasers  (not  shown)  provide  light  through  an  excitation  fiber. 
The  collimated  beam  is  deflected  by  a  dichroic  mirror  (BSP  1)  towards  the  scan 
unit  (upper  left  corner).  The  scan  unit  consists  of  two  minors  that  deflect  the 
beam  in  two  orthogonal  planes  to  scan  it  in  the  object  plane.  The  fluorescence 
light  emitted  by  the  sample  (not  shown)  is  descanned  by  the  scan  unit,  trans¬ 
mitted  by  the  dichroic  mirror  BSP  1,  and  split  into  a  short-  and  a  long- 
wavelength  component  by  another  dichroic  mirror  (BSP  2).  Each  of  the  two 
beams  is  focused  into  a  pinhole  (pinholes  1  and  2).  Behind  the  pinhole,  each 
divergent  beam  is  focused  by  a  lens  and  split  once  more  by  a  dichroic  mirror 
(BSP  3  and  4).  The  resulting  four  convergent  beams  each  pass  a  detection  filter 
(filter  wheels  FW  1^1)  and  eventually  hit  a  photomultiplier  tube  (PMT  1-4). 

scopic  center  of  such  an  instrument,  that  is,  the  assembly  that  is 
used  to  separate  the  excitation  from  the  emission  light  (Fig.  9.3). 

Class  1 

In  a  class  1  system  [Fig.  9.4(A)],  the  illumination  pinhole  is  at  the 
focal  point  of  the  first  two  lenses  that  spatially  filter  and  expand 
the  collimated  laser  beam.  The  expansion  is  determined  by  the 
ratio  of  the  focal  lengths  of  the  two  lenses  and  should  be  chosen 
to  fill  the  entrance  pupil  of  the  objective  lens.  The  spatially  filtered 
beam  is  then  deflected  by  the  dichroic  mirror  in  the  direction  of 
the  scan  unit.  Fluorescence  light  emitted  from  the  sample  passes 
the  dichroic  mirror,  and  a  lens  focuses  the  beam  into  a  pinhole  in 
front  of  the  detector  (Fig.  9.4). 

Refraction  in  the  dichroic  mirror  causes  the  emission  beam  to 
shift  as  it  passes  through.  This  shift  is  intrinsic.  But,  because  the 
light  beams  from  the  focal  plane  are  collimated,  any  such  shift  can 
be  corrected  by  displacing  the  detection  lens  and  the  detector  by  a 
similar  amount.  However,  this  arrangement  cannot  correct  for  any 
tilt  produced  by  wedge  error  in  the  dichroic  mirror,  which  must  be 
carefully  controlled.  In  order  to  obtain  a  backscattered  light  signal 
a  polarizing  beam-splitter  and  a  quarter-wave  plate  can  be  placed 
between  the  beam  expander  and  the  dichroic  mirror  without 
decreasing  the  efficiency  of  the  detection  path.  This  setup  is 
optimal,  as  it  uses  a  small  number  of  elements,  guaranteeing  high 
detection  efficiency.  Furthermore,  it  uses  each  optical  element  as 
it  should  be  used;  for  example,  a  collimated  beam  passes  the 
dichroic  mirror  and  a  planar  wavefront  enters  an  intermediate  lens. 

Class  2 

In  a  class  2  system  [Fig.  9.4(B)],  the  excitation  light  is  focused 
into  a  pinhole  that  is  at  the  focal  point  of  two  lenses  arranged  as 
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FIGURE  9.4.  Three  different  optical  arrangements.  In  all  arrangements  the 
laser  beam  comes  in  from  the  right-hand  side,  is  spatially  filtered  by  a  pinhole, 
and  deflected  by  a  dichroic  mirror  in  the  direction  of  the  tube  lens  (not  shown). 
Note  that  refraction  of  the  fluorescence  light  when  passing  the  dichroic  mirror 
causes  a  shift  of  the  light  path.  The  illumination  and  detection  pinholes  are 
placed  in  conjugate  image  planes.  (A)  In  a  Class  1  arrangement,  the  laser  beam 
is  expanded  and  collimated  before  it  reaches  the  dichroic  mirror.  The  collimated 
fluorescence  light  is  focused  into  the  detector  pinhole  by  a  separate  lens  after 
passing  the  dichroic  mirror.  This  is  the  optically  ideal  arrangement.  (B)  In  a 
Class  2  arrangement,  the  laser  beam  is  expanded  and  collimated  after  it  hits 
the  dichroic  mirror,  so  the  beam  is  divergent  at  the  dichroic  mirror.  The  fluo¬ 
rescence  light  is  focused  into  the  detector  pinhole  by  the  same  second  lens  of 
the  beam  expander  before  passing  the  dichroic  mirror.  (C)  In  a  Class  3  arrange¬ 
ment,  the  laser  beam  is  collimated  before  it  reaches  the  dichroic  mirror,  and  is 
expanded  in  the  light  path  common  to  illumination  and  detection.  Therefore, 
the  beam  expander  (shown  as  a  Galilean  telescope)  reduces  the  diameter  of  the 
fluorescence  light  beam  that  enters  the  detector  placed  at  a  long  distance  from 
the  image  plane. 


a  beam  expander.  However,  the  dichroic  mirror  is  placed  such  that 
the  divergent  beam  is  deflected  by  its  surface  before  being  colli¬ 
mated.  The  second  lens  of  the  beam  expander  is  also  used  to  focus 
the  fluorescence  emission  into  the  detector  pinhole.  A  convergent 
detection  beam  therefore  passes  the  dichroic  mirror,  which  causes 
a  shift  and  a  reduced  geometric  focal  length.  This  arrangement  is, 
therefore,  not  able  to  correct  for  either  a  shift  of  the  beam  or  a  tilt 
of  the  beam  at  the  dichroic  mirror.  It  is  also  not  possible  to  insert 
a  beam-splitter  arrangement  to  make  use  of  the  back-scattered  or 
reflected  light.  The  arrangement  is  sparse  and,  therefore,  efficient 
but  the  dichroic  mirror  is  placed  in  a  convergent  beam,  which 
might  cause  trouble  because  the  transmission/reflection  properties 
of  the  dichroic  mirror  depend  on  the  angle  of  incidence,  and  the 
spectral  response  will  be  different  for  axial  and  non-axial  rays. 

Class  3 

In  a  class  3  system  [Fig.  9.4(C)],  the  fluorescence  light  passes  the 
dichroic  mirror  and  enters  a  detector  that  is  some  distance  away.  The 
beam  is  shifted  in  the  dichroic  mirror.  Any  further  shift  of  the  beam 
can  be  corrected  in  this  arrangement,  whereas  a  tilt  at  the  dichroic 
mirror  cannot.  The  setup  is  obviously  efficient,  as  it  uses  the 
minimal  number  of  optical  elements,  and  a  collimated  beam  passes 
the  dichroic  mirror.  Not  having  to  use  a  detection  lens  reduces 
chromatic  effects,  but  this  must  be  balanced  against  the  need  for  dis¬ 
tances  of  more  than  1  m  before  the  beam  can  enter  the  photo¬ 
multiplier  tube  (PMT).  This  “optical  lever”  is  usually  implemented 
by  placing  three,  four,  or  more  mirrors  into  the  detection  path,  a 
process  that  makes  the  device,  at  least  in  principle,  more  sensitive 
to  mechanical  vibrations.  However,  to  a  first  approximation,  the  dis¬ 
tance  between  two  conjugate  image  planes  divided  by  the  diameter 
of  the  detector  is  a  constant,  so  the  performance  of  this  arrangement 
should  be  equivalent  to  that  described  as  Class  1  [see  Fig.  9.4(A)]. 
A  detection  lens  will  decrease  the  path  length  but,  unless  it  is  set  up 
as  a  telescope,  it  also  decreases  the  effective  area  of  the  detector  and 
hence  makes  the  introduction  of  a  small  pinhole  necessary. 

The  advisability  of  using  a  beam  expander  [as  shown  in  Figs. 
9.4(A)  and  9.4(B),  but  not  in  Fig.  9.4(C)]  is  debatable.  The  beam 
can,  of  course,  be  expanded  elsewhere  in  the  system  (e.g.,  in  a  scan 
lens  such  as  the  eye-piece).  It  is,  however,  inefficient  to  spatially 
filter  the  beam  in  the  common  detection/illumination  path.  The 
main  problem  is  that  the  laser  may  not  run  in  TEM00  mode 
(Chapter  5,  this  volume).  If  it  does  not  do  so,  it  must  be  spatially 
filtered  and  then  it  may  also  be  expanded  in  the  same  process.  If 
the  laser  does  run  in  TEM00  mode,  the  light  need  not  be  spatially 
filtered  because  it  would  all  pass  the  filter  anyway.  A  single-mode 
polarization  preserving  fiber  can  be  regarded  as  a  pinhole  with  a 
distinct  entrance  and  a  distinct  exit.  Thus  it  will  always  select  the 
TEM00  mode  of  a  laser. 

Not  shown  in  Figure  9.4  are  filters  and  special  equipment  for 
double-fluorescence  experiments.  Although  all  could  be  placed  in 
front  of  the  second  lens,  it  is  better  to  put  them  behind  the  detec¬ 
tor  pinhole  (see  Fig.  9.3).  It  is,  therefore,  not  necessary  to  include 
these  parts  in  this  discussion.  A  practical  problem  that  may  arise 
if  a  single  pinhole  is  used  for  both  channels  is  that  one  channel 
may  receive  less  signal  than  the  other.  If  two  pinholes  are  used, 
one  can  be  opened  to  increase  the  signal  while  the  other  remains 
set  to  the  optimal  size  (Chapters  22  and  35,  this  volume). 

Evaluation  of  Scanner  Arrangements 

The  simplest  of  all  confocal  optical  systems  is  exemplified  by  that 
used  in  the  Optiscan  confocal  endoscope.  In  this  system,  light 
emerging  from  the  end  of  a  single-mode  optical  fiber  is  focused 
by  an  objective  lens  into  a  spot  inside  the  specimen.  Returning 
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light  is  picked  up  by  the  same  fiber  tip  that  also  acts  as  the  detec¬ 
tor  pinhole.  Fluorescence  light  is  separated  from  excitation  light 
using  interference  filters  mounted  at  the  far  laser  end  of  the  fiber. 
Scanning  is  accomplished  by  actually  moving  the  fiber  tip  in  an 
xy-raster  using  an  electromagnetic  drive.  Although  this  system  is 
extremely  compact,  the  detector  pinhole  dimensions  are  fixed  and 
there  must  always  be  some  back-reflection  from  the  scanning  tip 
that  will  be  added  to  any  backscattered  light  signal  from  the  spec¬ 
imen  (see  also  Chapter  26,  this  volume). 

However,  conventional  beam-scanning  confocal  microscopes 
use  scan  mirrors,  and  this  section  describes  and  evaluates  four 
different  arrangements  of  xy-scanners.  Note  that  not  all  optical 
arrangements  described  above  should  be  combined  with  all 
scanner  arrangements. 

The  galvanometer  mirrors  used  for  confocal  scanning 
microscopy  must  be  custom-made  for  high  efficiency  in  fluores¬ 
cence  applications.  These  mirrors  have  conflicting  specifications: 
they  should  be  small,  lightweight,  mechanically  stable,  and  flat  to 
a  fraction  of  the  wavelength,  and  the  reflecting  surface  should  be 
centered  on  the  rotation  axis  of  the  scan  unit.  The  compromise 
depends  to  a  large  degree  on  the  application  of  the  confocal  fluo¬ 
rescence  microscope  and  the  arrangement  of  the  scan  unit.  The 
emphasis  should  always  be  on  efficiency  and  instruments  that  are 
tuned  to  perform  best  with  high-NA  high-magnification  lenses.  An 
excellent  choice  is  currently  the  VM500  produced  by  GSI  Lumon- 
ics.  Its  specifications  meet  essentially  all  requirements  if  one  uses 
mirrors  that  are  smaller,  thicker,  and  flatter  than  the  standard  parts 
(see  also  Chapter  3,  this  volume). 

The  scan  mirrors  are  effectively  also  aperture  stops  so  their 
diameter  must  match  the  diameter  of  the  conjugate  BFP  (see  Table 
9.2).  Therefore,  high  NA  and  low  magnification  that  requires  a  large 
beam  diameter  needs  large  scan  mirrors  to  fill  the  BFP,  a  factor  that 
limits  horizontal  (actually  fast  axis)  scan  speed  (Fig.  9.5). 


Two  Scan  Mirrors  +  Relay  Optics 

The  two  scanning  mirrors  rotate  on  perpendicular  axes,  so  their 
combined  movement  produces  an  xy-movement  of  the  light  spot 
in  the  image  plane  [Fig.  9.5(A)].  As  the  relay  optics  form  an  image 
of  the  first  mirror  on  the  axis  of  the  second  mirror,  both  mirrors 
will  be  in  conjugate  telecentric  planes  of  the  microscope,  as  long 
as  the  axis  of  either  mirror  is  in  such  a  telecentric  plane.  This 
arrangement  is  optically  perfect.  The  difficulty  is  that  the  require¬ 
ments  for  the  relay  optics  are  very  strict.  They  must  produce 
perfect  images  over  a  broad  range  of  wavelengths,  and  this  is  quite 
expensive  and  probably  also  not  very  efficient. 

To  have  a  well-corrected  image  across  a  widefield,  one  can  also 
use  two  plane-scanning  mirrors  with  two  intermediate  concave 
mirrors,  as  was  done  in  some  now  obsolete  Bio-Rad  scan  heads 
(e.g.,  Amos,  1991). 

Two  Closely  Spaced  Scan  Mirrors 

Two  mirrors  can  also  be  used  without  any  relay  optics  [Fig. 
9.5(B)],  as  implemented,  e.g.,  by  Zeiss  in  the  LSM  5  Series  and 
more  recently  by  Leica  and  almost  all  other  companies).  Again  the 
mirrors  rotate  on  perpendicular  axes,  but  now  they  are  placed  as 
close  together  as  possible  with  the  telecentric  plane  located  at 
the  midpoint  between  them.  The  closer  the  two  mirrors  come 
together,  the  more  this  arrangement  approximates  perfect  telecen- 
tricity.  The  actual  error  depends  on  the  magnification  of  the  inter¬ 
mediate  optical  system  and  is  probably  about  4  mm  over  100  mm. 
In  general,  this  will  only  have  an  effect  at  large  scan  angles  (low 
zoom  magnification)  where  optimal  optical  performance  is  less 
essential.  The  advantages  of  small  size  are  obvious. 

Center  Pivot/Off- Ax  is  Pivot 

The  arrangement  described  above  can  be  improved  by  moving  the 
axis  of  one  mirror  off  the  beam  axis  [Fig.  9.5(C),  as  implemented 


FIGURE  9.5.  Four  different  scanner  arrangements.  (A)  A 
system  consisting  of  two  scan  units  and  intermediate  imaging 
optics.  Two  mirrors  are  used  to  scan  the  beam  along  the  x  and  y 
axes.  The  optical  system  is  arranged  such  that  an  image  of  one 
scan  mirror  is  formed  on  the  other  mirror.  If  either  mirror  is  in 
a  telecentric  plane,  both  mirrors  are  in  conjugate  telecentric 
planes  of  the  microscope.  This  is  the  basic  and  ideal  optical 
arrangement.  (B)  The  scan  mirrors  are  placed  as  closely  as  pos¬ 
sible  and  the  geometric  midpoint  of  the  two  mirrors  is  in  a  con¬ 
jugate  telecentric  plane  of  the  microscope.  Both  mirrors  are, 
therefore,  only  approximately  in  a  telecentric  plane.  (C)  An 
arrangement  with  off-axis  and  on-axis  scan  units.  While  one 
mirror  rotates  around  the  center  of  its  axis,  the  other  has  a  pivot 
point  that  is  off-axis.  This  latter  movement  compensates  for  the 
missing  intermediate  imaging  optics.  The  two  mirrors  are,  there¬ 
fore,  both  in  an  approximate  conjugate  telecentric  plane  of  the 
microscope.  (D)  A  single  scan  mirror  is  used  to  scan  the  beam 
along  both  the  x  and  y  axes.  This  is  accomplished  by  mounting 
the  mirror  on  a  fast  galvanometer,  which  is  itself  mounted  on  a 
second,  orthogonal  scan  unit  whose  center  of  rotation  is  in  the 
center  of  the  mirror. 
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initially  by  Molecular  Dynamics].  The  second  mirror  shifts  as  it 
rotates.  The  combined  movement  can  reduce  the  apparent  position 
error  of  the  mirror  by  a  factor  of  up  to  25  (Carlsson  and  Liljeborg, 
1989).  It  also  changes  the  path  length  slightly  as  the  y-scanner 
moves  to  the  next  line.  This  scanner  arrangement  may,  therefore, 
not  be  used  with  the  class  2  optical  arrangement  [see  Fig.  9.4(B)]. 

Single  Mirror/Double  Tilt 

The  simplest  solution  is  to  use  a  single  mirror  that  tilts  around  two 
orthogonal  axes  [Fig.  9.5(D),  as  implemented  in  the  early  Leica 
systems].  A  small  galvanometer  is  mounted  on  a  larger,  slow-scan 
unit.  The  galvanometer  performs  the  fast  movement  and  the  scan 
unit  provides  the  tilt  along  the  slow  axis.  The  center  of  the  single 
mirror  is  easily  placed  in  a  telecentric  plane  of  the  microscope,  and 
the  efficiency  is  only  limited  by  the  reflection  coefficient  of  this 
mirror  (Stelzer  et  al.,  1988).  However,  this  solution  does  not  allow 
interchanging  the  x  and  y  axes  for  scanning  arbitrarily  formed  areas 
(Trepte,  1996). 

Another  possibility  is  to  mount  a  single  mirror  on  three  linear 
translators.  By  appropriately  driving  the  three  devices  one  can  tilt 
a  mirror  along  any  axis  and  can  even  place  the  point  of  rotation 
on  the  mirror  surface.  Such  piezo-driven  devices  are  available 
from  a  number  of  manufacturers.  Although  they  currently  lack 
either  a  sufficiently  high  speed  or  a  large  enough  rotation  angle, 
they  are  definitely  an  excellent  choice  if  these  issues  are  not  crit¬ 
ical  and  a  compact  device  is  anticipated. 

Scanner  arrangements  can  be  evaluated  on  a  number  of  crite¬ 
ria.  The  most  important  parameter  for  a  confocal  fluorescence 
microscope  is  photon  efficiency.  Other  requirements  might  be  the 
speed  or  precision  with  which  the  raster  is  scanned,  the  size  and 
shape  of  the  raster  area,  the  size  of  the  scan  unit,  xy-distortion, 
long-term  stability/repeatability,  and  its  flexibility  in  scanning  non¬ 
standard  patterns. 

Scanners 

Z-Scanners 

To  get  a  three-dimensional  image  of  a  sample,  it  has  to  be  scanned 
in  three  directions.  Therefore,  we  need  to  discuss  the  options  for 
scanning  the  sample  axially,  that  is,  up  and  down  in  a  standard 
microscope.  The  two  principal  solutions  are  moving  the  beam 
while  keeping  the  object  stationary,  and  moving  the  object  while 
keeping  the  beam  stationary. 

A  movement  of  the  focus  by  some  special  intermediate  optics 
can  be  ruled  out  in  microscopy.  The  reasoning  is  quite  simple.  In 
any  telecentric  system,  the  axial  magnification  Mz,  is  the  square  of 
the  lateral  magnification  M.  This  relation  tells  us  that,  when  using 
a  40x  lens,  an  axial  movement  of  the  focus  in  the  object  by  10  pm 
produces  a  shift  of  the  focus  in  the  intermediate  image  plane  of 
16mm!2  Therefore,  it  does  not  make  sense  to  change  focus  posi¬ 
tion  by  moving,  for  example,  the  tube  lens  of  the  microscope  or  the 
scan  lens  along  the  optical  axis.  The  required  movement  distances 
are  by  far  too  large.  Therefore  the  only  solution  for  beam  scanning 
along  the  optical  axis  is  the  axial  movement  of  the  objective  lens. 

The  most  common  method  for  producing  z-motion  is  to  drive 
the  fine-focusing  control  of  the  microscope.  Some  manufacturers 
add  an  internal  fixed  reference  for  full  control  over  the  exact  axial 
position.  Another  technique  is  to  mount  the  lens  (or  the  slide  holder 


2  Some  confocal  endoscopes  utilize  special-purpose  objectives  that  contain 
moveable  elements  capable  of  shifting  the  focus  plane  (see  Chapter  26,  this 
volume). 


in  case  of  object  scanning)  on  a  piezoelectric  driver.  This  offers 
excellent,  backlash-free  operation  over  a  range  of  <100  pm. 
Several  manufacturers  have  mounted  an  extra  galvanometer  on  a 
standard  specimen  stage.  This  moves  the  specimen  by  tilting  a  sub¬ 
stage  a  few  degrees. 

The  following  points  are  to  be  considered:  any  movement 
along  the  optical  axis  affects  the  immersion-oil  film.  A  rapid  move¬ 
ment  that  increases  the  distance  between  the  objective  lens  and  the 
coverslip  may  cause  the  sample  to  “stick”  to  the  lens  or  to  break 
the  film  so  that  air  bubbles  penetrate  the  volume  between  the  cov¬ 
erslip  and  the  objective  lens.  All  this  could  result  in  a  shift  of  the 
absolute  position  of  the  object.  If  the  absolute  position  of  the  object 
is  important,  one  needs  either  a  “hard”  connection  between  the 
driving  mechanism  and  the  motor  or  an  independent  position  feed¬ 
back  sensor.  A  potentiometer  connected  to  the  motor  is  insufficient. 

The  resolution  of  the  z-axis  motion  must  be  determined 
according  to  the  needs.  A  confocal  fluorescence  microscope  for 
biological  applications  needs  not  be  more  precise  than  ±50  nm, 
whereas  an  instrument  for  confocal  reflection  microscopy  should 
be  able  to  determine  axial  position  with  a  resolution  better  than 
±4  nm. 

Disk  Scanners 

As  pointed  out  above,  the  pinholes  in  a  beam-scanning  device  are 
located  in  the  conjugate  image  planes  of  a  conventional  micro¬ 
scope.  They  can  be  placed  directly  into  the  first  image  plane.  In 
the  tandem  scanner  (Petran  et  al,  1968;  see  also  Chapter  10,  this 
volume)  part  of  a  rotating  disk  with  holes  that  serve  as  both  sources 
and  detector  pinholes  is  located  in  the  intermediate  image  plane. 
The  holes  are  arranged  such  that  the  object  is  uniformly  covered 
by  light  passing  through  them  as  the  disk  rotates.  The  separation 
between  adjacent  holes  guarantees  little  overlap  between  the  detec¬ 
tion  and  the  illumination  path  when  viewing  specimens  in  which 
the  dye  is  confined  to  a  relatively  thin  layer.  Atto  Bioscience’s 
CARV  module,  that  converts  a  conventional  fluorescence  micro¬ 
scope  into  a  real-time  confocal  system,  uses  a  Nipkow  disk  with 
a  set  of  spirally  arranged  pinholes  to  produce  about  1000  beams 
in  the  field  of  view  at  one  time.  These  can  scan  the  entire  field 
simultaneously  at  a  rate  of  400  to  1000  Hz.  Because  disk  scanners 
form  a  real  image,  a  charge-coupled  device  (CCD)  camera  is 
placed  in  an  image  plane  of  the  microscope  to  detect  the  photons 
with  a  quantum  efficiency  that  is  5  to  10  times  better  than  that  of 
a  PMT.  The  main  problems  are  the  limited  brightness  of  a  con¬ 
ventional  light  source  and  the  loss  of  97%  to  99%  of  the  excita¬ 
tion  light  at  the  disk.  Source  brightness  can  easily  be  increased  by 
using  a  higher-powered  lamp  or  a  laser  as  the  light  source,  but 
eventually  absorption  by  the  disk  causes  it  to  overheat,  limiting  the 
effectiveness  of  this  approach  (see  also  Chapter  10,  this  volume). 

There  can  also  be  a  problem  in  matching  the  pinhole  diameter 
to  the  specifications  of  the  objective  lens  (e.g.,  30pm  diameter  for 
63X/1.4  or  45  pm  diameter  for  100x/1.4).  As  a  result,  one  disk  is 
optimal  for  only  a  few  lenses.  In  addition,  if  the  disk  resides  in  the 
intermediate  image  plane,  the  source  and  detector  pinholes  cannot 
be  optimized  independently  (Chapter  11,  this  volume).  Ideally,  pin¬ 
holes  of  this  size  should  also  be  very  thin,  but  this  is  in  conflict 
with  the  mechanical  stability  of  the  disk. 

However,  the  most  challenging  aspect  of  the  intermediate 
optics  of  the  tandem  scanner  is  the  difficulty  of  adjusting  the 
various  mirrors  so  that  the  returning  pattern  of  points  exactly  coin¬ 
cides  with  the  pattern  of  holes  on  the  detector  side  of  the  disk. 
Assuming  40  pm  holes  and  a  160  mm  fixed  conjugate  objective 
lens,  each  hole  in  the  field  should  be  aligned  with  an  accuracy  of 
±10  pm  in  x  and  y  over  a  field  of  ~25  mm.  Apart  from  the  need  to 
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FIGURE  9.6.  The  optical  system  of  the  Yokogawa  spinning  disk  microscope.  An  array  of  20,000  microlenses  is  mounted  on  a  disk.  Approximately  1000  of 
them  are  illuminated  at  one  time,  and  each  of  them  focuses  the  light  into  a  particular  pinhole  below  it  on  a  second  disk.  Both  disks  rotate  synchronously  at 
30  Hz,  and  because  12  complete  images  are  formed  per  rotation  (a  segment  of  30°  gives  a  complete  image),  an  image  acquisition  rate  of  360  Hz  is  achieved 
(Fukui  et  al. ,  1999;  Maddox  et  al.,  2003). 


exactly  orient  the  mirror  angles  so  as  not  to  distort  the  pattern  of 
dots,  the  total  length  of  the  illumination  and  detection  paths  must 
match  to  about  ±10|im  to  prevent  change  in  magnification  from 
causing  the  outer  spots  to  miss  their  holes.  Not  surprisingly, 
tandem  scanners  tend  to  use  fairly  large  pinholes. 

Some  of  the  problems  of  the  tandem  scanner,  such  as  disk 
thickness  and  alignment,  are  avoided  by  the  single-sided  disk 
scanner,  which  uses  a  tilted  quartz  disk  coated  with  an  opaque 
metal  film  except  where  the  pinholes  are  located  (Chapter  10,  this 
volume).  As  the  same  area  of  the  disk  is  used  for  both  excitation 
and  detection,  there  is  almost  no  alignment  problem,  but  there  can 
be  problems  with  the  illumination  light  reflecting  off  the  disk  and 
obscuring  the  image  data  coming  through  the  pinholes.  Clever 
polarization  techniques  can  reduce  these  reflections  to  a  remark¬ 
able  extent,  and  they  are  less  of  a  problem  when  recording  fluo¬ 
rescence  images  because  then  the  light  reflected  from  the  disk  can 
be  excluded  on  the  basis  of  its  wavelength.  However,  it  is  still  dif¬ 
ficult  to  convey  to  the  specimen  sufficient  light  in  a  narrow  wave¬ 
band  to  permit  high-resolution  fluorescence  images  to  be  recorded 
in  a  reasonable  time  span. 

The  optical  efficiency  problem  has  been  solved  by  Yokogawa 
in  a  design  that  is  used  by  PerkinElmer,  Solamere,  Andor  (Fig. 
9.6),  and  others.  A  microlens  array  is  mounted  on  a  rotating  disk. 
Each  lens  focuses  the  light  from  a  laser  into  a  particular  pinhole 
located  one  focal  length  below  it  on  a  second  disk,  such  that  more 
than  40%  of  the  light  passes  the  pinholes  (Tanaami  et  al.,  2002). 
The  dichroic  beam-splitter  is  placed  between  the  disks,  such  that 
the  detection  light  does  not  pass  the  microlenses  (Fig.  9.6). 

Object  Scanners 

The  first  operating  confocal  microscopes  were  stage  scanners 
(Minsky,  1961;  1988;  Brakenhoff  et  al.,  1979;  Marsman  et  al., 
1983;  Stelzer  and  Wijnaendts-van-Resandt,  1985),  and  those 
which  scanned  the  beam  by  moving  the  objective  lens  (Wilke, 
1985).  One  source  and  one  detector  pinhole  are  placed  in  conju¬ 
gate  image  planes.  The  optical  arrangement  is  stationary,  and  the 
object  is  scanned  through  a  single  beam  that  is  located  on  the 
optical  axis  of  the  lens.  Any  geometrical  distortion  in  the  recorded 
image  can  come  only  from  the  imperfect  control  of  the  motion  of 


the  object.  Object  scanners,  therefore,  offer  the  highest  quality 
image  for  data  processing  purposes  and,  because  of  their  simplic¬ 
ity,  they  are  very  photon  efficient.  Disadvantages  include  a  low 
scanning  speed  (10-150  lines/s)  and  the  special  specimen  prepa¬ 
ration  techniques  necessary  to  keep  the  moving  mass  low. 

The  two  overriding  considerations  that  have  limited  the  use  of 
this  approach  in  biological  confocal  microscopy  are  that  early 
instruments  employing  this  technique  did  not  permit  the  observer 
to  switch  to  normal  non-confocal  operation  easily,  in  order  to  scan 
the  specimen  and  find  the  area  of  interest,  and  that  biologists  were 
concerned  that  mechanical  resonance  within  the  specimen  would 
cause  it  to  move  relative  to  the  stage,  causing  disruption  of  the  cell 
and  distortion  of  the  image.  Although  the  force  on  the  specimen 
due  to  the  scanner’s  acceleration  is  usually  very  low  (about  1%  of 
the  gravity  for  an  amplitude  of  100|im  and  a  frequency  of  10Hz, 
but  1G  at  100  Hz),  it  can  affect  the  behavior  of  living  specimens 
suspended  in  viscous  media  (Fig.  9.7). 


FIGURE  9.7.  Optical  arrangements  using  fibers.  The  arrangement  of  Class  1 
uses  illumination  with  an  SMPP  fiber,  whose  tip  is  in  the  focal  point  of  a  lens 
that  collimates  the  beam.  Complete  plugs  for  laser-light  delivery  systems  consist 
of  the  tip  and  a  short-focus  achromatic  lens.  The  detection  can  be  performed 
with  a  conventional  system,  employing  a  multi-mode  fiber  placed  behind  a 
detector  pinhole,  or  using  the  same  single-mode  fiber  used  as  the  source. 
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Attachment  to  Microscopes 

A  conventional  microscope  offers  several  ports  through  which  a 
confocal  setup  can  be  attached  to  it  and  through  which  the  excita¬ 
tion  beam  can  enter  and  the  emission  beam  can  leave  the  instru¬ 
ment.  Commonly  used  are  the  video  port,  the  ocular,  or  the  port 
normally  used  for  the  photographic  camera.  Having  mentioned 
efficiency  as  one  of  the  main  issues  in  confocal  fluorescence 
microscopy,  it  should  be  obvious  that  it  is  preferable  to  use  the  port 
that  causes  the  beam  to  pass  the  smallest  number  of  optical  ele¬ 
ments.  Many  (but  not  all)  inverted  microscopes  guide  the  detec¬ 
tion  light  over  long  distances  and  many  optical  elements  between 
the  object  and  the  ocular  (or  video  port).  If  for  some  important 
reason  such  an  inverted  microscope  is  the  system  of  choice,  one 
should  consider  having  the  instrument  modified  to  permit  confo¬ 
cal  operation  using  a  simpler,  separate  path.  Such  modifications 
are  now  easier  because  both  Nikon  CF  objectives  and  Zeiss  ICS 
objective  lenses  (plus  a  tube  lens)  are  fully  chromatically  cor¬ 
rected,  and  the  attachment  can  be  performed  through  the  interme¬ 
diate  image  plane  without  the  need  for  any  additional  optics. 

Among  upright  microscopes,  the  video  port  of  the  Zeiss 
Axioimager  series  provides  direct  access  to  the  intermediate  image 
plane.  A  confocal  setup  may  be  interfaced  to  this  plane  with  no 
extra  optical  elements.  Microscopes  from  other  manufacturers  also 
have  this  capability.  The  advantage  of  scanning  through  video 
ports  and  eyepieces  is  that  no  modification  of  the  microscope  is 
required. 

Merit  Functions 

Functions  that  determine  the  performance  of  an  instrument  can  be 
object  dependent  and  object  independent.  An  object-dependent 
merit  function  is  the  product  of  all  transmission  and  all  reflection 
coefficients  weighted  with  the  emission  and  absorption  spectra  of 
a  particular  fluorophore  (Tables  9.3  and  9.4).  The  sum  over  all 
surface  errors  and  the  sums  over  all  tilts,  thickness  variations,  and 
deviations  from  planarity  are  object  independent. 

Object-dependent  functions  can  be  used  to  measure  the  per¬ 
formance  of  an  instrument  and  to  compare  the  data  with  those  gen¬ 
erated  at  later  times  or  by  other  instruments. 

This  is  not  so  easy  with  the  other  four  merit  functions  that  are 
more  likely  to  be  used  in  the  design  phase  of  an  instrument  to 
decide  which  development  path  should  be  followed.  The  sum  over 
all  surface  errors  should  include  all  elements  beyond  the  source 
pinhole.  It  determines  how  much  the  beam  deviates  from  a  perfect 


TABLE  9.3.  Typical  Intensity  Losses  in  the  Illumination  Path 


Element 

Loss  (%) 

Remainder  (%) 

Laser 

100 

Mirror 

1-3 

98 

1-3  dichroic  mirrors  for  several  lasers 

3-10  per  mirror 

88 

AOM 

10-15 

75 

Fiber 

25 

56 

Collimator/beam  expander 

4 

54 

Mirrors 

1-3  per  mirror 

52 

Polarizing  beam-splitter  and  A/4-plate 

4 

50 

Dichroic  mirror 

10-20 

42 

Scanner 

4 

41 

Relay  lens 

10 

37 

Inside  microscope 

10 

33 

BFP  of  objective  lens 

80 

7 

Total  intensity  entering  the  sample 

7 

TABLE  9.4.  Typical  Intensity  Losses  in  the  Detection  Path 


Element 

Loss  (%) 

Remainder  (%) 

Sample  emission 

100 

Collection  by  high-NA  lens 

75 

25 

Inside  microscope 

10 

23 

Relay  lens 

10 

20 

Scanner 

4 

19 

Dichroic  mirror 

25-30 

15 

Collimator/beam  expander 

4 

14 

Dichroic  mirror 

20 

11 

Filter 

40 

7 

Detector 

85 

1 

Total  intensity  detected 

1 

planar  wave  front,  hence  how  well  the  beam  can  be  focused  and 
how  much  the  objective  lens  must  be  over-illuminated.  As  a  bench¬ 
mark,  it  is  important  to  remember  that  a  wavefront  error  of  only 
^74  reduces  the  contrast  at  the  midpoint  of  the  spatial  frequency 
plot  by  almost  50%. 

MULTI-FLUORESCENCE 

Multi-fluorescence  experiments  are  those  in  which  one  observes 
specimens  containing  more  than  one  fluorophore  labeling  differ¬ 
ent  targets.  Such  experiments  can  be  performed  by  observing  each 
target  separately  or  by  observing  two  or  more  targets  at  the  same 
time.  In  the  first  case,  specific  filter  combinations  would  be 
selected  for  each  fluorophore,  matching  the  excitation  line  and  the 
emission  band.  In  the  second  case,  one  excitation  line  could  excite 
emission  from  two  or  more  fluorophores  and  this  signal  would 
have  to  be  separated  later  in  the  detection  path. 

The  main  disadvantage  of  the  first  approach  is  the  difficulty  of 
inserting  several  different  filter  sets  without  changing  the  illumi¬ 
nation  or  detection  paths.  Any  change  prevents  the  individual 
images  from  being  properly  aligned  with  each  other.  The  amount 
of  shift  that  can  be  tolerated  depends  on  the  resolution  of  the  objec¬ 
tive  lens  used.  In  addition,  as  was  pointed  out  above,  the  images 
of  the  source  and  the  detector  pinholes  must  overlap  in  the  object 
plane.  If  all  the  dichroic  mirrors  used  do  not  have  the  same  orien¬ 
tation  and  position,  the  signal  will  not  fall  into  the  pinhole  and  will 
be  lost.  On  the  other  hand,  observing  each  fluorophore  separately 
does  provide  the  necessary  degrees  of  freedom  needed  for  the 
optimal  choice  of  filter  sets  to  observe  each  dye. 

Separating  two  emission  bands  for  measurement  after  simul¬ 
taneous  excitation  with  a  single  line  is  possible  only  with  a  rela¬ 
tively  low  efficiency  because  the  narrow  bandpass  of  the  two  filter 
sets  will  cut  off  a  lot  of  otherwise  usable  spectrum.  In  addition,  if 
only  a  single  excitation  wavelength  is  used,  one  of  the  dyes  will 
almost  always  be  excited  less  efficiently  than  the  other.  However, 
the  advantage  of  simultaneous  excitation  is  that  the  corresponding 
pixels  of  the  two  images  are  definitely  in  register  because  a  single 
spot  of  incident  light  is  the  source  of  both  of  them.  If  the  color 
separation  filters  are  placed  behind  the  pinholes,  this  setup  is  also 
unaffected  by  changes  of  filter  sets.  A  problem  that  cannot  be  dealt 
with  so  easily  is  the  cross-talk  between  channels,  caused  by  the 
limited  spectral  performance  of  the  filters. 

Another  solution  that  also  adds  to  the  stability  of  the  instru¬ 
ment  is  the  use  of  double  and  triple  dichroics,  that  is,  dichroic 
mirrors  that  are  designed  to  deflect  several  laser  lines  and  let 
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several  emission  bands  pass  (or  vice  versa).  They  need  not  be 
moved  in  order  to  use  different  laser  lines,  whether  these  are  used 
simultaneously  or  not,  and  they  can  even  be  used  in  combination 
with  an  acousto-optical  element  to  switch  laser  lines  on  a  pixel- 
by-pixel  basis.  The  most  critical  element  for  the  proper  overlap  of 
the  illumination  and  the  detection  paths  is  the  dichroic  mirror,  and 
therefore  using  such  a  double  or  triple  dichroic  is  ideal  for  this 
purpose.  The  problem  is  that  multiple  dichroics  (tri-chroics,  tetra- 
chroics,  .  .  .)  have  slightly  reduced  transmission  between  the 
reflection  bands  because  of  the  design  compromises  needed  to 
produce  their  complex  spectral  properties,  and  that  they  only 
match  a  small  number  of  fluorophore  combinations.  As  with  every 
design,  the  user  has  to  decide  between  efficiency  and  alignment. 

The  acousto-optical  beam-splitter  (AOBS,  see  Chapter  3)  is  a 
system  recently  invented  by  Leica  (Birk  et  al,  2002),  which  can 
replace  the  dichroic  mirrors  that  usually  separate  the  illumination 
and  detections  paths  in  confocal  microscopes.  The  advantage  of 
the  AOBS  is  that  it  does  not  have  fixed  transmission  and  reflection 
bands,  but  that  it  can  be  programmed  to  adapt  from  single  to  multi- 
chromatic  characteristics  of  any  type.  This  adds  flexibility  for 
working  with  different  combinations  of  laser  lines  and  fluo- 
rophores.  In  practice,  the  AOBS  is  programmed  such  that  it  leaves 
most  of  the  fluorescent  light  undeflected,  while  the  laser  lines  are 
deflected  by  a  fixed  angle.  In  addition,  the  AOBS  is  more  efficient 
in  terms  of  transmittance  and  reflection.  This  allows  one  to  work 
with  less  laser  power  and  to  detect  with  an  improved  signal-to- 
noise  ratio.  These  benefits  are  particularly  valuable  when  working 
with  multiple  detection  channels. 

An  interesting  option  is  the  use  of  two-photon  excitation  (Denk 
et  al.,  1990;  Stelzer  et  al.,  1994;  see  also  Chapters  28  and  37,  this 
volume)  to  excite  not  only  UV  dyes  but  also  practically  any  cur¬ 
rently  used  visible  dye  (Fischer  et  al.,  1994).  The  required  filter 
sets  are  of  a  very  simple  design  and  extremely  efficient  because 
excitation  and  emission  lines  are  spectrally  very  far  apart.  It  should 
be  no  real  problem  to  use  this  technique  in  any  of  the  existing  con- 
focal  instruments  in  many  implementations  —  however,  because 
no  fluorescence  light  is  generated  away  from  the  plane  of  focus, 
the  detector  pinhole  is  unnecessary.  Instruments  used  for  a 
combination  of  single-photon  and  two-photon  excitation  should 
probably  have  provisions  for  both  confocal  and  widefield 
(non-descanned)  detection. 

SPECIAL  SETUPS 

Setups  for  Fluorescence  Recovery  After 
Photobleaching  Experiments 

Bleaching  experiments  use  lasers  to  bleach  a  spot,  an  area,  or  an 
arbitrary  pattern  into  a  fluorescent  sample.  Bleaching  refers  to  the 
removal  of  the  fluorescent  response  of  the  target,  ideally  without 
damaging  the  target  itself.  The  experiment  consists  of  the  obser¬ 
vation  of  time-dependent  changes  in  the  bleached  pattern,  that  is, 
recording  changes  in  the  structure  of  the  bleach  patterns  or  record¬ 
ing  the  fluorescence  intensity  recovery  in  certain  locations.  The 
confocal  beam  scanner  is  in  principle  an  ideal  device  for  generat¬ 
ing  bleach  patterns  or  for  applying  a  laser  light  pulse  of  a  well- 
defined  intensity  to  a  particular  location.  It  is  also  quite  well  suited 
to  observe  those  locations  before  and  after  the  application  of  the 
bleaching  light  pulse.  For  the  bleaching  process,  the  main  require¬ 
ment  is  the  capability  to  drive  the  scanners  in  a  more  or  less  arbi¬ 
trary  path  across  the  sample  and  to  control  the  laser  power. 
Although  bleaching  occurs  unavoidably  during  any  observation 


process  (excitation  consumes  a  fluorophore  with  a  certain  proba¬ 
bility)  it  is  usually  rather  slow.  For  typical  FRAP  experiments  one 
wishes  to  bleach  fast.  The  whole  pattern  should  be  generated 
within  much  less  than  a  second.  This  typically  requires  lasers  that 
provide  a  higher  power  than  those  required  for  simply  imaging 
(i.e.,  milliwatts,  not  microwatts).  For  FRAP  experiments  it  also 
makes  sense  to  provide  for  an  efficient  illumination  path.  So,  apart 
from  needing  a  higher-powered  laser  and  appropriate  software,  all 
confocal  beam-scanning  fluorescence  microscopes  should  be 
suited  for  FRAP  experiments. 

It  should  not  be  forgotten  that  single-photon  bleaching  occurs 
along  the  entire  illumination  path.  Hence  it  is  not  restricted  to  the 
focal  volume,  but  is  more  intense  there  if  only  a  small  area  is 
exposed.  Multi-photon  excitation  can  be  used  to  bleach  in  a  well- 
defined  three-dimensional  location  (Brown  et  al,  1999;  Chapters 
28  and  35,  this  volume). 

Setups  for  Fluorescence  Resonance  Energy 
Transfer  Experiments 

In  a  fluorescence  resonance  energy  transfer  experiment  (Clegg, 
1996),  one  tries  to  determine  whether  two  fluorophores  and  hence 
two  proteins  are  located  within  a  few  nanometers  of  each  other. 
Under  certain  circumstances,  an  excited  fluorophore  can  be 
quenched  by  a  neighboring  fluorophore  if  its  own  emission  band 
overlaps  the  excitation  band  of  the  neighbor.  In  the  spectral 
domain,  this  is,  in  principle,  a  multi-fluorescence  experiment 
where  only  one  fluorophore  is  excited  directly  by  the  laser  light, 
and  any  confocal  fluorescence  microscope  should  be  equipped  to 
perform  such  experiments.  In  the  temporal  domain,  one  requires 
special  equipment  (not  only  software  but  also  hardware)  that  con¬ 
trols  the  illumination  and  detection  timing. 


Setups  for  the  Integration  of  Optical  Tweezers 

A  particularly  interesting  application  of  lasers  in  biology  is  the  use 
of  focused  beams  to  capture  and  move  small  objects  such  as  cells 
or  bacteria  (Rohrbach  and  Stelzer,  2001;  Rohrbach  et  al.,  2004). 
One  usually  employs  lasers  that  provide  powers  in  the  milliwatt 
range.  To  reduce  damage  due  to  absorption  and  heating,  these  lasers 
operate  in  the  near-infrared  range  (NIR,  800-1  lOOnm).  Once  again 
in  principle,  the  beam-scanning  confocals  are  equipped  for  such 
purposes.  However,  optical  elements  are  optimized  for  the  visible 
range  of  wavelengths  (400-750nm)  and  may  fail  to  work  well  at 
the  particularly  important  wavelength  of  Nd:YAG  lasers,  1064  nm. 
It  is,  therefore,  more  appropriate  to  design  a  separate  optical  path 
with  a  separate  scanner  pair,  optimized  mirrors,  well-adapted  beam 
expanders,  and  an  appropriate  scan  lens  and  to  feed  the  NIR  light 
into  the  system  through  a  separate  port. 

Used  in  continuous  mode,  multi-photon  lasers  provide  the 
right  laser  powers  for  optical  tweezers.  When  operated  in  a  slow 
scanning  mode  (<100  lines/s)  they  can  move  a  lot  of  biological 
material  towards  the  borders  of  the  scan  area. 


Setups  for  the  Integration  of  Laser  Cutters 

Pulsed  UV  laser  beams  and,  to  a  certain  extent,  the  pulsed  NIR 
laser  beams  used  for  multi-photon  microscopy,  can  be  used  to  cut 
biological  material  (see  Chapter  38,  this  volume).  Once  again  being 
able  to  position  the  laser  beam  and  to  control  its  power  are  the 
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crucial  aspects.  However,  the  wavelength  of  the  UV  lasers 
(~355nm)  is  outside  the  specifications  of  the  visible  path  and  a 
separate  optimized  path  should  be  employed.  A  serious  issue  is  that 
the  axial  foci  of  the  UV  as  well  as  of  the  NIR  beams  can  be  several 
tens  of  micrometers  away  from  the  focal  plane  of  the  visible  light. 
Using  slightly  divergent  or  convergent  beams,  respectively,  is  a 
mediocre  solution.  Much  better  is  to  refocus  the  beam  during  the 
cutting  procedure,  that  usually  lasts  much  less  than  a  second,  by 
placing  the  lens  on  a  piezo-driven  positioner. 

Optical  tweezers  as  well  as  laser  cutters  work  very  well  with 
regular  widefield  microscopes.  It  makes  perfect  sense  to  use  a 
beam  steering  device  for  either  or  both  of  these  applications 
and  to  observe  the  effect  in  the  considerably  faster  widefield  video 
mode  (Grill  etal. ,  2001;  Grill  etal.,  2003;  Colombelli  etal.,  2004). 

Setups  for  the  Observation  of  Living  Specimens 

The  main  issue  in  the  observation  of  living  specimens  is  to  reduce 
the  exposure  of  the  sample  to  light  as  far  as  possible.  The  sample 
should  only  be  illuminated  when  a  signal  that  is  relevant  for  the 
experiment  is  recorded  or  when  the  system  is  set  up  prior  to  the 
experiment.  Hence,  speed  and  precision  of  operation  and  ease  of 
control  are  crucial  elements  of  a  high-quality  instrument.  An  effi¬ 
cient  light  path,  a  precise  control  of  the  laser  power,  and  the  ability 
to  turn  off  the  laser  beam  during  retrace  or  between  frames  are  just 
a  few  of  many  effective  procedures. 

MINIATURIZATION  AND 
COMPUTER  CONTROL 

Usually,  confocal  microscopes  are  built  around  a  conventional 
microscope  by  attaching  a  new  intermediate  optical  system  to  its 
body.  However,  because  image  acquisition  in  a  confocal  micro¬ 
scope  is  computer  controlled,  it  is  questionable  whether  this  solu¬ 
tion  is  appropriate.  Utilizing  the  body  and  the  basic  mechanical 
and  optical  elements  of  a  conventional  microscope  puts  mechani¬ 
cal  and  physical  constraints  on  the  design  of  the  optical  system  that 
are  mostly  dictated  by  anatomical  considerations.  Indeed,  the  size 
of  the  instrument  (and  thus  the  length  of  the  optical  paths)  is 
adapted  to  the  posture  of  the  user.  Optical  corrections  are  per¬ 
formed  to  ensure  that  the  image  of  the  sample  is  well  perceived 
through  the  observer's  eyes.  Because  the  most  important  issue  in 
confocal  fluorescence  microscopy  is  efficiency,  it  seems  reason¬ 
able  to  put  these  constraints  aside  and  to  start  a  new,  computer- 
centered  design,  in  which  principles  such  as  miniaturization  and 
computer-delegation  are  put  first.  This  also  provides  a  number  of 
other  advantages  such  as  higher  precision  and  long-term  stability, 
features  that  are  required  for  high-resolution  studies  of  living 
biological  specimens.  In  addition,  the  state  of  the  microscope  at 
any  point  in  time  is  known  to  the  computer.  Each  setting  is  docu¬ 
mented  and  therefore  reproducible.  Manual  intervention  other  than 
placing  the  sample  in  the  microscope  is  not  necessary  and  not 
desirable. 

Important  points: 

•  Use  of  digital  detectors/cameras 

•  Corrections  made  by  software 

•  Instrument  is  not  bigger  than  a  pile  of  books 

•  Feasibility  of  quantitative  microscopy 

•  Good  temperature  control  due  to  small  size 

•  Suitable  for  implementation  of  structured  illumination 
(Gustafsson,  1999) 


THERMAL  STABILITY 

The  increasing  use  of  confocal  microscopes  for  the  acquisition  of 
large  three-dimensional  data  sets  and  for  time-lapse  experiments 
generate  a  concern  about  the  long-term  thermal  stability  of  the 
instruments.  Recent  systematic  investigations  (Adler  and  Pagakis, 
2003,  of  the  effect  of  temperature  fluctuations)  have  shown  that 
images  are  distorted  due  to  movements  of  the  microscope  stage,  in 
particular  along  the  optical  axis.  For  room  temperature  variations 
of  several  degrees,  focal  shifts  of  about  2  pm  have  been  observed. 
This  can  result  in  a  stretched  or  compressed  image  of  the  sample, 
depending  on  the  direction  of  the  shift.  Such  temperature  variations 
can  easily  occur  in  laboratories  with  many  heat  sources  (comput¬ 
ers,  lasers,  lamps,  etc.),  or  where  the  temperature  is  cooled  down 
cyclically  by  air  conditioning.  For  long  time-lapse  experiments  that 
run  during  the  night,  one  should  remember  that  the  heat  load  will 
inevitably  be  lower  when  the  laboratory  is  less  busy. 

In  addition  to  the  large  heat  sources  already  mentioned,  heaters 
close  to  the  sample  are  obviously  a  concern,  in  particular  sample 
chamber  heaters,  stage  heaters,  and  objective  lens  heaters.  Other 
temperature-sensitive  parts  that  can  be  sources  for  thermal  distor¬ 
tions  of  the  image  are  the  immersion  oil.  mounting  liquids,  the 
light  sources,  the  acousto-optic  tuning  filter  (AOTF),  and  fiber- 
optics  (Swedlow  et  al.,  2002). 

But,  generally  speaking,  the  thermal  stability  of  modem  com¬ 
mercial  microscopes  is  quite  well  taken  care  of. 

VIBRATION  ISOLATION 

Although,  vibration-isolation  tables  just  mechanically  isolate  the 
instrument  from  the  environment,  usually  some  more  sophisticated 
versions  can  actively  damp  noise  sources  created  by  the  instrument 
itself.  The  only  real  vibration  sources  in  a  confocal  microscope  are 
the  scan  mirrors  and  the  cooling  system  of  the  laser  head.  The  latter 
source  can  be  eliminated  if  the  laser  head  is  not  mounted  on  the 
same  table  as  the  microscope  and  the  light  is  carried  to  the  micro¬ 
scope  by  an  optical  fiber.  Figure  9.7  shows  how  an  arrangement 
of  Class  1  uses  illumination  and  detection  with  fibers  (see  also 
Chapter  26). 

One  should  also  remember  that  mechanical  damping  only 
works  when  all  the  optical  elements,  particularly  heavy  ones,  are 
firmly  fixed  to  the  table.  Otherwise  the  movement  of  the  different 
elements  will  be  independent  and  not  in  phase.  Under  any  cir¬ 
cumstances,  damping  mechanical  vibration  is  a  complicated  task, 
and  it  is  often  fairly  easy  to  do  something  that  seems  straightfor¬ 
ward  but  that  actually  makes  things  worse. 

CONCLUSIONS  AND  FUTURE  PROSPECTS 

The  most  important  issue  in  any  kind  of  light  microscopy  is  photon 
efficiency,  as  this  determines  the  gap  between  photo-damage  and 
useful  contrast  in  the  recorded  image.  As  long  as  the  chosen  optical 
arrangement  maintains  the  necessary  diffraction  limit,  the  general 
rule  is  that  the  more  sparse  an  optical  path,  the  more  efficient  it 
will  be.  The  scan  system  should  be  chosen  in  accordance  with 
photon  efficiency.  Because  a  number  of  different  scanning  micro¬ 
scopes  have  been  realized,  each  of  them  employing  different 
optical  and  scanning  arrangements,  time  should  show  which  are 
the  most  efficient  and  the  most  versatile  to  use.  My  personal 
preferences  are  the  optical  arrangement  shown  in  Figure  9.4(A) 
and  the  scanner  arrangements  shown  in  Figures  9.5(B)  and  9.5(D). 
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Sparsity,  relative  simplicity,  and  good  illumination  efficiency 
are  advantages  of  single-beam  scanners  over  disk  scanners.  The 
disadvantage  of  single-beam  scanners  is  that  they  have  to  use  rel¬ 
atively  inefficient  PMT  detectors  to  achieve  high  recording  speeds. 
It  is  also  a  mistake  to  assume  a  technological  standstill.  The  large 
number  of  companies  and  the  short  product  lifetimes  indicate  that 
much  can  still  be  done.  In  the  short  term,  major  improvements  are 
more  likely  to  be  found  in  the  fluorescent  dyes,  in  the  biological 
application,  or  in  other  biochemical  developments.  Particularly 
useful  alternatives  are  the  disk  scanning-devices.  They  offer  an 
excellent  compromise  between  speed  and  efficiency,  and  the  new 
electron-multiplying  charge-coupled  device  (EM-CCD)  cameras 
give  the  disk  scanners  a  detector  that  is  very  comparable  to  the 
PMT  (see  Chapters  10  and  12,  this  volume). 

Ceterum  censeo  that  conventional  fluorescence  microscopes 
are  excellent,  reliable,  and  easy-to-use  optical  instruments.  A  high- 
quality,  conventional  optical  microscope  should  always  be  the  first 
instrument  of  choice.  Those  scientists  who  do  not  achieve  the 
expected  image  quality  should  consider  attaching  a  high-quality, 
possibly  cooled  CCD  camera  (Hiraoka  et  al.,  1987)  to  their  con¬ 
ventional  widefield  microscope.  Using  an  efficient  CCD  camera, 
a  powerful  computer,  and  appropriate  software  to  discriminate 
against  out-of-focus  light  will  in  some  cases  be  a  simpler,  and 
perhaps  an  even  more  efficient,  solution  to  many  relevant  biolog¬ 
ical  problems  (Agard  and  Sedat,  1983;  Fay  et  al.,  1989;  see  also 
Chapters  23  and  24,  this  volume). 
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Disk-Scanning  Confocal  Microscopy 
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INTRODUCTION 

Rapid  biological  imaging  of  faint  fluorophores  in  living  cells  — 
especially  in  four  dimensions  [three  dimensions  +  time]  — 
imposes  different  instrumentation  challenges  from  slowly  acquir¬ 
ing  a  single  high-resolution  confocal  snapshot  of  fixed  tissue.  High 
acquisition  speeds  with  acceptable  contrast  and  minimal  photo- 
bleaching  suddenly  become  essential,  all  without  losing  the  instan¬ 
taneous  optical  sectioning  that  a  confocal  microscope  affords.  Of 
particular  interest  here,  disk-scanning  confocal  microscopes  are 
proving  to  be  a  powerful  tool  in  rapid  imaging  of  live  cells  in  space 
and  time.  While  the  principle  is  relatively  old,  new  instrument 
developments,  both  in  optics  and  in  novel  ultra-sensitive  charge- 
coupled  device  (CCD)  cameras  are  greatly  expanding  the  versatil¬ 
ity  and  scope  of  this  approach.  However,  knowing  which  system 
is  best  for  a  given  question  and  understanding  the  inherent 
strengths  and  weaknesses  is  not  easy,  especially  as  many  of  the 
choices  involve  complex  trade-offs  between  resolution,  speed,  sen¬ 
sitivity,1  and  signal-to-noise  ratio  (S/N).  The  goal  of  this  chapter 
is  to  provide  both  a  theoretical  and  a  practical  guide,  with  more 
emphasis  on  the  latter  especially  when  theoretical  considerations 
are  covered  elsewhere  (Chapters  22,  23,  and  34,  this  volume).  Here 
the  relative  merits,  strengths,  and  weaknesses  of  using  a  disk-scan¬ 
ning  confocal  microscope  for  biological  imaging  will  be  examined 
and  sample  applications  shown. 

This  chapter  rests  heavily  on  that  written  by  G.  S.  Kino  for  the 
second  edition,  and  many  of  his  original  figures  and  text  have  been 
retained  or  abridged.  One  notable  distinction  is  that  we  have 
explicitly  chosen  to  emphasize  fluorescent,  rather  than  reflected  or 
backscattered  light  (BSL)  imaging,  as  the  former  represents  the 
most  common  use  of  this  instrument  in  the  biological  sciences.  We 
have  also  limited  discussion  on  microscopes  that  are  no  longer  in 
production  (or  are  very  rarely  used  by  biologists)  and  instead  have 
compared  and  contrasted  the  most  prevalent  current  commercial 
disk-scanning  confocal  systems  (Yokogawa,  BD  Bioscience,  and 
Olympus)  with  conventional  confocal  light-scanning  microscopy 
and  with  a  new,  fast  confocal  slit-scanning  microscope  from  Zeiss. 
Finally,  current  limitations  and  new  technological  advances  that  in 
our  opinion  will  markedly  influence  the  power  and  popularity  of 
this  approach  are  discussed. 

The  reader  is  advised  to  pay  special  attention  to  other  chapters 
in  this  volume  that  go  into  considerable  depth  about  related  subject 


1  Although  sensitivity  is  a  term  that  most  people  think  that  they  understand,  it 
lacks  a  specific  technical  meaning.  We  will  use  it  here  to  represent  a  mixture 
of  detector  quantum  efficiency  and  detector  noise  floor  (including  back¬ 
ground).  A  “sensitive”  detector  is  one  suitable  for  discriminating  a  very  weak 
light  level  from  “black.” 


areas:  pinhole  and  slit  detectors  (Wilson  and  Sheppard,  1984; 
Wilson  and  Carlini,  1987;  Chapters  25  and  34),  signal-to-noise 
ratio  (Chapters  22  and  23),  multi-focal  multi-photon  microscopy 
(Chapter  29),  photodetectors  (Chapters  2  and  12),  deconvolution 
(Chapters  23,  24,  and  25),  and  visualization  of  three-dimensional 
(3D)  datasets  (Chapters  13  and  14). 

BACKGROUND 

Live  Cell  Imaging:  Probing  the  Future 

Since  the  publication  of  the  second  edition  of  this  book  in  1995, 
there  has  been  a  virtual  explosion  of  live  cell  imaging  papers.  This 
is  in  no  small  part  due  to  widespread  use  of  genetically  encoded 
tags,  most  notably  green  fluorescent  protein  (GFP)  and  now  a 
literal  rainbow  of  spectral  mutants  including  CFP  (cyan),  YFP 
(yellow),  RFP  (red;  monomeric  and  tetrameric),  pH-(in)sensitive 
mutants,  and  fluorescence  resonance  energy  transfer  (FRET) 
probes  (Miyawaki,  2003;  Chapter  16,  this  volume).  The  immense 
popularity  of  these  fluorophores  derive  from  that  fact  that  nearly 
any  protein  can  be  specifically  tagged  and  followed  in  space  and 
time.  Illustrating  their  growing  prevalence,  in  1995,  2000,  and 
2003  there  were  respectively  ~60,  1000,  and  1600  citations  of 
papers  using  “GFP.”  New  categories  of  mixed  chemical/genetic 
probes  have  been  developed  including  FlAsH  and  ReAsH  dyes 
which  facilitate  pulse-chase  labeling  and  correlative  light  and  elec¬ 
tron  microscopy  (Gaietta  et  al.,  2002).  Of  course,  there  is  also  a 
plethora  of  traditional  vital  dyes  for  measurements  of  cellular 
physiology  (ion  concentrations,  membrane  voltage,  etc.),  organelle 
dynamics  (mitochondria,  lysosomes,  etc.),  and  fluorescently 
tagged  ligands,  lipids,  and  antibodies;  the  Molecular  Probe  cata¬ 
logue  (now  Invitrogen)  is  an  excellent  reference,  as  are  Chapters 
17  and  36. 2  Very  recently,  small,  semiconductor  nanocrystals, 
also  called  “quantum  dots,”  have  been  developed  that  are  much 
brighter,  more  photostable,  and  have  narrower  emission  spectra 
than  traditional  dyes  (Lidke  and  Arndt-Jovin,  2004). 3 

What  does  all  the  progress  in  the  probe  development  mean  to 
the  microscopist?  As  the  floodgates  of  new  specific  fluorescent 
molecules  and  multi-spectral  dyes  have  opened,  there  is  an  ever- 
increasing  desire  to  study  the  dynamics  of  molecules  in  living 
cells  with  high  spatial  and  temporal  resolution.  The  Holy  Grail 
of  such  imaging  is  to  study  biochemistry  at  the  single  molecule 


2  See  http://www.probes.com/. 

3  Nanocrystals  have  large  absorption  cross-sections  and  long  decay  times  (T 
—10ns).  This  makes  them  well  suited  for  low-dose,  2-photon  and  confocal 
lifetime  imaging,  but  can  also  cause  singlet-state  saturation  at  relatively  low 
beam  power. 
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level  in  living  cells.  While  this  may  sound  like  science  fiction,  the 
reality  is  that  such  scientific  studies  are  being  published  using 
state-of-the-art  probes  and  sophisticated  microscopes  (Sako  and 
Yanagida,  2003). 

A  Need  for  Speed  and  Less  Photobleaching 

What  are  the  live  cell  imaging  needs  of  the  biologist?  There  is  no 
easy  answer.  Is  it  enough  to  see  a  single  “snapshot”  of  a  cell  under¬ 
going  mitosis  or  is  the  process  only  revealed  by  time-lapse  or  four¬ 
dimensional  (4D)  imaging?  Only  the  experimentalist  can  judge, 
but  increasingly  the  desire  to  understand  the  complex  and  dynamic 
intra-  and  intercellular  organization  is  pushing  the  trend  towards 
larger  multi-dimensional  datasets,  that  include  the  following  dif¬ 
ferent  spatial/temporal  categories:4 

•  2D/3D  imaging:  A  single  two-dimenional  (2D)  snapshot  or  3D 
“stack”  or  projection  of  many  optical  sections 

•  2D  +  time:  Imaging  of  a  single  optical  section  over  time.  This 
varies  from  slow  “time-lapse”  (1  frame  every  s/min/h),  to 
“real-time”  (~30  frames/s),  to  the  ultrafast  (100-1000 
frames/s). 

•  4D  (3D  +  time):  Time-lapse  (or  real-time)  imaging 

Parameters  that  are  generally  important  here  are:  adequate  res¬ 
olution  (xv),  z-axis  optical  sectioning,  S/N,  minimal  photobleach¬ 
ing  of  fluorophores  or  phototoxicity  to  cells,  and  speed.  How  fast? 
This  depends  on  the  biological  process  being  studied.  For  example, 
if  studying  microtubule  dynamics,  an  acquisition  of  0.1  to  1  frames 
per  second  (fps)  may  suffice.  However,  for  fast  processes  such  as 
calcium  sparks  or  waves  that  happen  in  milliseconds,  acquisition 
rates  of  100  to  lOOOfps  may  be  needed. 

One  rule  of  thumb  is  to  acquire  images  fast  enough  so  that  only 
minimal  changes  occur  between  frames  (e.g.,  small  changes  in 
object  intensity  or  position).  In  this  way  no  action  is  missed  and 
automated  image  analysis  is  facilitated.  However,  this  “upper 
bound  of  the  speed  envelope”  is  often  unrealistic,  either  because 
the  imaging  system  cannot  respond  fast  enough  or  because  the 
sample  bleaches  too  rapidly  to  obtain  the  desired  information.  This 
“rule”  can  be  even  more  demanding  when  acquiring  live  3D  or  4D 
images  as,  in  this  case,  each  entire  3D  stack  must  be  acquired  fast 
enough  to  essentially  freeze  the  action.  For  example,  if  a  vesicle 
is  moving  at  1  micron/s  (a  common  speed)  and  one  wishes  to 
restrict  object  motion  blur  to  -0.25  microns,  then  ideally,  the  whole 
z-stack  should  be  acquired  in  less  than  250ms.  If  this  cell  is 
5  microns  thick  and  one  acquires  a  stack  of  10  optical  sections 
(500  nm  z-steps),  this  means  that  each  slice  needs  to  be  acquired 
in  under  25  ms!  This  is  not  even  considering  the  overhead  time  of 
fast  (piezoelectric)  focusing  or  the  excitation  of  multiple  dyes. 

The  other  problem  associated  with  4D  imaging  is  faster 
bleaching.  For  example,  if  a  fluorescent  sample  were  normally  to 
bleach  to  50%  after  500  (2D)  images,  it  would  now  photobleach 
after  only  20  3D  stacks  of  25  planes  each,  or  in  5  sec.  if  acquired 
at  4  stacks/s.  Anyone  who  has  attempted  fluorescent  time-lapse 
imaging,  especially  in  4D,  knows  all  too  well  the  problem  of 
photobleaching  and  the  proverbial  proclamation  that  “every 
photon  is  sacred”  and  that  3D  deconvolution  reduces  Poisson  noise 
substantially  by  effectively  averaging  over  -100  voxels  in 
Nyquist-sampled  data  (see  Chapters  19  and  25,  this  volume). 


4  This  is  equally  applicable  for  fixed  cells,  however,  viewing  living  cells  avoids 
fixation  artifacts  and  can  facilitate  rapid  screening  of  samples  (e.g.,  GFP- 
based  visual  screens). 


Advantages  and  Limitations  of  Confocal  Laser- 
Scanning  Microscopes 

Why  are  confocal  laser-scanning  microscopes  (LSM),  so  popular 
with  the  biological  community  in  spite  of  their  quarter  to  half 
million  dollar  expense?  The  single  major  advantage  of  the  confo¬ 
cal  LSM  is  the  ability  of  its  spatial  filter  (either  a  single  pinhole 
or  a  slit)  to  reject  out-of-focus  light.  This  improves  contrast- 
reducing  background  or  haze,  and  provides  better  optical  section¬ 
ing  (and  resolution)  along  the  z-axis,  although  a  -30%  increase  in 
lateral  (xy)  resolution  is  possible  if  the  pinhole  is  closed  to  about 
0.05  Airy  units5  (Wilson  and  Sheppard,  1984).  Given  the  many 
attributes  of  this  mature  technology,  why  should  one  use  any  other 
kind  of  light  microscope  for  fluorescent  imaging? 

Alas,  the  source  of  its  strength,  the  single  confocal  pinhole,  is 
also  its  Achilles’  heel,  creating  problems  that  while  of  little  concern 
with  bright,  fixed  cells,  can  be  disadvantageous  for  fast  or  long¬ 
term  imaging  of  living  cells.6  Let  us  start  by  considering  the  data 
rate.  As  is  clearly  demonstrated  in  Figure  35.19  (Chapter  35,  this 
volume),  the  most  important  factor  in  obtaining  a  good,  high- 
contrast  image  is  to  collect  enough  photons!  Otherwise  statistical 
noise  in  the  signal  makes  the  image  grainy  and  smaller  features 
become  invisible.  This  means  that  both  the  collection  of  light  (lens 
and  intermediate  optics)  and  its  detection  must  be  optimal.  Most 
confocal  LSM  use  photomultiplier  tubes  (PMTs)  for  detecting 
photons.  Although  they  have  a  rapid  response  and  good  dynamic 
range,  their  effective  quantum  efficiency  (QE)  is  low,  typically 
only  in  the  6%  to  15%  range  (in  the  400-600 nm  spectrum).  This 
stands  in  stark  contrast  with  the  up  to  90%  QE  achievable  with  a 
good  back-illuminated,  CCD  camera  (see  Chapter  13,  this  volume). 

Because  the  rate  at  which  data  can  be  derived  from  a  fluores¬ 
cent  specimen  depends  directly  on  the  number  of  excitation 
photons  striking  it,  to  image  at  the  same  frame  rate,  the  intensity 
of  the  light  in  the  single  spot  of  an  LSM  must  be  about  -105  times 
higher  than  that  used  in  a  widefield  (WF)  system.  This  very  intense 
excitation  light  can  lead  to  dye  saturation  (see  Chapters  2  and  16, 
this  volume),  a  phenomenon  that  occurs  when  the  excitation  is  so 
bright  that  most  of  the  dye  molecules  are  in  the  excited  state.  Using 
more  laser  power  only  raises  the  background  because,  as  mole¬ 
cules  away  from  the  focus  plane  are  now  excited  more  strongly, 
but  are  still  not  saturated,  they  will  produce  relatively  more  out- 
of-focus  light.  It  also  increases  photobleaching  without  increasing 
signal  level  (Chapters  38  and  39,  this  volume,  discuss  the  rela¬ 
tionship  between  bleaching  and  saturation).  Saturation  can  only  be 
minimized  by  using  a  dye  with  a  faster  singlet  decay  time  or  by 
using  less  intense  light. 

On  the  other  hand,  rapid  fluorescent  imaging  requires  more 
intense  light  so  as  to  produce  a  similar  number  of  emitted  photons 
in  a  shorter  time.7  Clearly  there  is  a  theoretical  maximum  imaging 
speed  set  by  the  total  rate  at  which  photons  are  generated 


5  Rarely  is  this  done  when  viewing  live  biological  samples  as  signal  level 
decreases  approximately  with  the  square  of  the  pinhole  diameter  and  low 
signal  is  much  more  a  barrier  than  simple  optics  to  seeing  even  the  Abbe 
resolution. 

6  Note  that  when  commercial  single-beam  confocal  microscopes  first  appeared 
two  decades  ago,  living  cell  fluorescent  imaging  was  uncommon  and  a 
premium  was  put  first  on  the  optimization  of  resolution  and  later  on  multi- 
spectral  excitation  and  detection,  rather  than  on  speed  or  sensitivity.  Para¬ 
doxically,  Zeiss’  2004  release  of  their  LSM  510  LIVE  confocal  is  based  on  a 
lower  resolution  slit  scanner. 

7  An  analogy  can  be  drawn  to  filling  up  buckets  with  water,  either  drip-by-drip 
over  hours  or  in  a  second  with  a  fire  hose. 
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(depending  on  the  number  of  beams,  the  excitation  intensity  in 
each  one,  and  the  photoproperties  and  concentration  of  the  dye) 
and  the  photon  efficiency  of  the  entire  detector  channel.  Although 
one  can  increase  signal  level  by  using  more  dye,  this  can  seldom 
be  done  without  affecting  functionality;  for  example,  protein  over¬ 
expression  can  cause  misfolding  or  mislocalization. 

One  alternative  is  to  scan  a  very  small  area  of  interest  or  even 
just  a  single  line  to  gain  frame  speed.  However,  when  sampling 
only  a  small  region,  information  on  a  larger  spatial  scale  is  lost.8 
In  fact,  weakly  labeled  samples  are  triple  hit;  more  intense  light 
is  needed  to  excite  the  dye  to  generate  enough  photons,  leading  to 
more  background,  phototoxicty,  and  increased  photobleaching. 

The  second  requirement  for  fast  confocal  imaging  is  that  both 
the  laser  beam  and  the  image  of  the  pinhole  must  be  raster-scanned 
over  the  sample  —  typically  using  linear  galvanometers.  The 
mechanical  properties  of  linear  galvanometers  limit  how  fast  a 
line  can  be  scanned  (low  kilohertz,  see  Chapter  3,  this  volume). 
Although  acousto-optical  deflectors  or  resonant  galvanometers  can 
go  faster,  they  have  their  own  limitations  (see  Chapter  9,  this 
volume).  Finally,  lasers  are  expensive  and  by  definition  only  excite 
at  very  discrete  “lines,”  limiting  the  fluorescent  dyes  that  can  be 
excited. 

Other  Imaging  and  Deconvolution 

Is  confocal  imaging  the  only  way  to  go?  As  mentioned  above,  the 
advantage  of  a  (1 -photon)  confocal  is  to  reduce  background  by 
rejecting  out-of-focus  emission  light  with  a  pinhole.  Alternatively, 
one  can  make  optical  section  images  by  limiting  the  excitation  to 
a  single  optical  plane  or  by  using  other  microscopical  techniques, 
for  example,  2-photon  excitation  (Chapters  28,  29,  and  37, 
this  volume)  or  total  internal  reflection  fluorescent  microscopy 
(Toomre  and  Manstein,  2001).  When  imaging  samples  in  which 
most  or  all  of  the  dye  is  confined  to  a  single  optical  plane  (e.g., 
some  in  vitro  assays),  widefield  (WF)  epi-fluorescent  imaging  may 
suffice. 

Alternatively,  one  may  acquire  a  stack  of  WF  epifluorescent 
images  and  use  a  measured,  or  theoretical,  point  spread  function 
(PSF)  to  “deconvolve”  the  3D  image.  The  principle  here  is  that  at 
least  some  of  the  out-of-focus  haze  can  be  converted  into  useful 
information.  While  a  detailed  analysis  of  deconvolution  is  pro¬ 
vided  in  Chapters  23  and  24,  it  is  worth  underscoring  the  follow¬ 
ing  limitations  of  deconvolution: 

•  Deconvolution  is  time  consuming  (especially  for  4D  datasets) 
and  does  not  give  on-the-fly  results. 

•  Small  misalignment  of  the  optics  can  prevent  good 
deconvolution. 

The  quality  of  the  deconvolution  depends  on  the  sample  and 
the  method  of  deconvolution  (there  are  several).  However,  there  is 
also  some  good  news:  deconvolution  will  also  improve  the  image 
quality  of  confocal  3D  datasets  by  suppressing  out-of-bandwidth 
“features”  caused  by  Poisson  noise  and  effectively  averaging 
signal  levels  over  many  voxels.  In  addition,  because  the  confocal 
PSF  is  confined  to  a  much  smaller  volume,  the  computer  process¬ 
ing  time  is  greatly  reduced. 


Often  one  only  determines  a  posteriori  what  region  of  an  image  was 
important. 


CONFOCAL  DISK-SCANNING  MICROSCOPY 
Nipkow  Disk  —  An  Innovation 

The  Nipkow  disk  that  is  central  to  the  design  of  all  modern  disk¬ 
scanning  confocal  microscopes,  was  developed  by  Paul  Nipkow  in 
1884  as  a  means  of  dissecting  an  image  into  a  single  continuous 
signal.  Nipkow  had  the  foresight  to  realize  that  if  a  series  of  pin¬ 
holes  (or  squares)  were  arranged  in  an  Archimedian  spiral  of  con¬ 
stant  pitch,  then  one  revolution  of  the  disk  would  be  equivalent  to 
scanning  the  object  (see  Fig.  1.3,  this  volume)  and  the  light 
transmitted  by  the  holes  could  be  measured  to  record  any  image 
projected  onto  the  disk.  It  was  an  early  competitor  of  the  televi¬ 
sion  we  know  today.  Apart  from  its  mechanical  complexity,  its 
main  weakness  was  that  at  any  given  time  light  is  only  detected 
(or  transmitted)  through  a  single  small  pinhole  making  it  ineffi¬ 
cient  for  detecting  or  displaying  image  information.  This  is  why 
televisions  no  longer  use  Nipkow  disks. 

More  recently,  Petran  designed  a  new  a  new  type  of  Nipkow 
disk,  one  having  many  more  holes  arranged  in  a  series  of  nested 
spirals  so  that  hundreds  or  thousands  were  present  in  the  frame  at 
any  one  time  and  in  such  a  way  that  the  entire  pattern  was  axially 
symmetrical  (Fig.  10.1). 

A  Renaissance  —  Advantages  of  Disk-Scanning 
Confocal  Imaging 

Disk-scanning  imaging  was  reborn  in  1967  in  Egger  and  Petran’ s 
first  implementation  of  a  tandem  scanning-disk  confocal  micro¬ 
scope.  The  purpose  of  the  Petran  disk  was  not  to  dissect  the  image 
(as  in  Nipkow’s  design)  but  to  perform  point-illumination/point- 
detection  confocal  imaging.  Light  transmission  through  a  Petran 
disk  is  hundreds  to  thousands  of  times  higher  than  through  a 
Nipkow  disk.9  New  designs  by  both  individuals  (e.g..  Kino,  Xiao, 


FIGURE  10.1.  Schematic  of  a  Nipkow/Petraii  disk  used  in  real-time  disk¬ 
scanning  confocal  microscopy.  Typically  20,000  to  200,000  pinholes  are 
arranged  in  spirals  of  constant  pitch  (Archimedian)  and  a  subregion  (shown  in 
blue)  corresponding  to  about  a  thousand  pinholes  are  illuminated  and  imaged. 
A  partial  rotation  (e.g.,  30°  for  the  Yokogawa  CSU10/22)  of  a  rapidly  disk¬ 
scanning  (~1 800-5000 rpm)  scans  the  sample;  arrows  show  how  holes  rotate 
to  new  positions  (red).  Images  can  be  seen  in  real-time  by  eye  or  acquired  using 
a  camera  that  is  synced  with  rotation  of  the  disk.  (Image  adapted  from  H.  Ishida 
et  al.,  Department  of  Physiology,  Tokai  University.) 


9  Note  on  nomenclature:  many  people  and  companies  call  spinning  confocal 
disks  “Nipkow  disks”  even  though  the  pinhole  spacing  and  design  may  be 
radically  different  from  the  original  Nipkow  disk.  Thus,  a  “Nipkow  disk” 
merely  denotes  a  spinning  disk  pinhole-  (or  slit-)  based  confocal  system 
rather  than  a  specific  design. 
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TABLE  10.1.  Comparison  of  Confocal  LSM,  Disk-Scanning,  and  Scanned  Slit  Microscopes 


Single-Beam  Laser  Scanning 

Spinning  Disk 

Scanned  Slit 

Advantages 

•  High  resolution 

•  High  speed 

•  High  speed 

•  Excellent,  adjustable  z-resolution 

•  Good  QE  detector  and  sensitivity 

•  Good  QE  detector  and  sensitivity 

•  FRAP/photoactivation 

•  Moderate  cost3 

•  Simultaneous  readout  of  several 

•  Simultaneous  readout  of  several  wavelength  channels 

•  Laser  not  required 

wavelength  channels 

Disadvantages 

•  High  cost 

•  Lower  resolution  and  z-resolutionb 

•  Lower  resolution  and  z-resolution 

•  Slow  to  moderate  speed 

•  Fixed  pinhole/slit 

•  Requires  lasers 

•  Low  QE  detector 

•  Low  transmission  of  excitation0 

•  High  cost 

•  High  excitation  intensity 

•  No 

•  More  photobleaching 

•  Requires  lasers 

FRAP 

“Large  price  range  from  -US$35, 000  to  US$300,000  (see  Table  10.2). 

b  Depends  on  the  pinhole/slit  setting  (see  text  and  Fig.  10.4)  and  operationally  on  the  light  budget. 
cAn  exception  is  the  Yokogawa  microlens  design,  which  transmits  more  excitation  light. 


Boyde,  Lichtman)  and  companies  have  further  expanded  the  pos¬ 
sibilities,  however  all  disk  scanners  share  common  features.  The 
optimal  pinhole  (or  slit)  size  and  spacing  is  important  and  will  be 
discussed  later  and  in  Chapter  1 1 . 

The  disk  is  located  in  a  conjugate  image  plane  and  a  partial 
rotation  of  the  disk  scans  (see  arrows)  the  specimen  with  thousands 
of  beams  of  light  that  can  cover  the  whole  image  plane  in  as  little 
as  1  ms.  While  a  single-beam  confocal  LSM  illuminates  and  col¬ 
lects  intensity  information  serially,  multiple  pinholes  allow  the 
specimen  to  be  sampled  in  parallel.  This  produces  a  multiplica¬ 
tive  gain  in  potential  scan  speed  proportional  to  the  number  of 
pinholes  present  in  the  illumination/detection  system. 

As  the  Petran  microscope  requires  the  illumination  of  a  wide- 
held  of  pinholes,  rather  than  a  single  small  spot,  a  conventional 
“white”  light  source  such  as  a  mercury  or  xenon  lamp  can  be  used 
instead  of  a  laser.  Disk  scanning  is  more  rapid  than  the  human 
visual  flicker  response  rate  (~18Hz)  and  a  “real”  confocal  image 
of  the  sample  can  be  seen  by  eye  or  detected  with  a  camera  in  true 
color. 

Some  of  the  earliest  applications  of  disk-scanning  confocal 
microscopes  (e.g.,  Petran,  Kino,  and  colleagues)  were  to  illumi¬ 
nate  the  sample  with  white  light  through  the  pinholes  and  collect 
light  reflected  from  the  confocal  plane.  When  such  a  system 
employs  objective  lenses  intentionally  designed  to  have  very  high 
chromatic  aberration,  light  of  different  wavelengths  forms  foci  at 
different  heights.  The  result  is  that  images  of  opaque,  reflective 
specimens  are  colored  with  each  color  representing  features  of  a 
particular  topographic  elevation.  By  inserting  diffraction  gratings 
into  the  eyepieces  of  the  microscope,  such  images  can  even  be 
viewed  as  real-time  stereo-pair  images.  As  the  gratings  displace 
the  location  of  features  horizontally  depending  on  their  color,  and 
as  their  color  is  coded  for  depth  by  orienting  the  gratings  in  oppo¬ 
site  directions  for  each  eye,  a  stereo-pair  image  is  presented  in 
which  the  actual  3D  position  of  the  specimen  surface  is  both  coded 
by  color  and  made  visible  by  stereo  disparity  (Chapter  15,  second 
edition). 

Advantages  of  disk-scanning  include: 

•  Up  to  100-  to  1000-fold  gain  of  speed:  multiple  points  of  light 
illuminate  the  sample  and  are  detected  in  parallel,  greatly 
reducing  fluorophore  saturation. 

•  Generating  a  real  image  that  can  be  detected  by  eye  or  with  a 
fast,  high-QE  CCD  camera. 

•  Less  photobleaching  (due  to  lower  local  excitation  intensity). 

•  No  strict  requirement  for  laser  illumination,  reducing  cost  and 
allowing  more  choices  of  excitation  wavelengths. 


Disadvantages 

Several  factors  must  be  considered  when  using  a  disk-scanning 
confocal  (see  Table  10.1).  First,  light  scattered  or  fluoresced  by 
structures  away  from  the  focus  plane  can  still  reach  the  detector 
through  an  adjacent  pinhole,  decreasing  the  z-resolution  of  the 
system.  Second,  low  transmission  of  light  through  the  disk  may 
hinder  imaging  of  dim  fluorescent  samples.  Although  this  can  be 
compensated  with  longer  exposure  times,  doing  so  negates  the 
speed  advantages  that  are  sought.  However,  a  high-QE  detector 
may  still  give  slow  confocal  imaging  with  less  bleaching  than  a 
confocal  LSM.  Third,  90%  to  99%  of  illumination  light  does  not 
go  through  the  disk  and  these  reflections  can  cause  high  back¬ 
ground,  especially  in  single-sided  bright-held  spinning  confocal 
microscopes.  Field  illumination  may  not  be  uniform  and  may 
require  a  correction  lens  and/or  homogenization  of  the  arc  source 
(e.g.,  a  liquid  light  guide).  Finally,  disk-scanning  systems  do  not 
allow  high  power  illumination  of  selected  regions  of  interest  for 
fluorescent  recovery  after  photobleaching  (FRAP)  experiments. 
These  disadvantages  can  be  partially  mitigated  by  judicious  choice 
of  intermediate  optics,  pinhole  or  slit  size  and  spacing,  detector 
type,  and  the  addition  of  a  separate  bleaching  system. 


CRITICAL  PARAMETERS  IN  PINHOLE  AND 
SLIT  DISKS 

Fill  Factor  and  Spacing  Interval  F 

Although  the  present  discussion  has  centered  on  confocal  pinholes, 
the  use  of  a  confocal  scanning  disk  with  slit  apertures  can  increase 
the  transmitted  light  budget.  Slits  can  either  be  arranged  on  the 
disk  as  spirals  or  in  linear  arrays.  The  main  trade-off  between  slit 
versus  pinhole  is  usually  between  brighter  illumination  and 
higher  signal  with  a  slit  design  and  better  z-resolution  with 
pinholes.  Slit-disk  confocals  have  a  larger  percentage  transmission 
(7),  or  fill  factor,  than  pinholes  when  the  ratio  of  the  slit/pinhole 
diameter,  D,  to  distance  between  slits/pinholes,  S,  is  fixed: 

rsU‘  =  (f)Xl0°  W  =  (f)2><100  (D 

For  example,  if  D  =  50  microns  and  S  =  500  microns,  then  10% 
and  1%  of  the  light  will  be  transmitted  by  a  disk  with  slits,  and 
pinholes,  respectively.  The  transmission  of  the  pinhole  disk  can  be 
increased  by  using  a  smaller  pinhole  spacing  interval.  However, 
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even  if  D/S  is  1/5,  only  4%  of  the  light  will  be  transmitted,  and 
smaller  D  causes  too  much  of  the  emitted  light  to  return 
through  neighboring  pinholes,  decreasing  ^-resolution,10  as  dis¬ 
cussed  below. 

Other  options  are  to  increase  the  pinhole  or  slit  diameter, 
however,  this  will  decrease  the  axial  resolution  (see  below).  Alter¬ 
natively,  intermediate  optics  employing  “microlenses,”  such  as 
those  used  in  the  Yokogawa  system,  can  focus  more  of  the  excita¬ 
tion  light  onto  the  pinholes  but  only  if  laser  light  is  used.  It  is  worth 
noting  that  when  the  slits  are  very  close,  this  imaging  system 
begins  to  resemble  structured  illumination  (e.g.,  Zeiss  Apo-tome; 
see  Chapter  13,  this  volume)  although  there,  signal  detection  is  not 
confocal. 


Lateral  Resolution 


The  ^-resolution  of  the  light  microscope  is  described  by  the  Abbe 
equation  and  is  a  function  of  the  emission  wavelength  and  numer¬ 
ical  aperture  of  the  objective: 


r..  =0.61 - 2 

^  NA 


Lobj 


(2) 


In  biological  fluorescence  confocal  microscopy,  where  overall 
performance  drops  rapidly  because  of  poor  signal  statistics  if  D  is 
reduced  below  one  Airy  unit,  xy-resolution  is  essentially  the  same 
as  that  of  the  widefield  instrument.  This  is  particularly  true  for  disk 
scanners  having  D  that  is  more  than  2  Airy  units  because,  in  this 
case,  the  CCD  will  record  a  mini-image  from  the  light  returning 
through  each  pinhole.11 


values  for  objective  lenses  with  different  M/NA  ratios.  In  com¬ 
mercial  systems  with  a  single  disk  (Yokogawa  and  CARV),  the 
pinhole  was  designed  for  high  magnification,  high  NA  objectives, 
and  even  then  the  50  to  70  micron  pinhole  diameter  is  excessively 
wide  (Table  10.2). 

The  effects  of  pinholes  (or  slits)  that  are  the  wrong  size  are 
shown  schematically  in  Figure  10.2.  A  small  pinhole  has  decreased 
transmission  due  to  its  small  area  and  also,  because  the  illumina¬ 
tion  light  diffracts  at  the  aperture,  the  light  from  it  overfills  the 
pupil  of  the  objective  lens,  wasting  more  light.  On  the  detection 
side,  an  overly  small  pinhole  rejects  more  light  than  necessary 
and  decreases  the  S/N  (identical  to  when  one  closes  the  pinhole 
too  much  in  a  confocal  LSM).  Alternatively,  if  the  pinhole  is  too 
big  then  and  the  (non-laser)  illumination  is  collimated  rather  than 
convergent,  then  it  may  not  be  diffracted  adequately  to  fill  the  pupil 
of  the  objective,  illuminating  a  larger  spot  at  the  focus  plane.13 
Signal  light  from  this  larger  spot  will  be  imaged  as  a  mini-image 
in  the  aperture  of  the  disk.  The  emitted  light  is  collected  efficiently 
but  the  large  pinhole  makes  the  z-resolution  worse.  Some  com¬ 
mercial  systems  allow  one  to  change  the  slits  to  match  the  objec¬ 
tive  (DSU  unit)  while  in  others  the  pinholes  are  fixed  (Yokogawa 
CSU  series  and  CARV)  and  only  the  magnification  of  the  objec¬ 
tive  or  the  tube  can  be  adjusted  to  change  the  effective  size  of  D. 

Use  of  the  same  pinhole  or  slit  for  both  illumination  and 
detection  causes  the  z-resolution  of  the  non-laser  disk-scanning 
confocals  to  fall  off  much  more  rapidly  as  D  increases  than  occurs 
with  single-beam  confocals.14  However,  when  the  D  is  optimal, 
the  curves  for  the  single-  and  multi-beam  scanners  converge,  as 
long  as  one  ignores  haze  reaching  the  CCD  through  adjacent 
pinholes. 


Pinhole/Slit  Size 

What  is  the  optimal  pinhole  size?  Unlike  confocal  LSMs,  which 
offer  a  diffraction-limited  spot  as  the  excitation  pinhole  and  sepa¬ 
rate  independently  adjustable  pinholes  for  each  detection  channel, 
disk-scanning  confocals  use  the  same  pinhole  for  excitation  and 
detection  (or  in  tandem  systems,  one  of  identical  size).  While  in 
both  types  of  confocal.  increasing  D  increases  signal  strength 
because  light  can  now  reach  the  detector  from  features  farther  from 
the  focus  plan,  only  in  disk  scanners  does  it  also  produce  more 
signal  because  there  is  more  excitation  light  striking  the  specimen 
in  the  larger  excitation  spot.  This  makes  pinhole  size  important. 

The  optimal  diameter  of  a  pinhole/slit.  DOPT,  sets  it  equal  to 
the  FWHM  of  the  Airy  figure  and  can  be  derived  from  the  Fraun¬ 
hofer  formula  (Goodman,  1968;  Kino,  1987;  Kino  and  Xiao,  1990) 
and  expressed  as  the  following  approximate  equation;12 


0.57^  _  0.5XM 
b  ~  NA 


(4) 


where  M  and  NA  are,  respectively,  the  magnification  and  numeri¬ 
cal  aperture  of  the  objective  lens.  Assuming  a  100X  1.4  NA  oil- 
immersion  lens,  and  X  =  560 nm,  DOPt  is  20pm  but  only  9.3  pm 
for  a  40x  1.2  NA  lens.  Thus,  one  needs  disks  with  different  D 


Axial  Resolution 


The  axial  resolution,  dz ,  near  the  focal  plane  of  slit  and  point  scan 
confocal  microscopes  with  very  small  pinhole/slit  dimensions  can 
be  expressed  by  the  equation 


d  = 


(3) 


where  X  is  the  emission  wavelength,  n  is  the  refractive  index  of 
the  medium,  NA  is  n  sina  (a  is  the  acceptance  angle  of  the  objec¬ 
tive  of),  and  K  is  a  scalar  correction  factor  where  for  a  slit  disk  Kslit 
=  0.95  and  for  a  pinhole  disk  /C'plntloie  =  0.67.  For  example,  if  X  = 
500 nm,  NA  =  1.2,  and  n  =  1.33  then  d  would  be  627  nm  and 
442  nm  for  a  slit  disk  and  pinhole  disk,  respectively.  As  long  as  D 
is  the  same  for  both  pinholes  and  slits  and  also  significantly  smaller 
than  the  calculated  Airy  limit,  slits  will  have  ~1.4x  worse  optical 
sectioning  than  pinholes. 

However,  this  statement  really  does  not  tell  the  whole  story.  If 
one  were  to  make  a  z-scan  through  a  thin,  horizontal  layer  of  dye. 


10  Also  applies  to  slits  (see  graph  in  Fig.  10.9). 

11  This  mini-image  will  have  dimensions  only  a  few  pixels  in  size  and  the  infor¬ 
mation  it  contains  can  only  be  recorded  as  long  as  the  magnification  of  the 
camera  lens  is  sufficient  to  match  these  pixels  with  the  size  of  the  pixels  on 
the  CCD. 

12  This  sets  the  pinhole  diameter  at  the  full-width  half-maximum  of  the  Airy 
figure  and  lets  through  about  75%  of  the  light  in  the  central  maximum.  In 
biology,  the  pinhole/slit  is  often  set  at  the  first  zero  of  the  Airy  figure,  about 
twice  as  big. 


13  In  laser-illuminated  disk  scanners,  the  high  coherence  of  the  source  insures 
that  the  light  is  focused  into  an  Airy  figure  no  matter  what  the  size  of  the 
aperture.  However,  the  size  of  this  figure  depends  on  the  extent  to  which  light 
from  each  aperture  fills  the  back-focal  plane  (BFP)  of  the  objective,  a  factor 
that  can  be  increased  by  the  convergence  angle  of  the  light  striking  the  disk. 

14  In  laser-illuminated  disk  scanners,  assuming  that  the  divergence  of  the  light 
leaving  each  pinhole  is  sufficient  to  fill  the  BFP  of  the  objective,  each  exci¬ 
tation  beamlet  is  focused  into  a  single,  diffraction-limited  spot,  regardless  of 
the  pinhole  size.  This  occurs  because  of  the  extremely  high  coherence  of  laser 
light. 
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TABLE  10.2.  Comparison  of  Some  Different  Microscope  Specifications3 


Disk-Scanning  Confocal 


Microscope 

Confocal 

Confocal 

Yokogawa 

Parameters 

Epi-fluorescence 

Point  LSM 

Line  LSM 

(CSU10/22) 

CARVb 

DSU 

Supplier/Distributor 

All  major 

Many:  Zeiss, 

Zeiss 

(Yokogawa)  / 

Atto 

Olympus 

manufactures 

Leica, 

(LSM510  Live) 

PerkinElmer, 

Bioscience 

Olympus, 

Solamere  Tech. 

Nikon  .  .  . 

McBain 

Instruments, 

Visiteec 

Scan  Type 

No 

Point  scan 

Line  scan 

Disk 

Disk 

Disk 

Pinhole/Slit 

None 

Pinhole  or  slit 

Slit 

Microlens  +  pinhole 

Pinhole 

Slit 

Adjustable 

n/a 

Yes 

Yes 

No 

No 

Yes 

pinhole/slit 

Illumination 

Hg  or  xenon 

Laser(s) 

Lasers 

Laser(s) 

Hg  or  xenon 

Hg  or  xenon  arc 

arc  lamp 

(diode) 

arc  lamp 

lamp 

UV  excitation 

Yes 

Yes,  but  need 

Yes 

Only  specified 

Yesd 

Yes 

expensive  laser 

(405  nm) 

for  400-650  nm 

Pinhole  or  slit 

n/a 

Adjustable 

Adjustable  slit 

Pinhole 

Pinhole 

Slits 

size/spacing 
diameter  (fim) 

pinhole  or  slit 

50/250 

70/-250 

13-38/140^100 

Disk  fill  factor 

n/a 

n/a 

n/a 

4% 

5%-7% 

5%-10% 

%  Transmission 

-100% 

-50% 

>90% 

-40%  with 

~5%-7% 

5%-10% 

pinhole/slit 

microlens 

Detector 

CCD 

PMT 

Linear  CCD 

(EM)  CCD 

(EM)  CCD 

(EM)  CCD 

Detector:  Max  QEe 

~50%-90% 

~10%-20% 

~50%-80% 

~50%-90% 

-50%-90% 

~50%~90% 

Full  image 

n/a 

-lfps 

~120fps 

1000 fps 

200  fps 

-50  fps 

scan  rate8 

(no  scan) 

(faster  for  ROI) 

@  512x512 

Max.  z-resolution 

(Very  poor) 

Best 

Moderate 

Good 

Good 

Moderate 

Cost 

Inexpensive 

Expensive 

Expensive  (TBA) 

Expensive 

~$50K 

~$35K 

($200  K-$400  K) 

($100K-$300  K) 

Microscope 

N/A 

Usually  sold  with 

Zeiss 

Many:  inverted 

Many: 

Olympus:  inverted 

compatibility 

microscope 

or  upright 

inverted  only 

or  upright 

Relative  space 

Small 

Variable: 

Medium 

Large 

Medium 

Small 

requirements 

medium/large 

including  laser 

Filter  options; 

Standard; 

Many  internal 

Many  internal 

1-3  filter  cubesf 

3  filter  cubes; 

6  filter  cubes; 

Exchangeable? 

yes 

filter  options;  no 

filter  options;  no 

yes 

yes 

a  Not  an  exclusive  list  and  parameters  may  change. 
b  A  CARV2  instrument  will  have  5x  faster  scanning  and  FRAP  capacity. 

c  Supplied  by  Yokogawa  (Japan)  and  distributed  by  PerkinElmer  as  part  of  the  Ultraview  microscope,  Solamere  Technology  Group,  McBain  and  Visitec  as  stand-alone  or 
integrated  units  with  lasers,  integration  software,  etc. 

d  Post-disk  intermediate  optics  do  not  transmit  well  in  the  lower  UV  (cutoff  ~370nm). 
e Depend  on  camera/PMT  model  and  spectral  response  curve  (e.g.,  see  Fig.  10.8). 

f  Acquisition  speed  may  be  limited  by  camera  speed  and/or  number  of  photons  collected.  Note:  When  a  line  is  scanned  only  once  (maximum  speed),  it  is  possible  that 
information  may  be  lost  or  duplicated  unless  sychronization  between  disk  position  and  camera  exposure  time  is  perfect. 


hole  too  big 


D 

point 

reflector 


FIGURE  10.2.  Ray  paths  with  different  pinhole  sizes.  (A)  Pinhole  too  small 
wastes  transmitted  light  with  too  wide  a  beam.  (B)  Pinhole  too  big  yields  a 
narrow  beam  and  low  effective  aperture.  (C)  Pinhole  too  small  wastes  received 
light.  (D)  Pinhole  too  large  gives  poor  definition. 
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there  would  be  a  peak  as  the  focus  plane  coincided  with  the  dye 
layer  and  Eq.  3  would  describe  the  width  of  this  peak.  However, 
this  peak  would  have  a  very  long  tail  because  an  amount  of  light 
about  equal  to  the  square  of  the  gross  transmission  of  the  disk 
would  still  reach  the  CCD  through  the  wrong  pinhole,  even  when 
the  dye  layer  was  very  far  out  of  focus.  It  is  this  tail  of  diffuse 
background  haze  that  will  limit  overall  performance  whenever  one 
observes  specimens  that  are  both  thick  and  heavily  stained  and  the 
amount  of  this  background  is  proportional  to  the  transmission  of 
the  disk. 

Therefore,  while  Eq.  3  may  provide  a  good  estimate  of  the 
accuracy  with  which  one  can  determine  the  surface  height  of  a 
highly  reflective  semiconductor  using  a  disk  scanner  with  small 
pinholes  placed  on  a  sparse  grid  and  a  dry  objective  of  relatively 
low  NA,  it  fails  to  account  for  the  many  important  practical  matters 
that  affect  fluorescence  imaging  in  transparent  specimens.  As  the 
signal  reflected  from  a  semiconductor  can  easily  be  105x  higher 
than  that  produced  by  a  fluorescent  specimen,  D  can  be  much  less 
than  one  Airy  unit.  In  addition,  as  the  height  variations  of  the  semi¬ 
conductor  are  both  known  and  fairly  small,  it  is  possible  to  choose 
S  large  enough  so  that  little  light  reflected  from,  for  example,  the 
highest  feature,  is  sufficiently  out  of  focus  to  pass  through  neigh¬ 
boring  pinholes  when  the  focus  plane  coincides  with  the  lowest 
feature.  Last  but  not  least,  as  the  structures  of  semiconductors  tend 
to  be  relatively  horizontal  and  opaque,  one  is  usually  more  inter¬ 
ested  in  determining  the  location  of  the  outer  surface  (^-position, 
the  height  of  peak  reflection)  than  trying  to  see  one  feature  below 
another  (z-resolution). 

However,  in  biological  disk-scanning  where  the  NA  is  high, 
the  specimens  are  transparent  and  the  signal  levels  are  so  low  that 
D/S  >0.1,  the  fact  that  Eq.  3  takes  no  account  of  light  coming 
through  the  wrong  pinholes  is  a  serious  limitation.  Let  us  call  this 
light  WP  light  (for  wrong  pinhole). 

Clearly,  the  amount  of  WP  light  that  reaches  the  detector 
depends  on  the  staining  pattern  in  the  specimen.  If  the  stain  is  con¬ 
fined  to  a  horizontal  plane  that  is  thin  in  the  manner  defined  above 
for  a  semiconductor,  then  there  is  little  problem.  However,  because 
D  and  NA  are  now  larger  and  S  is  smaller,  the  band  of  z-heights  for 
which  this  is  true  is  thinner.  As  the  exact  manner  in  which  this  plays 
out  is  both  important  to  understand  if  one  is  to  appreciate  the  optical 
sectioning  properties  of  disk  scanners  and  is  also  poorly  conveyed 
by  equations,  we  will  attempt  a  more  heuristic  explanation. 

Figure  10.3(A)  shows  a  red  fluorescent  point  object  emitting 
light  in  all  directions.  We  will  not  yet  discuss  how  it  is  being 
excited  except  to  note  that  the  process  follows  exactly  the  same 
rules  as  are  described  below,  except  in  reverse.  Only  some  of  the 
light  leaving  the  point  hits  an  objective  lens  having  an  acceptance 
half-angle  of  45°  (e.g.,  an  NA  1.0  lens).  The  broken  black  hori¬ 
zontal  line  represents  a  disk  with  pinholes  in  it.  Although  the  actual 
disk  is  located  in  the  intermediate  image  plane  on  the  other  side 
of  the  objective,  we  can  see  its  masking  effect  just  as  well  in  any 
image  plane,  including  the  focus  plane  in  the  specimen  as  shown 
here.  In  fact,  it  is  rather  instructive  to  imagine  that  the  pinhole  is 
sampling  the  light  field  near  the  focus  plane. 

In  Figure  10.3(A),  the  point  object  is  seen  lying  in  the  focus 
plane  and,  as  we  have  assumed  that  D  is  1  Airy  unit  in  diameter 
(i.e.,  ~0.5  pm,  when  referred  to  the  focus  plane),  and  that  essen¬ 
tially  all  of  the  light  from  the  central  maximum  of  the  Airy  disk 
that  represents  the  point  object  goes  through  the  pinhole  and  pro¬ 
ceeds  (via  optics  that  are  not  shown)  to  the  CCD.  Figure  3(B) 
shows  the  layout  of  the  disk  viewed  enface.  The  D/S  is  about  1/5, 
implying  a  S  spacing  of  about  2.5  pm  and  a  total  transmission  of 
-4%,  about  normal  for  many  disk  scanners  used  in  biology. 


When  the  stage  moves  down,  the  point  object  drops  below  the 
focus  plane,  and  the  amount  of  light  going  through  the  central 
pinhole  in  the  disk  drops  [Fig.  10.3(C)]  because  the  light  leaving 
the  point  object  at  any  angle  that  the  objective  can  accept  now 
mostly  strikes  the  bottom  of  the  disk,  as  is  represented  by  the  pink 
disk  in  Figure  10.3(D). 

When  the  point  object  is  2.5  pm  from  the  focus  plane  of  the 
NA  1  objective,  the  defocus  equals  S  and  some  of  the  light  from 
the  point  begins  to  pass  through  the  first  ring  of  neighboring  pin¬ 
holes,  but  not  very  much.  In  simple  terms,  6x  more  light  will  now 
reach  a  CCD  pixel  that  is  focused  on  the  wrong  pinhole  than 
reaches  it  through  the  proper  pinhole.  However,  the  light  from  the 
object  that,  at  focus,  was  concentrated  into  a  spot  0.5  pm  in  diam¬ 
eter  is  now  spread  over  one  with  five  times  the  diameter  and  25x 
the  area  and,  as  the  same  diminution  in  intensity  is  occurring  on 
the  excitation  side,  the  number  of  fluorescent  photons/square  cen¬ 
timeter  at  the  plane  of  the  disk  is  now  625x  lower  than  it  was  at 
focus.  As  the  pinholes  are  laid  out  as  equilateral  triangles,  one  can 
see  from  Figure  10.3(B)  that  only  about  D/S  of  the  light  striking 
the  annulus  with  radius  S  and  thickness  D  actually  passes  through 
the  disk.  This  fraction  would  be  much  higher  were  we  referring  to 
an  array  of  parallel  slits. 

As  the  point  moves  farther  out  of  focus,  the  absolute  number 
of  photons  reaching  the  CCD  from  the  point  continues  to  dimin¬ 
ish  until  some  light  starts  going  through  the  ring  of  second  neigh¬ 
bors,  about  4  to  5  pm  away  from  the  focus  plane. 

We  can  see  from  Figure  10.3  that  if  the  NA  is  higher,  this  same 
process  will  occur  at  lower  defocus  values,  and  vice  versa. 
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FIGURE  10.3.  Diagrams  showing  how  light  originating  from  out-of-focus 
planes  passes  through  adjoining  pinholes  to  create  a  haze  level  and  how  this 
process  is  affected  by  the  geometry  of  the  pattern  of  the  holes  in  the  disk  and 
the  convergence  angle  of  the  objective,  as  described  in  the  text. 
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So  far  so  good.  But  what  if  instead  of  having  a  point  object, 
we  have  an  unpoint,  a  black  hole  in  a  sea  of  dye.  When  the  unpoint 
is  in  the  focus  plane,  a  pinhole  that  should  measure  zero  will  not 
do  so  because  other  beams  will  be  exciting  dye  above  and  below 
the  specimen  and  some  of  this  light  will  be  out-of-focus  enough 
to  come  through  the  pinhole  that  should  be  dark.  Long  before  the 
dark  point  is  distance  S  from  focus,  it  will  become  invisible.  The 
geometry  that  helped  reduce  the  brightness  of  the  out-of-focus 
point  object  by  625x  no  longer  works  because,  beyond  a  defocus 
of  +D,  the  diminution  in  intensity  that  occurs  as  the  diameter  of 
the  cone  of  light  from  a  single  point  is  supplemented  by  bright¬ 
ness  from  the  now-overlapping  cones  of  fluorescent  light  from 
nearby  holes. 

Although  Figure  10.3  discusses  round  apertures,  similar  rea¬ 
soning  can  be  applied  to  linear  ones,  with  the  exception  that,  when 
the  defocus  equals  D,  a  larger  fraction  of  the  light  will  go  through 
the  next  slit  than  went  through  the  six  nearest  pinholes.  This  fact 
is  often  taken  as  evidence  that  “slit-scanners  do  a  worse  job  of 
optical  sectioning  than  point-scanners.”  However,  if  we  keep  the 
transmission  of  the  two  disks  the  same,  the  D/S  of  a  slit  disk  will 
be  smaller  and,  assuming  that  we  choose  the  same  D  value  based 
on  the  resolution  of  the  objective,  S  will  be  larger  and  in  the  final 
situation  stray  light  situation  will  be  much  the  same  as  with  pin¬ 
holes.  Yes,  a  greater  fraction  of  the  light  goes  through  the  next  slit 
but  now  this  will  not  happen  until  the  defocus  is  greater  and,  con¬ 
sequently,  the  light  intensity  in  photons/square  centimeter,  is  lower. 
Indeed,  as  a  slit  scanner  with  a  larger  S  value  has  a  deeper  band 
of  “-values  for  which  no  overlap  occurs,  it  could  be  said  to  provide 
better  optical  sectioning. 

Clearly,  any  curves  plotting  the  ^-resolution  of  disk  scanners 
inherently  include  assumptions  about  the  stain  distribution  in  the 
specimen,  not  only  its  thickness  and  pattern,  but  in  the  case  of 
some  line  scanners,  also  its  orientation  in  the  xy-plane.  In  other 
words,  any  simple  statements  regarding  the  optical  sectioning  abil¬ 
ities  of  disk  scanners  that  fail  to  discuss  the  nature  of  the  speci¬ 
men  do  not  convey  much  useful  information  to  the  biologist. 
Figure  10.3  shows  that 

•  For  specimens  in  which  stained  structures  are  confined  to  a 
thin  layer,  or  for  thicker  specimens  in  which  the  stain  is  con¬ 
centrated  in  clumps  that  represents  only  a  small  minority  (say 
<1%)  of  voxels,  disk  scanners  can  approach  the  optical  sec¬ 
tioning  performance  of  single-beam  confocals. 

•  Choosing  a  larger  S  increases  the  zone  of  best  sectioning  per¬ 
formance  but  also  reduces  signal  levels. 

•  Specimens  composed  of  heavily  stained  structures  that  extend 
over  a  range  of  depths  will  pose  more  problems. 

•  In  the  final  analysis,  optical  sectioning  performance  has  more 
to  do  with  the  transmission  of  the  disk  than  with  whether  the 
apertures  are  slits  or  pinholes. 

As  will  be  discussed  below  in  the  section  on  electron  multi¬ 
plying  CCD  (EM-CCD),  it  is  significant  that  all  geometries  will 
permit  more  light  from  objects  that  are  far  from  the  focus  plane  to 
reach  the  detector  than  would  be  the  case  with  single-beam 
scanners. 
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reflected  or  backscattered  light,  it  is  important  to  prevent  excita¬ 
tion  light  from  reflecting  off  the  disk  and  reaching  the  detector. 
Although  most  of  the  90%  to  99%  of  the  light  that  does  not  pass 
through  the  disk  (except  the  Yokogawa  design)  is  absorbed  by  it, 
the  amount  reflected  from  even  the  blackest  surface  is  immense 
compared  with  that  returning  from  the  specimen.  This  reflected 
light  can  be  eliminated  using  barrier  filters  as  long  as  the  system 
is  used  to  detect  a  fluorescent  signal  but  other  techniques  must  be 
used  when  the  microscope  is  used  to  collect  backscattered  or 
reflected  light.  The  disk-scanning  units  are  often  attached  to  the 
side  port  (camera)  or  back  port  (lamp)  of  the  microscope  and  either 
a  mercury/xenon  arc  lamp  or  laser  fiber-optic  (Yokogawa)  is 
attached  to  the  unit.  A  detailed  comparison  of  the  hardware  spec¬ 
ifications  of  common,  commercial,  epi-fluorescent,  confocal  LSM 
and  disk-scanning  microscopes  is  shown  in  Table  10.2. 

Disk  Scanners  for  Backscattered  Light  Imaging 
The  Tandem-Scanning  Confocal  Microscope 

The  earliest  design  of  a  disk-scanning  confocal  microscope  used  a 
tandem  pinhole  design  (Petran  et  al.,  1968,  1985)  in  which  pin¬ 
holes  arranged  on  the  disk  in  a  symmetric  pattern  of  interleaved 
right-  and  left-handed  Archimedian  spirals,  illumination  and  detec¬ 
tion  occur  in  tandem,  through  separate  pinholes  on  opposite  sides 
of  the  disk.  The  illumination  entering  one  pinhole  goes  through  the 
objective,  and  light  reflected  from  the  sample  is  optically  rerouted 
via  a  set  of  mirrors  and  a  pellicle  beam-splitter  to  a  second  set  of 
conjugate  pinholes  located  on  a  diametrically  opposed  spiral  (see 
Fig.  10.4).  This  convoluted  light  path  was  used  to  prevent  glare 
reflected  from  the  disk  from  reaching  the  ocular.  This  was  a 


Nipkow  disk 
top  view 


In  disk-scanning  microscopes,  the  scanning  disk  is  positioned  in  FIGURE  10.4.  The  tandem  scanning  reflected  light  microscope  (Petran et  al, 

an  image  plane.  For  all  systems,  but  especially  for  those  using  1985). 
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FIGURE  10.5.  The  single-sided  scanning  confocal  light  microscope  (Xiao  and 
Kino,  1990). 


tremendous  advantage  when  looking  for  weak  backscattered-light 
signals  but  the  difficulty  of  making  and  orienting  a  dichroic  mirror 
on  the  surface  of  the  pellicle  beam-splitter  made  this  design  diffi¬ 
cult  to  use  for  fluorescence.  Not  only  was  it  hard  to  change  the 
beam-splitter  without  ruining  the  alignment,  but  the  intensity  was 
so  low  that  one  still  needed  to  make  long  exposures,  thereby  negat¬ 
ing  the  advantages  of  the  high  scan  speed.  The  other  major  limi¬ 
tation  of  this  microscope  was  its  mechanical  complexity  and  the 
difficulty  of  properly  translating  the  mirrors  in  three  dimensions 
and  tilting  them  along  two  axes.  In  fact,  scan  lines  were  always 
visible.  A  commercial  version  of  this  microscope  using  40  to  70 
micron  square  pinholes  was  produced  by  Noran  (now  Thermo- 
Noran,  Middleton,  WI,  model  TSM,  now  discontinued). 

Single-Sided  Disk  Scanning 
Confocal  Microscope 

The  single-sided  disk-scanning  confocal  microscope  designed  by 
Gordon  Kino  (Xiao  and  Kino.  1987;  Xiao  et  al.,  1988;  Lichtman 
et  al.,  1989)  uses  the  same  set  of  pinholes  for  illumination  and 
detection  (Fig.  10.5).  It  has  a  simple  optical  path  that  incorporates 
a  number  of  features  to  minimize  glare  from  the  disk.  The  holes 
are  etched  in  a  polished  black  chrome  coating  evaporated  onto  a 
quartz  disk  and  this  disk  is  placed  in  the  intermediate  image  plane 
at  a  slight  angle  so  that  excitation  light  is  reflected  off  the  optical 
axis  to  where  it  can  be  trapped  by  a  beam  stop.  For  BSL  imaging, 
the  input  light  is  polarized  and  after  being  transmitted  through  the 
disk  it  passes  through  a  quarter  wave  plate  in  front  of  the  objec¬ 
tive  lens  so  as  to  produce  circularly  polarized  light.  Light  reflected 
by  the  specimen  undergoes  the  reverse  effect  and  an  analyzer 
placed  in  front  of  the  eyepiece/camera  selects  against  light 
reflected  from  the  disk  and  passes  light  reflected  from  the 
specimen. 

The  disk  on  Kino’s  microscope  contained  -200,000  pinholes 
of  25  micron  diameter  and  spun  at  -2000  rpm,  giving  a  scanning 
speed  of  up  to  700  frames/s.  As  the  pinhole  is  near  the  optimal 
diameter  for  use  with  a  high  magnification,  high  NA  lens,  the 
resolving  power  of  this  system  is  similar  to  that  of  a  single-beam 
confocal  LSM,  but  much  faster!  Kohler  illumination  (Born  and 
Wolf,  1998)  is  used  so  that  standard  epi-fluorescent  (or  bright- 
field)  imaging  can  be  achieved  by  removal  of  the  disk.  Important 


optimization  parameters  include  adjusting  the  mercury  (or  xenon) 
arc  so  that  illumination  is  uniform  over  the  field  of  view.  An 
aperture  stop  limits  stray  light  and  a  field  lens,  positioned  below 
the  disk,  decreases  vignetting.  For  fluorescent  imaging  the  50/50 
beam-splitter  can  be  replaced  by  an  appropriate  fluorescent  filter 
cube,  as  in  epi-fluorescent  microscopy. 

Disk  Scanner  Designs  for  Use  in  Fluorescence 

Because  of  its  simplicity,  the  single-sided  disk  scanner  is  by  far 
the  most  popular  style  of  confocal  disk-scanning  microscope. 
Tricks  such  as  tilting  of  the  disk  and  the  use  of  anti-flex  techniques 
are  unnecessary  because  modern  dichroics  and  barrier  filters  can 
prevent  almost  all  excitation  light  from  reaching  the  CCD.  Several 
commercial  systems  suitable  for  fluorescent  imaging  use  variations 
of  this  design. 


CARV,  DSU,  and  Other  Disk-Scanning 
Confocal  Microscopes 

Early  commercial,  fluorescent,  single-sided  disk  systems  were  sold 
by  Technical  Instruments  system  (K2-Bio)  and  Newport  Systems 
(VX100  confocal  adaptor).15  In  the  former,  the  upper  part  of  a  stan¬ 
dard  microscope  was  replaced  with  the  disk  and  illumination 
system.  The  later  system  is  based  on  an  instrument  developed  by 
Jeff  Lichtman  (Lichtman  et  al. ,  1989)  and  was  sold  as  an  attach¬ 
ment  for  an  upright  microscope.  Briefly,  a  4x  low  NA  lens  is  used 
to  image  the  disk  scanner,  mounted  on  a  secondary  external  stage, 
onto  the  CCD.  The  secondary  stage  contains  compact  intermedi¬ 
ate  optics  and  a  high  NA  main  objective  is  used  to  image  the  spec¬ 
imen  onto  the  disk.  In  both  systems,  the  disk  had  separate  tracks 
with  different  sized  pinholes  or  slits  so  as  to  match  the  magnifica¬ 
tion  of  the  objective  or  light  budget  of  the  application,  but  only  the 
KCS  Bio  used  normal,  highly  corrected  microscope  objectives 
under  optimal  optical  conditions.  The  CARV  (Boston,  MA;  Fig. 
10.6;  Table  10.2)  uses  a  mercury  arc  lamp  for  excitation  and 
mounts  onto  the  left-side  port  of  an  inverted  microscope.  It  uses 
60  pm  diameter  pinholes  at  -250  pm  spacing,  scans  at  200  fps,  and 
has  -5%  to  8%  transmission.  As  these  pinholes  are  well  above  the 
Airy  limit  of  even  the  highest  magnification  lens,  z-resolution  is 
moderate  even  when  one  utilizes  the  1 .5x  auxiliary  magnifier  now 
found  on  some  microscope  stands  to  change  a  lOOx  objective  into 
one  with  150x.  The  user  can  see  the  live,  confocal  image  through 
a  second  binocular  head,  but  this  system  is  relatively  bulky  and 
does  not  work  with  upright  microscopes.  The  CARV2  with  a  faster 
scanning  speed  (1000  fps),  automated  filters,  and  the  ability  to  do 
FRAP  by  switching  the  Nipkow  disk  with  a  photobleaching  slit, 
has  just  been  introduced. 

A  new  disk-scanning  unit,  the  DSU  by  Olympus  (Figs.  10.7 
and  10.8;  Table  10.2)  uses  a  multi-slit  design  in  which  half  the 
slits  are  horizontal  and  the  other  half  are  vertical  on  the  disk.  The 
user  can  switch  between  five  different  disks  (DSU1-5)  each 
having  a  different  fill  factor  (5%  to  10%)  and  slit  width  (13  to 
38  pm)  for  use  with  both  low  and  high  magnification  objectives. 
The  confocal  head  is  compact,  but  as  it  attaches  to  the  back  Hg 
lamp  port  of  either  inverted  or  upright  Olympus  microscopes, 
direct  viewing  through  the  eyepiece  is  not  possible.  It  has  an 
adjustable  field  stop  and,  as  the  intermediate  optics  are  part  of  the 
conventional  illumination  path,  it  is  fully  UV  compatible.  As 
shown  in  Figure  10.8  (bottom),  S  >  85  (12.5%  fill  factor),  other¬ 
wise  the  resolving  power  decreases  substantially. 


15  See  schematics  in  Appendix  2  of  the  second  edition. 
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FIGURE  10.6.  CARV  (Atto  Bioscience)  disk-scanning  confocal  microscope. 
Schematic  of  the  CARV2  optical  path  (BD  Bioscience),  a  pinhole-based  disk¬ 
scanning  confocal  system.  (Image  adapted  from  that  provided  by  BD  Bio¬ 
science.) 
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FIGURE  10.8.  Characteristics  of  Olympus  DSU  slit-based  confocal  disk  and 
importance  of  a  low  fill-factor  for  adequate  axial  resolution.  (Top,  left)  Layout 
of  the  slit  disk  that  is  about  the  size  of  a  CD.  Approximately  half  the  slits  are 
horizontal  (green)  and  half  vertical  (red)  so  that  when  rotated  the  lateral  reso¬ 
lution  is  matched  in  the  x-  and  y-axis.  (Top,  right)  Blowup  of  a  region  of  the 
disk.  The  slit  width  ( D )  and  distance  between  slits  (5)  are  indicated  and  a  table 
shows  the  different  D  and  S  (in  micrometers)  of  interchangeable  disks. 
(Bottom)  Plots  show  that  axial  resolution  worsens  as  S  decreases  (for  a  fixed 
D )  and  in  practice  S  >  8 D  (12.5%  fill  factor).  (Images  and  data  adapted  from 
those  kindly  provided  by  Olympus.) 


FIGURE  10.7.  Olympus  disk-scanning  unit  confocal  instruments.  (Left)  Schematic  of  the  optical  path.  (Right)  Picture  of  the  DSU  (compact  box)  mounted  on 
an  upright  microscope  (can  also  be  mounted  on  an  inverted  Olympus  microscope).  (Images  adapted  from  those  provided  by  Olympus  USA.) 
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The  Yokogawa  Microlens  —  An  Illuminating 
Approach 

The  CSU10/22  disk-scanning  head  design  by  Yokogawa  Electric 
(Japan)  is  an  innovative  twist  on  the  single-sided  disk-scanning  flu¬ 
orescence  confocal  microscope  that  greatly  improves  the  ex¬ 
citation  light  budget  (Fig.  10.9).  Yokogawa  focuses  the  light 
from  a  beam-expanded  laser  onto  a  disk  containing  20,000 
microlenses.  This  disk  is  mounted  on  the  same  axis  as  the  pinhole 
disk  and  in  such  a  way  that  each  lens  focuses  its  light  onto  a  dif¬ 
ferent  50|im  diameter  pinhole  (Fig.  10.9,  blue  spindle).  As  the 
spacing  interval  between  pinholes  (pitch)  is  250  microns,  the  exci¬ 
tation  transmission  would  be  only  4%  without  the  microlens.  With 
the  microlens,  the  total  transmission  through  both  disks  is 
-40%,  an  order  of  magnitude  improvement  (Tanaami  et  al., 
2002).  Thus,  even  for  poorly  stained  samples  there  is  adequate 
light.16 


FIGURE  10.9.  Schematic  of  Yokogawa  CSU-10/22  spinning  disk.  Yokogawa 
has  a  microlens  array  top  (blue  disk)  that  focuses  excitation  laser  light  (green) 
onto  20,000  pinholes  in  a  Nipkow  disk  located  in  an  intermediate  image  plane, 
illuminating  the  sample.  Emitted  light  (red)  returns  through  the  same  pinhole 
and  is  reflected  by  a  dichroic  pellicle  mirror  positioned  between  the  microlenses 
and  the  Nipkow  disk  and  is  then  focused  onto  the  camera.  The  positioning  of 
the  dichroic  ensures  that  excitation  light  scattered  by  the  disks  is  not  detected. 
(Schematics  adapted  from  those  kindly  provided  by  Yokogawa  Electric 
Corporation,  Japan.) 


The  disk  rotates  at  1800  to  5000  rpm,  depending  on  the  model 
(CSU-10  vs.  CSU-21/22,  respectively)  and  the  hole  pattern  scans 
12  frames  in  each  rotation.  This  gives  a  theoretical  maximum 
frame  rate  of  360  to  1000  fps,  assuming  that  the  camera  will  read 
out  this  fast  and  that  enough  signal  can  be  produced  in  such  a  short 
period.  In  practice,  operation  at  anything  above  100  to  200  fps 
requires  careful  synchronization  between  the  disk  and  the  camera 
so  that  all  holes  move  an  integral  number  of  frame  crossings. 
Failure  to  do  so  will  cause  streaking  in  the  image.  Moire  effects 
can  occur  between  the  curved  scan  lines  of  the  pinhole  and  the  rec¬ 
tilinear  layout  of  the  CCD,  especially  at  short  exposure  times,  and 
this  problem  is  worse  when  the  CCD  uses  frame-transfer  readout 
(Chong  et  al.,  2004). 

At  any  one  time  -1000  pinholes  (6%  of  total)  are  illuminated. 
Direct  viewing  is  possible  through  a  monocular  but  laser  safety 
concerns  have  eliminated  this  feature  in  some  systems.17 

A  dichroic  beam-splitter  mounted  between  the  two  disks 
directs  fluorescent  light  emerging  from  the  pinholes  to  the  CCD 
camera  via  a  projection  lens.  The  CSU-10  uses  only  a  single  filter 
cube  that  passes  only  up  to  three  laser  lines  but  this  can  be  removed 
with  some  difficulty;  the  CSU-21/22  accommodates  up  to  three 
dichroics  and  three  emission  filters  but  these  must  be  installed  at 
the  factory  and  can  not  be  changed.  Current  filters  sold  in  the 
PerkinElmer  version  of  the  Yokogawa18  system  include  GFP, 
CFP/YFP,  GFP/Texas  red/Cy5.  Cross-talk  can  be  an  issue  and  thus 
an  additional  wheel  of  barrier  filters  is  desirable.  Several  com¬ 
panies  sell  Yokogawa  spinning  disks  units  individually  or  as  a 
complete  package  with  hardware  [lasers,  acousto-optic  modulator 
(AOM),  cameras,  z-piezodrivers,  etc.]  and  software.19 

New  Fast  Line  Scanner  —  Zeiss  LSM510  LIVE 

Although  not  a  disk-scanning  system,  the  new  Zeiss,  LSM5  LIVE, 
line-scanning  confocal  microscope  may  soon  emerge  as  a  com¬ 
petitor  to  existing  disk-scanning  confocals.  A  schematic  of  the 
LSM5-LIVE  is  shown  in  Figure  10.10  and  some  parameters  are 
included  in  Table  10.2.  The  largest  departure  from  the  popular 
point-scanning  LSM510  is  that  the  system  scans  a  line  of  excita¬ 
tion  over  the  focus  plane  and  then  detects  the  signal  in  parallel 
using  a  1  x  512  pixel  linear-array  CCD.  The  Gaussian  beam  from 
the  laser  is  transformed  into  a  plane  of  parallel  light  rays  using 
anamorphic  optics.  This  line  of  light  reflects  off  a  narrow,  reflec¬ 
tive  patch  in  the  center  of  an  otherwise  clear  beam-splitter,  called 
the  Achrogate.  The  light  sheet  reflected  by  the  Achrogate  strikes  a 
galvanometer  scanning  mirror  located  in  an  aperture  plane,  so  that 
as  it  rotates,  the  line  of  light  is  scanned  over  the  specimen  in  a 
direction  perpendicular  to  its  length.  The  objective  focuses  this  line 
of  light  into  another  line,  perpendicular  to  the  first,  at  its  focus 
plane.  Light  leaving  the  specimen  at  any  angle  that  strikes  the 


16  The  total  illumination  power  depends  on  the  power  of  the  laser  line  (e.g., 
5-50  mW),  its  coupling  efficiency  to  the  single-mode  fiber  (-50%),  and  trans¬ 
mission  through  the  beam  expander,  microlens,  and  pinhole  (-40%).  Light 
striking  the  170  X  120|rm  field  covered  by  a  100X  objective,  should  be  in 
the  range  of  l-10mW. 

17  PerkinElmer  Ultraview  5-line  instrument. 

18  We  have  extensively  tested  the  Ultraview  5-laser  line  spinning  disk  system 
from  PerkinElmer  and,  with  exception  of  a  few  bugs  in  the  file  nomencla¬ 
ture,  the  hardware  is  well  integrated  and  easy  for  new  users  to  use.  The 
system  from  Solamere  offers  more  flexibility  of  setup  configuration.  If  one 
integrates  one’s  own  system,  attention  should  be  paid  to  camera  synchro¬ 
nization  and  the  CSU/22  with  a  variable  speed  disk  should  facilitate  this. 

19  Other  purveyors  include:  Solamere  Technologies,  Salt  Lake  City,  UT; 
Visitech,  Sunderland,  UK;  Andor  Technologies,  Belfast,  UK,  etc. 
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FIGURE  10.10.  Schematic  of  Zeiss  LSM5-LIVE,  line-scanning  confocal.  1, 
Laser  fibers;  2—3,  mirrors;  4,  linear  beam  shaper;  5,  Achrogate;  6,  zoom  lens; 
7,  scan  mirror;  8,  tube  lens;  9,  objective;  10,  specimen;  11,  dichroic  wheel;  12, 
lens;  13,  adjustable  slit  mask;  14,  barrier  filter;  15,  linear  1  X  512  pixel,  CCD. 
(Schematic  kindly  provided  by  Carl  Zeiss  Inc.,  Jena,  DE.) 


objective  leaves  it  as  a  round  beam,  most  of  which  does  not  strike 
the  reflective  patch  and  proceeds  toward  the  detectors.  It  is  then 
separated  into  two  fluorescent  channels  by  a  dichroic  beam¬ 
splitter  and  each  channel  passes  through  a  “camera”  lens  that 
focuses  the  round  beam  to  form  an  image  of  the  illuminated  line 
on  the  specimen  at  the  surface  of  a  glass  aperture  plate.  The  met¬ 
allized  reflective  layer  on  the  aperture  plate  is  etched  so  as  to  form 
a  number  of  slits  having  different  widths.  Normally,  one  would 
choose  a  slit  having  a  width  of  1  to  2  Airy  units  for  the  objective 
in  use.  Only  the  light  passing  the  slit  is  transmitted  to  the  linear 
CCD  detector.  The  advantages  of  this  system  seem  to  be: 

•  Parallel  illumination  of  512  pixels  increases  the  rate  that  data 
can  be  accumulated  without  reaching  singlet-state  saturation. 

•  Galvanometer  scanning  in  a  single  direction  increases  the 
maximum  frame  rate  to  that  set  by  the  readout  speed  of  the 
detector. 

•  The  Achrogate  reflects  all  the  laser  light,  regardless  of  wave¬ 
length,  but  obstructs  only  about  5%  of  the  returning  signal. 

•  The  detection  slit  can  be  adjusted  independently  of  the  illumi¬ 
nation  system  to  match  the  properties  of  the  objective  and  the 
specimen. 

•  The  linear  CCD  has  a  higher  QE  than  any  PMT  and,  if  the 
present  version  is  eventually  replaced  with  an  EM-CCD 
version  (as  discussed  below),  the  noise  level  will  be  similar. 

The  trade-off  in  xy-  and  z-resolution  is  the  same  as  for  other 
laser-based  slit-scanners.  While  the  design  is  new  and  intriguing. 


more  details  on  light  efficiency  and  sensitivity  are  needed  as  is 
testing  on  dim  fluorescent  samples  head-to-head  with  spinning 
disks. 

NEW  DETECTORS  -  A  CRITICAL  COMPONENT 

A  great  benefit  of  disk-scanning  confocals  is  their  ability  to 
produce  a  real-time,  2D  confocal  image  directly.  A  PMT  senses 
a  signal  that  varies  in  time,  not  position,  and  requires  a  computer 
to  assemble  a  viewable  image.  The  ability  to  look  down  an  eye¬ 
piece  (depending  on  the  model)  and  see  a  live  image  that  permits 
you  to  select  the  best  part  of  the  specimen  is  a  great  convenience. 
Our  eyes  are  exquisite  detectors  with  an  excellent  dynamic  range 
and  ability  to  detect  fine  detail  and  motion.  Although  their  spec¬ 
tral  range  is  limited  to  -400  to  600  nm  and  the  QE  is  not  very  good 
(<10%),  if  the  sample  is  bright  enough,  your  eyes  are  the  quickest 
way  of  finding  a  good  region  of  the  specimen. 

Standard,  front-illuminated  CCDs  have  a  QE  of  -20%  to  65% 
in  the  green,  while  with  more  expensive  back-thinned  CCD  detec¬ 
tors,  QE  can  approach  95%,  far  superior  to  a  PMT.  Most  stan¬ 
dard,  scientific  CCDs  have  a  12  to  16  bit  dynamic  range  and  a  noise 
floor  of  ±  5  to  ±  15  electrons/pixel,  depending  on  read  speed  and 
other  design  parameters.  As  a  result,  weak  signals,  such  as  those 
from  very  dim  samples  or  single  molecules,  are  lost  in  the  readout 
noise. 

A  CCD  camera  can  have  good  QE  and  still  have  poor  noise 
performance;  it  turns  photons  into  photoelectrons  efficiently,  but 
this  signal  is  then  obscured  by  the  three  sources  of  background 
noise:  shot  noise,  dark  charge  noise,  and  electronic  read-noise.20 
Shot  noise  is  unavoidable  and  is  due  to  the  random  Poisson  sta¬ 
tistics  of  collecting  photons.  Dark  noise  is  the  square  root  of  the 
charge  that  leaks  into  the  pixel  during  the  exposure  and  depends 
exponentially  on  the  CCD  operating  temperature  (it  is  halved  for 
every  8°C  of  cooling).  Electronic  noise  increases  with  the  capac¬ 
itance  (area)  of  the  read  node  and  the  square  root  of  the  readout 
speed.  The  readout  noise  of  a  good  scientific  CCD  is  <  ±10  elec¬ 
trons  at  a  lMHz+  readout  rate  (see  Table  10.3).  This  is  much  too 
high  to  detect  single  photons  or  the  low  light  levels  from  a  confo¬ 
cal  scanner,  where  many  pixels  image  voxels  of  the  specimen 
contain  no  dye.  (For  more  details  about  CCDs,  see  Appendix  3). 


Image  Intensifiers 

For  the  last  two  decades,  the  detector  of  choice  for  fast  imaging  of 
low  intensity  dyes  (e.g.,  single  molecules)  coupled  an  image  inten- 
sifier  with  a  CCD  readout.  In  an  image  intensifier,  photons  strike 
a  photocathode,  releasing  free  electrons  into  a  vacuum.  Each 
electron  is  amplified  usually  by  impact  multiplication  in  a 
microchannel  plate,  producing  a  cloud  of  -106  electrons  that  is 
then  accelerated  onto  a  phosphor  layer  where  each  one  gives  up 
its  energy  by  producing  an  even  larger  number  of  photons.  These 
photons  are  then  conveyed  to  the  CCD  by  a  fiber-optics  plate.  The 
gain  of  the  intensifier  is  so  high  that  the  signal  in  the  CCD  output 
corresponding  to  a  single  photoelectron  is  greater  than  the  CCD 
readout  noise. 

The  QE  of  an  image  intensifier  is  determined  by  the  material 
used  for  the  photocathode.  The  most  modern  cathodes  use  GaAsP 
and  have  an  intrinsic  QE  of  up  to  30%  to  40%,  well  into  the  red 


20  All  detectors,  including  PMTs,  suffer  from  this  noise. 
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TABLE  10.3.  Relative  Parameters  of  CCD  and  EM-CCD  Detectors 


CCD  and  EM-CCD3  Detectors 

Traditional  CCD:  Orca  ERb 

EM-CCD:  DV887BV 

EM-CCD:  DV860BI 

EM-CCD:  DV885VP 

Chip:  Front  or  back  illuminated 

Front 

Back 

Back 

Front 

Camera  manufacturer 

Hamamatsu 

Andor 

Andor 

Andor 

and  model 

(C4742-80- 1 2AG) 

E2V  CCD  87 

E2V  CCD  60 

TI  TC285 

Active  pixels 

1344  x  1024 

512x512 

128  x  128 

1000  X  1000 

Pixel  sizec 

6.45  pm 

16pm 

24  pm 

8  pm 

Image  area 

8.67  x  6.6  mm 

8.2  x  8.2  mm 

3.1  x  3.1  mm 

8.0  x  8.0mm 

Raw  QE 

70%  @  500  nm 

92%  @  575  nm 

92%  @  575  nm 

>65%  @  600  nm 

Readout  speed 

14.7  MHz 

10,  5,  3,  1  MHz 

10,  5,  3,  1  MHz 

35,  27,  13,  5  MHz 

Maximum  cooling 

-30°C 

-90°C 

-90°C 

-90°C 

Maximum  EM  gain  (actual) 

1 

X1000 

X1000 

X1000 

Dark  current  (minimum) 

0.003  e/p/s 

0.0035  e/p/s 

0.004  e/p/s 

0.001  e/p/s 

Read  noise  (r.m.s.)d 

8e  @  14.7  MHz 

62e  @  10  MHz 

60e  @  10  MHz 

25e  @  35  MHz 

<le  with  EM 

<le  with  EM 

<le  with  EM 

CCD  clocking 

Multi-phase 

Multi-phase 

Multi-phase 

Virtual  phase 

Vertical  shift  speed  (per  row) 

Variable 

3.4-6  (As  variable 

0.1-6ps  variable 

0.1-6ps  variable 

Maximum  full  frame  rate 

9fps 

>34  fps 

>500  fps 

>3 1  fps 

Maximum  frame  rate  (4  x  4) 

>27  fps 

>130fps 

>1300  fps 

>100  fps 

Binning 

2x2,  4x4,  8x8 

Full  serial  and  parallel 

Full  serial  and  parallel 

Full  serial  and  parallel 

(vertical) 

(vertical) 

(vertical) 

Fill  factor 

-75% 

100% 

100% 

100% 

Single  pixel  full  well 

18,000e 

220,000e 

200,000e 

40,000e 

Dynamic  range  of  digitizer 

2,250:1 

12  bits  @  10  MHz 

12  bits  @  10  MHz 

12  bits  @  35  MHz  13 

>15  bits  @ 

>15  bits  @ 

bits  @  5  MHz 

1  MHz 

1  MHz 

A/D  bit  depth6 

12  bits 

14/16  bits 

14/16  bits 

14  bits 

CCD  type  &  mount 

Progressive  scan  interline 

Masked  w/full  frame 

Masked  w/full  frame 

Masked  w/full  frame 

CCD  with  microlens: 

transfer;  C-mount 

transfer;  C-mount 

transfer;  C-mount 

C-mount 

aEM-CCD  cameras  are  sold  by  several  companies  including  Andor  (shown  here),  Hamamatsu,  Roper,  PCO,  and  others.  Many  use  identical  chips  from  E2V  or  Texas  Instru¬ 
ments,  however,  other  properties  such  as  chip  cooling  temperature,  vacuum  and  sealing,  amplifier,  hardware  (e.g.,  TTL  integration),  software  control  and  integration,  inter¬ 
nal  shutters,  etc.,  can  vary  considerably  between  manufactures  and  can  strongly  affect  camera  and  versatility.  Data  kindly  provided  by  Colin  Coates  at  Andor  Tech.  (Belfast, 
Northern  Ireland).  All  specifications  based  on  manufacturers  claims. 

bThis  is  a  popular  high  readout  speed,  scientific  CCD  and  chosen  as  a  respective  point  of  reference.  There  are  hundreds  of  CCDs  to  choose  from  and  specifications  will 
vary  considerably  and  are  occasionally  optimistic. 

c  Cameras  with  18  pm  pixels  will  require  the  use  of  a  2x  phototube  to  produce  Nyquist-sampled  images  with  40x  and  60x  high  resolution  objectives. 

dManufacture  specifications;  note  often  some  of  this  bit  depth  is  empty  and  the  true  dynamic  range,  which  is  determined  by  the  ratio  of  the  full  well  depth  over  the  noise 

floor  of  the  camera,  is  considerably  smaller  (e.g.,  if  in  the  Orca  ER  18,000e/8e  =  2250,  while  212  =  4096). 


end  of  the  spectrum.  However,  as  good  red  performance  often  goes 
along  with  high  dark  current  unless  the  photocathode  is  cooled, 
many  intensifiers  use  the  S-20  photocathode  material  commonly 
used  in  PMTs  and  having  a  QE  at  least  2x  lower.  In  addition,  as 
only  about  50%  of  the  photoelectrons  reaching  the  micro-channel 
plate  (MCP)  are  actually  multiplied  at  all,  and  as  the  amount  by 
which  each  of  the  remainder  are  actually  amplified  varies  greatly 
(creating  multiplicative  noise),  the  effective  QE  is  about  half  that 
of  the  photocathode  itself.  Finally,  because  both  the  MCP  and  the 
process  of  accelerating  the  charge  onto  the  phosphor  to  make  light, 
involve  loss  of  spatial  precision,  the  resolution  of  the  entire  camera 
is  substantially  less  than  would  be  indicated  by  the  array  size  of 
the  CCD. 

On  the  other  hand,  the  intensified  CCD  is  the  only  common 
image  sensor  that  can  be  gated  on  and  off  in  nanoseconds,21 
permitting  the  extremely  short  acquisition  bursts  needed  in  spe¬ 
cialty  applications  such  as  fluorescent  lifetime  imaging  (FLIM). 

Intensifier  Advantages  (+)  and  Disadvantages  (-): 

(+)  Noise  level  very  low;  able  to  “see”  signal  from  a  single 

photoelectron. 


21  The  camera  readout  time  is  much  longer  (e.g.,  <10()fps;  Kindzelskii  and 
Petty,  2003). 


(+)  Fast  gating  possible. 

(±)  Effective  QE  of  10%  to  25%. 

(— )  Pixel  blooming  of  bright  signals  can  occur. 

(— )  Scintillation  or  hot  pixels  can  occur. 

(— )  Photocathode  can  be  permanently  damaged  by  brief  overex¬ 
posure  to  light. 

On-Chip  Electron  Multiplying 
Charge-Coupled  Device 

Recently,  the  problem  of  increasing  the  signal  from  a  single 
photoelectron  above  the  noise  level  of  the  CCD-read  amplifier 
has  led  to  the  perfection  of  on-chip  electron  multiplication  (EM). 
Although  high-gain  (108x),  avalanche  electron  multiplication  in  a 
reverse-biased  semiconductor  photodiode  has  been  used  for  some 
time  to  turn  a  single  photoelectron  into  a  current  pulse  large 
enough  to  be  easily  measured  by  electronics,  efforts  to  apply  this 
technique  to  the  output  of  a  CCD  failed  until  recently. 

The  main  distinction  between  an  EM-CCD  and  a  normal,  sci¬ 
entific  CCD  is  the  addition  of  a  second  horizontal  register,  called 
the  gain  register,  between  the  chip  and  the  read  amplifier  (see  Fig. 
10.11.  top).  In  the  EM  gain  register,  a  higher  voltage  on  one  of  the 
three  sets  of  the  charge-transfer  electrodes  sets  up  electric  fields  to 
produce,  not  the  high  gain  impact  multiplication  found  in  the 
avalanche  photodiode,  but  very  low  gain.22  By  then  repeating  the 
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FIGURE  10.11.  (Top)  Anatomy  of  an  electron  multiplying  CCD  (EM-CCD) 
chip.  Illumination  of  the  imaging  register  (orange),  read  register  (pink),  and 
horizontal,  readout  register  is  to  the  same  as  any  frame-transfer  CCD  chip. 
The  main  addition  is  an  electron  multiplying  (EM)  gain  register  (in  red)  before 
the  output  amplifier.  (Bottom)  Raw,  spectral  response  curves  of  back- 
illuminated  visible  (7:»'V )  and  /ronf-illuminated  visible  (FV)  EM-CCD  chips. 
The  emission  wavelengths  of  some  common  dyes  are  indicated.  (Data  kindly 
provided  by  Colin  Coates  at  Andor  Tech. )  When  these  chips  are  used  with  the 
EM  register  activated,  the  effective  QE  is  only  50%  of  that  shown. 


process  500  or  1000  times  over  a  line  of  many  pixels,  a  gain  of 
more  than  1000  can  be  obtained.  This  makes  the  signal  from  even 
one  photoelectron  easily  visible  above  the  noise  of  the  CCD  read 
amplifier,  even  one  operating  at  a  very  high  read  speed.23 

With  read  noise  effectively  absent,  the  main  remaining  noise 
sources  are  dark  current  and  clock-induced  charge  (CIC).  Dark 
current  can  be  substantially  reduced  by  cooling  the  chip  with  a 
multi-stage  Peltier  cooler.  At  -80°C  spurious  dark  current  and  CIC 
signals  are  reduced  to  about  one  count  in  250  pixels.  CIC  is  caused 
when  lattice  electrons  are  pulled  into  the  valence  band  by  the 
charge  transfer  process.  Unmeasurable  before  the  advent  of  the 
EM-CCD,  CIC  is  now  an  important  noise  source  that  can  be 
reduced  somewhat  by  carefully  shaping  the  charge  transfer  control 
pulses  or  made  slightly  worse  by  using  back-thinned  chips. 

Amplifying  the  signal  “on-chip”  is  appealing  as  it  reduces  the 
number  of  components  and  one  can  take  advantage  of  the  good 
QE  of  the  CCD.  As  the  signal  from  each  pixel  is  amplified  sepa¬ 


rately,  the  spatial  resolution  (MTF)  of  the  EM-CCD  is  identical  to 
that  of  the  same  CCD  chip  with  the  electron-multiplier  turned  off. 

There  is,  of  course,  a  price  to  pay  and  the  price  is  increased 
multiplicative  noise.  While  in  a  normal  CCD,  each  photoelectron 
is  counted  the  same,  because  of  the  statistical  rules  governing  the 
impact  ionization  process  in  the  EM-CCD,  some  electrons  are 
amplified  much  more  than  others.  As  the  noise  term  describing  this 
process  is  of  the  same  form  as  Poisson  noise,  and  as  such  terms 
are  “added”  as  the  square  root  of  the  sum  of  the  squares,  the  overall 
effect  is  to  increase  the  “Poisson"  noise  level  to  1.4x  what  it  should 
be  based  on  the  number  of  photoelectrons  actually  produced.  As 
the  only  way  to  reduce  Poisson  noise  1.4x  is  to  count  twice  as 
many  photoelectrons,  it  is  perhaps  easiest  to  think  of  the  EM-CCD 
as  having  essentially  no  read  noise  but  operating  as  though  the 
effective  QE  were  only  50%  as  high  as  it  would  be  were  the  same 
CCD  used  without  the  electron  multiplier.  This  fact  must  be 
remembered  when  viewing  raw  EM-CCD  QE  specifications. 


Electron  Multiplication  Charge-Coupled 
Devices  and  Disk  Scanners24 

How  do  EM-CCDs  improve  disk-scanning  confocal  imaging? 
Good  resolution  and  QE  are  important,  but  single-photon  sensi¬ 
tivity  means  that  the  EM-CCD  can  detect  very  low  signals  when 
a  normal  CCD  would  only  read  noise.  This  is  important  not  only 
because  disk  scanners  without  microlenses  are  light  starved  (e.g., 
it  is  like  using  a  95%-98%  neutral  density  filter  on  your  WF  flu¬ 
orescence  system),  but  also  because  in  a  selectively  stained  spec¬ 
imen,  by  far  the  most  common  voxel  intensity  is  zero  and  the 
EM-CCD  measures  zero  very  well.25 

Figure  10.12  shows  that  even  using  a  disk  scanner  and  a  back- 
thinned  90%  primary  QE  EM-CCD,  the  exposure  time  for  ade¬ 
quate  S/N  is  significant  (150ms).  Were  a  normal  high  quality  CCD 
to  be  used  instead,  -3-  to  5-fold  longer  exposures  would  be  needed 
for  similar  S/N.  Even  if  one  has  enough  light  to  use  a  normal  CCD, 
using  a  more  sensitive  detector  allows  one  to  use  less  excitation 
and  produce  less  photobleaching. 

Currently,  EM-CCD  chips  are  manufactured  by  E2V  (Enfield, 
UK)  and  Texas  Instruments  (Houston,  TX)  and  are  used  in  cameras 
from  several  companies:  Andor,  Hamamatsu,  Roper,  PCO,  Red 
Shirt,  and  others.  Several  different  EM-CCD  chips  exist  and  one 
should  match  the  QE,  pixel  clock  rate,  pixel  size,  well  depth,  and 
total  chip  area  to  the  biological  process  being  investigated  (see 
Table  10.3).  For  fast  imaging,  a  smaller  chip  (e.g.,  256  x  256 
pixels)  will  provide  increased  frame  rate.  At  present,  all  the  avail¬ 
able  chips  use  full-frame  transfer.  While  good  for  QE,  the  fact  that 
the  charge  pattern  moves  past  the  image  as  it  proceeds  to  the  read 
register  can  cause  some  streaking  and  moire  effects  with  the  scan 
lines  of  disk  scanners  even  when  fast  vertical  transfer  is  used 
(Chong  et  al.,  2004).  This  can  be  avoided  if  EM-CCD  chips  incor¬ 
porating  interline  transfer  appear  because  such  chips  permit  elec¬ 
tronic  shuttering  at  very  high  speed.  However,  unless  such  chips 
are  also  fitted  with  micro-lens  arrays,  these  will  pay  a  price  in  fill- 
factor  and  at  least  exhibit  2x  lower  effective  QE. 

Our  tests  of  the  Olympus  DSU  disk-scanning  system  have 
shown  that  EM-CCDs  can  make  the  difference  between  getting  an 
acceptable  image  and  seeing  nothing  because  the  signal  level  is 
below  the  noise  threshold  of  our  normal  CCD.  In  addition,  it 


22  The  gain  depends  on  the  temperature  and  the  exact  voltage  applied  to  a 
special  set  of  charge  transfer  electrodes  in  the  gain  register,  but  is  commonly 
about  1  %/transfer,  that  is,  a  charge  packet  of  100  electrons  would,  on 
average,  become  101  electrons  after  one  transfer,  or  again  on  average,  a 
single  electron  would  become  two  electrons  after  about  100  transfers. 

23  The  new  chips  will  operate  up  to  35 MHz  or  140fps  for  a  512  x  512  array. 


24  See  discussion  at  http://www.emccd.com. 

25  However,  because  of  the  reduction  in  effective  QE  the  normal  CCD  begins 
to  outperform  the  EM-CCD  when  the  signal  level  gets  much  above  about 
100  photons/pixel.  The  exact  crossover  point  depends  on  the  read  speed  as 
this  seriously  changes  the  read  noise  of  conventional  CCDs. 
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FIGURE  10.12.  Disk-scanning  slit  confocal  improves  optical  sectioning  and 
S/N  compared  to  epi-fluorescent,  but  requires  longer  exposure  times.  Lamprey 
neurons  were  microinjected  with  fluorescent-tagged  phalloidin  to  label  axons 
(ring-like  structures).  A  single  15  ms  exposure  epi-fluorescent  image  was  taken 
using  an  Olympus  BX-5 1  microscope  ( 1 . 1 NA  60x  water-immersion  objective) 
and  an  Andor  887,  back-illuminated  EM-CCD  camera  (top).  After  inserting  the 
DSU  disk  #3  (10%  transmission)  a  150ms  image  was  acquired  (same  gain, 
bottom).  In  both  images,  the  same  number  of  photons  struck  the  specimen. 
(Images  kindly  provided  by  Dr.  Jennifer  Morgan,  Yale  University  Medical 
School.) 


increased  the  number  of  high-quality  images  that  could  be  acquired 
before  photobleaching.26 

EM-CCD  Advantages  (+)  and  Disadvantages  (-): 

(+)  Like  CCDs,  EM-CCDs  are  photon  efficient  (good  QE),  have 
good  contrast  (MTF),  are  mechanically  robust,  and  are  not 
harmed  by  exposure  to  bright  light. 

(+)  They  have  single-photon  sensitivity. 

(+)  Ergo,  they  can  greatly  improve  the  speed  or  number  of  images 
acquired  before  photobleaching. 

(— )  No  fast  gating. 

(— )  Higher  cost  than  normal  CCD  (roughly  equivalent  to  an  inten¬ 
sified  CCD). 


APPLICATIONS  AND  EXAMPLES  OF 
CONFOCAL  DISK-SCANNING  MICROSCOPES 

Comparison  with  Epi-Fluorescence  Imaging 

As  discussed  above,  a  simple  disk-scanning  confocal  should  have 
better  optical  sectioning  but  lower  light  throughput  than  a  WF  epi- 
fluorescence  microscope.  To  demonstrate  this  we  took  images  of 
the  same  specimen  with  both  an  epi-fluorescence  setup  and  with 
an  Olympus  DSU  spinning-slit  system  (10%  transmission)  using 
an  Andor  512x512  BI  EM-CCD  camera.  Ten  times  longer  expo¬ 
sures  were  used  with  the  disk  in  to  compensate  for  reduced  illu¬ 
mination  striking  the  specimen.  As  seen  in  Figure  10.12  (processed 


26  The  same  holds  for  image  intensifiers,  but  lower  QE  and  reduced  resolution 
mitigate  this  benefit. 


identically),  there  is  considerably  less  background  fluorescence 
with  the  scanning  disk  in  place. 


Fast  3D/4D  Imaging 

The  micro-lens-assisted  spinning  disks  in  the  Yokogawa  design 
transmit  more  illumination  to  the  specimen,  increasing  the  max¬ 
imum  imaging  speed.  Even  with  a  traditional  CCD  such  as  the 
Hamamatsu  Orca  ER,  one  can  record  high  resolution  multi-color 
confocal  stacks,  as  demonstrated  in  Figure  10.13.  In  ~20s  we 
acquired  a  3D  stack  with  200  sections;  a  3D  reconstruction  is 
shown  in  the  lower  panel.  Using  a  single-beam  confocal  such  as 
the  Zeiss  LSM510,  acquisition  of  a  data  stack  of  similar  size  and 
quality  would  take  ~5min+.  In  the  xz  -views,  one  can  see  that 
z-resolution  appears  roughly  on  par  with  that  of  a  good  con¬ 
focal  LSM. 

The  ability  to  acquire  confocal  images  rapidly  allows  fast  4D 
imaging.  An  example  of  such  an  application  is  shown  in  Figure 
10.14  where  3D  stacks  of  ~20  layers  were  each  acquired  in  under 
2.5  s  and  repeated  over  time.  The  scientific  advantage  of  4D 
imaging  is  that  one  gets  a  complete  view  of  the  process.  In  this 
example,  clathrin  dynamics  can  be  seen  on  the  upper  cell  surface, 
near  the  perinuclear  Golgi  region  and  also  at  the  bottom  of  the  cell; 
areas  where  dynamic  changes  occur  are  circled.  The  lower  panel 
shows  a  single,  3D  time  point  from  a  4D  stack  of  a  double-labeled 
cell  projected  in  space  at  two  angles.  As  it  is  hard  to  convey  in  a 
static  medium  (this  book)  the  visual  power  and  the  high  level  of 
biological  contextual  information  available  from  looking  at  a  3D 
image  over  time  from  any  angle.  A  movie  made  from  this  3D  data 
stack  will  be  available  on  the  Web  site  associated  with  this  book 
http://www.springer.com/0-387-25921-X. 

The  challenge  of  how  best  to  process  and  visualize  large 
datasets  will  likely  be  an  increasingly  significant  hurdle.  It  is  worth 
commenting  that  while  photobleaching  is  still  a  real  limiting  factor 
with  a  normal  CCD,  ultrasensitive  cameras  allow  us  to  push  this 
envelope  as  we  enter  the  realm  of  the  real-time  4D  confocal 
imaging. 


Blazingly  Fast  Confocal  Imaging 

Some  biological  processes  can  only  be  revealed  (and  understood) 
with  high-speed  imaging.  Calcium  sparks,  flashes,  and  waves  are 
examples.  The  3D  diffusion  is  so  rapid  that  even  a  1  ms  exposure 
may  not  be  fast  enough  and  the  process  is  definitely  obscured  in 
real-time  (30fps)  or  slower  imaging.27  Optical  sectioning  is  needed 
both  to  improve  S/N  and  also  to  sample  a  slice  through  the  middle 
of  a  cell  because  such  an  image  is  less  subject  to  volumetric 
changes  in  signal  than  is  a  normal  epi-fluorescence  image. 

Maximum  image  acquisition  speed  is  determined  by  four 
factors:  the  excitation  power  delivered  to  the  sample,  photophysi¬ 
cal  limits  such  as  saturation,  the  disk  scan  speed,  and  the  camera 
pixel  clock  and  sensitivity.  Because  the  read  noise  of  a  normal 
CCD  rises  rapidly  with  read  speed  while  that  of  the  EM-CCD  does 
not,  an  ultrasensitive  camera  is  essential.  In  Figure  10.15,  the  com¬ 
bination  of  a  Yokogawa  disk-scanning  confocal  and  EM-CCD 
camera  has  permitted  ~110fps  imaging  of  calcium  sparks  (top 
panel)  or  calcium  waves  induced  by  electrophysiological  depolar¬ 
ization  (bottom).  These  images  are  truly  just  the  start  of  seeing  life 
in  the  (ultra)fast  lane. 


27  Even  current  EM-CCDs  may  not  be  fast  enough  to  fully  visualize  this 
process,  as  experiments  with  50  ns  time  resolution  made  using  a  gated  inten¬ 
sified  camera  (but  -30  ms  repeat  rates)  show  very  fast  initiation  of  the 
calcium  spark  (Kindzelskii  and  Petty,  2003). 
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FIGURE  10.13.  Fast,  multi-color  3D  optical  sectioning  (and  reconstruction)  of  a  single  axon  within  an 
intact  lamprey  spinal  cord  recorded  using  a  Yokogawa  CSU-22  disk-scanning  confocal  microscope 
mounted  on  an  Olympus  DC-71  with  a  Hamamatsu  Orca  ER  camera.  (Left)  Alexa-488  phalloidin  and  FM 
4-64  were  microinjected  into  a  single  axon  of  a  lamprey  spinal  cord.  60  x  1.2NA  objective.  Merged 
image  clearly  reveals  that  F-actin  surrounds  the  synaptic  vesicle  clusters.  (Right)  A  3D  stack  of  ~200  z- 
sections  was  acquired  at  lOOnm  steps  (~  100  ms  exposures);  the  total  acquisition  time  was  only  ~20s 
(much  less  than  for  200  sections  with  a  LSM  confocal).  Reconstruction  made  using  Volocity  (Improvi- 
sion)  software.  (Images  kindly  provided  by  Dr.  Jennifer  Morgan,  Yale  University  Medical  School.) 


FUTURE  DEVELOPMENTS? 

Although  forecasting  future  development  can  be  risky,  it  is  always 
useful  to  consider  what  could  be  done  better.  One  of  the  most  chal¬ 
lenging  aspects  of  disk-scanning  confocals  is  the  low  light  budget 


for  fluorescent  excitation.  The  Yokogawa  system  avoids  this  at 
the  cost  of  laser  operation  (limited  wavelengths,  high  cost,  high 
power/cooling,  and  large  footprint).  Two  technological  develop¬ 
ments  may  mitigate  this  problem.  Small,  efficient  solid-state 
lasers  will  doubtless  become  less  expensive  and  even  more  con- 


FIGURE  10.14.  3D/4D  visualization  of  membrane  trafficking  with  a  disk-scanning  confocal.  (Left)  4D  movie  showing  clathrin-GFP  dynamics  in  PtK2  cells 
imaged  on  an  Ultraview  PerkinElmer  Yokogawa  spinning  disk  system  using  a  60  X  1.2NA  water  objective,  Hamamatsu  Orca  ER  CCD.  Stacks  of  21  --slices 
were  collected  at  200 nm  spacing,  with  exposure  times  of  lOOms/slice  or  <2.5  s/stack.  3D  stacks  were  acquired  at  30s  intervals.  Some  clathrin  endosomal  struc¬ 
tures  that  changed  over  time  are  highlighted  in  different  focal  planes  by  colored  circles.  (Right)  Living  PtK2  cells  were  labeled  with  clathrin-GFP  (green)  and 
transferrin  (red)  to  label  endosomes.  A  region  of  interest  (-500  X  500  pixels  X  10  slices),  corresponding  to  1  cell  was  very  rapidly  acquired  in  4D  at  1  s/stack 
and  repeated  over  time  (photobleaching  occurred  after  -50  stacks).  (Images  from  R.  Zoncu  and  D.  Toomre,  Yale  University  Medical  School.) 
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venient.28  High-intensity  light-emitting  diodes  (LED)  are  becom¬ 
ing  brighter  and  less  expensive  (see  Chapter  6,  this  volume)  and 
their  lifespan  is  up  to  100,000  h.  In  addition,  they  can  be  pulsed 
very  rapidly,  an  ability  that  may  make  them  useful  for  FLIM. 

At  the  other  end  of  the  system,  better  detectors,  such  as 
microlens-equipped,  interline-transfer  EM-CCDs,  would  permit 
better  coupling  to  present  disk  scanners.  If  interline-transfer  chips 
turn  out  to  be  hard  to  develop,  another  solution  to  the  smearing 
and  moire  problems  would  be  to  turn  off  the  illumination  during 
fast  vertical  charge  transfer.  This  could  be  implemented  either  by 
sending  a  blanking  signal  to  the  acousto-optical  wavelength  selec¬ 
tor  now  used  to  select  lines  and  power  levels  in  most  laser-based 
confocals,  or  by  pulsing  the  power  to  an  LED  illumination  system 
(see  Chapter  6,  this  volume). 

As  camera  performance  improves  (bigger  chips,  faster  data 
transfer,  better  integration  with  system  control  software,  etc.),  new 
dyes  are  developed,  and  photobleaching  becomes  a  less  significant 
factor,  HUGE  amounts  of  data  will  be  generated.  Current  cameras 
can  read  a  full,  1  K  X  1  chip  at  ~35fps,  or  ~35MB/s,  or  126GB/h! 
One  may  balk,  but  4D  imaging  can  easily  require  truly  enormous 
datasets  (see  Chapters  32  and  50,  this  volume). 

Better  data  integration,  storage,  visualization,  and  analysis  are 
already  extremely  important  areas  that  can  easily  become  THE  bot¬ 
tleneck  in  the  system.  More  transparent  data  formats,  such  as  those 
being  used  in  the  open  microscopy  environment  (OME)  may  help.29 
This  rapidly  emerging  problem  will  require  companies  to  produce 
cameras  that  are  highly  integrated  with  acquisition,  visualization, 
and  analysis  software  to  form  a  package  that  all  works  together. 
Both  scientists  and  manufacturers  have  a  vested  interest  here. 

One  may  even  ask  the  outrageous  question:  Will  disk  scanners 
start  to  replace  single-beam  confocals?  Although  it  is  hard  to  tell 
what  will  narrow  the  gap  between  disk-scanning  confocals  and 
single-beam  LSMs,  it  should  be  noted  that,  because  of  their  poten¬ 
tial  for  high-speed  imaging,  CCD-equipped  disk  scanners  are 
already  being  integrated  into  high  content  screening  workstations 
for  use  in  drug  discovery  and  proteomics  (outlined  in  Chapter  46, 
this  volume). 

While  present  disk  scanners  lack  the  ability  to  scan  only  a 
region  of  interest  (ROI)  to  either  photobleach  (FRAP)  or  photoac- 
tivate  probes,  at  least  one  manufacturer  has  already  added  a  sepa¬ 
rate  high-power,  pulsed-laser  system  to  a  normal  LSM  for  this 
purpose.  There  seems  no  reason  why  this  could  not  also  be  done 
with  a  disk  scanner. 

Future  disk  scanners  would  benefit  from  better  integration  that 
allowed  the  system  to  make  pixel-matched  images  using  other  con¬ 
trast  modalities  such  as  brightfield,  differential  interference 
contrast  (DIC),  total  internal  reflection  fluorescence  microscopy 
(TIRFM),  etc.,  and  it  seems  quite  likely  that  they  might  eventually 
be  adapted  to  be  sensitive  to  other  photostates  such  as  fluorescent 
lifetime  and  polarization.  It  will  be  of  interest  to  many  to  see  if 
this  old  approach  to  confocal  imaging  surges  to  the  forefront. 

SUMMARY 

•  Disk-scanning  microscopes  can  produce  back-scattered-light 
or  fluorescent  real-time  color,  confocal  images. 

•  Disk-scanning  confocals  can  scan  100  to  lOOOx  faster  than 
single-pinhole  confocal  LSMs. 
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FIGURE  10.1 5.  Very  fast  imaging  of  calcium  with  a  Yokogawa  disk-scanning 
confocal.  (A-C)  Images  are  from  a  guinea-pig  bladder  SMC  recorded  at  37°C 
at  113 fps  (9ms  interval)  on  an  Andor  iXon  camera  with  the  512  X  512  E2V 
EM-CCD  (binned  2  X  2).  The  plots  below  each  image  show  typical  calcium 
sparks  on  a  long  timescale  (B )  and  on  an  expanded  timescale  (C).  (D,  E)  Images 
are  from  a  rabbit  urethral  smooth  muscle  cell  that  was  held  at  —60  mV  and  then 
depolarized  to  OmV  for  500ms.  This  experiment  was  recorded  at  82fps  on  the 
same  camera  and  the  montage  in  (D)  shows  false-colored  consecutive  images 
of  changes  in  Ca2+  in  response  to  the  depolarization.  (E)  shows  a  record  of 
whole  cell  fluorescence  recorded  over  many  seconds  to  demonstrate  the  sta¬ 
bility  of  the  signal  and  the  S/N.  To  facilitate  direct  comparison  with  the  raw 
video  images  often  shown,  the  images,  kindly  provided  by  Dr.  M.  A.  Holly¬ 
wood,  Smooth  Muscle  Group,  Ireland  (http://www.smoothmusclegroup.org), 
are  shown  magnified  and  unprocessed,  rather  than  smoothed. 


28  As  is  evident  in  the  Zeiss  LSM5-Live  slit  scanner  that  only  uses  -10  to 
lOOmW  small  solid-state  lasers. 

29  http://www.openmicroscopy.org/. 
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•  Quantum  efficiency  of  detection  can  be  an  order  of  magnitude 
better  than  PMT-based  LSM,  reducing  photobleaching  and 
permitting  the  acquisition  of  long  time  series. 

•  Under  optimum  conditions,  the  lateral  and  axial  resolution 
of  a  pinhole-based  disk-scanning  confocal  is  comparable  to 
confocal  LSMs.  The  z-resolution  of  a  slit-based  confocal  is 
slightly  lower. 

•  Pinhole/slit  size  must  be  matched  to  the  magnification  and  NA 
of  the  objective  lens. 

•  Both  pinhole  and  slit  disk  designs  have  advantages.  Compared 
to  the  disk  using  microlenses,  the  simpler  slit  disks  are  easier 
to  interchange  to  properly  match  the  slit  width  to  the  imaging 
conditions. 

•  In  all  cases,  the  fill  factor  of  the  disk  limits  the  light  through¬ 
put  and,  in  general,  greater  throughput  tracks  with  less  effec¬ 
tive  optical  sectioning. 

•  Low-noise,  high  QE  EM-CCD  cameras  are  critical  for  systems 
that  have  tight  light  budgets  and  require  rapid  image  acquisi¬ 
tion.  Fortunately,  they  are  now  available. 

As  emphasized  in  Chapters  23,  24,  and  25,  image  deconvo¬ 
lution  of  confocal  data  can  be  very  powerful  in  overcoming  the 
effects  of  bad  imaging  statistics  attendant  on  low  signal  levels. 
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AFTERWORD 

Other  fast-scanning  confocal  designs  are  possible.  Two  instru¬ 
ments  that  are  expected  to  come  to  the  market  soon  are  the  “VT- 
infinity”  instrument  from  Visitech  (see  Appendix  2)  and  the 
“LiveScan  Swept  Field,”  designed  by  Prairie  Technologies  (sold 
by  Nikon).  Both  confocal  heads  use  lasers  for  illumination,  mirrors 
driven  by  piezoelectric  elements  and  galvanometers  to  scan  the 
sample  and  CCD  cameras  as  efficient  detectors.  The  VT-infinity 
system  is  somewhat  reminiscent  of  the  Yokogawa  system  in  that 
it  uses  thousands  of  microlens  and  pinholes;  these  however  are 
arranged  as  stationary  arrays  and  scanning  is  achieved  with 


galvo/piezo  mirrors.  The  “Swept  Field”  system  rapidly  scans  the 
sample  (over  200  frames/sec)  with  either  a  slit  or  row  of  32  pin¬ 
holes  (different  sizes  are  available).  In  the  return  path  a  separate 
slit  or  row  of  pinholes  is  used;  precise  mechanical  and  optical 
alignment  is  likely  essential  for  successful  operation  of  this 
system.  Both  fast-scanning  systems  are  intriguing  in  design  and  it 
will  be  interesting  to  see  how  they  compare  to  spinning  disk  con- 
focals  counterparts. 
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Measuring  the  Real  Point  Spread  Function  of  High 
Numerical  Aperture  Microscope  Objective  Lenses 

Rimas  Juskaitis 


INTRODUCTION 

Testing  and  characterization  of  high-quality  lenses  have  been 
perfected  into  fine  art  with  the  advent  of  lasers,  phase-shifting 
interferometers,  charge-coupled  device  (CCD)  cameras,  and 
computers.  A  bewildering  array  of  techniques  is  described  in 
Malacara’s  classic  reference  book  on  the  subject  (Malacara,  1992). 
Several  of  these  techniques,  in  particular  the  Twyman-Green  inter¬ 
ferometer  and  the  star  test,  are  applicable  to  testing  of  microscope 
objective  lens. 

Characterizing  high  numerical  aperture  (NA)  objective  lenses 
presents  unique  challenges.  Many  of  the  standard  approaches, 
including  Twyman-Green  interferometry,  are  in  fact  comparative 
techniques.  They  require  a  reference  object  —  an  objective  or  a 
concave  reflective  surface  —  of  the  same  or  larger  numerical  aper¬ 
ture  and  of  perfect  (comparatively  speaking)  optical  quality.  This 
is  problematic.  Even  if  two  lenses  of  the  same  type  are  available, 
a  problem  still  remains  of  apportioning  the  measured  aberrations 
to  the  individual  lenses.  The  star  test,  which  is  absolute,  hits  a 
similar  problem  in  that  the  Airy  disk  produced  by  the  lens  being 
tested  is  impossible  to  observe  directly,  and  hence  it  has  to  be 
magnified  by  a  lens  with  a  similar  or  better  resolution,  that  is, 
higher  NA.  Immersion  lenses  create  further  complications.  All  tests 
described  in  this  chapter  are  free  from  these  problems.  They  are 
absolute  and  use  a  small  point  scatterer  or  a  flat  mirror  to  create  a 
reference  wavefront  against  which  the  lens  aberrations  are  checked. 
Together  with  advanced  interferometric  techniques  and  processing 
algorithms,  this  results  in  a  range  of  techniques  suitable  for  routine 
characterization  of  all  available  microscope  objective  lenses. 

A  few  words  have  to  be  said  regarding  identity  of  the  lenses 
used  throughout  this  chapter.  Although  the  principal  data  for  the 
specimens  used  in  our  test  is  given,  the  name  of  their  manufac¬ 
turers  is  not.  This  is  done  for  several  reasons,  not  least  to  avoid 
accusations  of  not  using  most  up-to-date  lenses  from  company  A 
to  make  it  look  worse  than  company  B.  As  a  university  laboratory 
we  rely  on  the  stock  of  lenses  which  have  accumulated  over  the 
years.  A  few  people  also  brought  their  own  lenses  to  test.  Not  all 
of  these  are  necessarily  the  best  in  class.  This  work  is  about 
testing  techniques  and  typical  problems  with  high  NA  lenses,  not 
about  relative  merits  of  individual  lenses  or  their  respective 
manufacturers. 

Before  describing  specific  lens  testing  techniques  it  might  be 
useful  to  repeat  here  a  few  basic  facts  about  microscope  objective 
lenses  in  general.  Modern  objective  lenses  are  invariably  designed 
for  infinite  conjugate  ratio,  that  is,  the  object  of  observation  is 
placed  in  the  front  focal  plane  and  its  image  is  formed  at  infinity. 
In  order  to  obtain  a  real  intermediate  image,  a  separate  lens,  called 


the  tube  lens,  is  used.  The  focal  length  of  this  lens  F  (which  ranges 
from  165  mm  for  Zeiss  and  180  mm  for  Olympus  to  200  mm  for 
Leica  and  Nikon)  together  with  the  magnification  of  the  objective 
M  gives  the  focal  length  of  the  objective /=  F/M. 

One  of  the  basic  postulates  of  aberration-free  lens  design  is 
that  it  has  to  obey  Abbe’s  sine  condition.  For  a  microscope  objec¬ 
tive  treated  as  a  thick  lens,  this  can  be  interpreted  as  the  fact  that 
its  front  principal  surface  is  actually  a  sphere  of  radius  / 
centered  at  the  front  focus.  Any  ray  leaving  the  focus  at  an  angle 
a  to  the  optical  axis  is  intercepted  by  this  surface  at  the  height 
cl  =  /sin  a  and  emerges  from  the  back-focal  plane  parallel  to  the 
axis  at  the  same  height,  as  shown  in  Figure  11.1.  For  immersion 
lenses  this  has  to  be  multiplied  by  the  refractive  index  of  the 
immersion  fluid  n. 

In  most  high  NA  objective  lenses,  the  back-focal  plane,  also 
called  the  pupil  plane,  is  located  inside  the  lens  and  is  not,  there¬ 
fore,  physically  accessible.  Fortunately,  lens  designers  tend  to  put 
an  aperture  stop  as  close  to  this  plane  as  possible,  which  greatly 
simplifies  the  task  of  identifying  the  pupil  plane  when  re-imaging 
it  using  an  auxiliary  lens.  Any  extra  elements,  such  as  phase  rings 
in  phase  contrast  lenses  or  variable  aperture  iris  diaphragms,  will 
also  be  located  in  the  back-focal  plane. 

The  physical  aperture  of  an  objective  lens  D  is  related  to  its 
numerical  aperture  n  sin  a  via 


Ultimately  it  is  limited  by  the  size  of  the  objective  thread.  For  a 
modern  low  magnification  high  NA  immersion  lens  with,  say,  n  sin 
a  =  1  and  M  =20,  D  can  be  as  large  as  20  mm.  This  is  one  of  the 
reasons  why  some  lens  manufacturers  (notably  Leica  and  Nikon) 
have  now  abandoned  the  former  gold  standard  of  the  Royal 
Microscopical  Society  (RMS)  thread  and  moved  to  larger  thread 
sizes. 

Every  infinity-corrected  lens  needs  a  field  lens  to  produce  the 
intermediate  image.  Field  lenses  differ  considerably  between  man¬ 
ufacturers  and  even  between  their  various  microscope  ranges. 
Strictly  speaking,  one  should  test  each  objective  lens  together  with 
its  matching  tube  lens.  We  chose  not  to  do  this  for  several  reasons. 
First,  there  is  general  movement  to  confine  all  aberration  correc¬ 
tion  to  the  objective  lens.  The  only  aberration  that  is  still  some¬ 
times  corrected  in  the  tube  lens  is  the  lateral  color.  This  does  not 
affect  the  results  of  our  measurements.  Moreover,  there  is  a  ten¬ 
dency  to  use  objective  lenses  individually  without  their  respective 
field  lenses,  especially  in  laboratory-built  scanning  systems.  All 
this  implies  the  importance  of  knowing  imaging  properties  of  high 
NA  objective  lenses  as  individual  units. 
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FIGURE  1 1 .1 .  Schematic  diagram  of  a  typical  high  NA  planapochromat  objective  lens.  Principal  surfaces,  aperture  stop,  and  marginal  ray  are  indicated. 


MEASURING  POINT  SPREAD  FUNCTION 


A  perfect  lens  transforms  a  plane  wave  front  into  a  converging 
spherical  wave.  If  they  are  not  too  dramatic,  any  deviations  from 
this  ideal  behavior  can  be  described  by  introducing  a  complex 
Pupil  Function  P( p,  0),  where  p  is  the  normalized  radial  coordi¬ 
nate  in  the  pupil  plane  and  p  is  the  azimuthal  angle  in  the  same 
plane.  Both  amplitude  and  phase  aberrations  can  be  present  in  a 
lens,  but  it  is  the  latter  that  usually  play  the  dominant  role.  The 
amplitude  aberrations  are  typically  limited  to  some  apodization 
towards  the  edge  of  the  pupil;  these  are  discussed  in  more  detail 
in  the  section  “Apodization.” 

The  optical  field  distribution  produced  by  this  (possibly  aber¬ 
rated)  converging  wave  is  termed  the  point  spread  function,  or  PSF, 
of  the  lens.  This  distribution  can  be  obtained  in  its  most  elegant 
form  if  dimensionless  optical  coordinates  in  lateral 
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2  n 

=  — n  sm 


a  sjx2  +y2 


(2) 


and  axial 


h(u,v)  =  2k  A  exp 


(5) 


This  equation  is  readily  calculated  either  numerically  or  using 
Lommel  functions.  The  resulting  intensity  distributions  are  well 
known  and  can  be  found,  for  example,  in  Born  and  Wolf  (1998). 
Not  only  that,  but  also  PSFs  subjected  to  various  aberrations  have 
been  calculated  countless  times  in  the  past  and  are  instantly  rec¬ 
ognizable  to  most  microscopists.  It  is  precisely  for  this  reason  that 
a  relatively  straightforward  measurement  of  the  PSF  can  frequently 
provide  an  instant  indication  of  what  is  wrong  with  a  particular 
objective  lens. 

Equations  4  and  5  are,  of  course,  scalar  approximations.  This 
approximation  works  remarkably  well  up  to  angular  apertures  of 
about  60°.  Even  above  these  angles  the  scalar  approximation  can 
be  safely  used  as  a  qualitative  tool.  For  those  few  lenses  that  seem 
to  be  beyond  the  scalar  approach  (and  for  the  rigorous  purists) 
there  is  always  an  option  to  use  a  well-developed  vectorial  theory 
(Richards  and  Wolf,  1959). 
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directions  are  used.  In  these  coordinates  the  intensity  distribution 
in  the  PSF  is  independent  of  the  NA  of  the  lens,  and  the  surface 
u  =  v  corresponds  to  the  edge  of  the  geometric  shadow.  The  actual 
focal  field  distribution  in  these  newly  defined  cylindrical  coordi¬ 
nates  is  given  by  (Born  and  Wolf,  1998): 
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The  exponential  term  in  front  of  the  integral  is  nothing  else  than  a 
standard  phase  factor  of  a  plane  wave  2nnzfk. 

For  the  aberration-free  case  P  =  1  and  the  integral  over  0  can 
be  calculated  analytically  to  give  2jtJ0(vp).  Equation  4  now  sim¬ 
plifies  to 


Fiber-Optic  Interferometer 

The  requirement  to  measure  both  amplitude  and  phase  of  the  PSF 
calls  for  an  interferometer-based  setup.  The  fiber-optic  interfer¬ 
ometer,  Figure  11.2,  that  was  eventually  chosen  for  the  task  has 
several  important  advantages.  It  is  an  almost  common-path  system, 
which  dramatically  improves  long-term  stability.  It  is  also  a  self¬ 
aligning  system:  light  coupled  from  the  fiber  to  the  lens  and  scat¬ 
tered  in  the  focal  region  is  coupled  back  into  the  fiber  with  the 
same  efficiency.  For  convenience,  the  whole  setup  is  built  around 
a  single-mode  fiber-optic  beam-splitter,  the  second  output  of  which 
is  index-matched  in  order  to  remove  the  unwanted  reflection.  A 
helium-neon  (He-Ne)  laser  operating  at  633  nm  is  used  as  a  light 
source.  The  whole  setup  bears  cunning  resemblance  to  a  confocal 
microscope.  In  fact,  it  is  a  confocal  microscope,  or  at  least  can  be 
used  as  such  (Wilson  et  al.,  1994).  Provided  that  light  emerging 
from  the  fiber  overfills  the  pupil  of  the  objective  lens,  the  former 
acts  as  an  effective  pinhole  ensuring  spatial  filtering  of  the 
backscattered  light  (see  Chapter  26,  this  volume).  Thus,  if  the 
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object  can  be  regarded  as  a  point  scatterer,  then  the  amplitude  of 
the  optical  signal  captured  by  the  fiber 

R  ~  h2  =  \h\2  exp[2f  arg(/t)]  (6) 

that  is,  its  magnitude  is  equal  to  the  intensity  of  the  PSF  whereas 
the  phase  is  twice  the  phase  of  the  PSF.  In  order  to  measure  both 
these  parameters,  light  reflected  back  along  the  fiber  from  the  tip 
is  used  as  a  reference  beam  r.  Wavefronts  of  both  signal  and  ref¬ 
erence  beams  are  perfectly  matched  in  a  single-mode  fiber  —  this 
is  the  beauty  of  the  fiber-optic  interferometer.  Furthermore,  the 
fiber  tip  is  dithered  to  introduce  a  time-varying  phase  shift  between 
the  interfering  beams  0(f)  =  00  cos  cor.  The  interference  signal 
reaching  the  photodetector  is  now  given  by 

I  =  \r  +  R\2  =  r2  + 1/?|~  +  2r[Re(/?)  cos(0o  cos  cor)  - 

Im(R)cos(0ocoscor)] ,  (7) 

where  r  was  assumed  to  be  real  for  simplicity.  It  is  now  a  simple 
matter  to  extract  both  R e(R)  and  Im(R)  from  this  signal  by  using 
lock-in  detection  technique.  The  signal  of  Eq.  7  is  multiplied  by 
cos  cor  and  the  result  is  low-pass  filtered  to  give 

h  =  r-/i(0o)Im(^  (8) 

whereas  synchronous  demodulation  with  cos  2cor  yields 

I2  =  rJo^o)Re(R)-  (9) 

By  appropriately  adjusting  the  modulation  amplitude  0O,  it  is  easy 
to  achieve  /i(0o)  =  72(0o)  and,  by  substituting  Eq.  6,  to 
calculate 

h  ~  s]I2  + 12  exp^arctany-  j .  (10) 

Thus,  the  goal  of  obtaining  both  the  amplitude  and  phase  of 
the  PSF  of  the  objective  lens  has  been  achieved.  Of  course,  in  order 
to  obtain  full  two-  (2D)  or  three-dimensional  (3D)  PSF  corre¬ 
sponding  scanning  of  the  object,  the  point  scatterer  is  still  required. 

Point  Spread  Function  Measurements 

In  order  to  demonstrate  the  effectiveness  of  this  method  in  detect¬ 
ing  small  amounts  of  aberrations,  it  was  tested  on  a  special  kind 
of  objective  lens.  This  60  X  1.2  NA  water-immersion  plan- 


apochromat  was  developed  for  deconvolution  applications  and 
hence  was  specifically  designed  to  have  a  well-corrected  PSF.  It 
was  also  equipped  with  a  correction  collar  to  compensate  for  cover 
glass  thicknesses  in  the  range  0.14  to  0.21mm.  One  hundred 
nanometer  colloidal  gold  beads  mounted  beneath  a  #1.5  coverslip 
of  nominal  thickness  (0.17  mm)  acted  as  point  scatterers  in  this 
case.  The  coverslip  was  in  turn  mounted  on  a  microscope  slide  and 
a  gap  between  them  was  filled  with  immersion  oil  so  as  to  elimi¬ 
nate  reflection  from  the  back  surface  of  the  coverslip.  The  size  of 
the  bead  was  carefully  chosen  experimentally  in  order  to  maximize 
the  signal  level  without  compromising  the  point-like  behavior. 
Indeed,  a  control  experiment  using  40  nm  beads  yielded  similar 
results  to  those  presented  below  but  with  a  vastly  inferior  signal- 
to-noise  ratio. 

In  principle,  this  apparatus  is  capable  of  producing  full  3D 
complex  PSF  data  sets.  It  was  found  however  that  in  most  cases 
xt-cross-sections  provided  sufficient  insight  into  the  aberration 
properties  of  the  lens  without  requiring  too  long  acquisition  times. 
Such  results  are  shown  in  Figure  11.3  for  two  settings  of  the  cor¬ 
rection  collar.  In  order  to  emphasize  the  side  lobe  structure,  the 
magnitude  of  the  PSF  is  displayed  in  decibels  with  the  peak  value 
taken  to  be  OdB.  It  can  be  seen  that  a  collar  setting  of  0.165  mm 
gives  a  near-perfect  form  to  the  PSF.  The  axial  side  lobes  are  sym¬ 
metric  with  respect  to  the  focal  plane  and  the  phase  fronts  away 
from  this  plane  quickly  assume  the  expected  spherical  shape.  On 
the  other  hand,  a  small  10%  deviation  from  the  correct  setting 
already  has  a  quite  pronounced  effect  on  the  PSF  in  the  bottom 
row  of  Figure  11.3.  The  symmetry  is  broken,  the  axial  extent  of 
the  PSF  has  increased  by  about  30%,  and  distinct  phase  singular¬ 
ities  appeared  on  the  phase  fronts.  Everything  points  towards  a 
certain  amount  of  uncompensated  spherical  aberration  being 
present  in  the  system.  It  is  interesting  to  note  that,  as  the  phase 
map  of  the  PSF  seems  to  be  more  sensitive  to  the  aberrations  than 
the  magnitude,  this  can  be  used  as  an  early  warning  indicator  of 
the  trouble.  It  also  underlines  the  importance  of  measuring  both 
the  magnitude  and  phase  of  the  PSF. 

Although  so  far  the  measured  PSF  has  been  described  in  purely 
qualitative  terms,  some  useful  quantitative  information  about  the 
objective  lens  can  also  be  extracted  from  these  data.  One  parame¬ 
ter  that  can  be  readily  verified  is  the  objective’s  NA.  Axial  extent 
of  the  PSF  is  more  sensitive  to  the  NA  than  its  lateral  shape.  Using 
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FIGURE  11.2.  Schematic  diagram  of  the  fiber-optic  interferometer-based  setup  for  measuring  objective  PSFs. 
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FIGURE  11.3.  The  amplitude  and  phase  of  the 
effective  PSF  for  60  X  1 .2  NA  water-immersion  lens 
with  correction  collar.  Results  for  two  different  collar 
settings  are  shown.  Image  size  in  both  x  (horizontal) 
and  z  (vertical)  are  5  pm. 


the  axial  section  of  the  PSF  is  therefore  the  preferred  method  to 
determine  the  NA.  Besides,  the  interference  fringes  present  in  the 
z-scan  provide  a  natural  calibration  scale  for  the  distance  in  z. 
The  actual  measurement  was  obtained  by  finding  the  best  fit  to  the 
curve  in  Figure  1 1 .4.  A  somewhat  surprising  result  of  this  exercise 


z,  pm 

FIGURE  11.4.  Measured  (dots)  and  calculated  (line)  amplitude  axial 
responses  for  the  same  lens. 


was  that  the  best  fit  corresponded  to  NA  of  1.15,  rather  than  the 
nominal  value  of  1.2.  This  is  not  a  coincidence:  such  discrepan¬ 
cies  were  found  with  other  high  NA  objective  lenses  as  well.  The 
reason  for  this  kind  of  behavior  will  become  clear  in  the  section 
“Pupil  Function.” 

CHROMATIC  ABERRATIONS 

Chromatic  aberrations  constitute  another  class  of  aberrations  that 
can  adversely  affect  the  performance  of  any  microscopy  system. 
These  aberrations  are  notoriously  difficult  to  overcome  in  high  NA 
objective  lens  design.  The  reason,  at  least  in  part,  is  the  relative 
uniformity  of  dispersion  properties  of  common  glasses  used  in 
objective  lenses.  Ingenious  solutions  have  been  found  at  an 
expense  of  dramatic  increase  of  the  number  of  lens  elements  — 
typically  to  more  than  a  dozen  in  apochromats.  Even  then,  part  of 
the  correction  may  need  to  be  carried  out  in  the  elements  external 
to  the  objective. 

Lateral  and  axial  color,  as  they  are  called  by  lens  designers, 
are  usually  treated  as  separate  chromatic  aberrations.  The  former, 
which  manifests  itself  as  the  wavelength-dependent  magnification, 
is  easy  to  spot  in  conventional  microscopes  as  coloring  of  the  edges 
of  high-contrast  objects.  Lateral  chromatic  aberration  is  also  the 
more  difficult  of  the  two  to  correct.  Traditionally  this  has  been 
done  by  using  the  tube  lens,  or  even  the  ocular,  to  offset  the 
residual  lateral  color  of  the  lens.  Some  of  the  latest  designs  claim 
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to  have  achieved  full  compensation  within  the  objective  lens  itself 
—  claims  to  be  treated  with  caution.  The  correct  testing  procedure 
for  the  lateral  color  should  include  at  least  a  matched  tube  lens. 
The  simplest  test  would  probably  be  to  repeat  the  experiments 
described  in  the  section  “PSF  Measurements”  for  several  wave¬ 
lengths  at  the  edge  of  the  field  of  view  and  record  the  shift  of  the 
lateral  position  of  the  point  image. 

In  confocal  microscopy,  where  the  signal  is  determined  by  the 
overlap  of  the  effective  excitation  and  detection  PSFs,  the  loss  of 
register  between  them  should  lead  to  a  reduction  of  signal  towards 
the  edge  of  the  field  of  view.  It  has  to  be  said,  though,  that  in  most 
confocal  microscopes,  almost  always,  only  a  small  area  around  the 
optical  axis  is  used  for  imaging,  hence  this  apodization  is  hardly 
ever  appreciable.  Axial  color,  on  the  other  hand,  is  rarely  an  issue 
in  conventional  microscopy,  but  it  can  be  of  serious  consequence 
for  confocal  microscopy,  especially  when  large  wavelength  shifts 
are  involved,  such  as  in  multi-photon  or  second  and  third  harmonic 
microscopy.  Mismatch  in  axial  positions  of  excitation  and  detec¬ 
tion  PSFs  can  easily  lead  to  degradation  or  even  complete  loss  of 
signal,  even  in  the  center  of  the  held  of  view.  Below  we  describe 
a  test  setup  which  uses  this  sensitivity  of  the  confocal  system  to 
characterize  axial  chromatic  aberration  of  high  NA  objective 
lenses. 


Apparatus 

Ideally,  one  could  conceive  an  apparatus  similar  to  that  in  Figure 
11.2,  whereby  the  laser  is  substituted  with  a  broadband  light 
source.  One  problem  is  immediately  obvious:  it  is  very  difficult  to 
couple  any  significant  amount  of  power  into  a  single-mode  fiber 
from  a  broadband  light  source,  such  as  an  arc  lamp.  Using  multi¬ 
ple  lasers  provides  only  a  partial  (and  expensive)  solution.  Instead, 
it  was  decided  to  substitute  the  point  scatterer  with  a  plane  reflec¬ 
tor.  Scanning  the  reflector  axially  produces  the  confocal  signal 
(Wilson  and  Sheppard,  1984): 


sinw/2 

m/2 


(11) 


The  maximum  signal  is  detected  when  the  plane  reflector  lies  in 
the  focal  plane.  This  will  change  with  the  wavelength  if  chromatic 
aberration  is  present.  Using  a  mirror  instead  of  a  bead  has  another 
advantage:  the  resulting  signal  is  one-dimensional,  function  of  u 
only,  and  hence  a  dispersive  element  can  be  used  to  directly  obtain 
2D  spectral  axial  responses  without  the  necessity  of  acquiring 
multiple  datasets  at  different  wavelengths. 

The  resulting  apparatus,  depicted  in  Figure  11.5  and  described 
in  more  detail  in  Juskaitis  and  Wilson  (1999),  is  again  based 
around  a  fiber-optic  confocal  microscope  setup,  but  the  interfer¬ 
ometer  part  is  now  discarded.  Instead,  a  monochromator  prism 
made  of  SF4  glass  is  introduced  to  provide  the  spectral  spread  in 
the  horizontal  direction  (i.e.,  in  the  image  plane).  Scanning  in  the 
vertical  direction  was  introduced  by  a  galvo-mirror  moving  in  syn¬ 
chronism  with  the  mirror  in  the  focal  region  of  the  objective  lens. 
The  resulting  2D  information  is  captured  by  a  cooled  16-bit  slow- 
scan  CCD  camera.  A  small-arc  Xe  lamp  is  used  as  a  light  source 
providing  approximately  0.2  |iW  of  broadband  visible  radiation  in 
a  single-mode  fiber.  This  is  sufficient  to  produce  a  spectral  snap¬ 
shot  of  a  lens  in  about  10  s. 

Axial  Shift 

Typical  results  obtained  by  the  chromatic  aberration  measurement 
apparatus  are  shown  in  Figure  11.6.  Because  the  raw  images  are 
not  necessarily  linear  either  in  z  or  X,  a  form  of  calibration  proce¬ 
dure  in  both  coordinates  is  required.  To  achieve  this,  the  arc  lamp 
light  source  was  temporarily  replaced  by  a  He-Ne  and  a  multi-line 
Ar+  lasers.  This  gave  enough  laser  lines  to  perform  linearization  in 
A,.  As  a  bonus,  coherent  laser  radiation  also  gave  rise  to  inter¬ 
ference  fringes  in  the  axial  response  with  the  reflection  from  the 
fiber  tip  acting  as  a  reference  beam,  just  as  in  the  setup  shown  in 
Figure  1 1 .2.  When  a  usual  high  NA  objective  lens  was  substituted 
with  a  low  NA  version,  these  fringes  covered  the  whole  range  of 
the  z-scan  and  could  be  readily  used  to  calibrate  the  axial  coordi¬ 
nate.  The  traces  shown  in  Figure  11.6  have  been  normalized  to 
unity  at  each  individual  wavelength.  The  presence  of  longitudinal 
chromatic  aberration  is  clearly  seen  in  both  plots.  Their  shapes  are 


galvo-mirror  mirror 


FIGURE  11.5.  Experimental  setup  for  measuring  axial  chromatic  aberration  of  objective  lenses. 
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FIGURE  1 1 .6.  Experimental  results  from  (A)  32  X  0.5  NA  plan-achromat  objective,  displayed  as  3D  plot,  and  (B)  50  X  0.8  NA  plan-achromat  in  pseudo-color 
representation.  Note  the  change  in  z-scale. 


characteristic  for  achromats  in  which  the  longitudinal  color  is  cor¬ 
rected  for  two  wavelengths  only.  It  is  interesting  also  to  note  the 
change  of  the  shape  of  the  axial  response  with  the  wavelength.  This 
is  noticeable  for  a  32  x  0.5  NA  plan-achromat  but  it  becomes  par¬ 
ticularly  dramatic  for  a  50  x  0.8  NA  plan-achromat  [Fig.  11.6(b)]. 
Clearly  the  latter  lens  suffers  from  severe  spherical  aberration  at 
wavelengths  below  550  nm,  which  results  in  multiple  secondary 
maxima  at  one  side  of  the  main  peak  of  the  axial  response. 

An  interesting  problem  is  posed  by  the  tube  lens  in  Figure  1 1.5. 
This  lens  may  well  contribute  to  the  longitudinal  chromatic  aber¬ 
ration  of  the  microscope  as  a  whole  and,  therefore,  it  is  desirable 
to  use  here  a  proper  microscope  tube  lens  matched  to  the  objec¬ 
tive.  In  fact  it  transpired  that  in  some  cases  the  tube  lens  exhibited 
significantly  larger  axial  color  aberration  than  the  objective  itself. 
This  is  hardly  surprising:  a  typical  tube  lens  is  a  simple  triplet  and, 
sometimes,  even  a  singlet.  Clearly,  it  is  impossible  to  achieve  any 
sophisticated  color  correction  in  such  an  element  taken  separately, 
therefore  the  objective  lens  would  have  to  be  designed  to  take  this 
imperfection  into  account.  Further  information  on  this  can  be 
found  in  Chapter  7. 


wavelength,  nm 


In  this  experiment,  however,  the  main  task  was  to  evaluate  the 
properties  of  the  objective  lens  itself.  A  different  approach  was 
therefore  adopted.  The  same  achromatic  doublet  (Melles  Griot  01 
LAO  079)  collimating  lens  was  used  with  all  objective  lenses. 
Because  the  chromatic  aberrations  of  this  lens  are  well  documented 
in  the  company  literature,  these  effects  could  be  easily  removed 
from  the  final  results  presented  in  Figure  11.7.  Figure  11.7(B)  pre¬ 
sents  the  same  data  but  in  a  form  more  suited  to  confocal 
microscopy,  whereby  the  chromatic  shift  is  now  expressed  in 
optical  units  as  defined  in  Eq.  3.  The  half  width  of  the  axial 
response  to  a  plane  mirror  is  then  given  by  2.78  optical  units  at  all 
wavelengths  and  for  all  NAs.  This  region  is  also  shown  in  the 
figure.  The  zero  in  the  axial  shift  is  arbitrarily  set  to  correspond  to 
X  =  546  nm  for  all  the  objectives  tested. 

As  could  be  expected  these  results  show  improvement  in  per¬ 
formance  of  apochromats  over  achromats.  They  also  show  that 
none  of  the  tested  objectives  (and  this  includes  many  more  not 
shown  in  Figure  11.7  for  fear  of  congestion)  could  meet  the 
requirement  of  having  a  spectrally  flat  —  to  within  the  depth  of 
field  —  axial  behavior  over  the  entire  visible  range.  This  was  only 


FIGURE  1 1 .7.  Wavelength  dependence  of  the  axial  response  maxima  represented  in  physical  (A)  and  optical  (B)  axial  coordinates.  The  four  traces  correspond 
to:  32  X  0.5  NA  plan-achromat  ( dotted  line),  50  X  0.8  NA  plan-achromat  ( long  dashes),  100  X  1.4  NA  plan-apochromat  (short  dashes),  and  the  same  lens  stopped 
down  to  0.7  NA  (solid  line). 
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possible  to  achieve  by  stopping  down  a  1 .4  NA  apochromat  using 
a  built-in  aperture  stop  —  the  trick  to  be  repeated  several  times 
again  before  this  chapter  expires. 


PUPIL  FUNCTION 


One  of  the  mirrors  in  the  reference  arm  of  the  interferometer 
was  mounted  on  a  piezoelectric  drive  and  moved  in  synchronism 
with  the  CCD  frame  rate  to  produce  successive  interferograms  of 
the  pupil  plane  shifted  by  2tc/3  rad 


r  +  P(  P,  6)exp^i 


/  =  0,1,2.  (12) 


Pupil  function  is  the  distribution  of  the  phase  and  amplitude  across 
the  pupil  plane  when  the  lens  is  illuminated  by  a  perfect  spherical 
wave  from  the  object  side.  It  is  related  in  scalar  approximation  to 
the  PSF  via  a  Fourier-like  relationship  (Eq.  4).  It  would  appear, 
therefore,  that  they  both  carry  the  same  information  and  therefore 
the  choice  between  them  should  be  a  simple  matter  of  convenience. 
Reality  is  a  bit  more  complicated  than  that.  Calculating  the  pupil 
function  from  the  PSF  is  an  ill-posed  problem  and  therefore  very 
sensitive  to  noise.  Measurements  of  the  pupil  function  provide 
direct  and  quantitative  information  about  the  aberrations  of  the  lens 
—  information  that  can  only  be  inferred  from  the  PSF  measure¬ 
ments. 

The  trouble  with  mapping  the  pupil  function  is  that  a  source 
of  a  perfect  spherical  wave  is  required.  Such  a  thing  does  not  exist 
but,  fortunately,  the  dipole  radiation  approximates  such  a  wave 
rather  well,  at  least  as  far  as  phase  is  concerned.  The  approach 
described  in  this  section  is  based  on  using  small  side-illuminated 
scatterers  as  sources  of  spherical  waves.  The  actual  pupil  func¬ 
tion  measurement  is  then  performed  in  a  phase-shifting  Mach- 
Zender  interferometer  in  a  rather  traditional  fashion. 


Using  these  three  measurements  the  phase  component  of  the  pupil 
function  was  then  calculated  as 

arg[P(p,  0)]  =  arctan-^y— (13) 
A  + 12  ziQ 

The  lens,  together  with  the  prism,  was  mounted  on  a  pivoting 
stage  which  could  rotate  the  whole  assembly  around  the  axis 
aligned  to  an  approximate  location  of  the  pupil  plane.  Thus,  the 
off-axis  as  well  as  on-axis  measurements  of  the  pupil  function 
could  be  obtained.  A  set  of  such  measurements  is  presented  in 
Figure  11.9,  which  clearly  demonstrates  how  the  performance  of 
the  lens  degrades  towards  the  edge  of  the  field  of  view.  Not  only 
appreciable  astigmatism  and  coma  are  introduced,  but  also 
vignetting  becomes  apparent.  The  presence  of  vignetting  would  be 
very  difficult  to  deduce  from  direct  measurements  of  the  PSFs 
shown  in  Figure  11.10,  as  it  could  be  easily  mistaken  for  astig¬ 
matism.  Not  that  such  vignetting  is  necessarily  an  indication  that 
something  is  wrong  with  the  lens;  it  may  well  be  deliberately  intro¬ 
duced  there  by  the  lens  designer  to  block  off  the  most  aberrated 
part  of  the  pupil. 


Phase-Shifting  Interferometry 

The  experimental  setup  depicted  in  Figure  11.8  comprised  a  fre¬ 
quency  doubled  Nd:YAG  laser  which  illuminated  a  collection  of 
20  nm  diameter  gold  beads  deposited  on  the  surface  of  a  high 
refractive  index  glass  prism  acting  as  dipole  scatterers  (Juskaitis 
et  al.,  1999).  Because  the  laser  light  suffers  total  internal  reflec¬ 
tion  at  the  surface  of  the  prism,  no  direct  illumination  can  enter 
the  objective  lens.  The  gold  scatterers  convert  the  evanescent  field 
into  the  radiating  spherical  waves  that  were  collected  by  the  lens 
and  converted  into  plane  waves.  These  waves  were  then  superim¬ 
posed  on  a  collimated  reference  wave.  A  4-f  lens  system  was  then 
used  to  image  the  pupil  plane  of  the  lens  onto  a  CCD  camera.  A 
pinhole  in  the  middle  of  this  projection  system  served  to  select  a 
signal  from  a  single  scatterer.  The  size  of  this  pinhole  had  to  be 
carefully  controlled  so  as  not  to  introduce  artifacts  and  degrade 
resolution  in  the  image  of  the  pupil  function.  A  second  CCD 
camera  was  employed  to  measure  the  PSF  at  the  same  time. 


Zernike  Polynomial  Fit 

Traditionally,  the  phase  aberrations  of  the  pupil  functions  are 
described  quantitatively  by  expanding  them  using  a  Zernike  circle 
polynomial  set: 

arg[/>(p,  6)]  =  j^fl;Z;( p,  6),  (14) 

i=i 

where  a,  are  aberration  coefficients  for  corresponding  Zernike 
polynomials  Z,(p,  9).  Significant  variations  between  different 
modifications  of  Zernike  polynomials  exist.  In  this  work  a  set  from 
Mahajan  (1994)  was  used.  The  first  22  members  of  this  set  are 
listed  in  Table  11.1  together  with  their  common  names.  This  list 
can  be  further  extended.  In  practice,  however,  expansion  beyond 
the  second-order  spherical  aberration  is  not  very  reliable  due 
to  experimental  errors  and  noise  in  the  measured  image  of 
P(p,  6). 


lens 


FIGURE  1 1 .8.  Phase-shifting  Mach-Zehnder  interferometer  used  for  the  pupil  function  measurements.  Laser  is  frequency-doubled  Nd:YAG  laser. 
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FIGURE  1 1.9.  Phase  distributions  in  the  pupil  plane  of  a  representative  range  of  objective  lenses.  Left  to  right:  10  x  0.5  NA,  25  x  0.8  NA  multi-immersion, 
40  x  1.2  NA  water-immersion,  and  63  x  1.4  NA  oil-immersion  objectives.  Performance  at  the  different  positions  in  the  field  of  view  as  measured  in  the  inter¬ 
mediate  image  plane  is  shown.  Regular  phase  tilt  added  to  facilitate  visualization. 


FIGURE  1 1 .10.  Point  spread  functions  for  the  same  lenses  as  in  Figure  11.9. 


Measuring  the  Real  Point  Spread  Function  of  High  Numerical  Aperture  Microscope  Objective  Lenses  •  Chapter  11  247 


TABLE  11.1.  Orthonormal  Zernike  Circle  Polynomials 


i 

n 

m 

Ur,  6) 

Aberration  Term 

i 

i 

0 

1 

Piston 

2 

i 

1 

2rcos0 

Tilt 

3 

i 

-1 

2rsin0 

Tilt 

4 

2 

0 

V3(2  r2-  1) 

Defocus 

5 

2 

2 

V6rcos20 

Astigmatism 

6 

2 

-2 

V6rsin29 

Astigmatism 

7 

3 

1 

2V2(3P  -  2r)cos9 

Coma 

8 

3 

-1 

2V2(3r3  —  2r)sin0 

Coma 

9 

3 

3 

2V2;Jcos30 

Trefoil 

10 

3 

-3 

2V2psin30 

Trefoil 

11 

4 

0 

V5(6r4  —  6 r2  +1) 

Primary  spherical 

12 

4 

2 

VT0(4r4  -  3r)  cos  20 

13 

4 

-2 

VTO(4r4-3r)sin20 

14 

4 

4 

Vl0r4cos49 

15 

4 

-4 

VlOr4  sin  40 

16 

5 

1 

2a/3(10;-5-  12F  +  3r)cos0 

17 

5 

-1 

2V3(10P-  12r'  +  3r)sin0 

18 

5 

3 

2V3(5r5-4r3)cos30 

19 

5 

-3 

2V3(5P-4r3)sin30 

20 

5 

5 

2V3pcos59 

21 

5 

-5 

2V3r5sin50 

22 

6 

0 

V7(20r6  -  30r4  +  12 r2  -  1) 

Secondary  spherical 

The  determination  of  the  expansion  coefficients  a,  should,  in 
principle,  be  a  simple  procedure,  given  that  the  Zernike  polyno¬ 
mials  are  orthonormal.  Multiplying  the  measured  pupil  function  by 
a  selected  polynomial  and  integrating  over  the  whole  pupil  area 
should  directly  yield  the  corresponding  aberration  coefficient. 
The  real  life  is  a  bit  more  complicated,  especially  when  process¬ 
ing  the  off-axis  data,  such  as  shown  in  Figure  1 1 .9.  One  obstacle 
is  vignetting:  the  standard  Zernike  set  is  no  longer  orthonormal 
over  a  non-circular  pupil.  Even  without  vignetting,  the  2n  phase 
ambiguity  poses  a  problem.  Before  the  expansion  procedure  can 
be  applied,  the  phase  of  the  pupil  function  has  to  be  unwrapped  — 
not  necessarily  a  trivial  procedure. 

An  entirely  different  expansion  technique  was  developed  to 
overcome  these  difficulties.  This  technique  is  based  on  a  simulated, 
iterative  wavefront  correction  routine,  originally  conceived  to  be 
used  in  adaptive  optics  applications  together  with  a  diffractive 
optics  wavefront  sensor  (Neil  et  al.,  2000).  The  essence  of  the 
method  is  that  small  simulated  amounts  of  individual  Zernike  aber¬ 
rations  are  applied  in  turns  to  the  measured  pupil  function.  After 
each  variation,  the  in-focus  PSF  is  calculated  and  the  whole 
process  iteratively  repeated  until  the  Strehl  ratio  is  maximized.  The 
final  magnitudes  of  the  Zernike  terms  are  then  taken  to  be  (with 
opposite  signs)  the  values  of  the  Zernike  expansion  coefficients  of 
the  experimentally  measured,  aberrated  pupil  function.  This  pro¬ 
cedure  is  reasonably  fast  and  sufficiently  robust,  provided  that  the 
initial  circular  aperture  can  still  be  restored  from  the  vignetted 
pupil. 

The  power  of  this  technique  is  demonstrated  in  Figure  11.11, 
where  a  40  x  1.2  NA  water-immersion  lens  was  investigated  at 
three  different  settings  of  the  correction  collar.  As  expected,  adjust¬ 
ing  the  collar  mainly  changes  the  primary  and  secondary  spheri¬ 
cal  aberration  terms.  Variations  in  the  other  terms  are  negligible. 
The  optimum  compensation  is  achieved  close  to  d  =  0.15  mm 
setting,  where  small  amounts  of  both  aberrations  with  opposite 
signs  cancel  each  other.  The  usefulness  of  the  Zernike  expansion 
is  further  demonstrated  by  the  fact  that  the  main  residual  term  in 


this  case  was  the  defocus,  which,  although  not  an  aberration  itself, 
could  be  easily  mistaken  for  a  spherical  aberration  upon  visual 
inspection  of  the  interference  pattern. 


Restoration  of  a  3D  Point  Spread  Function 

Nowhere  is  the  power  of  the  pupil  function  approach  to  the  objec¬ 
tive  lens  characterization  more  apparent  than  in  cases  when  the  full 
3D  shape  of  the  PSF  needs  to  be  determined.  Such  need  may  arise, 
for  example,  when  using  deconvolution  techniques  to  process 
images  obtained  with  a  confocal  microscope. 

As  is  clear  from  Eq.  4,  it  is  not  only  possible  to  calculate  an 
in-focus  PSF  from  a  measured  pupil  function,  but  the  same  can  be 
done  for  any  amount  of  defocus  by  choosing  an  appropriate  value 
for  the  axial  coordinate  u.  Repeating  the  process  at  regular  steps 
in  u  yields  a  set  of  through-focus  slices  of  the  PSF.  These  can  then 
be  used  to  construct  a  3D  image  of  the  PSF  in  much  the  same 
manner  that  3D  images  are  obtained  in  a  confocal  microscope. 
Compared  to  the  direct  measurement  using  a  point  scatterer, 
advantages  of  this  approach  are  clear.  A  single  measurement  of  the 
pupil  function  is  sufficient  and  no  scanning  of  the  bead  in  three 
dimensions  is  required.  Consequently,  exposures  per  image  pixel 
can  be  much  longer.  As  a  result,  this  method  provides  much 
improved  signal-to-noise  ratio  in  the  final  rendering  of  the  PSF, 
allowing  even  the  faintest  sidelobes  to  be  examined. 

Obviously,  presenting  a  complete  3D  image  on  a  flat  page  is 
always  going  to  be  a  problem  but,  as  Figure  11.12  shows,  even  just 
two  meridional  cross-sections  of  a  3D  PSF  provide  infinitely  more 
information  than  a  plain  2D  in-focus  section  of  the  same  PSF  at 
the  bottom  of  Figure  11.9.  Thus,  for  example,  the  yz-section  clearly 
shows  that  the  dominant  aberration  for  this  particular  off-axis  posi¬ 
tion  is  coma.  Comparing  the  two  sections  it  is  also  possible  to  note 
different  convergence  angles  for  the  wavefronts  in  two  directions 
—  a  direct  consequence  of  vignetting. 


d  =  0.14  mm  d  =  0.15  mm  d=  0.16  mm 
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FIGURE  11.11.  Variations  in  wavefront  aberration  function  expressed  via 
Zernike  modes  when  correction  collar  of  a  water-immersion  lens  is  adjusted. 
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FIGURE  11.12.  Three-dimensional  PSF  restored  from 
pupil  function  data,  shown  here  via  two  meridional 
sections. 
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Empty  Aperture 

Testing  objective  lenses  with  highest  NAs  ( 1 .4  for  oil,  1 .2  for  water 
immersion),  one  peculiar  aberration  pattern  is  often  encountered. 
As  shown  in  Figure  11.13,  the  lens  is  well  corrected  up  to  90%  to 
95%  of  the  aperture  radius,  but  after  that  we  see  a  runaway  phase 
variation  right  to  the  edge.  Speaking  in  Zernike  terms,  residual 
spherical  aberration  components  of  very  high  order  are  observed. 
Because  of  this  high  order,  it  appears  unlikely  that  the  aberrations 
are  caused  by  improper  immersion  fluid  or  some  other  trivial 
reason.  These  would  manifest  themselves  via  low-order  spherical 
as  well.  More  realistically,  this  is  a  design  flaw  of  the  lens. 

The  portion  of  the  lens  affected  by  this  feature  varies  from  few 
to  about  10%.  For  the  lens  in  Figure  11.13,  the  line  delimiting  the 
effective  aperture  was  somewhat  arbitrarily  drawn  at  NA  =  1.3. 
What  is  undeniable  is  that  the  effect  is  not  negligible.  In  all  like¬ 
lihood,  this  form  of  aberration  is  the  reason  for  the  somewhat  mys¬ 
terious  phenomenon  when  a  high  NA  lens  exhibits  a  PSF  that  is 
perfect  in  all  respects  except  for  an  apparently  reduced  NA.  This 
was  the  case  in  “PSF  Measurements”  and  also  described  by  other 
researchers  (Hell  et  al.,  1995). 

It  is  quite  clear  that  competitive  pressures  push  the  lens  design¬ 
ers  towards  the  boundary  (and  sometimes  beyond  the  boundary) 
of  the  technical  possibilities  of  the  day.  A  few  years  ago  no  micro¬ 
scope  lens  manufacturer  could  be  seen  without  a  1.4  NA  oil- 
immersion  lens  when  the  Joneses  next  door  were  making  one.  The 


FIGURE  11.13.  Pupil  function  of  63  X  1.4  NA  oil  immersion  lens  with  both 
nominal  (outer  ring)  and  effective  working  (inner  ring)  apertures  indicated. 


plot  is  likely  to  be  repeated  with  the  newly  emerging  1.45  NA 
lenses.  It  is  also  true  that  a  hefty  premium  is  charged  by  the  man¬ 
ufacturers  for  the  last  few  tenths  in  the  NA.  It  is  quite  possible  that 
in  many  cases  this  is  a  very  expensive  empty  aperture,  which, 
although  physically  present,  does  not  contribute  to  the  resolving 
power  of  the  lens. 

This  discussion  may  seem  to  be  slightly  off  the  point  because 
many  users  buy  high  NA  lenses  not  because  of  their  ultimate  res¬ 
olution,  but  because  of  their  light  gathering  efficiency  in  fluores¬ 
cence  microscopy.  This  property  is  approximately  proportional  to 
NA 2  and  therefore  high  NA  lenses  produce  much  brighter,  higher 
contrast  images.  At  first  glance  it  may  seem  that  the  aberrated  edge 
of  the  pupil  will  not  affect  this  efficiency  and  hence  buying  a  high 
NA  lens,  however  aberrated,  still  makes  sense.  Unfortunately,  this 
is  not  true.  Because  the  phase  variation  at  the  edge  is  so  rapid,  the 
photons  passing  through  it  reach  the  image  plane  very  far  from 
the  optical  axis.  They  do  not  contribute  to  the  main  peak  of  the 
diffraction  spot,  instead  they  form  distant  sidelobes.  In  terms  of 
real  life  images,  it  means  that  the  brightness  of  the  background, 
and  not  the  image  itself,  is  increased.  Paradoxically,  blocking  the 
outermost  portion  of  the  pupil  would  in  this  case  improve  the 
image  contrast! 

The  last  statement  may  well  be  generalized  in  the  following 
way:  in  the  end,  the  only  sure  way  of  obtaining  a  near-perfect  high 
NA  objective  lens  is  to  acquire  one  with  larger-than-required 
nominal  NA  and  then  stop  it  down.  Incidentally,  in  certain  situa¬ 
tions  this  may  be  happening  even  without  our  deliberate  interven¬ 
tion.  Consider  using  an  oil  immersion  lens  on  a  water-based 
sample:  no  light  at  NAM. 33  can  penetrate  the  sample  anyway  and 
hence  the  outer  aperture  of  the  lens  is  effectively  blocked.  Not  that 
such  use  should  be  ever  considered  unless  in  dire  need:  see  Chapter 
20  for  a  graphic  description  of  the  Bad  Things  that  will  happen. 

MISCELLANEA 

In  this  section  a  few  more  results  obtained  with  the  pupil  function 
evaluation  apparatus  are  presented.  These  need  not  necessarily  be 
of  prime  concern  to  most  microscope  users  but  might  be  of  inter¬ 
est  to  connoisseurs  and,  indeed,  could  provide  further  insight  into 
how  modern  objective  lenses  work. 

Temperature  Variations 

Many  microscopists  should  recall  seeing  23  °C  on  a  bottle  of 
immersion  oil  as  a  temperature  at  which  the  refractive  index  is 
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temperature,  °C 

FIGURE  11.14.  Temperature  variation  of  primary  (red)  and  secondary 
(purple)  spherical  aberrations  as  well  as  magnitudes  of  astigmatism  (blue), 
coma  (orange),  and  trefoil  (green).  The  latter  three  were  defined  as  a  = 
a  =  Jaf+HJ,  where  a1,  and  a]  are  the  sine  and  cosine  components  of  the  cor¬ 
responding  aberrations. 


specified,  typically  n  =  1.518  at  X  =  546  nm.  But  how  important  is 
this  standard  laboratory  temperature  to  the  performance  of  high 
NA  lenses?  In  order  to  answer  this  question,  the  pupil  function  of 
a  100  x  1.4  NA  oil-immersion  lens  was  measured  at  a  range  of 
temperatures  and  the  results  were  processed  to  obtain  the  variation 
of  the  primary  aberration  coefficients  with  the  temperature. 

The  major  factor  in  the  degradation  of  the  imaging  qualities 
of  the  immersion  lenses  with  temperature  is  the  variation  of  the 
refractive  index  of  the  immersion  oil,  usually  at  the  level  of  dnldT 
=  3  -  4  x  1CT4.  The  effect  of  this  change  is  similar  to  that  of  intro¬ 
ducing  a  layer  of  a  refractive-index-mismatched  material  between 
the  lens  and  the  sample.  The  resulting  aberrations  are  well  under¬ 
stood,  their  exhaustive  analysis  can  be  found  in  Torok  and  col¬ 
leagues  (1995).  In  short,  spherical  aberrations  of  various  orders 
will  be  generated;  the  relative  weight  of  higher-order  terms  rises 
dramatically  with  the  NA  of  the  lens.  This  is  corroborated  by  the 
experimental  data  in  Figure  11.14  which  show  steady,  almost  linear 
variation  in  both  the  primary  and  secondary  spherical  aberration 
terms  with  temperature.  Less  predictably,  the  same  plot  also  reg¬ 
isters  significant  variations  in  the  other  first-order  aberrations: 
coma,  astigmatism,  and  trefoil.  Because  of  their  asymmetry,  these 
aberrations  cannot  be  explained  by  the  oil  refractive  index 
changes.  They  are  probably  caused  by  small  irregular  movements 
of  individual  elements  within  the  objective  lens  itself. 


Strictly  speaking,  aberrations  caused  by  refractive  index 
changes  in  the  immersion  fluid  should  not  be  regarded  as  lens  aber¬ 
rations.  In  practice,  however,  the  lenses  are  designed  for  a  partic¬ 
ular  set  of  layers  of  well-defined  thicknesses  and  refractive  indexes 
between  the  front  of  the  lens  and  the  specimen.  Any  change  in 
these  parameters  upsets  the  fine  optical  phase  balance  in  the  system 
and  results  in  aberrated  PSF.  This  might  be  an  argument  why  it 
makes  sense  to  treat  the  immersion  medium  as  being  a  part  of  the 
objective  lens.  The  temperature  effect  depends  dramatically  on  the 
type  of  the  immersion  fluid  used.  Water  with  dnldT  =8x  10-5  is 
far  less  sensitive;  dry  lenses,  of  course,  are  not  affected  at  all. 
Long-working-distance  lenses  will  be  at  a  disadvantage  too  due  to 
longer  optical  paths  in  the  immersion  fluid. 

Closer  analysis  of  Figure  11.14  reveals  that  the  aberrations  are 
indeed  minimized  around  23  °C,  where  the  combined  primary  and 
secondary  spherical  aberrations  are  close  to  their  minimum.  A 
small  but  noticeable  hysteresis  effect  was  also  noted  when,  after  a 
few  temperature  cycles,  the  aberration  coefficients  failed  to  return 
to  their  low  initial  values.  It  is  tempting  to  connect  this  effect  to 
the  fact  that  the  imaging  properties  of  even  the  best  of  lenses 
always  deteriorate  with  age  —  although  accidental  straining  during 
experiments  is  still  likely  to  remain  the  prevailing  factor. 

Polarization  Effects 

Polarization  effects  encountered  when  imaging  a  dipole  with  a 
high  NA  lens  have  been  covered  elsewhere  (Wilson  et  al,,  1997). 
For  the  purposes  of  this  investigation  they  are  interesting  inasmuch 
as  the  imperfections  of  the  lens  contribute  to  them.  When  an  image 
of  a  small  scatterer  is  viewed  between  the  crossed  polarizers,  a 
characteristic  “clover  leaf’  pattern  emerges.  An  image  of  the  pupil 
plane.  Figure  11.15,  is  particularly  telling.  It  shows  that  only  the 
rays  that  travel  very  close  to  the  edge  of  the  pupil  pass  through  the 
polarizer. 

This  happens  because  the  polarization  of  light  in  these  four 
segments  is  slightly  rotated  from  its  original  x-direction.  The 
reason  for  this  rotation  is  2-fold.  First  of  all  the  radiation  of  a  dipole 
is  linearly  polarized  in  the  meridional  plane,  which  can  only  be 
approximated  by  a  uniformly  polarized  plane  wave  for  small 
angles  a.  When  this  spherical  wavefront  is  straightened  by  the  lens 
and  projected  into  the  pupil  plane,  only  light  propagating  in  xz  and 
yz  possesses  perfect  ^-polarization,  the  remainder  is  rotated  to 
some  degree.  The  degree  of  rotation  increases  dramatically  with 
higher  aperture  angles.  This  effect  is  fundamentally  geometrical; 
its  close  cousin  is  a  well-known  problem  of  how  to  comb  a  hairy 
ball. 

The  second  reason  for  the  polarization  rotation  is  more  prosaic: 
Fresnel  losses.  These  tend  to  be  higher  for  s  than  for  p  polarized 


FIGURE  11.15.  Polarization  and  apodization  effects  in  a  high  NA  lens.  An  image  of  a  dipole  scatterer  viewed  at  the  pupil  plane  of  a  100  X  1.4  NA  lens  with 
crossed  (left)  and  parallel  (center)  analyzers.  Principal  cross-sections,  together  with  the  theoretical  predictions  according  to  Eq.  15  (red  line),  are  also  shown. 
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light.  For  a  beam  travelling  in  a  plane  bisecting  the  xz-  and  yz- 
planes,  the  overall  tendency  would  be  to  rotate  the  polarization 
towards  the  radial  direction.  Hence,  this  effect  seems  to  work  in 
the  opposite  direction  to  that  caused  by  the  geometric  factor,  which 
favors  the  azimuthal  direction. 


Apodization 


So  far  the  emphasis  of  this  investigation  has  been  on  measuring 
the  phase  aberrations.  This  is  justified  by  the  fact  that  the  phase 
deviations  from  an  ideal  spherical  wavefront  have  considerably 
more  impact  on  the  shape  of  the  PSF  than  similar  imperfections  in 
amplitude.  Nevertheless,  for  completeness’  sake  it  might  be  inter¬ 
esting  now  to  have  a  closer  look  at  the  apodization  effects  occur¬ 
ring  in  high  NA  lenses.  Using  dipole  radiation  as  a  probe  offers 
unique  advantages  in  this  task.  Because  the  angular  intensity  dis¬ 
tribution  of  the  dipole  radiation  is  well  defined,  any  deviations 
from  perfect  lens  behavior  should  be  easy  to  spot. 

Let’s  assume  that  the  polarization  vector  of  a  dipole  situated 
in  the  focus  is  aligned  in  the  .r-direction.  Angular  intensity  distri¬ 
butions  in  the  xz-  and  yz-  (i.e.,  meridional  and  equatorial)  planes 
will  be  given  by,  respectively,  Ix  —  cos2  a  and  Iy  =  const.  Due  to 
purely  geometric  reasons,  these  distributions  will  change  when 
light  propagates  to  the  pupil  plane  even  if  the  lens  is  perfect.  With 
reference  to  Figure  11.11  and  to  the  sine  condition  d  =  n/sin  a,  it 
is  not  too  difficult  to  show  that  an  extra  factor  of  sec  a  has  to  be 
introduced  when  going  from  the  object  to  the  pupil  side  of  the 
objective  in  order  to  satisfy  the  energy  conservation  law.  This 
factor  has  been  well  known  since  the  early  days  of  high  NA  lens 
theory  (Richards  and  Wolf,  1959).  Intensity  distributions  in  the 
pupil  plane  should  therefore  look  like  Ix  -  cos  a  and  Iy  ~  sec  a  or, 
with  the  help  of  sine  condition: 
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An  experiment  to  measure  these  distributions  was  carried  out 
on  the  setup  shown  in  Figure  11.8  by  simply  blocking  the  refer¬ 
ence  beam  and  capturing  the  pupil  intensity  image  alone.  To 
produce  the  results  shown  in  Figure  11.15  images  of  eight  indi¬ 
vidual  scatterers  were  acquired  and  averaged  in  the  computer. 
Intensity  distributions  in  the  two  principal  planes  were  then 
extracted.  They  follow  the  theoretical  predictions  rather  well  up  to 
about  half  of  the  pupil  radius.  After  that,  apodization  is  apparent, 
and  increases  gradually,  reaching  about  30%  to  50%  towards  the 
edge  of  the  pupil. 

The  cause  of  this  apodization  in  all  likelihood  is  the  Fresnel 
losses  in  the  elements  of  the  objective  lens.  Broadband  anti¬ 
reflection  coatings  applied  to  these  elements  are  less  effective  at 
the  higher  incidence  angles  that  the  high  aperture  rays  are  certain 
to  encounter.  Because  of  the  nature  of  these  losses,  they  are  likely 
to  be  very  individual  for  each  particular  type  of  objective  lens.  It 
is  also  worth  noting  a  slight  polarization  dependence  of  the  losses 
that  contributes  to  polarization  effects  described  in  the  previous 
section. 


CONCLUSION 

The  range  of  experimental  setups  and  techniques  dedicated  to  the 
characterization  of  the  high  NA  objective  lenses  could  be  contin¬ 
ued.  For  example,  the  lateral  chromatic  aberration  has  not  been 
considered  thus  far.  One  has  to  be  practical,  however,  and  draw  a 
line  at  some  point. 


From  the  tests  described  in  this  chapter,  the  measurement  of 
the  pupil  function  provides  the  most  detailed  and  insightful  infor¬ 
mation  about  the  capabilities  of  a  particular  lens.  In  many  cases, 
two  or  three  pupil  functions  measured  across  the  field  of  view  and, 
perhaps,  tabulated  in  the  form  of  Zernike  coefficients,  would  be 
more  than  sufficient  to  predict  the  lens  performance  in  most  prac¬ 
tical  situations.  It  is  disheartening  to  think  of  how  much  wasted 
time,  frustration,  and  misunderstandings  could  be  avoided  if  such 
information  were  to  be  supplied  with  the  objective  lenses  by  their 
manufacturers. 

My  overall  conclusion  is  that  the  vast  majority  of  currently 
designed  objective  lenses  perform  really  well.  Any  imperfections 
visible  in  a  microscope  image  are  far  more  likely  to  be  a  result  of 
a  sloppy  practice  (e.g.,  tilted  coverslip,  incorrect  immersion  fluid, 
etc.)  than  of  a  fault  of  the  lens  itself.  That  said,  cutting-edge 
designs  are  always  going  to  be  a  problem  and  the  very  highest  NA 
lenses  should  be  approached  with  caution.  It  is  also  worth  point¬ 
ing  out  that  elements  of  the  microscope  other  than  the  objective 
lens  may  also  be  a  factor  in  imaging  quality.  The  tube  lens  is  one 
such  element  of  particular  concern.  Having  evolved  little  over  the 
last  years,  this  lens  simply  has  too  few  elements  to  achieve  the 
aberration  correction  on  par  with  that  of  an  all-singing  all-dancing 
objective  lens.  This  situation  is  further  exacerbated  by  the  advent 
of  a  new  breed  of  low  magnification  high  NA  objectives  with  their 
enormous  back  apertures. 
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Photon  Detectors  for  Confocal  Microscopy 

Jonathan  Art 


INTRODUCTION 

In  its  most  general  form,  confocal  microscopy  encompasses  all 
optical  techniques  whose  illumination  and  detection  scheme  exam¬ 
ines  each  point  in  an  object  in  the  absence  of  interfering  informa¬ 
tion  from  neighboring  points.  Recently  the  technique  has  expanded 
to  encompass  not  only  morphology  but  also  disciplines  as  far  afield 
as  physiology,  spectroscopy,  fluorescence  lifetime  analysis,  and 
even  DNA  sequencing.  As  a  result,  the  requirements  and  design 
constraints  on  appropriate  detection  systems  are  as  varied  as  the 
fields  to  which  the  technique  is  applied. 

In  the  familiar  case  of  fluorescent  imaging,  for  example,  con¬ 
focal  illumination  and  detection  is  used  to  reduce  the  back¬ 
ground  fluorescence  from  out-of-focus  planes  to  obtain  better 
image  quality  than  that  achieved  in  conventional  fluorescence 
microscopy  using  Kohler  illumination.  For  many  confocal  micro¬ 
scopes  employing  either  disk  (Petraii  et  al.,  1968)  or  line  scan  tech¬ 
niques,  the  primary  detector  is  the  human  eye.  It  is  attractive  in 
terms  of  its  quantum  efficiency:  roughly  16%  of  the  photons  inci¬ 
dent  on  the  cornea  are  perceived.  The  number  and  size  of  detector 
elements  is  high,  on  the  order  of  106,  and  this  massive  parallelism, 
joined  with  higher-order  processing,  results  in  the  relatively  rapid 
perception  of  an  image.  Any  confocal  technique  using  visible  light 
that  projects  points  in  an  object  coherently  onto  conjugate  points 
in  an  image  plane  can  employ  this  detector.  If  the  scanning  is 
rapid  enough,  a  stable,  full-field  image  will  be  seen.  When  such 
microscopy  is  combined  with  focusing  through  the  object,  it  is 
possible  to  readily  identify  and  characterize  its  three-dimensional 
(3D)  structure.  Under  relatively  high  illumination,  such  full-field 
confocal  systems  are  in  many  ways  the  most  general  and  suitable 
for  a  wide  variety  of  tasks.  The  detection  problem,  if  you  will,  has 
been  solved  for  us  over  the  last  109  years  of  evolution. 

Under  low-light  conditions  and  for  other  more  specialized 
tasks  such  as  spectroscopy  or  fluorescent  lifetime  analysis,  the  eye 
is  far  from  the  perfect  detector.  Because  it  is  highly  adaptive  and 
can  work  over  nine  log  units  of  intensity,  it  is  also  less  than  ideal 
at  quantitative  comparisons  of  absolute  intensity.  Nor  is  the  eye 
able  to  implement  averaging  and  filtering  techniques  to  enhance 
the  signal-to-noise  ratio  (S/N)  in  the  image.  Finally,  and  often  most 
significantly,  some  confocal  systems  have  slow  scanning  rates  or 
are  designed  to  project  points  in  the  object  plane  back  onto  a  single 
point,  and  it  is  impossible  in  such  cases  for  the  eye  to  perceive  a 
coherent  image. 

For  these  reasons  and  others,  it  is  necessary  to  develop  tech¬ 
niques  that  approach,  and  in  many  respects  surpass,  the  signal  pro¬ 
cessing  capabilities  of  the  eye  and  that  are  amenable  to  further 
digital  enhancement.  The  task  is  important  because,  at  present,  the 
design  goal  of  achieving  the  highest  S/N  in  a  confocal  image  for 


a  given  dose  to  the  specimen  is  often  degraded  by  elements  in  the 
detection  system.  For  example,  when  light  levels  are  low  (<20 
photons/pixel),  significant  improvements  in  S/N  can  often  be 
made  by  employing  the  greatest  possible  gain  in  the  detector  or 
by  exploiting  photon-counting,  rather  than  analog  measurement 
schemes.  It  should  also  be  remembered  that  the  particular  detec¬ 
tion  scheme  chosen  is  constrained  not  only  by  the  detectors  but 
also  by  the  particular  confocal  technology  employed,  the  levels  of 
illumination  that  can  be  used,  and  limitations  imposed  by  the 
nature  specimen. 

At  the  outset  it  should  be  noted  that  simple  point  detectors, 
such  as  photomultiplier  tubes  (PMT),  are  often  used  in  conjunc¬ 
tion  with  bandpass  filters  to  quantify  the  number  of  photons  at  dif¬ 
ferent  wavelengths  in  a  typical  two-  or  three-channel  laser 
scanning  microscope.  However,  the  recent  interest  in  characteriz¬ 
ing  the  full  spectrum  of  dyes  in  situ,  as  well  as  a  desire  to  sepa¬ 
rate  labels  whose  wavelength  of  peak  emission  may  differ  by  a  few 
nanometers,  has  led  to  the  development  of  sophisticated  spectro¬ 
scopic  array  detectors  that  divide  the  incident  light  into  bands  a 
few  nanometers  wide.  In  addition,  continued  improvements  in 
multi-point  or  line-scanning  confocal  microscopy  have  resulted  in 
the  use  of  sophisticated  array  detectors  that  simultaneously  accu¬ 
mulate  data  across  the  image  plane.  In  all  of  these  contexts,  the 
overarching  goal  is  to  choose  a  detection  scheme  that  yields 
the  highest  quantum  efficiency,  the  lowest  background  levels,  and 
the  highest  S/N. 

In  this  chapter  we  will  consider  the  quantal  nature  of  light 
and  the  interaction  of  photons  with  materials.  We  will  compare  a 
number  of  possible  detectors  in  terms  of  their  quantum  efficiency, 
responsivity,  spectral  response,  inherent  noise,  response  time,  and 
linearity.  We  will  then  consider  the  design  constraints  in  terms  of 
the  front-end  circuitry  that  digitizes  the  data.  The  figures  of  merit 
for  detection  are  usually  that  the  estimate  of  the  signal  is  limited 
either  by  the  noise  within  the  signal  or  by  the  background  radia¬ 
tion.  No  physical  detector  can  improve  on  these  limits.  Finally,  we 
will  suggest  future  directions  toward  more  perfect  detectors  with 
signal  processing  capabilities  limited  only  by  the  stochastic  nature 
of  the  signal.  We  will  begin  by  considering  the  kinetic  energy  of 
photons. 

THE  QUANTAL  NATURE  OF  LIGHT 

At  very  low  light  levels,  two  aspects  of  the  quantal  nature  of  light 
can  be  demonstrated.  First,  each  particle,  or  photon,  has  an  asso¬ 
ciated  kinetic  energy.  An  incident  photon  stream  transfers  kinetic 
energy  to  a  material  and  gives  rise  to  the  variety  of  effects  used  in 
light  detection.  Second,  at  low  light  levels  it  is  apparent  that  even 
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with  perfect  detection,  the  estimate  of  the  intensity  of  a  source  is 
limited  by  the  statistics  of  photon  arrival  at  the  detector.  The  sta¬ 
tistical  nature  of  photon  flux  will  be  considered  in  a  later  section. 

First  consider  the  kinetic  energy  of  a  photon  incident  on  a 
detector.  The  classic  frequency  of  electromagnetic  radiation,  v 
(s_1),  is  related  to  the  quantum  mechanical  kinetic  energy,  E(J),  of 
an  individual  photon  by  E=  hv,  where  /;  is  Planck’s  constant  (6.626 
x  10”34in  Js).  The  relation  between  the  frequency,  v,  and  the  wave¬ 
length,  X  (in  m),  is  given  by:  v  =  c/X,  where  c  is  the  velocity  of 
light  (2.997  X  10s  m/s).  Each  photon  in  monochromatic  green  light, 
X  =  550nm,  has  a  kinetic  energy  of  3.58  X  10~19 J,  or  2.25  electron 
volts  (eV).  The  human  eye  is  sensitive  to  light  with  /is  roughly 
between  400  and  750  nm.  In  confocal  microscopy,  the  range  of 
molecular  probes  used  in  biological  experiments  will  most  likely 
fluoresce  at  /is  that  range  from  the  ultraviolet  to  the  near-infrared, 
say  from  300  nm  to  1200nm.  The  associated  photons  will  have 
kinetic  energies  between  4.16  and  1.03  eV.  These  modest  energy 
levels  will  necessarily  restrict  the  types  of  materials  and  the  tech¬ 
niques  that  can  be  used  for  photon  detection. 

INTERACTION  OF  PHOTONS  WITH  MATERIALS 

When  metals  or  semiconductors  are  illuminated,  the  photons  may 
be  either  reflected  or  absorbed.  If  they  are  absorbed,  the  kinetic 
energy  (KE)  of  the  photon  is  imparted  to  the  structure.  This  energy 
may  be  transformed  into  a  random  motion  as  heat,  or  it  may  have 
a  direct  effect  that  changes  the  arrangement  of  charges  in  the 
crystal  lattice  of  the  material  (Kittel,  1986).  These  direct  effects 
are  the  bases  of  a  number  of  internal  and  external  techniques  of 
photon  detection. 

Thermal  Effects 

Thermal  effects  do  not  depend  on  the  photon  nature  of  light,  and 
the  response  of  the  material  depends  on  the  radiant  power,  not  the 
spectral  content  of  the  radiation.  Thermal  effects  are  characterized 
by  changes  in  properties  of  a  material  arising  from  an  increase  in 
its  temperature  due  to  the  absorption  of  radiation.  Heating  and 
cooling  of  a  macroscopic  detector  is  slower  than  techniques  in 
which  photons  interact  directly  with  electrons  in  the  material. 
Roughly  speaking,  thermal  effects  are  on  a  millisecond  time  scale, 
and  photon  effects  are  on  microsecond  or  nanosecond  time  scales. 
In  confocal  microscopy,  the  slow  response  and  lower  sensitivity  of 
thermal  detectors,  as  compared  to  direct  photodetectors,  would 
restrict  their  use  as  primary  detectors.  They  could,  however,  be 
used  to  calibrate  the  system  because  their  response  is  independent 
of  the  radiation  X.  The  performance  of  thermopiles,  Golay  cells, 
bolometers,  and  pyro-electric  thermal  detectors  have  been 
reviewed  by  Putley  (1977)  and  will  not  be  considered  further  here. 

Direct  Effects 

In  direct  photodetection,  the  photons  interact  directly  with  elec¬ 
trons  in  the  material.  These  electrons  may  either  be  free  or  bound 
to  lattice  or  impurity  atoms.  For  reasons  of  speed  and  sensitivity, 
the  detectors  used  in  confocal  microscopy  will  generally  be  of  the 
photon  detection  class.  In  metals  and  semiconductors,  electrons  are 
bound  to  their  atoms  within  the  material  by  electrostatic  force.  The 
average  strength  of  this  force  is  described  by  the  material’s  ion¬ 
ization  energy  or  work  function.  An  electron  with  energy  greater 
than  this  can  escape  from  the  atom.  The  smallest  ionization  energy 
for  any  elemental  solid  is  ~  2. 1  eV  of  metallic  Cs.  If  an  electron  in 


solid  Cs  absorbs  a  photon  with  a  KE  >2.1  eV,  a  direct  photoeffect 
can  occur.  The  ionization  energies  of  a  number  of  materials  are 
given  in  Table  12.1.  Given  the  low  KE  of  photons  in  the  visible 
range,  the  materials  employed  for  direct  photodetection  will  almost 
always  be  semiconductors. 

Direct  photon  effects  can  be  divided  into  two  classes,  internal 
and  external.  Internal  effects  are  those  in  which  the  electrons  or 
holes  stay  within  the  material.  Internal  effects  can  be  further  sub¬ 
divided  in  three  ways:  (1)  those  in  which  the  incident  photon  inter¬ 
acts  with  a  bound  electron;  (2)  those  in  which  the  photon  interacts 
with  carriers  that  are  already  free;  and  (3)  those  in  which  the 
photon  produces  a  localized  excitation  of  an  electron  into  a  higher 
energy  state  within  the  atom.  The  external  photoeffect,  or  photo- 
emissive  effect,  is  one  in  which  the  photon  causes  emission  of  an 
electron  from  the  surface  of  the  absorbing  material  known  as  the 
photocathode.  External  devices  often  charge-multiply  this  free 
electron,  producing  gain.  Of  all  the  possible  direct  detection 
methods,  the  most  often  used  are  the  photoconductive,  photo¬ 
voltaic,  and  photoemissive. 

Photoconductivity 

In  photodetectors  exploiting  the  phenomenon  of  photoconductiv¬ 
ity,  a  photon  effect  is  manifested  by  an  increase  in  the  free  electron 
or  hole  concentration  of  a  material,  causing  an  increase  in  its  elec¬ 
trical  conductivity.  Intrinsic  photoconductivity  occurs  if  the  photon 
has  enough  kinetic  energy  to  produce  an  electron/hole  pair  [Fig. 
12.1(A)].  Extrinsic  photoconductivity  occurs  when  an  incident 
photon  produces  excitation  at  an  impurity  center  in  the  form  of 
either  a  free  electron/bound  hole  or  a  free  hole/bound  electron  [Fig. 
12.1(B)].  In  general,  photoconductivity  is  a  majority  carrier  phe¬ 
nomenon  that  corresponds  to  an  increase  in  the  number  of  electrons 
in  an  n-type  semiconductor  or  an  increase  in  the  number  of  holes 
in  a  />-type  material.  The  minority  carriers  will  also  contribute,  but 
because  of  their  shorter  lifetimes,  they  contribute  less.  A  circuit  to 
detect  the  photoconductive  effect  is  shown  in  Figure  12.2(A). 

Photovoltaic 

Unlike  the  photoconductive  effect,  the  photovoltaic  effect  requires 
an  internal  potential  barrier  and  field  to  separate  the  photoexcited 
hole-electron  pair.  This  occurs  at  a  simple  p-n  junction  but  can  be 
observed  in  p-i-n  diodes,  Schottky  barrier  diodes,  and  avalanche 
diodes  as  well.  The  field-induced  charge  separation  accompanying 
photoexcitation  is  shown  in  Figure  12.1(C).  By  definition,  the  pho¬ 
tovoltaic  effect  is  obtained  at  zero  bias  voltage,  that  is,  the  open 


TABLE  12.1.  Work  Functions  and  Forbidden  Energy  Gaps 
for  Common  Materials  Used  in  Direct  Photodetectors 


Metals 

Work  function,  emission 
from  bulk  material,  eV 

Semiconductors 

Forbidden  energy 
gaps,  eV 

Be 

3.67 

CdS 

2.40 

Mg 

3.66 

CdSe 

1.80 

Ca 

2.71 

CdTe 

1.50 

Sr 

2.24 

GaP 

2.24 

Li 

2.49 

GaAs 

1.35 

Ba 

2.38 

Si 

1.12 

Na 

2.29 

Ge 

0.67 

K 

2.24 

Rb 

2.16 

Cs 

2.14 
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FIGURE  12.1.  Direct  photodetector  energy-band  models  (energy  vs.  dis¬ 
tance).  (A)  Energy  profile  of  intrinsic  photoconductivity  producing  an  electron 
( filled  circle )  and  a  hole  (open  circle).  (B)  Energy  profile  of  extrinsic  photo¬ 
conductivity  producing  either  a  free  electron  or  a  free  hole.  (C)  Field-induced 
separation  of  charge  in  the  photovoltaic  effect.  (D)  Energy  profile  at  the  metal 
semiconductor  interface  of  a  Schottky  diode.  (E)  Energy  profile  for  emission 
from  a  classic  photocathode  material.  (F)  Energy  profile  for  emission  from 
semiconductor  photocathodes.  (G)  Energy  profile  for  emission  from  negative 
electron  affinity  photocathodes. 


circuit  voltage  [Fig.  12.2(B)].  However,  these  detectors  are  fre¬ 
quently  operated  under  reverse  bias,  so  that  the  observed  signal  is 
actually  a  photocurrent  (see,  e.g.,  the  following  avalanche  diode 
section)  rather  than  a  photovoltage  [Fig.  12.2(C)],  In  contrast  to 
the  photoconductive  phenomenon,  the  photovoltaic  effect  is 
dependent  on  the  minority  carrier  lifetime  because  the  presence  of 
both  the  majority  and  minority  carriers  is  necessary  for  the  intrin¬ 
sic  effect  to  be  observed.  Consequently,  because  the  lifetimes  of 
the  minority  carriers  are  less  than  those  of  majority  carriers,  the 
frequency  response  of  photovoltaic  detectors  is  higher  than  that  of 
photoconductive  devices. 

p-i-n  Diodes  differ  from  p-n  diodes  in  that  a  region  of  intrin¬ 
sic  material  is  incorporated  between  the  two  doped  regions  [Fig. 
12.2(D)].  Absorption  of  incident  radiation  in  the  intrinsic  region 
produces  electron/hole  pairs.  Because  of  the  high  collection 
voltage  and  the  small  distance  across  the  intrinsic  material,  the 
electron/hole  pairs  will  drift  rapidly  through  this  region.  As  a  con¬ 
sequence,  the  frequency  response  of  a  p-i-n  diode  will  be  higher 
than  a  comparable  p-n  diode  (Mathur  et  al.,  1970). 

The  Schottky  diode  is  formed  at  a  metal-semiconductor  inter¬ 
face  [Fig.  12.1(D)],  Such  an  interface  forms  a  potential  barrier  that 
causes  separation  of  electrons  and  holes.  These  devices  are  espe¬ 


cially  useful  at  ultra-high  frequencies  and  are  often  used  as  optical 
receivers  operating  in  the  GHz  range  (Sharpless,  1970). 

Avalanche  occurs  in  p-n  diodes  of  moderate  doping  under 
reverse  bias.  Photoexcited  or  thermally  excited  electrons  or  holes 
are  accelerated  in  the  high-field  region  of  the  junction.  As  they  are 
accelerated,  they  collide  with  the  structure  and  free  more  electrons. 
Thus,  as  the  name  implies,  an  avalanche  of  electrons  occurs  in  the 
high-held  region.  The  advantage  of  the  avalanche  photodiode  over 
other  types  of  photodiodes  is  that  it  has  internal  gain.  Under  iden¬ 
tical  conditions,  the  response  of  such  a  diode  is  larger  than  that  of 
the  p-n  diode.  This  internal  gain  cannot  increase  the  S/N  inherent 
in  the  detector,  but  it  reduces  the  stringency  of  the  noise  and  gain 
requirements  of  following  stages.  The  high  amplitude  of  the  output 
and  the  speed  of  the  response  make  avalanche  diodes  attractive  for 
confocal  microscopy,  but  the  avalanche  multiplication  factor  varies 
with  the  position  of  the  initial  photon  absorption  so  the  signal  must 
be  discriminated  to  remain  a  true  measure  of  photon  number.  Addi¬ 
tional  difficulties  with  implementing  avalanche  diodes  focus  on  the 
design  of  active  drive  circuits  [Fig.  12.2(E)]  that  rapidly  quench 
the  avalanche  response  (Brown  et  al.,  1986,  1987).  Because  of 
their  inherent  speed  and  gain,  avalanche  diodes  are  often  used  in 
extremely  rapid  kinetic  measurements  performed  at  low  repetition 
rates.  Low  repetition  is  necessary  because  the  response  to  an  inci¬ 
dent  photon  must  be  quenched  before  the  arrival  of  a  subsequent 
one. 


FIGURE  12.2.  Direct  photodetection  circuits.  (A)  Photoconductivity  mea¬ 
surement  using  a  bias  voltage  and  measuring  the  voltage  drop  across  the  series 
resistor.  (B)  Photovoltaic  effect  measures  the  potential  difference  generated 
across  a  simple  p-n  junction.  (C)  Measurement  of  the  voltage  drop  across  a 
series  resistor  produced  by  the  photocurrent  through  a  reverse  biased  photodi¬ 
ode.  (D)  Geometry  of  p-i-n  photodiode  with  narrow  intrinsic  region  between 
the  p  and  n  regions.  (E)  Active  quenching  circuit  for  an  avalanche  photodiode 
(Brown  et  al.,  1987).  (F)  Charge  multiplication  as  photoelectrons  are  acceler¬ 
ated  along  a  dynode  chain  by  high  potential  between  cathode  and  anode.  Charge 
is  converted  to  a  potential  difference  across  the  load  resistor  at  the  anode. 
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Charge-Coupled  Devices 

Photoconductive  or  photovoltaic  effects  can  be  used  to  produce 
free  carriers,  which  can  then  be  injected  into  the  transport 
structure  of  a  charge-coupled  device  (CCD;  Boyle  and  Smith, 
1970).  The  metal-insulator-semiconductor  and  metal-oxide- 
semiconductor  capacitors  in  these  devices  can  be  used  to  store 
photogenerated  electrons  [Fig.  12.3(A)].  Appropriate  doping  of 
the  semiconductor  substrate  can  be  used  to  match  the  device  to 
the  X  of  interest,  and  quantum  efficiencies  of  greater  than  85%  can 
be  achieved  with  commercially  available  devices  when  back- 
illuminated  through  the  substrate  (Fairchild,  1987).  The  maximum 
charge  (in  electrons)  that  each  capacitor  can  store  without  over¬ 
flowing  into  nearby  pixels  can  be  estimated  by  multiplying  the 
pixel  area  (|im2)  by  600.  Scientific  CCD  arrays  range  from  45  to 
900  |im2,  corresponding  to  a  maximum  of  27,000  to  540,000 
electrons/pixel. 

Generally  these  devices  are  packaged  as  either  linear  (Coutures 
and  Boucharlat,  1988)  or  two-dimensional  (2D)  arrays  (Flier  et  al., 
1988;  Mackay,  1988;  McMullan,  1988)  and  are  not  available  as 
simple  elemental  detectors.  Because  these  devices  have  such  high 
apparent  quantum  efficiency,  they  are  suited  to  the  low  light 
requirements  of  confocal  microscopy.  For  full-field,  rotating  disk, 
or  line  scan  confocal  microscopes,  the  2D  and  one-dimensional 
(ID)  arrays  are  a  natural  geometry,  allowing  the  response  of  inde¬ 
pendent  pixels  to  be  integrated  simultaneously.  This  is  often  an 
advantage  at  low  light  levels. 

There  are  a  number  of  ancillary  considerations  when  using 
CCD  arrays.  First  it  should  be  noted  that  by  their  design  [Fig. 
12.3(B)],  the  2D  arrays  will  not  have  an  available  detector  over  the 
entire  surface  of  the  array.  A  percentage  of  the  substrate  will  be 
occupied  by  the  transport  electronics  and  will  be  unavailable  for 
primary  imaging.  Consequently,  though  the  material  may  naturally 
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FIGURE  12.3.  Charge-coupled  devices  used  in  photodetection.  (A)  Energy 
versus  distance  profile  in  metal-insulator-semiconductor  «-channel  structure. 
The  initial  bands  (1 )  are  changed  after  application  of  a  gate  voltage  VG  (2)  and 
charge  builds  up  at  the  insulator-semiconductor  interface  until  equilibrium  is 
approached  (3).  (B)  (1)  Two-dimensional  devices  can  be  built  from  an  array  of 
discrete  photosites  and  overlying  CCD  electrode  arrays  arranged  to  sequen¬ 
tially  shift  out  and  read  the  charge  packets  collected  at  each  photosite.  (2) 
Method  of  transfer  of  charge  packet  along  MIS  CCD  structure.  Overlapping 
clocks  transfer  charge  between  neighboring  electrodes. 


have  high  quantum  efficiency,  a  large  fraction  of  the  surface  area 
of  the  device  may  be  unavailable  for  photon  capture.  Second, 
because  the  difference  in  the  number  of  electrons  in  each  potential 
well  represents  the  spatial  variation  of  light  incident  on  the  detec¬ 
tor,  it  is  crucial  that  this  variation  be  preserved  as  the  packets  of 
charge  are  transferred  along  the  CCD  to  the  output  [Fig.  12.3(B2)]. 
Incomplete  transfer  of  charge  packets  and  the  inability  to  transfer 
charge  down  the  CCD  structure  in  the  absence  of  bias-,  thermal-  or 
photon-generated  electrons  in  all  the  wells  presently  limited  the 
utility  of  early  devices  in  applications  in  which  large  expanses  of 
the  field  are  dark  (Coutures  and  Boucharlat,  1988).  Third,  at  low 
light  levels  the  detection  of  the  signal,  the  contrast,  and  the  S/N  are 
particularly  dependent  on  the  existence  of  low  dark  levels,  and 
detectors  that  generate  the  fewest  thermal  electrons  during  photon 
capture  and  electron  transport  are  to  be  preferred. 

Photoemissive 

The  third  direct  photon  detection  technique  is  photoemissive,  also 
known  as  the  external  photoeffect.  The  incident  radiation  causes 
the  emission  of  an  electron  from  the  surface  of  the  absorbing  mate¬ 
rial  (Einstein,  1905),  the  photocathode,  into  the  surrounding  space 
[(Fig.  12.1(E,F,G)],  from  whence  it  is  collected  by  an  anode. 
Photoemissive  effects  are  used  in  simple  vacuum  phototubes  com¬ 
posed  of  a  simple  cathode  and  anode.  They  are  very  fast.  They  are 
also  found  in  devices  that  interpose  gain,  such  as  gas-filled  photo¬ 
tubes  that  rely  on  the  avalanche  effect  of  photoelectrons  striking 
the  gas  within  the  tube.  In  the  familiar  photomultiplier  tube,  the 
primary  photoelectron  strikes  a  series  of  emissive  dynode  surfaces 
interposed  between  the  photocathode  and  the  anode.  At  each 
surface,  the  number  of  electrons  emitted  is  multiplied  until,  at  the 
anode,  millions  of  electrons  may  be  collected  for  each  photoelec¬ 
tron  originally  emitted  from  the  photocathode  [Fig.  12.2(F)]. 

Three  special  forms  of  photoemissive  devices  should  be  men¬ 
tioned  —  the  vacuum  avalanche  photodiode,  the  image  dissector, 
and  the  microchannel  plate.  The  first  is  a  single  gain-stage  replace¬ 
ment  for  traditional  PMTs  that  is  particularly  suited  for  photon 
counting.  The  latter  two  devices,  in  addition  to  the  fast  response, 
wide  spectral  sensitivity,  and  photon-counting  capabilities  of  the 
usual  types  of  PMTs,  have  position  sensitivity  and  can  be  used  as 
full-field  detectors. 

Vacuum  Avalanche  Photodiode 

The  vacuum  avalanche  photodiode  (VAPD)  combines  an 
avalanche  photodiode  (APD)  with  a  photocathode  in  a  vacuum 
tube  (Advanced  Photonix).  In  the  configuration  shown  in  Figure 
12.4(A),  the  electrons  produced  when  photons  are  absorbed  by  the 
photocathode  are  accelerated  by  the  large  imposed  electric  field 
and  bombard  the  underlying  APD.  A  new  electron  is  promoted  for 
every  3.6 eV  of  incoming  energy,  and  a  typical  8k  eV  field  would 
yield  about  1800  new  electrons  striking  the  APD.  These  secondary 
electrons  are  subject  to  the  internal  APD  field  and  will  drift  towards 
the  junction  and  experience  the  gain  inherent  in  the  conventional 
avalanche  phase.  The  total  gain  of  such  a  device  can  be  as  much 
as  106,  but  because  much  of  the  gain  is  in  the  first  stage,  the  noise 
figures  are  superior  to  conventional  PMTs.  Circuits  suitable  for 
single-photon  counting  or  Raman  spectroscopy  are  shown  in 
Figure  12.4(B). 

Image  Dissector 

The  use  of  the  image  dissector  tube  in  confocal  microscopy 
(Goldstein,  1989)  is  a  novel  approach  that  combines  the  aperture 
and  detection  functions.  The  image  dissector  consists  of  the  usual 
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FIGURE  1 2.4.  (A)  Schematic  of  VAPD  structure.  Photons  incident  on  the  pho¬ 
tocathode  release  photoelectrons  that  accelerate  toward  the  underlying  APD. 
(B )  VAPD  circuit  suitable  for  photon  and  low-light-level  spectroscopy. 


photocathode  and  photomultiplier  dynode  chain.  Between  the  two, 
an  external  magnetic  held  focuses  the  photoelectrons  onto  an 
image  plane  in  which  a  small  aperture  is  centered.  Deflection  coils 
sweep  the  image  across  the  aperture,  and  only  those  electrons 
passing  through  the  aperture  undergo  multiplication.  All  others  are 
discarded.  This  type  of  field  detector  differs  from  normal  vidicon 
tubes  and  CCD  arrays  in  that  integration  does  not  take  place  in  the 
detector  between  scans.  Rather,  it  functions  as  a  movable  point 


detector  and,  in  this  capacity,  can  be  used  to  track  the  illuminated 
spot  in  a  confocal  microscope  electronically. 

MicroChannel  Plate 

The  microchannel  plate  photomultiplier  (MCP-PMT)  differs  from 
the  usual  PMT  by  replacing  the  chain  of  discrete  dynodes  with  a 
channel  of  continuous  photomultiplication.  The  MCPs  are  sec¬ 
ondary  multipliers  consisting  of  an  array  of  capillary  tubes,  or 
channels,  fused  together  to  form  a  plate.  The  tube  diameter  can  be 
as  small  as  a  few  micrometers,  and  the  interior  of  each  is  coated 
with  a  photoemissive  compound.  A  high-voltage  electric  field  is 
imposed  along  the  length  of  the  tube.  Photons  entering  one  end  of 
the  tube  at  an  angle  strike  the  channel  wall  with  sufficient  energy 
to  generate  secondary  electrons.  These  electrons  are  accelerated 
and  in  turn  strike  the  wall  farther  down  the  tube,  and  the  process 
is  repeated.  Gains  of  104  to  106  can  result  from  this  series  of  col¬ 
lisions  (Kume  et  al.,  1988).  The  MCP  can  thus  serve  as  an  inter¬ 
mediate  gain  stage  in  front  of  lower  sensitivity  detectors. 

Comparison  of  Detectors 

Detectors  vary  widely  in  their  response  properties.  A  comparison 
of  the  quantum  efficiency,  responsivity,  spectral  response,  response 
time,  and  linearity  of  direct  photodetectors  is  given  in  Table  12.2. 
Unlike  thermal  detectors,  the  response  of  direct  photodetectors 
depends  on  the  photon  nature  of  light.  Therefore,  the  response  of 
the  detector  varies  with  the  X  of  radiation.  The  direct  detectors 
differ  widely  in  their  maximum  sensitivity  and  quantum  efficiency. 
Graphs  of  the  spectral  response  for  examples  of  direct  detectors 
are  given  in  Figure  12.5.  PMTs  with  classic  photocathode  materi¬ 
als  have  greatest  sensitivity  in  the  blue  to  UV  (QE  <  35%).  The 
short  X  limits  are  functions  of  the  envelope  materials  rather  than 
the  photocathode  materials  and  can  be  extended  by  using  quartz 
and  fluorite  faceplates.  Semiconductors  are  attractive  photoemis¬ 
sive  materials  because  they  have  high  quantum  efficiency  at  longer 
Xs  and  can  be  designed  with  ionization  energies  or  work  functions 
that  match  the  X  of  interest.  As  such  they  will  be  useful  either  as 
negative  electron  affinity  (NEA)  emissive  materials  for  cathodes 
and  dynodes  in  PMTs  or  as  solid-state  detectors  (QE  <  85%). 


TABLE  12.2.  Response  Properties  of  Sample  Direct  Photodetectors 


Quantum 

Radiant 

Response  peak 

Response 

Type  of  detector 

efficiency 

sensitivity  (AAV) 

Spectral  range 

(nm) 

(seconds) 

Range 

Type 

Silicon 

0.4 

190-1000 

720 

5  x  10-7 

10~n  -  10~3W 

Hamamatsu  S  1227 

photodiode 

80.0% 

0.5 

190-1100 

920 

2  x  10~7 

10~n  -  10~3W 

Hamamatsu  S  1337 

PIN  silicon 

83.0% 

0.6 

190-1060 

900 

3  x  10'9 

10~n  -  10~2W 

Hamamatsu  S  1721 

photodiode 

GaAsP 

0.18 

190-680 

610 

1  x  10"6 

10~13-  10"13W 

Hamamatsu  S  1025 

photodiode 

(Schottky) 

0.22 

190-760 

710 

0.8  x  10~6 

10-|3_  10"3W 

Hamamatsu  S  1745 

Avalanche 

photodiode 

PMTs 

70.0% 

350-1050 

830 

0.2  x  10~9 

2  x  106cts/s 

Hamamatsu  S  2381 

S-l 

0.5% 

2.5  x  10~3 

300-1100 

800 

15  ns  (FWHM) 

5  x  107cts/s 

Thom-EMI-9684 

S-20 

23.0% 

64.0  x  10"3 

300-850 

420 

25  ns  (FWHM) 

3.3  x  107cts/s 

Thom-EMI-9890 

GaAs :  Cs-0 

17.0% 

120.0  x  10^3 

200-950 

850 

RCA-Type  60  ER 

CCD  linear 

85.0% 

4.5V/mJcnT2 

330-950 

680 

50  ns/pixel 

7500:1 

Fairchild  CCD- 134 

readout 

CCD  Areal 

15.0% 

0.08  V/mW  cnT2 

500-1000 

800 

@  60  Hz  field 

1000:1 

Fairchild  CCD-222 

488  x  380 

elements 

rate 
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FIGURE  12.5.  Typical  spectral-response  curves  for  internal  and  external 
direct  photodetector  materials. 


NOISE  INTERNAL  TO  DETECTORS 

An  ideal  detector  would  be  one  in  which  detection  is  limited  by 
the  noise  or  background  of  the  radiant  source  and  no  additional 
noise  is  added  by  the  detector  or  subsequent  amplification  and  con¬ 
version  electronics.  With  present  techniques,  it  is  possible  to 
design  amplification  and  conversion  stages  with  noise  below  that 
of  the  zero-signal  output  of  the  detector  (Siliconix,  1986).  There¬ 
fore,  the  detection  scheme  will  generally  be  limited  by  the  noise 


of  the  detector  stage  itself.  All  detectors  are,  in  principle,  in  one  of 
their  two  regimes.  At  very  low  light  levels,  their  utility  is  limited 
by  noise  within  the  detector.  At  higher  levels,  the  certainty  in  the 
estimate  is  limited  by  the  statistics  of  photon  flux.  We  will  con¬ 
sider  the  noise  inherent  in  both  internal  and  external  devices  in 
turn.  Formal  expression  of  each  noise  contribution  is  given  in 
Table  12.3  as  the  noise  power  obtained  by  summing  the  contribu¬ 
tions  of  the  independent  noise  sources  in  quadrature  (Schwartz, 
1970). 

Noise  in  Internal  Detectors 

Internal  detection  with  semiconductor  devices  is  associated  with 
two  general  classes  of  noise.  The  first  is  measurable  in  the  absence 
of  an  applied  bias  voltage.  It  arises  from  the  thermal  motion  of 
charge  carriers  within  a  material  and  is  known  as  Johnson, 
Nyquist,  or  thermal  noise.  The  second  category  of  noise  sources 
is  only  measurable  in  the  presence  of  a  bias  voltage  and  is 
specific  to  the  type  of  device  and  the  variable  measured  in  the 
photoconversion  process.  In  photoconductors  there  also  exists 
generation-recombination  (g-r)  noise,  which  is  due  to  fluctuations 
in  the  rates  of  thermal  generation  and  recombination  of  charge 
carriers,  giving  rise  to  a  fluctuation  in  the  average  carrier  concen¬ 
tration.  This,  in  turn,  is  measured  as  a  variation  in  average  resis¬ 
tance.  Complete  analysis  of  g-r  noise  can  be  found  in  Van  der  Ziel 
(1959)  and  Burgess  (1956).  In  diodes  with  an  applied  bias,  shot 
noise  is  apparent.  This  noise  is  due  to  the  quantal  nature  of  current 
carriers  and  results  in  a  statistical  variation  in  the  amplitude  of  the 
measured  current.  The  final  form  of  noise,  which  is  not  strictly 
amenable  to  analytic  treatment,  is  1/f  noise,  so  named  because  the 
power  varies  more  or  less  inversely  with  frequency. 


Noise  in  Photoemissive  Devices 

For  external  photoemissive  devices,  the  limit  to  detection  is  given 
by  the  dark  current.  This  current  has  three  origins:  ohmic  leakage, 
thermionic  emission  from  the  cathode  and  dynodes,  and  regener¬ 
ative  effects.  Ohmic  leakage  is  a  breakdown  between  active  leads 


TABLE  12.3.  Noise  Currents  in  Direct  Photodetectors3 


Value  at 

Generation- 

Shot 

1/f 

which  SNR 

Type  of  detector 

Johnson  noise* 

recombination  noise 

noisec 

Noise3 

Multiplication  noise 

=  r 

Extrinsic  photo-detectors 

Same 

Same 

Same 

N.A 

Intrinsic  photo-detectors 

Same 

1  =2IBfp  S  ) 

Era  N  l.  ’  1  +  co2t2J 

Same 

Same 

N.A 

Avalanche  photodiodes 

Same 

N.A. 

N.A. 

Same 

f'M  =  M(2eIBM)‘'2 
/2=M(2eIB)I/2(^l  +  AIj/ 

1CT14 

PMT  alone 

Same 

N.A. 

N.A. 

N.A. 

-nr17 

I-V  converter  noise 

PMT  and  electronics 

(4kTgB(l  +  RLg  +  (Anvi) 

N.A. 

N.A. 

Same 

N.A. 

(^N:g)B2C2))1<2 

“All  sources  considered  to  be  independent,  and  total  noise  current  is  the  square  root  of  the  sum  of  the  squares  of  the  contributing  currents.  Abbreviations:  B,  bandwidth  of 
measurement;  C,  shunt  capacitance  of  anode  lead;  d,  dynode  gain;  e,  charge  on  an  electron  f,  frequency;  g,  shunt  conductance  of  the  anode  lead;  I,  dark  current;  Ib,  bias 
current;  k,  Boltzman’s  constant;  K,  multiplicative  constant;  M,  charge  multiplication  or  gain;  N,  number  of  electrons;  P,  number  of  holes  RL,  load  resistance;  RN,  equiva¬ 
lent  noise  resistance  of  amplifier  input;  T,  temperature  in  degrees  Kelvin;  carrier  lifetime;  angular  frequency,  NEP  calculations  are  rough  estimates  for  each  class  of  detec¬ 
tors.  Noise  effects  due  to  the  noise-in-signal  and  that  resulting  from  converter  Q.E.  I  have  not  been  included. 
b  if  =  (4kTB/Rt)1/2  peak  sensitivity  (AAV)  due  to  resistive  components. 
cit=  (2eIB)1/2  due  to  bias  current. 

diF  =  (KIb  —  2B/f  ~  l)m  due  to  conjunctions  with  external  conductors. 

‘’NEP  (W/Hz"1/2  noise  equivalent  power). 
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along  the  insulating  material  of  the  tube.  It  can  be  identified  at  low 
acceleration  voltages  by  the  linear  relation  between  the  voltage  and 
the  measured  current.  At  higher  voltages  the  thermionic  compo¬ 
nent,  formally  equivalent  to  shot  noise,  becomes  apparent  because 
thermally  generated  electrons  from  the  cathode  (or  first  dynode) 
undergo  the  same  multiplication  as  do  photoelectrons,  and  the 
amplitude  of  this  component  will  increase  in  parallel  with  the 
amplitude  of  the  PMT  response  under  low  light  conditions.  This 
component  of  the  noise  is  dependent  on  temperature  and  can  be 
reduced  by  cooling  the  tube. 

At  higher  voltages,  regenerative  effects  can  come  in  and  can 
include  the  glow  of  the  dynodes  under  electron  bombardment. 
Glowing  of  the  glass  envelope  also  occurs  as  photoelectrons 
impact  the  tube  wall  if  it  is  not  at  the  cathode  potential.  This  effect 
can  be  eliminated  either  by  keeping  the  envelope  at  a  negative 
potential  or  by  providing  it  with  a  shield  at  such  a  potential.  After¬ 
pulses  can  also  occur.  One  type  is  the  result  of  feedback  from  the 
dynodes  to  the  photocathode  (due  to  light  given  off  from  the 
dynode  that  impacts  the  photocathode  and  consequently  gives  off 
more  electrons).  This  after-pulsing  has  a  short  delay  (40-50ns) 
after  the  initial  pulse.  A  second  type  of  after-pulse  is  due  to  the 
ionization  of  residual  gases  that  are  in  the  tube  or  are  adsorbed  in 
the  envelope.  Most  prevalent  are  N2+  and  H2+  ions  that  are  accel¬ 
erated  back  toward  the  negative  cathode,  giving  rise  to  further  elec¬ 
tron  emission.  (The  delay  is  somewhat  longer,  on  the  order  of  a 
few  100ns.)  Tubes  that  are  run  near  a  cooled  helium  environment 
will  have  troubles  because  the  helium  will  permeate  through  the 
glass  envelope  and  remain  within  the  tube  (RCA,  1980). 

Further  sources  of  noise  arise  from  radioactive  elements  within 
the  glass,  such  as  40K,  or  are  due  to  cosmic  rays.  Additional  noise 
is  often  produced  after  exposure  of  the  tube  to  blue  or  near¬ 
ultraviolet  (UV)  light.  Normally  this  is  due  to  fluorescent  room 
lights,  though  it  is  likely  to  be  an  increasing  problem  in  experiments 
where  near-UV  light  is  used  to  stimulate  the  chromophors  such  as 
the  Fura-2  and  Indo-1  indicators  used  in  the  study  of  [Ca2+]  (see 
Chapter  42,  this  volume). 

The  statistics  of  the  effects  of  these  noise  sources  will  be  con¬ 
sidered  next,  along  with  the  necessary  probabilistic  effect  of  the 
finite  quantum  efficiency  of  the  original  production  of  photoelec¬ 
trons  at  the  photocathode. 

Statistics  of  Photon  Flux  and  Detectors 

Several  of  the  parameters  of  direct  photodetector  behavior  were 
given  in  Table  12.2.  Many  of  the  noise  contributions  for  each 
detector  are  summarized  in  Table  12.3.  In  practice,  the  utility  of 
the  various  detectors  will  be  determined  not  only  by  the  speed  of 
the  response  necessary  to  represent  the  unique  values  at  each  pixel 
but  also  the  absolute  sensitivity  of  the  device.  In  this  section  we 
will  consider  a  figure  of  merit,  the  achievable  S/N  of  the  PMT,  to 
demonstrate  how  the  threshold  sensitivity  of  a  device  is  analyzed 
and  predicted.  This  is  not  the  only  figure  of  merit  that  could  be 
used  for  detectors.  At  the  outset  we  suggested  that  confocal 
microscopy  might  examine  light  with  Xs  from  the  UV  to  the  near- 
infrared  (IR).  At  the  longer  Xs,  the  performance  is  limited  by  the 
background  level  of  IR  radiation.  Comparisons  of  alternative 
figures  of  merit  for  detector  function  can  be  found  in  Seib  and  Auk- 
erman  (1973).  In  general,  noise  sources  will  either  be  additive  or 
multiplicative.  Additive  noise,  for  example,  is  the  noise  added  by 
the  process  of  photoemission  at  the  photocathode  to  the  noise 
already  present  in  the  original  photon  flux.  We  can  perform  a 
straightforward  analysis  of  the  behavior  of  a  detector  in  terms  of 
the  degradation  of  the  S/N  (Robben,  1971). 


The  detection  process  is  limited  at  the  front  end  by  the  S/N  of 
photon  flux.  If  the  average  rate  of  emission  of  photons  is  7P,  then 
during  the  observation  interval  X  the  average  observation  is  np  = 
/pi,  and  the  variance  is  given  by  op2  =  7px  =  np.  The  S/N  is  given 
by:  S/N  =  np/ap  =  (7px)lc  =  (wp)1/2. 

For  any  photon  incident  on  the  photocathode,  the  quantum  effi¬ 
ciency  (QE  or  QE)  is  the  product  of  three  probabilities.  The  first  is 
the  probability  that  a  photon  will  be  absorbed  rather  than  trans¬ 
mitted  or  reflected.  The  second  is  the  probability  that  this  photon 
will  produce  a  free  electron.  The  third  stands  for  the  probability 
that  the  free  electron  will  actually  reach  the  surface  of  the  mater¬ 
ial  and  escape.  Therefore,  for  all  real  materials,  QE  <  1.  Because 
the  probability  of  release  of  a  photoelectron  per  incident  photon  is 
related  by  the  quantum  efficiency,  QE,  then  wpe  =  QE  7px.  The  vari¬ 
ance  of  the  number  of  photoelectrons,  ope2  =  QE  7px.  Consequently, 
at  the  stage  of  emission  of  photoelectrons  from  the  photocathode, 
we  can  reduce  the  signal-to-noise  ratio  to  (S/N)pe  =  (QE  7px)1/2.  The 
only  way  to  improve  on  this  S/N  would  be  to  use  a  material  with 
the  highest  possible  quantum  efficiency,  such  as  a  NEA  material. 

During  any  recording  period,  x,  thermionic  electrons  will  be 
emitted.  The  average  number  of  these  emissions  is  nd,  and  their 
variance,  ope2  =  /?d.  In  all  practical  cases,  the  S/N  of  photoemission 
is  determined  from  the  total  emission,  which  is  a  combination  of 
the  terms  due  to  photoelectrons  and  thermionic  electrons.  The 
number  of  photoevents,  npe  =  nc  -  nd  =  QE  7px,  where  c  and  d  indi¬ 
cate  composite  and  dark  terms,  respectively.  Because  the  photo¬ 
events  and  the  thermal  ones  are  independent,  the  square  of  the 
variances  add,  ope2  =  QE  7x  +  nd.  Thus,  in  the  presence  of  dark 
noise,  the  S/N  of  a  realizable  PMT  is  given  by  (S/N)pe  =  (QE 
7px)/(QE  7px  +  «d)1/2. 

Now  consider  what  happens  if  we  interpose  j  dynode  stages 
all  having  identical  gains,  5j,  and  Poisson  statistics  of  secondary 
emission.  The  gain  seen  at  the  anode  will  be  m  =  51  •  52  .  .  .  5j. 
The  anode  variance,  oa2  =  QE  npm2[{  1  +  1/(5  -  1)}  •  { 1  -  (l/5j)}]. 
If,  for  example,  we  use  negative  affinity  materials  and  in  each  stage 
the  gain,  5,  is  large,  then  term  l/5j  will  be  very  small.  Thus, 
Ga2  =  QE  npm2  [1  +  1/(5  -  1)].  The  corresponding  S/N  is  given  by 
S/N  =  [QE  np(5  -  1  )/5] 1/2,  which,  if  the  gains  are  high,  reduces  to 
S/N  =  [QE  «p]1/2. 

Thus,  for  any  device  with  high  gain  at  each  stage,  the  perfor¬ 
mance  will  be  dominated  by  the  quantum  efficiency  of  the  first 
stage  or  perhaps  by  the  photocathode  and  the  statistics  of  the  first 
dynode.  In  terms  of  current,  the  anode  (S/N)a  at  the  anode  for  a 
PMT  is  given  for  two  cases  in  Table  12.3.  The  first  is  for  the  current 
alone,  and  the  second  includes  the  noise  contributed  by  the  output 
circuit  noise  in  the  amplifier.  A  complete  analysis  using  generat¬ 
ing  functions  to  predict  the  total  noise  of  a  detector  system  based 
on  additive  and  multiplicative  noise  sources  is  given  in  Jorgensen 
(1984)  and  Prescott  (1966). 

It  should  be  noted  that  the  implementation  of  spectroscopic 
imaging  demanded  the  development  of  multi-channel  detectors, 
and  as  is  the  case  for  the  Zeiss  510  Meta,  a  multi-channel  PMT. 
In  array  detectors  such  as  the  Hamamatsu  H7260,  32  photomulti¬ 
plier  channels  are  arrayed  in  parallel  within  a  single  glass  enve¬ 
lope.  The  face  of  each  channel  of  the  PMT  is  a  scant  0.8  by  7  mm, 
on  a  1mm  pitch,  and  the  10  dynodes  occupy  a  linear  array  less 
than  15  mm  in  extent.  As  implemented,  the  accelerating  voltage 
between  photocathode  and  the  first  dynode  is  equal  to  the  voltage 
drop  between  subsequent  dynodes,  so  the  QE  and  noise  advantage 
obtained  by  having  the  greatest  acceleration  at  the  first  stage 
between  the  cathode  and  dynode  is  lost.  In  more  conventional 
single-channel  PMTs,  the  acceleration  voltage  at  this  first  stage  is 
typically  three  times  that  of  subsequent  stages.  Consequently,  all 
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other  things  being  equal,  we  would  expect  single-channel  PMTs, 
to  have  better  multiplicative  noise  performance  than  multi-channel 
arrays,  if  simply  because  the  larger  physical  spacing  between 
dynodes  permits  higher  acceleration  voltages  at  the  initial  stage. 

A  second  issue  concerning  multi-channel  arrays  used  for  quan¬ 
titative  spectroscopy,  while  not  concerning  individual  photon 
noise,  per  se,  arises  from  possible  cross-talk,  and  the  variations 
in  quantum  efficiency,  amplification,  and  noise  characteristics 
between  channels.  When  a  single  detector  is  used  to  characterize 
the  spectrum  of  a  fluorescent  molecule,  narrow  filtering  selects 
photons  of  different  energies  to  play  sequentially  across  the  surface 
of  the  detector.  Once  the  spectral  characteristics  of  a  given  PMT 
are  known,  this  information  can  be  used  to  correct  the  experimen¬ 
tally  determined  spectra  of  interest.  The  requirements  for  multi¬ 
channel  PMTs  are  no  less  exacting,  and  in  principle  the  standard 
curve  for  each  channel  of  the  PMT  needs  to  be  determined.  During 
manufacture  variations  on  the  order  of  10%  in  the  mean  quantum 
efficiency  and  amplification  are  obtained.  Obviously,  because  the 
photons  in  narrow  energy  bands  will  be  directed  to  different  detec¬ 
tors  simultaneously  in  such  a  scheme,  then  to  accurately  determine 
the  spectrum  requires  that  the  characteristics  of  each  detector 
channel  have  been  measured,  and  this  information  is  used  to  nor¬ 
malize  the  raw  data  collected  along  the  spectrum  (see  Chapter  36, 
this  volume,  for  a  simple  test). 


Representing  the  Pixel  Value 

In  the  majority  of  confocal  microscopes,  the  image  is  constructed 
sequentially  from  the  output  of  a  point  detector.  Ideally,  the  digi¬ 
tized  value  at  each  pixel  should  reflect  the  average  detector 
response  during  the  time  the  beam  dwells  on  a  point  in  the  field. 
Two  problems  must  be  considered.  First,  can  we  define  a  suitable 
means  of  sampling  the  response  of  the  detector  over  the  period 
used  to  estimate  a  pixel  value?  Second,  because  the  goal  of  con- 
focal  microscopy  is  to  achieve  the  maximum  resolution,  how  do 
we  avoid  degrading  the  image  with  spurious  correlation  between 
the  sampled  values  at  neighboring  pixels?  Ideally  the  measure  of 
intensity  would  be  an  average  of  the  detector  response  over  the 
time  of  interest.  For  ease  of  manufacture  and  implementation, 
however,  such  a  scheme  is  often  approximated  by  the  use  of  a 
capacitive  integration  [Fig.  12.6(A)].  The  time  constant,  T,  of  such 
an  integrator  is  often  chosen  to  be  1/4  the  pixel  dwell-time.  Such 
an  integrator  has  the  advantage  that,  in  some  sense,  it  represents  a 
running  average  of  the  detector  output.  Unfortunately,  it  can  be 
shown  (Jones  et  al.,  1971)  that  this  form  of  integration  results  in 
a  2%  correlation  between  neighboring  pixels.  (If  the  Nyquist  band¬ 
width,  as  defined  in  Chapter  4,  this  volume ,  has  been  correctly 
chosen  to  be  1/27TT,  the  signal  will  correlate  between  adjacent 
pixels  by  the  same  amount.)  This  correlation  can  be  avoided  by 
the  use  of  true  integration  of  the  detector  signal,  whether  this  be 
in  photon  counting  [Fig.  12.6(B,C)]  or  analog  detection  schemes 
[Fig.  12.6(D)].  Other  advantages  of  full  integration  will  be 
considered  specifically  with  photon  counting  and  with  analog 
detection. 

At  low  light  levels  (<108  photons/s),  direct  photon-counting 
strategies  have  advantages  over  analog  methods  of  light  intensity 
estimation.  The  large  gain  inherent  in  most  PMT  designs  is  itself 
a  distinct  advantage  in  many  photon-counting  techniques  because 
these  tubes  produce  large  pulses  that  are  well  above  other  sources 
of  noise  in  the  tube  or  in  the  subsequent  electronics.  In  PMTs 
designed  and  selected  for  photon  counting,  the  pulses  produced  by 
single-photon  events  are  tightly  distributed  and  widely  separated 
from  the  amplitude  of  pulses  produced  by  multiple  photon  events. 
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FIGURE  12.6.  Methods  of  photon  quantification.  (A)  RC  integration.  Charge 
or  current  from  photodetector  is  changed  into  a  voltage,  and  a  running  average 
is  stored  on  a  capacitor.  The  resistor  in  parallel  should  insure  that  the  voltage 
across  the  capacitor  decays  with  a  time  constant  appropriate  for  the  pixel  dwell- 
time,  but  as  this  changes  with  scan  zoom  setting,  the  condition  is  not  always 
met.  (B)  Analog  method  of  photon  counting.  During  alternate  pixels,  the  upper 
and  the  lower  half  of  the  schematic  circuit  are  used.  During  odd  pixels,  charge 
pulses  generated  by  detected  photons  are  stored  on  Q.  At  the  end  of  the  pixel, 
S3  is  opened  and  the  value  on  capacitor  Q  is  converted  to  a  digital  word.  This 
value  is  held  for  the  duration  of  the  pixel  and  then  the  capacitor  is  reset.  While 
this  conversion  is  taking  place,  the  photon  pulses  for  the  next  even  pixel  are 
accumulating  on  capacitor  C2,  and  the  process  is  repeated.  (C)  Digital  method 
of  photon  counting.  ECL  or  TTL  logic  is  used  to  count  the  photon  pulses  and 
output  is  by  way  of  digital  word  or  DAC  conversion  back  to  a  video  voltage. 
(D)  Direct  integration  method  used  to  store  an  analog  output  voltage.  The 
circuit  assumes  that  subsequent  digital  processing  of  the  signal  is  used  to  nor¬ 
malize  the  response  with  respect  to  pixel  dwell-time. 


It  is  possible  with  modest  acceleration  voltages  to  produce  well- 
defined  charge  packets  that  develop  millivolt  pulses  of  nanosec¬ 
ond  duration  across  a  50Q  load.  Further,  removal  of  any  variation 
in  pulse  amplitude  at  the  load  resistor  can  be  accomplished  by 
comparing  the  height  of  the  pulse  across  the  resistor  with  two  well- 
defined  thresholds.  The  lower  threshold  defines  the  minimum 
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height  of  a  pulse  considered  to  be  due  to  a  single  photoelectron 
emitted  from  the  photocathode,  multiplied  along  the  dynode  chain, 
and  collected  by  the  anode.  The  upper  threshold  is  greater  than  the 
pulse  height  produced  by  a  majority  of  single  photoelectron  events, 
and  pulses  exceeding  this  height  are  likely  to  represent  multiple 
photon  events  or  other  artifacts.  The  output  of  this  two-stage 
detection  scheme  can  be  used  to  trigger  a  well-defined  and  tightly 
controlled  pulse  of  known  amplitude  and  duration.  A  minimum 
interval  between  pulses  can  be  imposed  to  eliminate  possible  PMT 
artifacts  such  as  those  produced  by  correlated  after-pulsing.  Typi¬ 
cally,  pulses  as  short  as  15  ns  with  a  dead  time  between  pulses  of 
5  ns  can  be  used.  Both  Schottky  transistor-transistor  logic  (TTL) 
and  emitter-coupled  logic  (ECL)  families  are  fast  enough  to  count 
at  the  implied  maximum  rate  of  50  MHz. 

True  integration  of  standard  pulses  can  be  accomplished  as 
shown  in  Figure  12.6(B,C).  In  the  analog  method,  pulses  are 
directly  integrated  on  a  capacitor.  Prior  to  integration,  the  capaci¬ 
tor  is  shorted  to  ground  by  a  switch.  During  the  integration,  the 
shorting  switch  is  open.  In  response  to  a  series  of  input  pulses,  the 
voltage  measured  across  the  capacitor  rises  in  a  staircase  fashion, 
with  the  rise  at  each  step  determined  by  the  size  of  the  capacitor 
and  the  magnitude  and  duration  of  the  input  pulses.  At  the  end  of 
the  integration  period,  the  final  analog  voltage  is  converted  to  a 
digital  value  using  an  analog-to-digital  converter  (ADC),  and  the 
capacitor  is  then  reset.  A  duplicate  capacitor  and  switch  are 
arranged  so  that  the  capacitors  integrate  on  alternate  pixels, 
avoiding  the  loss  of  photon  counts  during  ADC  and  capacitor 
resetting. 

In  the  digital  method,  the  pulses  increment  a  TTL  or  ECL 
counter.  At  the  end  of  the  integration  period,  the  count  is  latched 
in  an  output  register.  The  counter  is  reset  and  begins  accumulat¬ 
ing  counts  for  the  next  pixel.  Additional  logic  can  be  configured 
to  eliminate  the  loss  of  photon  counts  during  output  latching.  The 
output  of  the  digital  latch  can  be  used  directly  as  an  input  for  the 
image  buffer,  or  the  value  can  be  converted  back  into  a  standard 
video  voltage  with  a  digital-to-analog  converter  (DAC).  One 
attraction  of  the  digital  technique  is  that  once  the  PMT  pulses  have 
been  converted  to  logic  levels,  the  process  of  integration  with 
digital  counters  is  immune  to  noise  problems  inherent  in  further 
analog  processing.  For  either  method  of  photon  counting,  with 
pixel  dwell-times  on  the  order  of  a  microsecond,  we  would  expect 
at  most  a  few  tens  of  photons  to  be  counted.  The  small  number  of 
distinct  signal  levels  represented  by  these  photons  could  easily  be 
converted  to  the  range  of  standard  video  between  0  and  0.75  V  and 
used  as  input  to  a  typical  8-bit  video  ADC  and  frame  grabber.  Such 
a  device  would  convert  each  voltage  to  a  number  between  zero  and 
255. 

For  high  light  levels  (>108  photons/s)  that  might  be  typical  of 
reflectance  confocal  microscopy  as  used  in  metallurgy  or  in  the 
inspection  of  integrated  circuits,  the  extreme  measures  needed  to 
maximize  the  S/N  under  low  light  levels  are  unnecessary.  A  variety 
of  different  diode  detectors  with  current-to-voltage  converters  can 
be  used  as,  in  these  detectors,  each  photon  absorbed  deposits  an 
identical  amount  of  charge.  In  these  examples,  the  output  is  con¬ 
tinuous  rather  than  discrete.  Traditionally,  low-pass  filtering  of  the 
form  of  Figure  12.6(A)  (x  =  T/4)  is  used  to  create  a  running  average 
of  the  light  intensity.  The  difficulty  of  pixel  correlation  remains 
whenever  pixels  are  bigger  than  resels.  It  can  be  avoided  by  cre¬ 
ating  a  technique  similar  to  the  analog  method  of  integrating  pulses 
in  the  photon  detection  scheme  [Fig.  12.6(D)].  With  the  exception 
of  the  fact  that  the  input  to  the  capacitors  is  a  continuous  voltage 
rather  than  a  series  of  discrete  pulses,  the  technique  and  the  process 
is  identical  to  the  previous  case  in  Figure  12.6(B),  though  the 


results  will  be  somewhat  degraded  by  the  fact  that  some  photons 
contribute  larger  pulses  to  the  total  signal  than  others. 

For  either  discrete  or  continuous  estimates  of  luminance,  the 
full  integration  technique  is  preferred  over  bandwidth  limitation 
because  it  avoids  the  correlations  inherent  in  using  a  running 
average.  A  second  advantage  is  the  ease  with  which  different  rates 
of  confocal  scanning  can  be  implemented.  With  capacitive  inte¬ 
gration,  the  time  constant  of  the  integrator  must  be  changed  to 
match  different  rates  of  scan  and  pixel  dwell-time  and  also  to  com¬ 
pensate  for  changes  in  the  relative  sizes  of  pixels  and  resels.  With 
true  integration,  the  match  is  automatic  because  the  integration 
begins  and  ends  at  times  defined  by  a  pixel  dwell-time.  A  third 
advantage  of  this  technique  is  that  the  relative  variance  of  the 
intensity  estimate  using  full  integration  is  half  that  obtained  using 
capacitive  integration  with  x  =  T/4  (Jones  et  al.,  1971).  Conse¬ 
quently,  for  any  desired  level  of  precision,  full  integration  will 
require  4x  fewer  frames  for  an  average.  This  fact  is  important  both 
in  the  speed  of  data  acquisition  and  in  minimizing  photodamage 
to  the  sample. 

CONVERSION  TECHNIQUES 

The  resolution  of  the  analog-to-digital  conversion  required  for 
confocal  microscopy  is  determined  in  part  by  the  detector.  For  the 
photon-counting  example  above,  all  possible  pixel  values  could 
be  described  uniquely  with  an  8-bit  binary  word.  In  full-field 
microscopy,  however,  a  converter  would  need  the  resolution  of  a 
14-bit  binary  word  to  take  advantage  of  the  linearity  and  large 
dynamic  range  of  slow-scan  CCD  arrays.  In  essence,  an  ADC 
needs  to  have  sufficient  resolution  to  avoid  degrading  the  S/N 
inherent  in  the  conversion  of  the  signal  up  to  that  point.  The  choice 
of  converter  is  further  restricted  by  the  rate  at  which  conversion  is 
required.  In  general,  confocal  microscopy  is  characterized  by  a  rel¬ 
atively  high  degree  of  resolution  and  high  rates  of  digitization. 

Of  the  types  of  converters  available,  only  flash  and  successive 
approximation  converters  (Fig.  12.7)  will  be  considered.  The  flash 
converter  uses  multiple  comparators  and  simultaneously  compares 
the  analog  input  with  all  analog  code  levels.  The  speed  of  conver¬ 
sion  is  limited  only  by  comparator  speed  and  the  propagation 

A  B 


FIGURE  12.7.  Converter  types.  (A)  Parallel  A/D  converter  in  which  the 
analog  input  is  simultaneously  compared  with  four  reference  levels.  The  delay 
through  the  system  is  determined  by  the  logic  delay,  and  conversion  is  very 
rapid.  (B)  Successive  approximation  A/D  converter.  On  each  clock  cycle  the 
digital  input  to  the  DAC  is  toggled  starting  with  the  most  significant  bit.  The 
output  of  the  DAC  is  compared  with  the  analog  input  and  if  the  DAC  voltage 
is  greater,  the  test  bit  is  set  to  zero.  The  next  significant  bit  is  then  tested  in 
turn  until,  for  an  n-bit  conversion,  after  n  cycles  the  closest  n-bit  approxima¬ 
tion  to  the  analog  input  is  found.  Since  n  clock  cycles  are  used,  the  time  used 
with  this  method  of  conversion  may  be  relatively  long. 
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FIGURE  1 2.8.  Aperture  time  versus  frequency  of  a  full-scale  sinusoidal  input 
for  system  accuracies  to  within  one-half  of  one  least-significant  bit  error. 


delays  through  the  logic.  The  successive  approximation  technique, 
on  the  other  hand,  operates  on  a  feedback  principle.  The  analog 
input  and  the  output  of  a  DAC  are  compared  with  a  single  com¬ 
parator.  The  DAC  is  controlled  by  the  output  of  a  digital  register 
whose  bits  are  changed  successively  until  the  closest  possible 
match  is  achieved  between  the  DAC  output  and  the  analog  input. 
This  technique  is  relatively  time  consuming,  and  conversion  often 
takes  as  much  as  a  microsecond. 

The  rate  of  conversion  and,  therefore,  the  time  window  during 
which  a  conversion  takes  place,  is  of  prime  importance  when 
choosing  the  method  and  resolution  of  the  converter.  During  the 
time  that  the  conversion  is  taking  place,  it  is  critical  that  the  input 
change  by  less  than  the  amount  coded  by  the  least  significant  bit. 
If  the  input  changes  by  more  than  this  amount,  the  conversion  will 
be  incorrect.  This  problem  arises  due  to  the  uncertainty  in  the  time 
aperture  during  which  the  conversion  is  performed.  For  the  true 
integration  schemes  illustrated  above,  the  problem  does  not  arise 
because  the  output  of  the  integrators  is  arranged  to  be  constant  for 
a  large  percentage  of  the  pixel  interval,  and  all  that  is  required  is 
that  the  conversion  take  place  within  this  interval.  For  all  the  tech¬ 
niques  that  use  a  variant  of  the  capacitive  integration,  the  voltage 
will  be  changing  during  the  period  of  conversion.  In  the  capaci¬ 
tive  integration  we  chose  a  x  equal  to  1/4  the  pixel  dwell-time.  For 
sampling  periods  of  1  ps,  x  =  250  ns,  the  integrator  is  equivalent  to 
an  RC  filter  with  half-power  bandwidth,  or  3  dB  point,  of  roughly 
640  kHz.  It  is  sobering  to  realize  that  to  convert  a  sinusoidal 
voltage  accurately  to  8-bit  resolution,  at  this  frequency,  the  sam¬ 
pling  aperture  uncertainty  would  need  to  be  shorter  than  1  ns.  The 
relationship  between  the  desired  resolution  of  the  conversion,  the 
frequency  of  a  sinusoidal  input,  and  the  necessary  sampling  aper¬ 
ture  is  given  in  Figure  12.8. 

To  minimize  problems  created  by  the  conversion  aperture, 
either  the  converter  must  be  extraordinarily  fast  or  an  additional 
component  that  samples  and  holds  the  input  signal  must  be  inter¬ 
posed  between  the  detector  and  the  converter.  Such  devices  are 
similar  to  our  integrator  schemes  in  that  they  sample  the  analog 
input  on  a  capacitor  for  a  brief  interval,  and  then  the  capacitor  is 
disconnected  from  the  input  during  the  period  that  the  conversion 
takes  place.  Consequently,  the  problems  of  rapidly  sampling  the 
input  are  transferred  from  the  converter  to  the  sampling-and- 
holding  device.  The  ideal  device  for  confocal  microscopy  must  not 
only  have  a  very  short  aperture  uncertainty,  but  must  also  obtain 
the  input  voltage  on  the  sampling  capacitor  to  the  desired  resolu¬ 
tion  within  a  small  fraction  of  the  pixel  dwell-time. 


Because  the  output  of  this  conversion  is  only  certain  to  within 
1/2  of  a  least  significant  bit,  the  converter  can  be  viewed  as  an 
additional  source  of  uncertainty  or  noise.  With  an  8-bit  converter, 
this  noise  is  relatively  small  compared  to  the  sources  mentioned 
previously.  This  is  a  necessary  result  of  the  finite  resolution  of  the 
converter.  In  general,  the  dynamic  range  of  the  converter  should 
be  matched  to  the  dynamic  range  of  the  detector.  For  example,  if 
the  S/N  in  the  detector  output  is  1000,  or  60 dB,  then  the  converter 
output  should  be  at  least  10  bits  to  permit  an  equivalent  resolution 
(see  also  Chapter  4,  this  volume). 

ASSESSMENT  OF  DEVICES 

Now  that  a  number  of  laser-scanning  and  disk-scanning  confocal 
microscopes  have  been  reduced  to  practice  and  made  available  in 
the  marketplace,  it  is  proper  to  consider  how  well  the  implemented 
detection  systems  have  been  optimized.  We  shall  consider  one  pos¬ 
sible  detector  scheme  for  each  type  of  microscope:  the  PMT  for 
point  detection  and  the  CCD  for  full-field  detection. 

Point  Detection  Assessment  and  Optimization 

Of  current  devices  available  for  point  detection,  the  PMT  in 
photon-counting  mode  is  the  most  suitable  for  low-light  tasks  with 
fluxes  of  fewer  than  10s  photons/s  and  X  <  500nm.  Most  present 
microscopes  using  PMTs  convert  the  anode  current  into  a  voltage 
and  use  this  output  as  a  measure  of  the  incident  illumination.  Gain 
can  be  achieved  in  such  a  system  by  varying  the  acceleration 
voltage  along  the  dynode  chain,  thereby  increasing  the  photoelec¬ 
tron  multiplication  within  the  tube  [Fig.  12.9(B)].  However,  unfor¬ 
tunately,  with  progressively  larger  acceleration  voltages,  there  is  a 
tendency  for  the  internal  noise  of  the  PMT  to  increase.  This  is  par¬ 
ticularly  noticeable  under  low  or  absent  light  conditions  when, 
with  sufficient  acceleration,  the  output  of  the  headstage  amplifier 
will  display  large-amplitude  noise  pulses.  At  lower  voltages  this 
noise  source  will  be  seen  to  disappear.  This  is  a  simple  demon¬ 
stration  in  the  analog  mode  that  the  PMTs  have  a  preferred  oper¬ 
ating  range  beyond  which  the  noise  of  the  device  increases  and  the 
S/N  decreases. 

Most  of  the  noise  sources  associated  with  anode  current  mea¬ 
surement  can  be  avoided  by  using  photon  counting.  In  this  mode, 
S/N  is  maximized  by  finding  and  using  the  optimal  setting  for  the 
acceleration  voltage.  Ideally,  such  operations  are  performed  using 
a  pulse  height  analyzer  to  characterize  the  amplitude  of  pulses  at 
the  anode  both  in  the  dark  and  under  low  light  conditions.  Though 
such  analyzers  are  rarely  part  of  a  typical  microscopy  laboratory, 
an  alternate  strategy  can  be  employed.  For  pulse  counting,  the 
threshold  detectors  are  first  fixed  at  some  nominal  value,  say  1  mV, 
for  the  lower  detector.  The  output  of  the  discriminator  circuit  can 
be  counted  digitally.  Shown  in  Figure  12.9(A)  is  a  comparison  of 
the  number  of  counts  versus  acceleration  voltage  acquired  under 
no-light  and  two  low-light  conditions  for  the  Bio-Rad  MRC  500. 
In  the  dark  with  the  laser  off,  the  count  rate  rises  gradually  and 
then  plateaus  over  an  extended  range  (open  boxes).  Even  at  accel¬ 
eration  voltages  in  excess  of  1  kV,  there  are  fewer  than  400  events 
detected  by  a  Thom  C604  amplifier/discriminator.  Under  low  light 
conditions  with  the  laser  on  and  without  neutral  density  filters 
attenuating  the  beam,  the  count  rate  versus  acceleration  voltage 
was  determined  under  two  conditions.  In  the  first  case  the  count 
was  taken  without  a  specimen  in  the  object  plane  (filled  triangles). 
Ideally,  in  this  case  no  light  would  be  reflected  back  through  the 
system  and  onto  the  PMT.  Apparently,  significant  stray  light  is 
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FIGURE  12.9.  (A)  Behavior  of  Bio-Rad  PMT  (Thom  9828b) 
versus  voltage.  Confocal  laser  head  mounted  on  a  Zeiss  UEM 
with  a  40x,  NA  0.75  water-immersion  objective.  A  Thorn  C604 
head  amplifier  was  used  to  generate  and  detect  1  mV  pulses.  The 
head  amplifier  output  consisted  of  15  ns  ECL  pulses  with  5  ns 
deadtime.  Responses  were  counted  with  the  laser  off,  with  the 
laser  on  and  no  object  in  the  field,  and  with  a  FITC- stained  spec¬ 
imen.  Fewer  than  400  events  were  counted  in  the  dark.  The 
number  of  events  versus  PMT  acceleration  voltage  increased 
without  an  object  in  the  plane  of  focus  when  the  laser  was  ener¬ 
gized  and  the  beam  entered  the  confocal  head  producing  stray 
light.  The  number  of  counts  increased  again  when  a  FITC- 
stained  object  was  in  the  plane  of  focus.  The  maximum  S/N  of 
near  60  was  achieved  at  PMT  voltages  between  750  and  850  V. 
On  our  Bio-Rad  head,  this  corresponds  to  gain  position  between 
5.5  and  6.5,  but  these  exact  readings  may  vary  from  instmment 
to  instrument.  (B)  Gain  versus  acceleration  voltage  for  PMT. 
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scattered  back  through  the  system  and  is  detected  even  in  the 
absence  of  a  specimen.  When  a  fluoroscein  isothiocyanate  (FITC)- 
stained  specimen  was  placed  in  the  object  plane  of  the  microscope, 
the  number  of  signal  photons  detected  (filled  circles)  was  always 
greater  than  the  background  count.  Perhaps  because  of  the  scat¬ 
tered  light,  the  S/N  of  detection  was  a  maximum  of  60  at  relatively 
low  acceleration  voltages  between  750  and  850V.  The  S/N 
dropped  dramatically  for  higher  accelerations  near  1  kV.  For  most 
PMTs,  the  curve  under  low  light  conditions  is  similar  to  the  curve 
recorded  in  the  dark,  but  displaced  towards  higher  counts.  There¬ 
fore,  in  general  there  is  a  particular  voltage  for  each  photomul¬ 
tiplier  for  which  the  separation  between  the  signal  and  the 
background  noise  is  the  greatest.  The  tube  should  be  operated  at 
this  voltage.  Operation  at  higher  voltages  will  degrade  the  S/N  and, 
in  early  models  of  the  Bio-Rad  500,  degradation  of  the  S/N  is  com¬ 
pounded  by  the  additional  problem  of  scattered  light  within  the 
confocal  head.  Figure  12.9(B)  plots  the  gain  versus  the  accelera¬ 
tion  voltage.  However,  using  a  fixed,  optimal  PMT  voltage  pro¬ 
duces  a  requirement  for  variable  gain  in  the  pre-ADC  amplifier  so 
that  the  signal  output  level  matches  the  input  of  the  ADC. 

The  output  of  the  headstage  PMT  amplifier  is  often  used  as 
input  to  a  video-rate  frame-buffer  running  in  a  slow-scan  mode. 
The  transmission  and  digitization  stage  can  often  be  another  source 
of  problems.  In  one  commercial  microscope,  we  noticed  that  in 
response  to  a  step  change  from  white  to  black,  the  detector/digiti¬ 
zation  system  produced  a  series  of  damped  oscillations  or  sec¬ 
ondary  bright  levels.  The  problem  could  be  eliminated  by  insuring 
that  the  transmission  cable  terminates  with  a  matching  impedance 
at  the  input  of  the  video-board  digitization  stage. 

As  mentioned  earlier,  problems  arise  from  using  a  capacitive 
integration  technique  and  the  consequent  wide  variation  in  output 
that  occurs  under  certain  conditions.  Consider  the  difference  in 
response  that  results  when  a  photon  is  incident  on  the  PMT  at  the 
beginning  and  at  the  end  of  a  pixel  dwell-time.  Under  full  inte¬ 
gration,  the  output  in  either  case  would  be  the  same,  and  the  result 
would  be  proportional  to  a  single  photon.  Under  capacitive  inte¬ 
gration,  however,  the  voltage  resulting  from  the  photon  that  arrived 
at  the  beginning  of  the  pixel  dwell-period  would  have  decayed  to 
2%  of  its  value  by  the  time  that  the  pixel  value  was  sampled.  By 
comparison,  the  photon  arriving  just  before  the  detector  voltage 
was  sampled  would  be  at  full  amplitude.  Consequently  the  output 
of  such  a  detector/digitization  scheme  carries  with  it  a  random 
variation  of  the  output  dependent  on  the  time  of  arrival  of  the  inci¬ 
dent  photons  during  the  integration  period.  This  increase  in  noise 


will  often  prevent  achieving  a  satisfactory  S/N  with  an  acceptable 
number  of  image  scans. 

The  behavior  of  the  PMT  amplifier  and  conversion  system  can 
be  tested  directly  by  connecting  a  constant-current  pulse  genera¬ 
tor  capable  of  producing  low-level  steady  currents  and  pulses  com¬ 
parable  to  those  produced  by  the  PMT  with  optimal  acceleration 
voltage.  If  a  series  of  pulses  comparable  to  single  photon  events 
are  produced  at  a  constant  rate  near,  but  not  equal  to,  the  inverse 
of  the  pixel  dwell-time,  then  capturing  an  image  frame  will  reveal 
the  method  used  to  integrate  during  the  pixel.  If  true  integration  is 
used,  then  the  response  can  be  one  of  three  levels,  with  each  pixel 
having  a  value  equivalent  to  zero,  one,  or  two  pulses.  Often  it  is 
convenient  to  map  these  values  to,  say,  0,  127,  or  255  for  an  8-bit 
frame  grabber.  On  the  other  hand,  if  capacitive  integration  is  used, 
the  pixel  values  will  assume  all  values  between  0  and  255.  Such  a 
result  would  suggest  that  the  detection  and  digitization  scheme 
would  converge  more  slowly  towards  an  unbiased  estimate  of  the 
luminance  at  the  measured  point  than  the  full  integration  method. 

Field  Detection  Assessment  and  Optimization 

The  line-scanning  or  field-scanning  confocal  microscopes  require 
multi-element  line  or  full-field  detection  systems  rather  than  point 
detectors.  As  mentioned  in  the  Introduction,  the  eye  is  often  a  suit¬ 
able  device  but  suffers  under  very  low  light  conditions.  Alterna¬ 
tively,  one  could  use  the  normal  types  of  field  detectors  currently 
used  in  video  microscopy.  A  recent  review  of  the  field  is  available 
(Inoue  and  Spring,  1997).  Many  of  the  present-generation  imaging 
devices  are  inappropriate  due  to  the  speed  of  the  response.  In  addi¬ 
tion,  in  many  of  the  most  sensitive  tubes  there  is  significant  lag  or 
carry-over  between  the  image  recorded  during  one  frame  scan  and 
the  next.  Alternatively,  a  cooled  CCD  camera  could  be  used  as  the 
detector,  given  its  large  dynamic  range  and  extraordinarily  high 
quantum  efficiency.  As  noted  above,  the  difficulty  that  arises  with 
CCD  devices  is  the  generation  of  a  noise-equivalent  photoelectron 
signal  due  to  thermal  generation  during  the  integration  time,  to 
variations  in  the  charge-coupled  transfer  efficiency,  or  to  random 
variations  in  the  conversion  at  the  output.  To  be  sure,  this  noise 
can  be  minimized  and,  for  some  CCD  chips  with  slow  scan 
readout,  the  noise  can  be  reduced  to  a  few  photon  equivalents 
(rms).  However,  this  still  yields  peak  noise  (5a)  readings  of  nearly 
60  photon  equivalents  in  the  dark.  Such  a  figure  is  not  ideal  under 
low  light  conditions.  In  older  CCDs,  a  further  problem  arose  from 
difficulties  in  transferring  the  signal  through  the  charge-coupled 
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process  when  one  of  the  detection  sites  contains  too  few  photo¬ 
electrons.  In  this  case,  transfer  along  the  device  may  be  interrupted 
until  sufficient  charge  accumulates  from  adjacent  pixels  in  the 
column  to  permit  transfer.  This  problem  is  likely  to  be  exacerbated 
for  fluorescent  images  in  which  there  are  likely  to  be  large  numbers 
of  pixels  without  any  photon  hits  during  the  integration  period. 

Because  the  distance  between  detector  elements  in  a  2D  CCD 
is  on  the  order  of  10  pm,  it  is  possible  in  principle  to  add  a  gain 
stage  by  interposing  a  microchannel  plate  (such  as  the  Hamamatsu 
HPK  R2809,  built  with  6  pm  elements)  in  front  of  the  CCD  (Kume 
et  al.,  1988).  A  modest  gain  of  100  would  bring  us  clearly  above 
the  noise  floor  of  the  CCD  and,  because  these  CCDs  can  store  as 
many  as  5  X  105  electrons  in  a  well,  each  would  still  be  able  to 
detect  5000  photoevents  before  becoming  saturated.  Unfortu¬ 
nately,  because  the  MCP  is  manufactured  with  S 1 ,  multi-alkali,  and 
bialkali  photocathodes,  the  quantum  efficiency  of  the  composite 
device  will  be  much  less  than  that  of  the  CCD.  In  addition,  because 
of  statistical  variations  in  MCP  gain,  all  photons  will  no  longer 
deposit  identical  charge  in  the  sensor.  However,  a  benefit  of  such 
a  strategy  is  that  the  MCP  can  be  gated  on  and  off  and  perhaps 
phased-locked  to  periods  following  an  illuminating  pulse.  In  this 
manner,  the  kinetics  of  the  response  to  illumination  could  be  fol¬ 
lowed  on  a  nanosecond  time  scale.  The  response  to  such  an  exper¬ 
iment  could  be  integrated  simultaneously  over  the  entire  field,  and 
after  sufficient  repetitions  the  value  could  be  digitized  in  the  usual 
fashion  (see  Chapter  27,  this  volume). 

A  more  straightforward  approach  to  providing  gain  and  reduc¬ 
ing  the  dark  count  in  CCDs  has  resulted  in  cameras  for  full-field 
work  that  have  high  quantum  efficiency,  extremely  low  production 
of  thermal  electrons,  and  black  levels  reminiscent  of  PMTs.  Such 
electron-multiplying  CCDs,  or  EM-CCDs,  have  been  commer¬ 
cially  produced  by  both  E2V  (Chelmsford,  UK),  and  Texas  Instru¬ 
ments  (Houston,  TX,  USA),  under  the  names  L3Vision  and 
Impactron,  respectively.  These  devices  incorporate  a  silicon-based 
electron  multiplication  stage  in  addition  to  a  conventional  CCD 
structure.  The  electron  multiplication  is  provided  by  the  addition 
of  a  multiple-well  gain  register  that  is  similar  to  the  conventional 
shift  register  (see  Figs.  2.15  and  10.11,  this  volume),  except  that 
the  second  phase  electrode  is  replaced  by  two  electrodes,  one  at  a 
fixed  potential,  and  the  other  clocked  at  a  voltage  of  40  to  50  V,  a 
value  much  higher  than  necessary  to  transfer  charge.  The  result¬ 
ing  intense  electric  field  is  sufficiently  high  to  cause  impact  ion¬ 
ization  as  the  electrons  are  transferred,  and  rather  modest 
multiplication  by  a  factor  of  1.01  to  1.015  per  stage  can  be 
achieved.  Repeated  over  a  few  hundred  wells,  however,  this  mul¬ 
tiplication  is  raised  to  the  power  of  the  number  of  wells  in  the  reg¬ 
ister.  The  resulting  gain  in  the  Marconi  device,  with  591  wells,  is 
on  the  order  of  6600.  Clearly,  such  gain  is  sufficient  to  overcome 
the  readout  noise  that  is  the  main  noise  factor  at  transfer  rates 
greater  than  1  MHz  necessary  for  full-frame  microscopy.  Sufficient 
cooling  is  also  used  in  these  cameras  to  reduce  the  thermally  gen¬ 
erated  background  to  a  few  electrons  per  frame,  and  the  result  is 
the  creation  of  an  image  at  low  light  levels  that  has  the  potential 
to  combine  the  low  background  inherent  in  PMTs  with  the 
quantum  efficiency  possible  with  present  generation  CCDs. 

There  is  one  problem  with  this  picture:  the  charge  amplifica¬ 
tion  process  is  very  noisy  in  the  sense  that  not  all  the  electrons  in 
the  packet  are  amplified  by  the  same  amount.  The  multiplicative 
noise  that  this  causes  produces  an  uncertainty  in  the  measurement 
of  the  signal  in  the  charge  packet  that  is  very  similar  in  magnitude 
and  form  to  that  caused  by  Poisson  noise.  As  the  two  noise  terms 
are  added  as  the  square  root  of  the  sum  of  the  squares,  the  total 
noise  becomes  1 .4x  greater  than  it  would  have  been  if  each  elec¬ 


tron  in  the  packet  were  amplified  by  the  same  amount;  alterna¬ 
tively.  the  total  noise  is  the  same  as  if  only  50%  of  the  electrons 
in  the  original  packet  were  counted  perfectly.  What  this  means 
practically  is  that  the  EM-CCD  operates  as  though  the  read  noise 
is  essentially  zero  but  the  “effective  QE”  is  only  50%  of  the  actual 
QE. 

A  second  less  fundamental  problem  relates  to  the  fact  that  all 
current  EM-CCD  chips  use  frame  transfer  readout.  This  means 
that,  at  the  end  of  the  exposures,  the  accumulated  charge  pattern 
is  translated  past  the  light  field  of  its  way  to  the  read  register,  a 
process  that  can  cause  streaking  unless  the  light  is  turned  off,  either 
by  using  a  shutter  or  by  turning  off  the  microscope  illumination. 

As  with  all  new  devices,  one  expects  a  learning  curve  with 
respect  to  fully  understanding  the  inherent  noise.  Two  other 
sources  of  noise  that  became  apparent  only  when  the  EM-CCD 
“abolished”  read  noise  are  those  due  to  clocking  and  charge  trans¬ 
fer.  Even  in  conventional  CCDs,  spurious  electrons  are  produced 
every  100  or  so  transfers,  and  clocking  induced  charge,  or  CIC,  is 
rather  low  compared  to  read  noise.  In  the  EM-CCD  special  care 
must  be  taken  with  clock  amplitudes  and  edges  to  minimize  this 
effect,  given  the  high  acceleration  voltages  in  the  gain  register.  The 
other  source  of  noise  in  the  image  results  from  the  expected  charge 
transfer  efficiency  (CTE)  problems  that  result  when  wells  are  filled 
with  50,000  to  100,000  electrons  and  a  small  amount  of  charge  is 
left  behind  during  each  transfer.  This  results  in  a  streaking  in  the 
image,  most  apparent  in  fluorescent  images  where  often  one 
encounters  high  intensity  pixels  surrounded  by  black.  Under  such 
conditions  the  problem  can  be  ameliorated  by  reducing  the  gain, 
because,  at  high  pixel  intensities,  only  modest  gain  is  necessary  to 
overcome  either  the  thermal  noise  or  that  due  to  high-frequency 
readout. 


DETECTORS  PRESENT  AND  FUTURE 

In  conclusion,  we  will  consider  two  general  topics.  The  first  is  the 
means  of  optimizing  the  performance  of  this  generation  of  confo- 
cal  microscopes.  The  second  is  the  specification  of  the  qualities  of 
the  ideal  detector  to  use  in  confocal  microscopy.  Our  goal  is  to 
produce  images  with  the  least  possible  illumination  of  the  speci¬ 
men.  In  terms  of  constructing  an  image,  this  requires  that  the  signal 
being  recorded  be  limited  only  by  the  uncertainties  of  photon  flux. 
From  the  analysis  of  the  contributions  of  noise  sources,  the  S/N  is 
seen  to  be  degraded  at  each  step  in  the  conversion  process.  As  was 
shown  in  the  section  on  the  statistics  of  the  photon  flux  and  the 
detector,  the  first  point  of  departure  from  the  ideal  when  using  a 
PMT  as  the  detector  is  the  fact  that  the  quantum  efficiency  of  real¬ 
izable  PMTs  is  much  less  than  unity.  Consequently,  the  emission 
of  the  photoelectron  is  itself  a  second  probabilistic  event  in  the 
cascade,  and  the  S/N  is  reduced  by  (QE)1/2,  where  QE  is  the 
quantum  efficiency.  The  S/N  could  be  enhanced  simply  by  increas¬ 
ing  the  QE  of  the  photocathode  and  the  gain  of  the  first  dynode  in 
the  multiplication  chain  and,  towards  this  goal,  it  is  common  (but 
sadly  not  universal)  to  place  a  zener  diode  between  these  elec¬ 
trodes  so  that  the  gain  of  this  step  is  high  regardless  of  the  total 
PMT  voltage  used. 

The  S/N  in  PMTs  could  be  enhanced  further  by  cooling  the 
tubes  to  reduce  the  effects  of  thermionic  emission,  by  running  the 
cathode  at  near  ground  potential  to  reduce  the  regenerative  effects 
due  to  collisions  between  electrons  and  the  tube  wall,  by  reducing 
the  effective  cathode  area  and  focusing  of  the  photoelectron  beam, 
and,  finally,  by  reducing  the  number  of  stages  in  the  dynode  chain 
and  increasing  the  gain  at  each  stage.  All  of  these  argue  for  the  use 


Photon  Detectors  for  Confocal  Microscopy  •  Chapter  12  263 


of  more  efficient  negative  affinity  materials,  such  as  gallium- 
arsenide  (GaAs)  and  gallium-arsenide-phosphide  (GaAsP),  a 
feature  that  would  also  move  us  towards  the  use  of  the  less- 
damaging,  longer  Xs.  These  maneuvers  would  also  decrease  the 
dispersion  in  the  time-of-flight  of  the  photoelectron  cloud  pro¬ 
duced  by  a  single  incident  photon,  increasing  the  temporal  resolu¬ 
tion  of  the  detector  and  making  it  easier  to  perform  studies  on 
fluorescence  kinetics  (Chapter  27,  this  volume). 

Many  present  confocal  microscopes  use  rather  crude  tech¬ 
niques  to  create  a  running  average  of  the  photon  flux.  Perhaps  the 
two  most  straightforward  improvements  of  the  devices  for  use  in 
biological  or  low-light  measurements  would  be  the  following. 
First,  one  should  eliminate  the  use  of  anode  current  measurements 
and  the  associated  capacitive  integration  methods  of  pixel  inten¬ 
sity  estimation.  Second,  one  should  avoid  the  inappropriate  use  of 
the  dynode  acceleration  voltage  as  a  means  of  varying  the  inten¬ 
sity  of  the  image.  The  S/N  degradation  produced  by  using  these 
methods  is  too  severe  to  warrant  their  continual  use.  The  alterna¬ 
tive  to  adjusting  the  gain  in  this  way  is  to  increase  the  incident 
light  levels  or  to  use  variable  digital  or  electronic  gain.  As  other 
authors  in  this  volume  point  out,  increasing  the  light  level  can  often 
result  in  severe  photodamage  to  the  specimen  and  bleaching  of  the 
photoprobe,  again  resulting  in  an  inability  to  create  a  meaningful 
image  in  a  minimum  number  of  successive  scans. 


A  convenient  solution  both  to  the  problems  of  pixel  convolu¬ 
tion  and  to  problems  inherent  in  PMT  operation  can  be  imple¬ 
mented  simply  by  using  photon  counting.  The  dark  counts  of 
photon-counting  tubes  are  often  below  100  photoelectrons/s,  and 
in  an  image  that  has  perhaps  106  pixels/s,  this  contribution  is  neg¬ 
ligible.  Thus,  integrating  over  the  pixel  interval,  the  S/N  will  be 
limited  by  the  photon  statistics  and  the  QE  of  the  cathode  mater¬ 
ial.  The  spurious  noise  due  to  regenerative  effects  and  breakdown 
within  the  tube  when  run  at  high  acceleration  voltages  can  be  elim¬ 
inated.  These  effects  can  be  demonstrated  by  comparing  the  aver¬ 
aged  images  collected  under  extremely  low  light  conditions  using 
the  anode  current  measurement  provided  with  the  confocal  micro¬ 
scope  to  a  technique  of  direct  photon  counting.  Figure  12.10 
demonstrates  the  S/N  problem  viewed  as  a  measurement  task  after 
the  primary  detector.  In  Figure  12.10(A),  the  signal  is  presumably 
buried  somewhere  within  the  extraordinary  broadband  noise  in  the 
trace.  Because  this  is  the  input  to  the  ADC  stage,  the  signal  will 
be  sampled  at  the  end  of  each  pixel  as  an  estimate  of  the  pixel 
value.  The  result  of  averaging  512  frames  acquired  using  this  tech¬ 
nique  is  shown  in  Figure  12.10(C).  By  comparison,  the  16  levels 
from  the  photon-counting  circuit  are  shown  in  Figure  12.10(B). 
The  result  of  averaging  128  images  acquired  using  only  10%  as 
much  illumination  is  shown  in  Figure  12.10(D).  From  the  view¬ 
point  of  capturing  a  useful  image,  photon  counting  is  to  be  pre- 


FIGURE  12.10.  Comparison  of  the  use  of  Bio-Rad  MRC  500  confocal  microscope  in  anode  current  measurement  and  photon-counting  modes  at  low  light 
levels.  Pedal  ganglion  of  Aplysia  stained  with  antibody  raised  to  pedal  peptide  (Pep)  conjugated  with  FITC.  (A)  Background  noise  using  anode  current  mea¬ 
surement  apparent  in  the  512  scans  of  the  single  line  indicated  in  (C).  Successive  scans  displaced  vertically.  (B)  Output  of  photon-counting  circuit  for  the  field 
line  highlight  in  (D)  demonstrating  relatively  noise-free  operation.  (C)  Average  of  512  fields  using  anode  current  measurement  technique.  Display  restricted  to 
values  between  160  and  255  and  expanded  to  the  full  dynamic  range  of  the  video  memory  before  photographing.  (D)  Average  of  128  fields,  at  1/1 0th  the  light 
level  of  (C),  using  photon  counting.  Sixteen  quantal  levels  expanded  to  the  full  dynamic  range  of  the  monitor  before  photographing.  Scale  bar  =  100pm. 
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ferred.  Furthermore,  because  in  this  photon-counting  unit  the 
detection  circuitry  saturates  at  107  photoelectrons/s  (or  4  x  107 
photons/s),  the  likelihood  of  bleaching  the  fluorescent  probe  is 
very  low. 

In  the  future,  we  should  consider  enhancing  the  quantum  effi¬ 
ciency  of  the  detectors.  PMTs  are  presently  limited  to  QE  =  to 
levels  near  35%  for  blue  light  but  QE  <3%  for  rhodamine.  An 
alternative  approach  would  be  to  examine  further  the  feasibility  of 
using  the  solid-state  detectors  normally  used  in  CCD  arrays  as  the 
primary  detector.  In  the  present  generation  CCD  devices,  QE 
approaches  80%  well  into  the  near  IR,  and  this  fact  alone  suggests 
that  the  solid-state  direct  photodetectors  will  ultimately  come 
closest  to  achieving  a  S/N  in  the  detected  signal  that  is  limited  by 
the  statistics  of  the  photon  flux.  Presently,  the  sources  of  noise  in 
realized  devices  are  those  due  to  thermal  generation  both  in  trans¬ 
fer  along  the  readout  registers  and  in  resetting  the  readout  capac¬ 
itor.  The  second  source  of  uncertainty  is  due  to  the  readout  noise. 
Because  the  readout  of  the  voltage  due  to  a  packet  of  electrons  can 
be  made  non-destructively,  it  is  possible  to  shift  each  packet  down 
a  normal  charge-coupled  shift  register  and  to  make  multiple  read¬ 
ings  along  the  register  as  the  packet  passes  by.  In  this  way,  esti¬ 
mates  of  the  pixel  value  would  increase  in  precision  with  the 
square  root  of  the  number  of  estimates  performed  for  each  packet 
and  with  no  apparent  cost.  Alternatively,  the  EM-CCD  approach 
makes  almost  zero  read  noise  available  now,  albeit  at  the  cost  of 
a  50%  drop  in  QE. 
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Structured  Illumination  Methods 

Rainer  Heintzmann 


INTRODUCTION 

In  this  chapter,  I  discuss  methods  of  obtaining  optical  section  data 
using  methods  other  than  standard  confocal  microscopy.  These 
techniques  have  the  potential  of  being  more  light  efficient,  faster, 
and  of  providing  better  resolution  than  standard,  unprocessed  con- 
focal  microscopy. 

Confocal  microscopy  depends  on  an  uneven  distribution  of  the 
illumination  on  the  sample,  and  this  is  accomplished  by  directing 
the  beam  to  a  single  focus  that  is  then  scanned.  The  resulting  signal 
is  detected  pointwise  (voxel-by- voxel).  In  other  systems,  the  focus 
is  multiplexed  and  many  points  are  scanned  by  a  spinning  disk 
(Petran  et  al.,  1968;  Bewersdorf  et  al.,  1998;  Andresen  et  al.,  2001 ; 
Majoul  et  al.,  2002)  or  by  the  two-dimensional  regular  movement 
of  a  rectangular  mask  (Verveer  et  al.,  1998;  Egner  et  al.,  2002). 

Multiplexing  takes  advantage  of  the  fact  that  many  photode¬ 
tectors  operate  in  parallel  to  increase  data  throughput.  In  addition, 
by  spreading  the  illumination  over  many  pixels,  parallel  readout 
avoids  problems,  such  as  singlet-state  fluorescence  saturation,  that 
can  limit  the  data  rate  from  single-beam  confocal  microscopes. 

Multiplexed  confocal  systems  operating  with  single  photon 
excitation  (Ichihara  et  al.,  1996;  Verveer  et  al.,  1998)  employ  a 
structured  mask  (e.g.,  a  pattern  of  linear  or  circular  holes  in  a 
opaque  plane  or  a  pattern  of  microlenses  on  a  rotating  disk)  for 
generating  structured  illumination  as  well  as  spatially  filtering  the 
fluorescent  or  scattered  light  emitted  by  the  sample  before  it 
reaches  the  detector.  The  confocal  signal  is  usually  obtained  by 
integrating  this  spatially  filtered  signal  on  the  charge-coupled 
device  (CCD)  camera. 

In  contrast,  the  structured-illumination  methods  discussed  here 
are  based  on  the  acquisition  of  a  set1  of  individual  images  at  a  given 
focus  plane,  each  made  with  a  different  position  of  an  illumination 
mask,  but  made  with  no  mask  in  the  detection  beam  path,  that 
is,  widefield  detection  (Fig.  13.1).  The  detection  pinholes  can  then 
be  defined  computationally  in  order  to  produce  a  confocal-like 
image  from  this  set  of  images  (as  performed  in  Fig.  13.4).  Other 
ways  of  processing  the  data  to  yield  optically  sectioned  images  are 
also  presented  below. 

Some  advantages  and  disadvantages  of  this  strategy  are: 

1.  Because  a  full  image2  is  recorded  for  each  position  of  the 
illumination  mask,  a  much  larger  amount  of  data  needs  to  be 
acquired  per  reconstructed  slice.  In  this  sense,  patterned  illumina¬ 
tion  with  widefield  detection  also  constitutes  a  “scanning”  system 


1  A  set  is  a  series  of  images  made  at  one  focus  plane  but  with  different  illumi¬ 
nation  conditions. 

2  Or  at  least  a  substantial  amount  of  signal  originating  from  near  the  positions 

of  the  illumination  spots. 


and  sample  movement  must  be  avoided  during  the  acquisition 
time.  Taking  multiple  images  can  be  a  disadvantage  in  speed,  but 
as  modern  CCD  cameras  employ  fast  readout  systems  with  negli¬ 
gible  readout  noise  (e.g.,  with  electron-multiplier  amplification), 
this  does  not  pose  a  major  problem. 

2.  This  large  amount  of  data  offers  more  flexibility  for  the 
treatment  of  the  data.  In  contrast  to  standard  confocal  microscopy, 
it  is  unnecessary  to  choose  a  pinhole  diameter  during  data  acqui¬ 
sition  with  all  the  advantages  and  disadvantages  of  this  choice. 
Once  you  have  chosen  the  illumination  pattern,  the  trade-off 
between  sectioning  strength  (small  hole)  and  better  signal-to-noise 
ratio  (large  hole)  can  be  dealt  with  after  data  acquisition  (see  Fig. 
13.4)  or  be  avoided  entirely  during  data  processing. 

3.  In  most  structured-illumination  systems,  the  multi-spot 
approach  distributes  the  power  of  the  illumination  light  across  mul¬ 
tiple  positions.  In  a  single-spot  confocal  system,  fluorescence  sat¬ 
uration  fundamentally  limits  the  amount  of  collectable  fluorescent 
light  one  can  excite  per  unit/time  and  thus  defines  a  minimum  time 
required  for  the  acquisition  of  an  adequate  fluorescence  image. 
Given  illumination  sources  of  sufficient  brightness,  the  simultane¬ 
ous  utilization  of  one  thousand  spots  can  potentially  shorten  the 
acquisition  time  substantially.3 

4.  An  advantage  of  standard  confocal  scanning  systems  is  that 
the  scan  is  relatively  localized.  Thus,  sample  movement  only 
causes  major  image  distortion  if  the  feature  actually  being  scanned 
moves  by  an  amount  that  is  large  compared  to  the  beam  size.  In 
contrast,  patterned  illumination  systems  with  widefield  detection 
require  that  each  part  of  the  object  be  stable  within  the  limits  of 
the  achievable  resolution  throughout  the  acquisition  of  an  entire 
set  of  raw  data  images. 

EXPERIMENTAL  CONSIDERATIONS 

For  the  sake  of  simplicity,  an  identical  schematic  setup  (Fig.  13.1) 
will  be  assumed  throughout  this  chapter.  In  this  setup,  the  sample 
is  illuminated  through  an  illumination  mask  [e.g.,  Fig.  13.2(A-C)], 
which  can  be  shifted  by  well-defined  displacement  to  produce  a  set 
of  regularly  spaced  light  patterns.  At  each  position  of  the  illu¬ 
mination  mask,  an  image  of  the  sample  is  detected  with  a  CCD 
camera  at  a  conjugate  image  plane,  usually  maintaining  a  fixed,  in¬ 
plane  position  with  respect  to  the  sample.  The  common  features  of 
all  structured-illumination  arrangements  are  that  the  sample  is  illu¬ 
minated  with  structured  light,  and  fluorescence  or  the  scattered 


3  The  amount  by  which  the  total  acquisition  time  is  shortened  depends  on  how 
many  images  must  be  obtained  at  each  plane,  a  factor  that  varies  with  the 
sparcity  of  the  mask  used. 
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FIGURE  13.1.  Possible  experimental  setup  for  acquiring  structured- 
illumination  data  in  the  widefield  fluorescence  detection  mode.  The  illumina¬ 
tion  mask  is  projected  by  the  microscope  optics  onto  the  specimen  in  the  sample 
plane,  thereby  defining  the  illumination  pattern.  The  fluorescence  light  from 
the  sample  is,  however,  directly  imaged  onto  the  CCD  camera  without  any 
mask  in  the  detection  path. 


light  is  detected  in  a  widefield-type  arrangement  without  the 
use  of  any  detection  mask.  The  various  structured-illumination 
methods  differ  greatly  in  how  the  pattern  of  light  is  generated  (e.g., 
by  interference  patterns  or  even  using  a  single-spot  laser  illumina¬ 
tion)  and  also  in  the  detection  system  [e.g.,  either  widefield  or  de¬ 
scanning  with  detection  of  a  small  area  near  the  illumination  spot 
(Bertero  et  al.,  1984;  Bertero  et  al.,  1989;  Sheppard  and  Cogswell, 
1990;  Pawley  et  al.,  1996)].  Below,  I  describe  different  pattern  gen¬ 
eration  techniques  along  with  possible  sources  of  errors. 

Pattern  Generation 

Patterns  of  illumination  can  be  generated  by  placing  the  illumina¬ 
tion  mask  at  the  plane  of  the  field  stop  in  the  illumination  path  of 
the  microscope  (Fig.  13.1).  The  mask  can  be  a  diffraction  grating, 
or  a  spatial  light  modulator  (SLM)  such  as  a  programmable  liquid 
crystal,  a  digital  mirror,  or  a  liquid-crystal-on-silicon  (LCOS) 
device,  and  can  be  illuminated  either  by  incoherent  light  (Neil  et 
al.,  1997;  Heintzmann  and  Cremer,  1999a)  or  by  a  coherent  laser 
(e.g.,  as  in  Lanni  et  al.,  1993  or  Gustafsson,  2000). 

For  the  optimal  signal-to-noise  ratio  at  the  highest  in-plane  res¬ 
olution,  it  is  advantageous  to  produce  the  highest  possible  contrast 
(degree  of  modulation)  of  the  light  pattern  in  the  sample  plane.  The 


FIGURE  13.2.  Different  pattern  masks  (top  row)  along 
with  the  illumination  patterns  (rows  below)  generated  in  the 
plane  of  focus.  (A)  Regular  spaced  dots;  (B)  illumination 
dots  placed  on  a  hexagonal  unit  cell  such  as  often  used  in 
spinning  disk  systems;  (C)  Line-grid.  The  unit-cell  length 
D  is  indicated.  (D-F)  show  the  illumination  pattern  as  gen¬ 
erated  by  incoherent  imaging  of  (A-C)  at  the  sample  plane. 
(G-I)  correspond  to  coherent  imaging  with  suppression  of 
the  zero-order  in  the  diffraction  pattern.  Note  the  improved 
degree  of  modulation,  yielding  a  darker  background. 
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light  diffracted  from  a  coherently  illuminated  grid  can  be  focused 
to  well-defined  diffraction  orders.  To  achieve  the  maximum  degree 
of  modulation,  it  can  be  advantageous  to  block  the  zero  diffraction 
order  (stemming  from  non-diffracted  light)  with  the  help  of  a 
central  beam  stop  at  an  appropriate  position  in  the  illumination 
path.  It  is  also  possible  to  generate  illumination  patterns  by  inter¬ 
fering  beams  of  coherent  light  that  have  been  separated  by  beam¬ 
splitters  (Lanni  et  al.,  1986;  Lanni  et  al.,  1993;  Frohn  et  al.,  2000). 
However,  in  this  case  mechanical  stability  becomes  a  major 
concern  because  the  relative  length  of  the  two  beam  paths  formed 
by  the  splitter  must  remain  stable  to  subwavelength  precision 
during  the  time  needed  to  record  an  entire  set  of  images. 

When  working  with  microscopes  that  possess  high  aperture 
angles  in  the  sample  plane,  it  is  important  to  keep  the  vector  nature 
of  light  in  mind  (Fig.  13.3).  Beams  converging  onto  the  focus  plane 
at  an  angle  of  90°  with  respect  to  each  other  do  not  generate  an 
intensity  interference  pattern  in  p-polarization  (meaning  the  polar¬ 
ization  vector  is  in  the  plane  defined  by  the  two  interfering  beams), 
whereas  the  interference  of  s-polarized  beams  (polarization  vector 
perpendicular  to  the  plane  that  contains  the  two  interfering  beams) 
results  in  optimum  mask  contrast.4  At  a  high  numerical  aperture, 
this  consideration  makes  line-grid  diffraction  masks  [e.g..  Fig. 
13.2(C)]  with  s-polarized  illumination  (polarization  vector 
parallel  to  the  lines)  preferable  to  two-dimensionally  structured 
masks. 

The  ideal  source  of  this  light  is  less  clear.  Because  laser  light 
is  both  polarized  and  very  bright,  it  may  seem  an  obvious  choice. 
If  a  grating  or  SLM  is  used  to  generate  an  array  of  lines,  the  zero- 
order,  undiffracted  beam  can  be  easily  discriminated  and  removed 
to  achieve  the  maximum  possible  modulation  at  the  focus  plane. 
Furthermore,  laser  light  can  be  coupled  into  the  microscope  very 
efficiently  even  at  high  diffraction  angles  and,  in  combination  with 
holographic  techniques,  it  can  be  used  to  generate  sparse  patterns 
without  a  severe  loss  in  total  power.  However,  at  present,  laser  light 
is  only  available  at  certain  wavelengths  and  these  do  not  always 
match  the  absorption  spectra  of  the  dyes  of  choice.  In  addition,  the 
high  lateral  coherence  of  laser  light  can  cause  strong  interference 
effects  between  the  main  beams  generating  the  structure  and  light 
reflected  by  various  optical  surfaces,  particularly  the  water-glass 
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FIGURE  13.3.  Influence  of  the  vector  nature  (polarization/electric  field)  of 
light  at  high  numerical  aperture.  The  interfering  electric  field  vectors  ( bold 
arrows )  perpendicular  to  the  light  rays  ( thin  arrows)  can  even  be  perpendicu¬ 
lar  to  each  other  at  ^-polarization,  and  an  aperture  half-angle  of  45°,  as  dis¬ 
played  here,  leads  to  no  intensity  modulation  in  the  sample  plane.  In  contrast, 
v-polari/ation  with  the  electric  field  vector  pointing  towards  the  reader  would 
yield  two  parallel  vectors  and  thus  100%  contrast  in  the  intensity  pattern  gen¬ 
erated  by  constructive  and  destructive  interference. 


4  Whereas  p-polarized  rays  that  converge  at  a  relative  angle  of  40°  are  still 
capable  of  interfering  well  enough  to  produce  90%  of  full  contrast. 


surfaces  of  the  coverslip  and  the  front  surface  of  any  non¬ 
immersion  objective.  Although  lateral  coherence  can  be  scrambled 
in  a  number  of  ways  (as  discussed  in  Chapter  6,  this  volume), 
implementing  these  strategies  adds  so  much  complexity  and  cost 
that  they  have  so  far  been  used  only  in  homemade  systems. 

Arc  sources  usually  are  less  expensive,  less  coherent,  and 
much  less  bright  than  laser  sources  (see  Chapter  6,  this  volume). 
However,  they  emit  over  a  very  wide  range  of  wavelengths  and, 
particularly  at  the  peaks  of  the  Hg  emission  lines,  they  are  bright 
enough  for  many  studies.  Because  arcs  can  be  considered  as  lon¬ 
gitudinally  and  transversally  incoherent,  stray  interference  fringes 
are  seldom  a  factor. 

Their  problem  is  insufficient  brightness.  If  the  illumination 
system  is  adjusted  so  that  it  only  uses  the  light  emerging  from  a 
small  area  of  the  source,  the  blocking  of  the  zero  order  would  still 
be  possible.  However,  doing  so  reduces  the  intensity  so  much  that 
long  exposures  are  needed  to  maintain  a  decent  signal-to-noise 
ratio  in  each  image  in  the  data  set.  To  make  matters  worse,  sparse 
illumination  patterns  can  obstruct  more  than  90%  of  the  incident 
light  striking  the  pattern  mask.  Finally,  as  sheet  polarizers  trans¬ 
mit  only  about  one  third  of  the  incident  light,  it  is  a  matter  of  some 
interest  as  to  whether  or  not,  given  a  fixed  exposure  time,  the 
increased  contrast  of  the  pattern  is  offset  by  the  lower  signal  level 
caused  by  the  use  of  the  polarizer.  As  a  rule  of  thumb,  unless  it  is 
essential  to  collect  data  very  rapidly,  the  3x  lower  illumination 
intensity  that  occurs  when  you  use  a  sheet  polarizer  is  recom¬ 
mended  for  aqueous  specimens  whenever  the  objective  numerical 
aperture  (NA)  is  >0.4,  using  high  spatial  frequency  gratings  for 
in-plane  resolution  enhancement,  or  >0.8,  when  using  lower  spatial 
frequency  gratings  for  optical-sectioning  applications. 

On  the  other  hand,  if  one  uses  an  arc  source  to  project  a  pattern 
large  enough  so  that  its  image  is  only  marginally  affected  by  dif¬ 
fraction,  one  can  use  an  illumination  system  that  utilizes  light  from 
a  larger  area  of  the  source  and  the  photon  flux  becomes  sufficient 
for  rapid  optical  sectioning  using  patterned  illumination.  Indeed, 
the  Zeiss  ApoTome  is  such  a  device.  In  the  future,  it  may  be  pos¬ 
sible  to  use  an  array  of  high-power,  light-emitting  diodes  (LEDs) 
as  a  bright,  incoherent  source  of  patterned  illumination. 

The  choice  of  the  illumination  pattern  density  very  much 
depends  on  the  type  of  sample  as  well  as  on  the  intended  method 
of  data  processing.  A  comparably  thick  sample  and/or  a  sample 
with  volume-like  staining  is  more  difficult  to  process  when  using 
densely  patterned  illumination  because  the  large  amount  of  out-of- 
focus  fluorescence  will  dominate  the  small  amount  of  modulated 
fluorescence  stemming  from  the  focal  plane  (see  Appendix,  this 
chapter).  Although  using  a  sparse  pattern  reduces  this  signal-to- 
noise  problem,  it  always  requires  the  acquisition  of  more  raw  data 
images  to  produce  a  single  set.  This  implies  a  longer  image  acqui¬ 
sition  time.  An  elegant  way  to  achieve  sufficient  flexibility  and  to 
optimize  this  trade-off  between  relatively  noise-free  sectioning  and 
acquisition  speed  is  to  generate  the  pattern  using  a  programmable 
array  (Verveer  et  al.,  1998;  Hanley  et  al.,  1999;  Heintzmann  et  al., 
2001;  Fukano  and  Miyawaki,  2003). 

Out-of-focus-light  can  be  reduced  by  closing  the  field 
diaphragm  (Hiraoka  et  al.,  1990)  and  this  technique  may  be  very 
valuable.  However,  when  using  non-local  reconstruction  methods 
(e.g.,  the  Fourier-space  based  approaches)  the  illumination  pattern 
is  assumed  to  be  periodic  at  all  out-of-focus  positions.  A  small  field 
diaphragm  violates  this  assumption  and  may  lead  to  problems 
during  Fourier-based  reconstruction. 

The  illumination  pattern  can  be  displaced  relative  to  the 
sample  by  either  translating  the  mask  by  a  well-defined  distance 
(Neil  et  al.,  1997;  Gustafsson,  2000,  2005),  by  reprogramming  the 
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pattern  of  a  programmable  diffraction  (amplitude  or  phase)  mask 
(Verveer  et  al.,  1998;  Hanley  et  al.,  1999;  Heintzmann  et  al.,  2001; 
Fukano  and  Miyawaki,  2003),  by  altering  the  relative  phases  of 
interfering  beams  (e.g.,  by  splitting  the  beam  and  repositioning  a 
piezo-actuated  mirror  in  one  of  the  beam  paths)  (Lanni  et  al. ,  1993; 
Frohn  et  al.,  2000;  Failla  et  al.,  2002),  or  by  translating  the  sample 
past  a  fixed  pattern,  followed  by  a  computational  correction  to 
account  for  this  sample  movement  (Heintzmann  and  Cremer, 
1999a).  In  the  latter  case  appropriate  interpolation  kernels  or 
Fourier-space  based  resampling  approaches  (leading  to  a  sine 
kernel)  must  be  used  to  reduce  interpolation-induced  artifacts 
(Yaroslavsky,  2003)  that  occur  as  features  on  the  object  move 
across  the  pixels  of  the  CCD. 

Promising  attempts  to  replace  the  confocal  pinhole  by  a  mul¬ 
tiple  element  detector  (Bertero  et  al.,  1984;  Barth  and  Stelzer, 
1994)  with  fast  readout  (Sheppard  and  Cogswell,  1990;  Pawley  et 
al.,  1996)  also  belong  to  the  same  category  of  structured  illumi¬ 
nation  with  somewhat  wideheld-like  detection.5  The  data  can  thus 
be  treated  with  methods  similar  to  the  descriptions  given  below,  as 
long  as  the  descanning  is  accounted  for. 


COMPUTING  OPTICAL  SECTIONS  FROM 
STRUCTURED-ILLUMINATION  DATA 

The  data  acquired  with  a  setup  similar  to  the  one  shown  in  Figure 
13.1  consists  of  a  z-series  of  sets  of  images.  Each  set  is  taken  at 
one  focus  position  and  each  member  of  a  set  is  taken  at  a  differ¬ 
ent  position  of  the  illumination  light  pattern.  The  data  from  each 
set  is  first  processed  to  yield  an  optically  sectioned  image  using 
one  of  the  methods  described  below. 

Most  methods  for  deriving  an  optical  section  from  a  set  of 
structured-illumination  images  try  to  estimate  the  degree  of 
modulation  at  each  pixel.  For  simplicity,  in  the  case  of  a  one¬ 
dimensional  grid  [e.g.,  Fig.  13.2(C),  top  row  in  Fig.  13.7,  p.  272, 
Fig.  13.11,  p.  278]  it  is  assumed  that  N  images  are  acquired,  each 
with  the  pattern  shifted  by  1  IN  with  respect  to  the  replicative  unit 
cell.6  In  the  top  row  of  Figure  13.7,  the  interleaved  positions  of  the 
three  illumination  patterns  are  indicated  on  the  left  side  of  each 
image  for  N  =  3.  As  displayed  in  Figure  13.11,  those  parts  of  the 
sample  that  are  out  of  focus  are  more  homogeneously  illuminated. 
Light  emitted  from  these  parts  will  undergo  an  additional  blurring 
when  imaged  onto  a  detector  conjugate  to  the  in-focus  plane.  Thus, 
in  contrast  to  the  in-focus  light,  light  from  out-of-focus  areas  will 
exhibit  very  little  modulation  upon  variation  of  the  jc-positions  of 
the  excitation  pattern.  Computing  the  degree  of  modulation  over 
the  multiple  images,  in  a  pixel-by-pixel  fashion,  will  permit  the 
discrimination  of  the  non-modulated,  out-of-focus  from  the 
modulated,  in-focus  information. 

The  degree  of  modulation  can  be  calculated  locally,  consider¬ 
ing  only  the  modulation  at  a  single  pixel  position  over  time,  by 
various  approaches: 

1.  Dodt  (Dodt,  1990;  Dodt  and  Becker,  2003)  suggests  emu¬ 
lating  “synthetic  pinholes”  by  summing  up  the  thresholded  images 


5  For  speed  reasons,  the  suggested  detector  arrays  are  usually  relatively  small 
(e.g.,  5  X  5  elements),  yielding  a  confocal  operation  “bias”  for  thick  specimens. 

6  The  unit  cell  is  defined  as  the  smallest  (vectorial)  translation  that  can  be 

applied  to  the  pattern  to  reproduce  its  structure  (e.g.,  the  distance  D  indicated 
in  C,  Fig.  13.2). 


of  a  series  of  data  acquired  by  transmission  imaging  with  struc¬ 
tured  illumination.  This  technique  has  been  successfully  applied  to 
the  transmission  infrared  (IR)  imaging  of  onion  skin  (Dodt  and 
Becker,  2003)  and  300|dm  unstained,  freshly  prepared  thick  rat 
hippocampal  slices  in  which  single  spines  on  dendrites  could  be 
resolved  throughout  the  sample  (40x,  NA  0.8  objective  at  780 nm, 
Dodt  et  al.,  2001).  The  advantage  of  this  approach  is  that,  while 
the  optical  aberrations  induced  by  such  samples  render  the  opera¬ 
tion  of  a  standard  confocal  microscope  in  transmission  close  to 
impossible,  the  data  processing  strategy  employed  is  very  adaptive 
to  optical  aberrations  and  thus  is  able  to  yield  a  useful  image  con¬ 
taining  information  mainly  from  the  focal  slice. 

2.  Benedetti  and  co-workers  (Benedetti  et  al.,  1996)  deter¬ 
mined  the  reconstructed  slice  7rec  by  calculating  the  difference 
between  maximum  and  minimum  measured  intensity  I \  in  each 
pixel  i:  [Eq.  1,  see  also  Fig.  13.7(B)].  This  approach7  is  robust  with 
respect  to  various  artifacts  that  often  arise  in  imaging  such  as 
readout  noise  and  especially  fixed-pattern  noise  from  the  CCD. 
This  noise,  which  usually  does  not  vary  systematically  over  time 
but  rather  randomly  from  pixel  to  pixel,  is  efficiently  eliminated, 
as  is  scattered  light  from  out-of-focus  planes  and  from  anywhere 
in  the  optical  path.  It  is  observed  that  this  approach  generates  spu¬ 
rious  patterns  if  the  number  of  scanning  steps  is  too  low  for  the 
size  of  the  illumination  pattern  as  is  seen  by  the  residual  horizon¬ 
tal  stripes  visible  in  Figure  13.7(B).  However,  it  should  be  noted 
that  the  selected  width  for  this  pattern  was  extraordinarily  wide 
and  only  three  steps  were  chosen.  For  a  denser  pattern  or  an 
increased  number  of  scanning  steps,  the  visible  performance  of  this 
method  is  similar  to  approach  3  [Fig.  13.7(C)],  which  is  discussed 
below. 

If  the  illumination  pattern  is  sufficiently  sparse  [i.e.,  the  spot- 
to-spot  or  line-to-line  distance  D  in  Figure  13.2(C)  is  large  enough 
or,  more  generally,  if  the  ratio  of  open  to  opaque  area,  the  so-called 
mark/space  ratio,  is  low],  simply  computing  the  maximum  (Eq.  2) 
yields  fairly  good  optical  sections  (Bendedetti  et  al.,  1996),  albeit 
without  the  background  suppression  advantages  of  Eq.  1.  These 
approaches  have  been  applied  to  epi-fluorescence  microscopy 
(Benedetti  et  al.,  1996)  and  transmission  IR  imaging  (Dodt  and 
Becker,  2003).  By  Eq.  3  (termed  super-confocal),  a  further  increase 
in  optical  sectioning  quality  is  obtained.  However,  Eqs.  2  and 
3  yield  satisfying  results  only  for  data  acquired  using  sparse 
illumination. 

3.  Other  ways  of  determining  the  degree  of  modulation  (Eqs. 
4,  5)  are  described  by  Neil  (Neil  et  al.,  1997).  Equation  4  is  based 
on  square-law  detection  [Fig.  13.7(C)]  and  Eq.  5  emulates  a  homo¬ 
dyne  detection  scheme.  Another  possible  approach  is  the 
computation  of  the  absolute  magnitude  of  the  pixel-by-pixel 
Fourier-transform  over  each  set  of  images  (Ben-Levy  and  Peleg, 
1995).  For  the  case  of  three  such  images,  this  method  is  identical 
to  Eq.  5.  It  should  be  noted  that  both  of  these  methods  would  not 
be  able  to  reconstruct  the  high  ry-spatial  frequency  given  in  the 
moire  example  discussed  later  [Fig.  13.5(A)],  in  which  the  Fourier- 
transform  of  the  illumination  grid  has  its  peaks  just  outside  the 
range  of  detectable  spatial  frequencies.  This  is  the  case  when  the 
highest  possible  fluorescent  excitation  spatial  frequency  is  passed 
through  the  objective.  Such  light  will  consist  of  two  beams  passing 
through  the  edges  of  the  back-focal  plane.  Because  of  the  Stokes 
shift  of  the  excited  fluorescence  light,  this  spatial  frequency  will 


7  A  system  (ViCo)  based  on  this  and  related  concepts  is  marketed  by 
Biomedica  Mangoni  s.n.c.,  Pisa,  Italy. 
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not  be  imaged  by  the  objective  on  the  return  trip.  A  similar  situa¬ 
tion  occurs  when  employing  grazing  incidence  illumination  as  in 
Frohn  and  colleagues  (2000).  Most  strategies  assume  equally  dis¬ 
tributed  phases  between  the  images,  but  this  is  not  a  strict  require¬ 
ment.  Fukano  and  Miyawaki  (Fukano  and  Miyawaki,  2003)  use  a 
three-phase  scheme  with  a  relative  phase  of  2tc/4  between  the  three 
images,  and  a  modified  Eq.  4  for  sectioning.  With  unequal  phase 
approaches,  care  has  to  be  taken  (e.g.,  by  shifting  the  0-phase 
between  series  of  images)  to  avoid  bleaching  illumination  struc¬ 
ture  into  the  sample  over  time. 

4.  Scaled  subtraction  of  the  background  is  yet  another  way  to 
process  the  raw  images.  Assuming  that  the  nominal  position  and 
width  of  the  illumination  pattern  is  known,  a  pixel  at  the  plane 
of  focus  is  either  illuminated  or  not  in  each  of  the  images.  First 


1.  /  (x,y)  =  Max/ (x.v)-Min/ (*,v) 

rec  st  n=0...N  n  n=0...N  n 

2.  /  (*,y)  =  MaX/  (x,y) 

rec  st  n=o...  jv 


3.  Kec (*..v)  =  Max  /„ (x,y)  +  Min  /„ (x,y)~ 2 Avg  in (x,y) 


4 ■  4c(*a)  =  J^(4(*aW( 


(n+N  div  2) mod  N 


(x,y)f 


5- 


'y'  exp(2nin/  N) 


6a.  Ira.(x,y)  =  ^ Jn{x,y)Mask°"(x,y)-y^'  I lt(x,y)Mask°ff {x,y) 

n= 0  n=0 

1 


y- 


MAR 


--1 


N  N 

In ( x,y)Mask°n (x,y)  I n (x,y)Mask,°ff  (x,_y) 

o  »= 0  n=0 

H  N 

^  Mask.™ (x,y) 

n= 0 

N 

^  Mask°ff  (x,y) 

n= 0 

N 

Mask™ (x,y)  - 

M=0 

n  -y 

^  Mask™ (x,y) 

.  n=0 

N 

Ny^(Mask°"(x,y))2  - 

n= 0 

'  N 

^  Mask™ (x,y) 

-  n=0 

l2 

/P/(4  -  -s-7) 


7a. 


Y  a 

=-m  (O/OC/) 


1 


l=-m  (  CJ/ OC  /  ) 

7b.  = 


jyj) 

h  ,( k  +  s ,) 

em,l  \  l ) 


n:  index  of  the  acquired  images  (the  grid  position  varies  with  n) 

N:  number  of  acquired  images,  x,y:  in-plane  positions  in  the  image 
i:  imaginary  constant,  i.  indexing  the  components  (separated  order) 

Max,  Min,  Avg:  respective  maximum,  minimum  or  average  value, 

computed  at  each  pixel  over  all  N  successively  acquired  images. 

Mask0"  and  Mask0/  =  1  —  Mask/  define  which  pixel  was  illuminated  or  not 
illuminated  in  a  given  frame  n 

MAR  is  the  mark/area  ratio  describing  the  fraction  of  pixels  considered  to 
be  illuminated 


FIGURE  13.4.  Confocal  slices  computed  from  structured-illumination  data. 
(A)  Virtual  pinhole  diameter  chosen  to  be  half  the  spot-to-spot  distance.  (B) 
Virtual  pinhole  selected  as  0.15  of  the  spot-to-spot  distance.  (C)  Scaled  sub¬ 
traction  method  with  fixed  p  =  1  (see  Eq.  6  in  boxed  list  of  equations)  and  illu¬ 
mination  spot  of  Gaussian  shape  with  width  as  in  (B).  [The  experimental  data 
for  this  figure  was  kindly  provided  by  Pier  Alberto  Benedetti  (pollen  grain 
taken  at  1.3  NA,  -450  nm  excitation,  -550  nm  emission,  6  X  6  =  36  patterned 
images  acquired  for  this  single  slice,  200  nm  pixel  pitch  in  sample,  spot-to-spot 
distance,  2.1  |tm  in  sample).] 


the  value  of  each  pixel  is  averaged  over  multiple  images,  each 
recorded  at  one  of  the  positions  of  the  illumination  structure,  that 
illuminates  the  in-focus  parts  of  the  sample  in  this  pixel  (ON).  A 
second  average  is  taken  of  all  the  pixel  values  where  the  pixel  is 
nominally  not  irradiated  (OFF)  and  subtracted  from  the  first  ON 
average.  This  technique  estimates  and  removes  the  pinhole-to- 
pinhole  background  fluorescence,  assuming  that  in-focus  parts  of 
the  object  do  not  yield  fluorescence  when  not  irradiated  and  out- 
of-focus  parts  fluoresce  with  equal  brightness  independent  of  the 
position  of  the  illumination  structure.  It  is  termed  “scaled  subtrac¬ 
tion”  because  only  a  scaled  fraction  of  the  sum  of  the  pixel  inten¬ 
sities  in  the  non-illuminating  frames  has  to  be  subtracted  from  the 
sum  in  the  illuminating  frames.  Such  a  technique  is  commonly 
used  for  data  where  only  these  sums  are  acquired  (conjugate  and 
non-conjugate  light)  as  in  the  programmable  array  microscope 
(Hanley  et  a].,  1999;  Heintzmann  et  al.,  2001).  The  method  and 
the  scaling  factor  y  that  is  applied  to  the  non-illuminated  sum  is 
covered  by  Eq.  6.  Theoretically,  such  a  scaled  subtraction  may 
yield  negative  results  even  in  the  totally  noise-free  limit,  but  these 
are  so  small  that  they  can  be  neglected  in  practice.  Note  that  even 
though  Mask0N  and  Mask0FF  in  Eq.  6  were  initially  thought  of  as 
being  binary  masks,  this  technique  of  scaled  subtraction  can  also 
be  used  with  smooth  masks  [see  Eq.  6(b)],  in  which  case  the  binary 
mask  is  replaced  by  the  spatially  varying  excitation  probability  in 
the  computation.  Equation  6(b)  has  been  constructed  in  such  a  way 
that  small  spatial  variations  in  mask  intensity  (e.g.,  due  to  moire 
effects)  are  accounted  for.  If  this  is  not  an  issue,  [3  can  be  set  to  1. 
The  effect  of  Eq.  6(b)  with  [3  fixed  to  1,  is  shown  in  Figure  13.4(C). 
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Note  that,  in  contrast  to  filtering  approaches,  which  consider  infor¬ 
mation  in  neighboring  pixels,  this  processing  considers  the  multi¬ 
frame  data  pixel  by  pixel.  The  optical  sectioning  visible  in  Figure 
13.4  can  thus  not  be  explained  as  being  the  effect  of  a  high-pass 
filter  applied  to  the  image  data.  It  is  a  genuine  method  of  optical 
sectioning  obtained  by  processing  images  made  at  multiple  illu¬ 
mination  positions. 

The  scaled  subtraction  approach  has  also  been  used  in  a  slightly 
different  context  of  aperture  correlation  microscopy  (Wilson  et  al., 
1996).  In  the  first  image,  a  mask  with  holes  at  random  positions 
in  it  is  scanned  over  the  object  and  the  image  though  the  same 
mask  is  acquired.  The  second  image  consists  of  a,  usually  shorter, 
exposure  with  widefield  illumination.  Scaled  subtraction  of  the  two 
images  serves  to  remove  the  remaining  widefield  information, 
which  is  present  in  the  image  acquired  with  illumination  and  detec¬ 
tion  through  the  random  mask.  A  similar,  yet  more  signal-to-noise 
effective  approach,  obviating  the  need  for  a  separate  widefield 
image,  is  to  image  the  light  rejected  at  the  mask  onto  a  separate 
detector  (Hanley  et  al.,  1999;  Heintzmann  et  al,  2002)  and  also 
process  the  data  by  scaled  subtraction. 

All  of  the  above  techniques  process  the  data  locally  by  con¬ 
sidering  only  a  single  pixel  position  in  all  the  members  of  an  image 
set  for  its  own  reconstruction  and  ignoring  the  intensities  of  neigh¬ 
boring  pixels.  Some  of  the  above  methods  (Eqs.  1-5)  are  non¬ 
linear  in  the  sense  that,  during  reconstruction,  they  employ  at  least 
one  nonlinear  operation  such  as  thresholding,  squaring,  computing 
the  absolute  magnitude,  or  the  maximum.  In  the  absence  of  noise, 
they  do  remain  linear  with  respect  to  the  emitted  light  intensity.  As 
long  as  the  signal-to-noise  ratio  is  high,  the  deviations  from  a  linear 
treatment  remains  negligible  and  the  resulting  images  can  be  used 
for  quantification.  However,  in  low-signal  situations  (e.g.,  high 
out-of-focus  background)  the  deviation  of  the  output  from  the  true 
intensities  in  the  sample  will  be  severe.  The  nonlinear  steps 
(absolute  value,  square  root,  maximum,  minimum)  statistically 
bias  the  result  due  to  the  influence  of  the  noise,  for  example,  when 
the  true  degree  of  modulation  in  a  pixel  should  be  zero  but  a  high 
background  is  present  in  each  of  the  images,  methods  based  on 
taking  the  absolute  value  (Eqs.  4,  5)  will  yield  a  positive  result  just 
from  the  noise.  They  will  not  yield  zero  on  average,  whereas  a 
linear  method  would  be  expected  to  do  so. 

The  above  evaluation  techniques  have  the  advantage  of  being 
fast  and  easy  to  compute.  They  do  not  require  any  knowledge  of 
the  absolute  pattern  position  nor  do  they  need  to  estimate  this  infor¬ 
mation  from  the  measured  data.  Furthermore,  they  often  show  an 
inherent  robustness  to  inexact  pattern  positioning.  However,  the 
results  they  produce  are  generally  inferior  to  those  achievable  by 
approaches  for  the  linear  processing  of  structured-illumination 
data  as  outlined  below. 

RESOLUTION  IMPROVEMENT  BY 
STRUCTURED  ILLUMINATION 

In  addition  to  achieving  optical  sectioning  (Frohn  et  al. ,  2001), 
structured  illumination  can  also  yield  improved  lateral  resolution. 
The  reason  is  that  there  is  a  moire  effect  between  the  structured 
illumination  pattern  and  the  structure  of  the  object,  such  that  pre¬ 
viously  inaccessible  spatial  frequencies  of  the  sample  become 
detectable  (Fig.  13.5).  However,  to  yield  a  useful  reconstruction, 
the  illumination  pattern  must  first  be  disentangled  from  the 
detected  moire  fringes.  This  method  was  first  conceived  by  Lucosz 
for  the  case  of  a  rather  dense  line-grid  illumination  [Fig.  13.2(C)] 
and  detection  masks  (Lucosz  and  Marchand,  1963)  (the  unit-cell 
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FIGURE  13.5.  Moire  effect.  A  microscope  can  only  detect  information  up  to 
a  maximal  spatial  frequency.  In  other  words,  a  minimum  distance  between  the 
maxima  of  a  grid-like  feature  in  the  sample  is  required.  The  left  column  shows 
real-space  features,  whereas  the  right  column  shows  the  corresponding  situa¬ 
tion  in  Fourier-space.  The  circle  indicates  the  limiting  frequency  (pass-band  of 
the  transfer  function)  up  to  which  the  microscope  can  detect  information.  In 
(A),  a  grid  feature  of  the  sample  is  shown  that  cannot  be  resolved  with  evenly 
distributed  illumination,  yielding  equal  fluorescence  everywhere  in  the  detected 
image.  By  illumination  with  another  dense  spatial  grid  (B),  an  aliased  grid  (C) 
is  generated  (moire  effect)  that  can  then  be  partially  detected  (D).  However, 
this  detected  grid  (D)  has  “incorrect”  spacing.  Detailed  knowledge  about  the 
moire  effect  generated  by  the  illumination  pattern  (B)  can  be  used  to  reassign 
the  detected  spatial  frequency  to  the  correct  place,  thereby  reconstructing  (A). 


distance  in  the  sample  coordinate  system  was  on  the  order  of  the 
size  of  the  point  spread  function)  and  also  for  two-dimensional  grid 
[Fig.  13.2(A)]  patterns  (Lucosz,  1967).  These  arguments  are  based 
heavily  on  Fourier-space  considerations  and  form  the  basis  for  all 
the  computational  unmixing  systems  used  for  data  from  widefield 
detectors  recording  images  excited  by  structured  illumination  (as 
in  Fig.  13.1). 
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FOURIER-SPACE  —  AN  INTRODUCTION 

Fourier-space  is  extremely  useful  in  discussing  optical  imaging  because  the 
process  of  imaging  can  be  modeled  by  simply  multiplying  the  Fourier- 
transform  of  the  sample  distribution  by  the  Fourier-transform  of  the  point 
spread  function  of  the  microscope  (the  so-called  optical  transfer  function); 
a  process  that  is  carried  out  in  Fourier-space.  To  turn  this  result  into  a  sim¬ 
ulation  of  the  image,  this  product  must  be  “inverse-Fourier  transformed” 
to  bring  it  back  into  real  space. 

The  concept  of  Fourier-space  is  based  on  the  idea  that  any  function  in 
one-,  two-,  or  three-dimensional  space  (e.g.,  a  fluorophore  distribution)  can 
be  represented  as  a  sum  of  sinusoids  varying  in  spatial  frequency,  direc¬ 
tion,  strength,  and  phase  (position).  The  spatial  frequency  (or  wave 
number)  of  a  sine  wave  describes  the  number  of  bright  maxima  per  dis¬ 
tance  (e.g.,  with  the  unit  of  meter-1).  Its  component  along  the  x-axis  is  indi¬ 
cated  by  kx  while  the  number  of  maxima  counted  along  the  y-axis  is  termed 
kr  The  right  column  of  Figure  13.5  shows  the  magnitude  of  the  Fourier- 
transformation  of  simple  structures  consisting  of  one  (A,  B,  D)  or  only  a 
small  number  of  sine  waves  [Fig.  13.5(C)].  The  displayed  brightness  is 
proportional  to  the  magnitude  of  the  transform  at  each  location  in  Fourier- 
space  (also  called  reciprocal  space,  frequency  space,  or  k  space).  Such  a 
plot  shows  only  the  strength  of  the  sine  wave,  its  direction,  and  spatial  fre¬ 
quency.  The  exact  phase  (defining  the  position  of  the  first  maximum)  is  not 
displayed.  The  center  in  Fourier-space  is  located  in  the  middle  of  each 
image.  This  position  of  zero  spatial  frequency  corresponds  to  a  uniform 
brightness  in  real  space.  The  intensities  away  from  the  origin  represent 
smaller  and  smaller  spacings  the  farther  out  that  they  are. 

As  is  apparent  from  Figure  13.5(A,B,D),  a  sinusoid  shifted  to  all¬ 
positive  values  (by  adding  a  constant),  for  example,  the  emitted  sample 
intensity,  yields  three  peaks  in  its  Fourier-transformation.  The  central, 
zero-frequency  peak  represents  the  added  constant  and  the  two  other  peaks, 
taken  together  form  the  remaining  sine  wave  in  real  space.  Each  individ¬ 
ual  peak  in  Fourier-space  actually  forms  a  complex-valued  wave 
exp(i27ikr)  with  the  respective  k  vector  in  positive/negative  directions.  As 
a  sum  these  constitute  the  sine  in  real  space:  2sin(27tkr). 

For  a  preliminary  understanding  of  Fourier-space,  it  is  sufficient  to  know 
that  the  values  at  two  opposing  positions  in  k  space  are  complex  conju¬ 
gates  to  each  other  and  when  combined  always  form  a  sine  wave  in  real 
space.  Note  that  the  small  features  of  a  sample  are  represented  by  sums  of 
sine  waves  with  high  spatial  frequency  (small  distance  between  successive 
wave  maxima),  such  as  is  indicated  in  Figure  13.5(A).  A  coarser  sinusoid 
[e.g..  Fig.  13.5(D)  as  compared  to  Fig.  13.5(A)]  has  a  k  vector  closer  to 
the  origin  of  Fourier-space. 

The  process  of  imaging  is  represented  as  a  modification  of  the  Fourier- 
transform  of  the  sample  distribution  caused  by  multiplying  it  by  the  optical 
transfer  function  in  Fourier-space.  This  function  decays  smoothly  and 
amounts  to  zero  everywhere  beyond  a  certain  maximum  spatial  frequency. 
This  position,  beyond  which  no  information  can  be  transferred,  is  indicated 
by  the  white  circles  in  Figure  13.5.  The  decaying  optical  transfer  function 
of  a  widefield  microscope  is  also  indicated  by  the  dotted  line  in  Figure  13.6. 
For  a  widefield  microscope  this  in-plane  cut-off  frequency,  when  translated 
into  a  peak-to-peak  distance  of  a  sinusoid  corresponds  to  the  equation  d  = 
X/(2 NA),  with  the  vacuum  wavelength  X  and  the  numerical  aperture  NA. 

To  properly  measure  a  sample  feature  (sinusoid)  of  this  frequency,  the 
observer  needs  to  measure  at  a  pixels-to-pixel  spacing  of  below  half  this 
distance  to  avoid  misinterpretation  (aliasing)  of  the  result.  This  required 
maximal  pixel-to-pixel  distance  is  called  the  Nyquist  distance  dNq  = 
A/(4 NA).  For  a  different  introduction  to  Fourier  space,  see  the  Appendix  to 
Chapter  24,  this  volume. 


Imaging  can  be  treated  elegantly  in  Fourier-space  (see  box 
“Fourier-Space  —  An  Introduction”)  because  a  microscope  essen¬ 
tially  acts  as  a  Fourier-filter.  The  periodic  pattern  introduced  in  the 
illumination  path  of  the  microscope  yields  a  modification  of 
the  incident  and  thus  of  the  emitted  light.  For  fluorescence  or 
reflection-type  microscopy,  the  emitted  light  can  be  described  as  a 
multiplication  of  the  sample  structure  times  the  illumination  inten- 


FIGURE  13.6.  Scheme  of  the  linear  image  reconstruction.  (A)  Fourier- 
transform  of  the  structure  of  the  emitted  light.  The  optical  transfer  function 
defining  the  range  of  detectable  special  frequencies  is  indicated  by  the  dotted 
line.  (B)  Detection  sensitivity  for  the  various  reconstructed  orders  with  their 
zero-frequency  relocated  to  the  origin. 


sity  or  amplitude  structure,  respectively.  This  multiplication  in 
real-space  translates  into  a  convolution  in  Fourier-space.  Due  to 
the  periodicity  of  the  illumination  distribution,  its  Fourier- 
transform  is  a  number  of  (delta)  peaks  at  the  reciprocal  grid 
positions.  The  pattern  of  emitted  light  is  a  multiplication  (in  real 
space)  of  the  illumination  intensity  distribution  with  the  Fourier- 
transformed  object,  thus  its  Fourier-transformation  is  a  sum  of 
multiple  Fourier- transformed  objects  (termed  object  components). 
These  object  components  have  their  zero-frequency  displaced  to 
align  with  the  reciprocal  grating  of  the  illumination  distribution 
[Fig.  13.6(A),  Fig.  13.8(C)].  The  position  and  shape  of  the  illumi¬ 
nation  structure  in  real  space  determines  the  individual  position, 
strength,  and  phase  of  the  multiple  overlapping  object  components 
in  Fourier-space. 

The  imaging  of  this  emission  intensity  distribution  is  then 
described  in  Fourier-space  by  a  multiplication  with  the  optical 
transfer  function.  It  is  possible  to  computationally  unmix  the  sum 
of  the  displaced  copies  of  the  object  [Fig.  13.8(D)]  by  inversion 
(or  pseudo-inverse)  of  the  mixing  matrix  M,  which  mathematically 
describes  the  linear  superposition  of  displaced  object  components 
and  their  relative  phases  [Eq.  7(b);  see  also  Gustafsson,  2000; 
Heintzmann,  2003]  and  to  shift  the  displaced  position  of  their  zero 
object  frequency  back  to  the  real  zero  frequency.  This  shifting 
makes  it  obvious  that,  in  comparison  to  flat-illuminated  widefield 
microscopy,  the  pass-band  of  the  microscope  has  then  been 
increased  by  this  moire  effect  [Figs.  13.6(B),  13.8(E)].  Note  that 
as  opposed  to  the  previously  described  methods  (paragraphs  1-4), 
such  resolution  increase  is  even  possible  when  the  spatial  modu¬ 
lation  frequency  falls  outside  the  pass-band  and  is  thus  not  imaged. 

Acquiring  a  set  of  images  (similar  to  the  top  row  in  Fig.  13.7), 
makes  it  possible  to  unmix  the  overlapping  displaced  object  com¬ 
ponents  [Fig.  13.8(C)].  An  equation  system  for  this  unmixing  can 
be  constructed  [Eq.  7(b)]  by  using  the  unique  dependence  of 
the  complex  phase  of  each  object  component  on  the  position  of  the 
illumination  structure.  In  a  last  step  of  image  reconstruction,  the 
individual  unmixed  object  components  [e.g.,  Fig.  13.8(D)]  are 
shifted  to  their  proper  positions  in  Fourier-space,  and  multiple 
components  that  are  present  at  the  same  frequency  are  averaged 
with  frequency  dependent  weights  (e.g.,  with  the  inverse  variances 


FIGURE  13.7.  Reconstruction  results  obtained  by  different  strategies.  The  top  row  shows  the  three  individual  raw  data  images  taken  at  different  illumination 
pattern  positions  (as  indicated  by  the  white  lines  at  the  left  side  of  each  image)  with  a  Zeiss  ApoTome  setup  (Axiovert,  40  x  NA  1.3  objective,  ~3.4pm  pattern 
pitch  in  the  sample,  Axiocam  6.7  pm  pixel  pitch).  The  middle  row  shows  the  results  obtained  by  (A)  The  sum  (B)  max-min,  Eq.  1,  (C)  quadrature  method,  Eq. 
4.  Panel  (D)  shows  an  xz-cut  obtained  from  the  ApoTome  software  at  the  approximate  slicing  position  indicated  by  the  dashed  line  in  panel  (A).  The  three- 
dimensional  sectioning  capability  discriminating  between  layers  of  cells  is  seen  nicely  in  (B),  (C),  and  (D). 
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FIGURE  13.8.  Various  steps  in  the  image  reconstruction  process.  (A)  An  example  raw  data  image.  (B)  The  widefield  image  computed  by  summing  over  all 
partial  images.  The  region  of  interest  (ROI)  used  for  images  (G)  to  (K)  is  indicated  as  a  black  rectangle.  (C)  Magnitude  of  the  Fourier  transformation  of  (A), 
displaying  the  multiple  overlapping  components.  (D)  One  ( k  =  1.0)  object  component  separated  from  multiple  measurements  similar  to  (C)  taken  at  varying  illu¬ 
mination  mask  positions  [one  such  position  is  shown  in  panel  (A)].  The  region  of  support  in  detection  is  indicated  by  the  dotted  white  circle.  (E)  All  compo¬ 
nents  shifted,  averaged,  and  apodized  (border  indicated  by  white  circle).  (F)  Final  reconstruction  result  in  the  linear  case  (total  of  7  orders).  (G)  ROI  extracted 
from  raw  data  shown  in  panel  (A).  (H)  ROI  from  widefield-like  image  as  shown  in  (B).  (I)  Image  (H)  contrast-enhanced.  (J)  Contrast-enhanced  ROI  from  con- 
focal  processing  as  shown  in  Figure  13.4(B).  (K)  Equivalent  zoom  of  image  (F).  [The  raw  data  for  this  figure  was  kindly  provided  by  Pier  Alberto  Benedetti 
(for  acquisition  parameters  see  Figure  13.4).] 
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FIGURE  1 3.9.  Demonstration  of  resolution  extension  by  structured-illumination  microscopy.  The  sample  is  a  squash  preparation  of  a  polytene  chromosome  from 
a  salivary  gland  of  the  fruitfly  Drosophila  melanogaster ,  stained  for  DNA  with  Oligreen  to  show  the  characteristic  banding  pattern.  Many  band  features  that  are 
undetectable  in  the  conventional  microscope  image  (A)  are  clearly  visible  in  the  structured-illumination  microscopy  reconstruction  (B).  The  structured  illumina¬ 
tion  consisted  of  a  one-directional  pattern  of  parallel  lines  with  a  period  of  0.20  pm,  at  a  wavelength  of  457  nm.  This  pattern  frequency,  about  80%  of  the  resolu¬ 
tion  limit,  is  much  higher  that  those  used  in  the  rest  of  this  chapter.  Nine  raw  data  images  were  used:  three  images  with  a  relative  phase  shift  of  2k/3,  for  each  of 
three  pattern  orientations  spaced  120°  apart.  The  reconstruction  used  a  Fourier-based  algorithm  that  includes  compensation  for  the  detection  OTF,  as  described  in 
Gustafsson  (2000).  [These  images  were  kindly  provided  by  Mats  Gustafsson  (unpublished  data)  and  the  sample  was  prepared  by  Harry  Saumweber.] 


[Eq.  7(a);  Gustafsson,  2000;  Heintzmann,  2003;  Fig.  13.8(E)]. 
The  weights  of  this  averaging  in  Fourier-space  are  adjusted  as 
inverse  variances  of  the  noise  such  that  the  best  quality  recon¬ 
struction  at  each  frequency  is  preferred  over  more  noisy  recon¬ 
structions.  Finally,  there  is  an  inverse  Fourier  transformation  back 
to  real  space  [Figs.  13.8(F)].  The  resolution  improvement  obtained 
can  be  seen  in  Figure  13.8(K)  compared  to  what  is  obtained  by 
just  adding  the  widefield  detected  set  of  images  [Fig.  13.8(H)]  even 
after  contrast  enhancement  [Fig.  13.8(1)]  or  by  processing  the  same 
data  to  simulate  confocal  imaging  [Fig.  13.8(J)].  In  Figure  13.9  the 
resolution  improvement  is  even  more  obvious.  This  figure  was 
generated  by  illumination  with  a  grating  close  to  the  highest  trans- 
mittable  spatial  frequency,  which  leads  to  more  prominent  high- 
frequency  components  with  less  noise  after  their  extraction  (kindly 
provided  by  M.  G.  L.  Gustafsson).  In  some  of  the  figures  [espe¬ 
cially  in  Fig.  13.8(1)]  a  residual  patterning  can  be  observed.  This 
can  be  attributed  to  having  selected  too  few  pattern  positions  (here 
6x6  =  36)  in  the  set. 

It  is  interesting  to  note  that  the  sampling  of  the  raw  data  images 
need  only  satisfy  the  Nyquist  limit  of  the  widefield  microscope 
even  though  the  resolution  finally  achieved  extends  beyond  this 
limit.  During  the  relocation  process  of  the  object  components  in 
the  reconstruction,  the  discrete  frequency  space  can  be  “extended,” 
essentially  resampling  the  data  onto  a  different  grid.  This  recon¬ 
struction  does  not  contradict  information  theory  because  many 
images,  each  with  different  high-frequency  sample  information 
(but  downshifted  into  the  detection  pass-band)  are  used  to  con¬ 
struct  a  single  image  containing  information  outside  the  detection 
passband.  Although  negative  intensities  can  in  principle  result 


during  the  process  of  image  reconstruction,  this  does  usually  not 
pose  a  practical  problem. 

With  this  approach,  the  resolution  of  the  microscope  can 
theoretically  be  enhanced  by  a  factor  of  about  2  in-plane  (xy)  as 
well  as  along  the  optic  axis  (z)  compared  to  the  standard  epi- 
fluorescence  widefield  microscope.  Although  the  fundamental 
pass-band  limit  obtained  by  this  method  is  not  larger  than  the  pass- 
band  of  a  confocal  microscope,  a  substantial  practical  improve¬ 
ment  over  standard  confocal  microscopy  is  achieved.  The  high 
spatial  frequencies  of  the  object  are  detected  much  more  efficiently 
because  the  moire  effect  of  the  illumination  grid  shifts  them  into 
a  region  of  the  pass-band  that  is  more  efficiently  detected. 

Lucosz’s  formulation  (Lucosz  and  Marchand,  1963;  Lucosz, 
1967)  makes  it  possible  to  understand  both  the  confocal  micro¬ 
scope  and  multiple  dot-scanning  systems  in  the  nomenclature  of 
computational  reconstruction.  The  moving  detection  mask  of  these 
systems,  in  combination  with  the  integration  of  partial  images  on 
the  detector,  permits  the  required  unmixing  to  be  achieved  auto¬ 
matically,  the  positions  of  the  zero  frequencies  of  the  unmixed 
components  to  be  correctly  adjusted  and  integrated  (which  means 
essentially  summed)  in  the  detector  with  component-dependent 
weights.  Thought  of  in  this  way,  the  shape  and  size  of  the  detec¬ 
tion  pinhole  defines  the  relative  weights  of  individual  object 
components. 

As  opposed  to  Lucosz’  approach  or  scanning  disk  systems,  in 
which  the  decoding  is  achieved  by  a  detection  mask,  computa¬ 
tional  reconstruction  is  far  more  flexible.  Assume  as  a  gedanken 
experiment  that  we  illuminate  the  specimen  with  an  array  of  bright 
points  and  then  scan  these  points  over  the  field  of  view  so  that 
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FIGURE  13.10.  Performance  simulation  of  dense  structured  illumination  for  thick  samples.  (A)  Simulated  object.  (B)  Widefield  image  (1000  expected  photons 
in  maximum).  (C)  Confocal  image  (pinhole  size  0.5  Airy  units,  detector  efficiency  20%  that  of  panel  B).  (D)  yz- Section  computed  using  Eq.  1  from  dense  struc¬ 
tured  illumination,  three  phases.  Further  reconstructions  were  based  on  (E)  Eq.  4,  and  (F)  Eq.  6(b).  (G)  Widefield  image  with  a  background  fluorescence  added 
corresponding  to  approximately  five  layers  of  this  object  when  densely  packed.  (H)  Reconstruction  (Eq.  4)  of  a  structured-illumination  dataset  under  similar 
sample  conditions.  (I)  Reconstruction  (Eq.  4)  of  data  with  the  spacing  between  the  illuminating  bars  increased  3-fold  (total  illumination  dosage  was  kept  con¬ 
stant  for  all  simulations).  Simulation  parameters  as  described  in  the  text. 


eventually  it  is  all  covered.  If  we  record  a  separate  widefield  image 
of  the  specimen  for  each  location  of  the  excitation  array,  we  now 
have  a  stack  of  data  that  contains  all  the  information  we  need  to 
construct  an  optical  section  of  the  focus  plane  merely  by  summing 
the  signal  collected  at  virtual  “pinholes”  near  the  locations  of  the 
excitation  points  in  each  image  of  the  stack  (similar  to  the  approach 
chosen  for  Fig.  13.4).  Assuming  the  same  illuminating  power,  the 
same  scan  time,  and  the  same  detector  performance,8  this  optical 
section  will  be  in  every  way  identical  to  one  made  with  a  disk¬ 
scanning  confocal  microscope.  However,  this  admittedly  rather 
tedious  approach  has  the  added  advantage  that  we  could  have  used 
any  of  the  seven  methods  (Eqs.  1-7)  for  computing  optical 


8  Including  pixels  small  enough  to  accurately  delineate  the  image  of  each  round 
pinhole  in  the  image  of  the  disk. 


sections  discussed  above  and  in  addition  we  could  compute  results 
to  reflect  any  detector  pinhole  size  after  the  acquisition  of  the 
data. 

Although  this  approaches  pure  confocal  operation  (by  using 
very  sparse  structured  illumination),  it  would  be  extremely  time- 
consuming  to  take  the  many  individual  images.  In  addition,  taking 
so  many  high-speed  individual  images  will  increase  the  readout 
noise  with  obvious  consequences.  Other  problems  would  be 
caused  by  the  massive  amount  of  data  (a  full  image  acquired  for 
each  scan  position,  of  which  there  could  be  a  million).  Thus, 
despite  these  theoretical  advantages,  practical  limitations  still 
render  standard  confocal  microscopy  more  useful  for  imaging 
thick  specimens  (for  a  detailed  example,  see  box  “Imaging  of 
Thick  Specimens”). 

Approaches  such  as  the  programmable  array  microscope 
(Verveer  etaL,  1998;  Hanley  etal.,  1999;  Heintzmann  et  al,  2001) 
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IMAGING  OF  THICK  SPECIMENS 

In  Figure  13.10,  a  sample  consisting  of  a  vz-orienLed  star-shaped  stack 
(“wagon-wheel  pasta,"  see  also  Chapter  24,  this  volume )  and  its  images 
were  simulated  [diameter  of  object:  4um.  voxel  size  100  X  100  X  lOOnm, 
theoretical  point  spread  function  of  1.2  NA  water  immersion  (;i  =  1.33) 
objective.  Pattern  spacing  in  sample:  (B-H)  900  nm.  (I)  2.7  |Xm,  pattern 
width  300  nm.]  The  total  dosage  delivered  to  the  sample  for  the  acquisi¬ 
tion  of  all  of  the  raw  data  necessary  for  panels  (B)  to  (I)  was  kept  constant. 
This  yielded  a  total  of  1.2  X  1  06  detected  photons  for  the  central  slice 
(B,  D-F),  28  X  106  photons  (G-I),  and  36  X  103  photons  for  the  confocal 
image  (C). 

The  optical  sectioning  advantage  of  the  confocal  (C)  over  the  widefield 
image  (B)  is  clear  even  though  the  signal-to-noise  ratio  in  the  confocal 
image  is  lower  due  to  rejection  of  light  at  the  pinhole  and  lower  detection 
quantum  efficiency  (QE).  Note  that  in  the  widefield  image  only  -17%  of 
the  photons  in  the  central  slice  stem  from  the  in-focus  region  (3  voxels 
deep).  Images  obtained  from  processing  the  structured-illumination  data 
(D-F)  show  only  minor  differences  in  quality  and  are  comparable  also  to 
the  confocal  image.  However,  when  additional  background  is  added  (as  in 
G,  H)  stemming  from  a  distant  out-of-focus  region  in  the  form  of  five  addi¬ 
tional  densely  packed  layers  of  overlying  wagon-wheel  pasta,  the  recon¬ 
structions  from  dense  three-phase  structured-illumination  data  are  of  poor 
quality  (H),  whereas  the  confocal  image  quality  (C)  would  be  unaffected. 
For  this  amount  of  background  less  than  5%  of  the  collected  photons  stem 
from  the  in-focus  area. 

However,  by  adjusting  the  spacing  between  the  illumination  bars  to  yield 
a  more  sparse  illumination  (I),  a  better  sectioning  is  achieved  and  the 
quality  of  the  reconstruction  becomes  more  acceptable.  The  price  of  this 
improvement  is  that  data  acquisition  takes  longer  (if  limited  by  the  instru¬ 
ment)  and  the  raw  data  file  is  larger.  By  going  to  even  mores  sparse  illu¬ 
mination  and  a  two-dimensional  pattern,  confocal  quality  will  be  reached, 
but  the  long  acquisition  times  may  be  unacceptable.  Note  that  each  final 
reconstruction  (including  the  widefield  and  confocal  cases)  assumed  that 
the  same  total  number  of  photons  was  emitted  by  the  sample.  The  confo¬ 
cal  detector  (photomultiplier  tube  assumed)  was  assumed  to  have  a 
quantum  efficiency  20%  as  high  as  the  detectors  used  in  the  widefield 
detection  mode  (a  good  quality  CCD  assumed).  Read  noise  of  the  CCD 
was  not  accounted  for  because  it  would  be  of  marginal  influence  in  these 
bright  images.  For  a  further  discussion  on  the  effect  of  imaging  thick 
specimens  see  the  Appendix  of  this  chapter. 


that  can  rapidly  change  “pinhole  patterns”  using  a  digital  mirror 
device  for  both  illumination  and  detection,  can  make  use  of  nor¬ 
mally  rejected  light  by  sending  it  to  a  second  detector  (Heintzmann 
et  ah,  200 1 )  and  have  obvious  advantages  in  speed  and  reduction  of 
readout  noise  in  comparison  to  acquiring  all  the  individual  images 
of  a  set.  However,  when  integrating  over  a  set  of  pinhole  positions, 
these  devices  do  not  permit  the  widefield  detection  of  each  of  the 
images  of  a  set  of  structured-illumination  data  (e.g.,  changing  the 
pinhole  size  retrospectively  as  in  Fig.  13.4  is  usually  not  possible). 

Computational  reconstruction  of  structured-illumination  data 
is  in  some  ways  related  to  achieving  resolution  improvement  by 
analysis  of  a  series  of  intensity  distributions  at  the  pinhole  plane 
based  on  a  singular  functional  decomposition  (Bertero  et  ah, 
1989),  the  major  difference  being  that  aperture-modifying  filters 
influence  the  light  amplitude  and  not  only  the  intensity  as  does 
reconstruction. 

The  minimum  number  of  images  required  is  defined  by  the 
number  of  object  components  that  need  to  be  separated  [see  Figs. 
13.6(A)  and  13.8(D)].  For  a  single  optical  slice,  it  ranges  from 
three  images  when  using  a  one-dimensional  diffraction  grid  for 
improved  sectioning  (Neil  et  ah,  1997),  a  low  number  of  images 


(typically  7-9)  for  in-plane  resolution  improvement  with  succes¬ 
sive  rotation  of  the  illumination  grid  (Heintzmann  and  Cremer, 
1999;  Gustafsson,  2000)  to  many  tens  of  images  (Benedetti  et  al. , 
1996)  for  use  with  relatively  sparse  two-dimensional  patterns 
(pinhole  distance  >PSF  size). 

However,  looking  at  this  issue  from  the  perspective  of  informa¬ 
tion  theory  indicates  that  a  reduction  of  this  number  should  be  pos¬ 
sible  (Cox  and  Sheppard,  1986).  The  weighted  averaging  in  the 
reconstruction  process  also  hints  that  much  of  the  information  has 
been  acquired  multiple  times.  The  key  to  a  major  reduction  in  the 
minimum  number  of  images  required  is  to  modify  the  reconstruc¬ 
tion  process  so  that  both  order  separation  and  weighted  averaging 
are  combined  into  a  single  processing  step  (Heintzmann,  2003). 
When  this  idea  is  extended  to  three-dimensional  data,  another  sub¬ 
stantial  reduction  in  the  number  of  images  per  slice  can  be  expected. 
For  a  z-sectioning  image  stack  using  dense  grids,  this  could  reduce 
the  minimum  to  less  than  three  images  per  slice  on  average. 

When  acquiring  a  focus  series  (a  z-stack  of  image  sets),  the 
plane  of  focus  for  the  illumination  pattern  usually  coincides  with 
the  plane  of  focus  of  the  detection  path.  In  this  situation,  the  theory 
of  the  imaging  process  becomes  slightly  more  complex  than  out¬ 
lined  above.  In  the  description  above,  the  patterns  were  assumed 
as  being  “part  of  the  object,”  that  is,  as  its  multiplicative  modifi¬ 
cation.  However,  in  z-direction,  patterning  has  to  be  treated  as 
being  “part  of  the  point  spread  function”  because  the  illumination 
pattern  usually  stays  aligned  with  the  plane  of  best  focus  as  it  steps 
though  the  sample,  and  is  not  fixed  to  the  object. 

The  standing-wavefield  microscope  (Bailey  et  ah,  1993; 
Lanni  et  ah,  1993;  Krishnamurthi  et  ah,  1996;  So  et  ah,  2001)  and 
the  incoherent  illumination  image  interference  imaging  (I5M) 
microscope  (Gustafsson  et  ah,  1995,  1999)  generate  patterned  illu¬ 
mination  with  either  coherent  laser  light  or  an  incoherent  light 
source,  respectively,  and  illuminate  with  alternating  bright  and 
dark  xy-planes  stacked  along  the  optical  axis.  In  I5M,  the  maximum 
of  the  illumination  pattern  coincides  with  the  plane  of  best  focus 
in  detection.  The  illumination  pattern  stays  in  a  fixed  spatial  rela¬ 
tion  to  the  detection  point  spread  function  and  can  thus  be  treated 
as  part  of  it.  This  simplifies  the  reconstruction,  as  no  computational 
unmixing  is  required.  However,  standing-wavefield  microscopy 
with  illumination  solely  along  both  directions  of  the  optic  axis 
suffers  from  a  large  region  of  missing  intermediate  z-frequencies, 
essentially  rendering  three-dimensional  reconstruction  of  large 
features  impossible.  (In  real  space  this  corresponds  to  the  ambi¬ 
guity  problem  between  lobes  along  the  z-axis  of  the  PSF.) 

In  I5M,  the  situation  is  much  improved,  such  that  it  enables 
image  reconstruction  (Gustafsson  et  ah,  1999).  The  effect  of  noise 
on  I5M  and  4Pi  (which  is  essentially  a  point-scanning  technique, 
described  in  Chapter  30,  this  volume )  were  compared  by  Nagorni 
and  Hell  (2001a,  2001b)  indicating  the  superior  performance  of  the 
4Pi  approach,  especially  along  the  axial  direction.  However,  a 
combination  of  I5M  with  additional  patterning  along  the  in-plane 
directions  can  be  expected  to  yield  an  additional  increase  in  reso¬ 
lution.  This  may  also  have  the  potential  to  overcome  some  of 
the  signal-to-noise  difficulties  of  I5M  in  comparison  to  4Pi 
microscopy. 

All  of  the  reconstruction  techniques  described  above  suffer  to 
some  extent  from  photobleaching  of  the  fluorophores.  Because 
bleaching  is  caused  by  structured  illumination,  patterns  may  be 
bleached  into  the  sample  and  have  to  be  compensated  for  during 
reconstruction.  Efforts  to  compensate  for  patterned  bleaching  are 
treated  in  detail  by  Schaefer  and  colleagues  (2004). 

Although  the  three-dimensional  imaging  techniques  described 
above  have  usually  been  developed  for  fluorescence,  they  have 
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also  been  applied  to  incoherent  reflection  (Neil  et  al.,  1997)  and 
transmission  (Dodt  et  al.,  2003).  For  the  coherent  case,  approaches 
similar  in  spirit  to  the  Fourier-techniques  developed  for  synthetic 
aperture  radar  are  used  (Mermelstein,  1999;  Schwarz  et  al.,  2003; 
Nellist  et  al.,  1995). 

Nonlinear  Structured  Illumination 

As  described  by  Fleintzmann  and  colleagues  (2002),  the  methods 
of  structured  illumination  can  be  extended  to  the  nonlinear  regime 
of  the  experiment  in  a  straightforward  way,  and  doing  so  yields 
another  substantial  resolution  improvement  over  linear  structured 
illumination.  If  any  kind  of  nonlinearity  exists  between  the  illu¬ 
mination  intensity  and  the  emission  intensity  finally  measured, 
further  peaks  in  the  Fourier-transformation  of  the  effective  excita¬ 
tion  distribution  will  arise.  In  the  absence  of  noise,  this  allows  for 
any  number  of  components  to  separate  and  thus,  theoretically,  for 
infinite  resolution  in  the  reconstruction.  In  practice,  signal-to-noise 
issues  and  the  type  of  nonlinearity  limit  the  achievable  resolution, 
even  if  grid  quality,  positioning  accuracy,  and  detector  linearity  are 
perfect. 

One  of  the  nonlinear  effects  we  have  discussed  (Heintzmann 
et  al.,  2002)  is  fluorescence  saturation  (Sandison  et  al.,  1995).  In 
this  case,  the  sample  is  irradiated  with  structured  illumination  over 
the  full  field  of  view.  The  intensity  required  to  achieve  saturation 
is  extremely  high,  but  nevertheless  possible  by  using  pulse  lasers 
and  illuminating  only  a  few  nanoseconds.  Recently,  this  approach 
has  been  practically  demonstrated  by  Gustafsson  (2005)  who 
claimed  an  in-plane  resolution  of  ~50nm.  This  fluorescence  satu¬ 
ration  idea  is  related  to  the  saturation  of  the  stimulated  emission 
employed  in  stimulated  emission  depletion  (STED)  microscopy 
(Klar  et  al.,  2000;  see  also  Chapter  31,  this  volume). 

It  should  be  mentioned  that  multi-focal,  two-photon 
microscopy  yields  inherent  sectioning  due  to  the  two-photon  effect 
and  thus  does  not  require  a  detection  mask  (see  also  Chapter  29, 
this  volume ;  Andresen  et  al,  2001).  As  these  systems  that  illumi¬ 
nate  with  a  pattern  of  light  could  allow  the  detection  of  the  image 
at  every  pattern  position,  the  reconstruction  technique  described 
above  should  also  be  applicable,  with  its  potential  for  flexibility 
and  resolution  improvement.  If  one  neglects  the  extra  readout 
noise  associated  with  recording  all  those  images,  a  reconstruction 
based  on  such  a  set  of  structured-illumination  images  would 
increase  the  signal-to-noise  ratio  especially  for  high  frequencies  in 
comparison  to  just  acquiring  the  summed  images,  as  would  be 
done  in  the  standard  system.  However,  because  the  excitation 
probability  using  the  two-photon  effect  is  proportional  to  the 
square  of  the  incident  intensity,  analyzing  the  data  as  a  case  of 
structured  illumination  would  contribute  a  factor  of  2  increase  in 
resolution  from  the  excitation  side  of  the  scheme.  This  would  com¬ 
pensate  for  the  longer  excitation  wavelength  used  for  two-photon 
excitation.  Thus  structured  illumination  based  on  the  nonlinear 
two-photon  effect  leads  to  no  major  resolution  increase  compared 
to  single-photon  structured  illumination.  Other  nonlinear  effects, 
such  as  saturation  phenomena,  do  have  higher  orders,  which  can 
also  be  utilized  for  substantial  resolution  improvements  (such  as 
in  stimulated  emission  depletion  microscopy). 

As  any  type  of  nonlinearity  can  be  used  in  combination  with 
this  concept,  the  nonlinearities  considered  in  Hell  and  Kroug 
(1995)  and  Schonle  and  co-workers  (1999)  are  also  promising 
approaches  for  the  concept  described  by  Heintzmann  and  Cremer 
(1999b)  and  Heintzmann  and  colleagues  (2002).  Other  extremely 
promising  candidates  are  dyes  (Corrie  et  al.,  2001 ;  Giordano  et  al., 
2002)  or  proteins  (Ando  et  al.,  2004)  that  can  be  converted 


between  two  (or  multiple)  states  under  wavelength  selective  illu¬ 
mination.  These  compounds  constitute  multi-level  systems  in 
which  saturation  characteristics  can  be  utilized  without  requiring 
excessive  illumination  intensities  (Hell,  2004). 

SUMMARY 

•  Structured  illumination  in  combination  with  widefield  detec¬ 
tion  typically  requires  the  acquisition  of  a  large  amount  of 
data  in  comparison  to  standard  confocal  or  Nipkow-type  disk 
systems. 

•  One  need  not  select  a  “pinhole  size”  during  data  acquisition. 
The  data  acquired  can  be  processed  in  different  ways  to 
emphasize  different  contrast  and  resolution/noise  trade-offs. 
Depending  on  the  nature  of  the  pattern  employed,  a  practical 
resolution  improvement  of  a  factor  slightly  above  2  in  each 
direction  of  space  can  be  achieved  compared  to  standard  wide¬ 
field  microscopy.  Using  nonlinear  approaches  this  factor  can 
be  made  substantially  bigger. 

•  Practical  speed  limits  are  imposed  by  the  current  camera 
and  readout  technology,  but,  more  important,  by  the  absence 
of  bright  light  sources  suitable  for  incoherent  full-field 
illumination. 
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APPENDIX:  IMAGING  THICK  SPECIMEN  WITH 
STRUCTURED  ILLUMINATION 

What  illumination  pattern  do  we  choose  for  what  specimen?  If  we 
neglect  issues  of  resolution,  it  is  apparent  that  a  very  thin  sample 
(e.g.,  single  molecules  diffusing  in  a  flat  biomembrane)  can  be 
imaged  with  full-field  illumination  and  widefield  detection, 
whereas  a  very  thick  sample,  containing  a  lot  of  fluorophores  (e.g., 
a  fish  embryo  expressing  green  fluorescent  protein  (GFP)  through¬ 
out  all  the  cells)  requires  confocal  imaging.  Widefield  imaging  has 
the  advantage  of  being  relatively  fast  and  efficient  in  signal  detec¬ 
tion.  Confocal  imaging  allows  for  the  rejection  of  the  “bad,”  that 
is,  unemployable  out-of-focus  light  that  would  dominate  the  signal 
in  full-field  illumination. 

Structured  illumination  with  a  user-defined  illumination 
pattern  covers  the  entire  range  from  full-field  illumination  through 
dense  illumination  patterns  to  sparse  illumination,  approximating 
single-spot  confocal  illumination  conditions. 

The  crucial  task  is  to  obtain  an  estimate  of  the  noise  level  that 
we  should  expect  in  the  final  data  with  different  illumination 
patterns. 

If  we  first  assume  full-field  illumination  with  the  fraction  of 
the  illumination  area,  the  mark/area  ratio  MAR  =  1,  each  horizon¬ 
tal  plane  will  be  illuminated  with  the  same  intensity,  independent 
of  its  axial  position.  Depending  on  the  structure  of  the  sample,  a 
defined  thickness  Zeq  will  yield  an  equal  amount  of  foreground 
signal  and  out-of-focus  haze.  Structured-illumination  techniques 
and  even  widefield  deconvolution  aim  at  computationally  remov¬ 
ing  this  out-of-focus  haze.  However,  although  background  removal 
(e.g.,  by  subtraction)  can  be  achieved,  the  noise  contribution  of  the 
background  to  the  signal  will  still  be  present.  Thus,  specimens  con¬ 
siderably  thicker  than  Zeq  (e.g.,  more  than  10  times  as  thick)  will 
run  into  signal-to-noise  problems  (see  discussion  below). 

Consider  now  an  illumination  pattern  of  horizontal  bars  of  a 
thickness  of  d  =  5  pm  with  a  pitch  (distance  between  the  begin¬ 
ning  of  one  bar  and  the  beginning  of  the  next  bar)  of  D  =  50  (dm. 

,  d  1 

The  mark/area  ratio  is  defined  as  MAR  =  —  =  —  which  is  related 

D  10 

to  the  mark/space  ratio  by  MSR  =  — — —  =  — -r~ - •  If  these 

D~d  1  i 

MAR 

bars  are  sufficiently  wider  than  the  diffraction  limit,  such  that  dif¬ 
fraction  effects  can  be  neglected,  structured  illumination  allows  the 
acquisition  of  the  foreground  in  full  brightness  (knowing  which 
parts  of  the  in-focus  slice  are  illuminated)  while  reducing  the  out- 
of-focus  haze  by  the  mark/area  ratio. 

When  an  in-focus  structure  of  interest  is  not  illuminated,  which 
in  this  case  happens  in  90%  of  the  images  acquired  in  each  set  of 
frames,  the  background  described  above  is  still  present.  However, 
it  can  be  assumed  that  the  reconstruction  algorithm  can  account 
for  this  foreground  region  not  being  illuminated  because  it  knows 
the  current  illumination  structure  and  uses  this  information  merely 
for  a  precise  estimate9  of  the  amount  of  background.  This  estimate 
is  then  successively  removed  from  the  “foreground”  data,  acquired 
when  the  particle  was  illuminated  (e.g.,  using  Eqs.  6a  and  6b). 


9  The  quality  of  this  estimate  is  better  by  a  factor  of  J - 1  =  3  than  the 

V  MAR 

signal,  thus  exhibiting  only  1/3  of  the  noise. 


In  the  discussion  above,  it  is  assumed  that  out-of-focus  light 
only  influences  the  result  as  if  it  were  detected  as  an  additional, 
local  uniform  background  (with  its  associated  Poisson  noise),  and 
that  a  uniform  illumination  applies  even  to  fluorophores  situated 
directly  behind  or  in  front  of  the  plane  of  focus.  This  is  clearly 
NOT  true  for  the  patterned  illumination  geometry  given  in  Figure 
13.11(B).  The  parameters  d  and  D  define  three  illumination/detec¬ 
tion  regimes  depending  on  whether  the  fluorescent  structure  is  in 
focus,  close,  or  distant  from  the  focus  plane.  These  regimes  are 
characterized  as  follows: 

•  In  focus:  In  this  refined  model,  diffraction  effects  will  blur  the 
thin-line  illumination,  reducing  its  intensity.  As  a  result,  “back¬ 
ground”  contributions  will  arise  from  neighboring  regions 
where  illumination  was  not  intended.  These  in-plane  “spill¬ 
over”  effects  are  approximated  in  an  ad  hoc  way  by  the  fol¬ 
lowing  in-focus  contrast  reduction  factor  of  the  foreground: 

contrast  =  drJ(dlel  +  £), 

where  dlel  =  d/clm in  is  the  width  of  a  bar  compared  to  the  small¬ 
est  transmittable  spatial  wavelength  of  the  objective  at  the 
emission  wavelength  dmin  =  ftm/(2NA). 

The  number,  £  =  1.828  was  fitted  from  a  simulation  of  the 
image  of  a  bar  with  variable  width  at  NA  =  1.4. 


FIGURE  13.11.  Background  in  thick  samples.  The  sample  is  assumed  to 
resemble  a  “sea”  of  fluorescence.  The  image  depicts  the  illumination  bright¬ 
ness  distribution  in  the  sample,  viewed  from  the  side.  The  fraction  of  emitted 
light  from  a  specific  plane  falling  on  the  illuminated  grid-pattern  geometric 
positions  is  also  proportional  to  this  distribution,  which  has  to  be  integrated 
over  a  full  unit-cell  (here  distance  D).  The  region  very  close  to  the  illuminat¬ 
ing  bars  or  squares  should  start  with  a  widefield  -like  behavior.  However,  to 
keep  the  model  simple  the  whole  region  from  the  bars  to  the  distant  region  is 
denoted  close  in  which  a  linear  decay  in  the  background  stemming  from  an 
out-of-focus  plane,  detected  at  the  nominally  illuminated  in-focus  positions. 
Note  the  difference  for  dense  (A)  and  sparse  illumination  (B)  corresponding  to 
the  simulations  shown  in  Figure  13.10. 
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•  Close  [0  <  z  <  zi  =  DK2  tana)]:  Transition  occurs  between  in¬ 
focus  illumination  and  detection  and  dimmed-field  scaling. 
This  simplified  model  assumes  a  linear  transition  between  the 
foreground  in  the  in-focus  slice  and  the  background  contribu¬ 
tions  in  each  plane  of  the  distant  region. 

•  Distant  (41  <  z):  Dimmed-field  scaling.  Above  the  distance  Z\ 
=  D/( 2  tana)  from  the  plane  of  focus,  all  contributing  back¬ 
ground  is  assumed  to  be  illuminated  evenly  as  assumed  in  the 
discussion  above.  In  contrast  to  the  in-focus  regime,  features 
in  this  region  receive  illumination  intensity  dimmed  by  the 
mark/area  ratio  (MAR). 

In  Figure  13.12  the  relative  contribution  of  a  planar  fluores¬ 
cent  sheet  at  a  defined  distance  from  the  plane  of  focus  to  the  illu¬ 
minated  in-focus  pixels  is  shown  (simulation  on  a  400  x  400  pixel 
grid,  pixel  size  60  nm,  at  520  nm,  NA=  1.2  and  water  immersion 
n  =  1.33).  The  simplified  theory  agrees  reasonably  well  with  the 
detailed  simulation  (with  emission  wavelength  520  nm,  NA=  1.2, 
water  immersion  with  refractive  index  n  =  1.33).  Here  the  MAR 
was  kept  constant  as  %  (e.g.,  as  it  could  be  in  an  ApoTome)  and 


Line-Grid  Backround  at  MSR  =  1/4 


D=12pm 
model 
D=6pm 
model 
D=2.4pm 
model 
D=0.96|im 
model 
D=0.48|jm 
model 


O' - 1 - 1 - i - ' - 1 - 1 - 1 - 1 

0  0.5  1  1.5  2  2.5  3  3.5  4 

Distance  from  Focal  Plane  [pm] 

FIGURE  13.12.  Comparison  of  rough  model  (from  Fig.  13.11)  with  detailed 
calculations  at  MAR  =  1/4.  (A)  Shown  is  the  relative  contribution  each  plane 
of  focus  makes  to  the  signal  detectable  in  the  geometric  position  of  the  bars 
used  for  illumination.  The  '‘model  "  curves  are  calculated  according  to  the 
approximations  shown  in  Figure  13.11,  the  contribution  of  each  plane  to  the 
background  drops  linearly  (close  regime)  to  reach  a  relative  level  of  MAR  = 
d!D  at  distances  bigger  than  Zi  (distant  regime). 


the  pitch  and  width  of  the  bars  were  varied  synchronously.  As  the 
width  of  the  bars  gets  smaller,  the  foreground  signal  decreases.  It 
is  interesting  to  note  that,  for  samples  of  relatively  small  depth, 
there  is  an  optimal  illumination  pattern  that  provides  the  best 
signal-to-noise  ratio.  For  thicker  samples,  the  initial  curve  shape 
can  be  neglected  as  long  as  the  width  of  the  bars  is  well  above  the 
diffraction  limit.  The  maximal  achievable  thickness  then  scales 
reciprocally  with  the  chosen  MAR  as  in  the  simplified  initial 
model. 

Similar  simulations  performed  for  two-dimensional  patterns, 
revealed  that  the  model  can  safely  be  extended  in  an  ad  hoc  way 
by  assuming  the  out-of-focus  contributions  to  be  described  by  a 
product  of  the  results  from  the  appropriate  line  grids  (even  though 
the  problem  is,  strictly  speaking,  not  separable). 


Dependence  of  the  Maximal  Sample  Thickness 
on  the  Number  of  Collected  Photons 


Let  us  assume  that  a  total  of  1000  photons/pixel  are  detected  from 
a  feature  during  the  collection  of  as  many  images  as  are  needed  to 
illuminate  the  entire  plane.  In  other  words,  if  the  dark  space 
becomes  larger,  more  exposures  are  needed  to  illuminate  the  entire 
plane  but  exposure  must  now  involve  fewer  photons.  Problems 
arise,  if  the  foreground  (1000  photons)  cannot  be  distinguished 
from  the  noise  of  the  total  signal  s  =  fg  +  bg  (sum  of  back¬ 
ground  bg  and  foreground  fg).  The  noise  scales  with  the  square 
root  of  s.  To  achieve  a  defined  signal-to-noise  ratio  ( S/N )  for  the 
foreground-only  signal  ( fgrec )  reconstructed  by  subtracting  the 
background  (here  assumed  to  be  estimated  perfectly),  we  obtain: 

fgTec  _  fg 


S/N  = 


noise{fgKC)  - Jfg  +  bg 


thus 


r  =  I  -h i-  I  -  fe,  which 

'.S/N,1 


is  the  maximum  acceptable  background  level  given  the  specified 
foreground  level  (in  photons)  and  the  required  signal-to-noise 
ratio.  Assuming  a  signal-to-noise  ratio  of  10  gives  reasonable 
image  quality,  and  assuming  essentially  noise-free  background 
estimation,  the  background  level  bg  in  the  raw  data  can  be  as  high 
as  9000  photons/pixel  (i.e.,  the  background  signal  can  be  ~9  times 
higher  than  the  foreground).  However,  if  only  200  photons  can  be 
collected  from  the  foreground  object,  the  background  cannot  go 
above  200  photons  without  dropping  the  S/N  below  10. 

Therefore,  two  effects  determine  the  maximum  sample  thick¬ 
ness.  The  structure  of  the  illumination  pattern  will  define  the  break¬ 
even  distance  Zeq  at  which  the  out-of-focus  haze  equals  the 
foreground.  The  number  of  collected  photons  from  the  structure  of 

fg 

interest  then  finally  determines  the  factor  Y  = - y  - 1  by  which 

7  S/N2  J 

the  sample  thickness  can  exceed  Zeq,  while  still  maintaining  a 
signal-to-noise  ratio  above  the  given  limit  (S/N). 
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Visualization  Systems  for  Multi-Dimensional 
Microscopy  Images 

N.S.  White 


INTRODUCTION 

Rapid  developments  in  biological  microscopy  have  prompted 
many  advances  in  multi-dimensional  imaging.  However,  three- 
dimensional  (3D)  visualization  techniques  originated  largely  from 
applications  involving  computer-generated  models  of  macroscopic 
objects.  Subsequently,  these  methods  have  been  adapted  for  bio¬ 
logical  visualization  of  mainly  tomographic  medical  images  and 
data  from  cut  serial  sections  (e.g.,  Cookson  et  al.,  1989  and  review 
in  Cookson,  1994).  Most  of  these  algorithms  were  not  devised 
specifically  for  microscopy  images,  and  only  a  few  critical  assess¬ 
ments  have  been  made  of  suitable  approaches  for  the  most 
common  3D  technique,  laser-scanning  microscopy  (LSM)  (Kriete 
and  Pepping,  1992).  Ultimately,  we  must  rely  on  objective  visual¬ 
ization  of  control,  calibration,  and  test  specimens  in  order  to 
determine  which  visualization  algorithms  are  appropriate  for  a 
particular  analysis.  Hardware  developments  and  advances  in  soft¬ 
ware  engineering  tools  have  made  available  many  3D  reconstruc¬ 
tion  systems  that  can  be  used  to  visualize  multi-dimensional 
images.  These  are  available  from  instrument  manufacturers,  third 
party  vendors,  research  academics,  and  other  microscopists.  The 
author  has  attempted  to  collate  important  techniques  used  in  these 
programs  and  to  highlight  particular  packages  that,  not  exclusively, 
illustrate  various  techniques  described  throughout  the  text.  A  rep¬ 
resentative  collection  of  established  commercial  and  non¬ 
commercial  visualization  programs  available  at  the  time  of  writing 
is  listed  in  Table  14.1.  For  automatic  image  analysis  and  mea¬ 
surement,  see  Chapters  15  and  48,  this  volume. 

The  information  presented  in  this  chapter  about  the  various 
programs  is  not  the  result  of  exhaustive  tests  or  benchmarks  but  is 
merely  an  overall  view  of  some  key  issues.  The  speed  of  changes 
and  the  rapid  appearance  (and  loss)  of  particular  programs  and 
hardware  from  common  use  make  it  necessary  to  concentrate  on 
important  techniques  and  milestones  rather  than  intricate  details  of 
each  package. 

Multi-dimensional  microscopy  data  can  also  be  obtained  from 
instruments  other  than  LSM  configurations,  such  as  non-laser  con- 
focal  devices  and  widefield  (conventional)  systems  combined  with 
image  restoration.  Although  the  multi-dimensional  data  from  dif¬ 
ferent  instruments  may  have  different  characteristics,  the  same 
basic  visualization  methods  can  be  used  to  process  the  data. 

Definitions 

A  consistent  terminology  is  required  to  discuss  components  of 
any  visualization  system,  maintaining  a  fundamental  separation 


between  (1)  raw  images  and  subsequent  processed  stages,  (2)  data 
values  and  the  sampled  space  over  which  they  extend,  and  (3)  final 
results  and  the  presentation  form  of  those  results.  The  author  prefers 
the  following  terminology:  Original  intensities  from  the  micro¬ 
scope  comprise  an  image.  Subsequent  visualization  steps  produce 
a  view.  Intensities  in  an  image  or  view  represent  the  data  values, 
and  the  sampling  space  over  which  they  extend  constitutes  the 
dimensions  of  the  data.  Values  presented  on  a  screen,  hard  copy, 
etc.,  are  display  views.  Visualization  is  the  overall  process  by 
which  a  multi-dimensional  display  view  is  made  from  a  biological 
specimen,  although  we  will  only  be  concerned  with  the  software 
component  of  this  in  the  present  text.  Reconstruction  refers  to  the 
piecing  together  of  optical  sections  into  a  representation  of  the 
specimen  object.  Rendering  is  a  computer  graphics  term  that 
describes  the  drawing  of  reconstructed  objects  into  a  display  view. 

What  Is  the  Microscopist  Trying  to  Achieve? 

Human  visual  perception  and  cognition  are  highly  adapted  to  inter¬ 
pret  views  of  large-scale  (macroscopic)  objects.  The  human  eye 
captures  low  numerical  aperture  (NA)  views  like  photographic 
images.  We  get  information  from  such  views  by  calling  (largely 
subconsciously)  on  a  priori  information  about  both  the  object  and 
imaging  system.  High-NA  microscope  objectives  produce  images 
from  which  we  generate  views  with  properties  that  we  are  less 
equipped  to  interpret,  for  example,  the  apparent  transparency  of 
most  biological  samples.  This  arises  from  two  main  processes:  (1) 
Samples  are  mostly  thin,  non-absorbing,  and  scatter  only  some  of 
the  illuminating  light.  Reduced  out-of-focus  blur,  together  with 
this  real  transparency  enable  the  confocal  and  multi-photon  LSMs 
to  probe  deep  into  intact  specimens.  (2)  A  high-NA  microscope 
objective  collects  light  from  a  large  solid  angle  and  can  see  around 
small  opaque  structures  that  would  otherwise  obscure  or  shadow 
details  further  into  the  sample.  Such  intrinsic  properties  in  the 
resultant  images  must  be  sympathetically  handled  by  an  appropri¬ 
ate  visualization  system. 

The  goal  of  visualization  is  the  formation  of  easily  interpreted 
and,  sometimes,  realistic-looking  display  views.  While  pursuing 
these  aims,  several  conflicts  arise:  (1)  avoiding  visualization  arti¬ 
facts  from  inappropriate  a  priori  knowledge,  (2)  enhancing 
selected  features  of  interest,  and  (3)  retaining  quantitative  infor¬ 
mation.  Maintaining  these  goals  ensures  that  the  final  view  can  be 
used  to  help  obtain  unbiased  measurements  or  to  draw  unambigu¬ 
ous  conclusions.  The  microscopist  must  be  constantly  vigilant 
to  avoid  unreasonably  distorting  the  structural  (and  intensity) 
information  in  the  original  image  data  by  over-zealous  use  of 
image  processing. 
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Criteria  for  Choosing  a  Visualization  System 

Assessing  any  visualization  system  requires  a  judgment  of  (1)  fea¬ 
tures,  (2)  usability  or  friendliness,  (3)  price/performance,  (4)  suit¬ 
ability  of  algorithms,  (5)  integration  with  existing  systems,  and  (6) 
validation  and  documentation  of  methods  or  algorithms.  The  only 
way  to  determine  ease  of  use  is  by  testing  the  system  with  typical 
users  and  representative  data.  The  best  demonstration  images 
saved  at  a  facility  should  never  be  used  to  assess  a  visualization 
system  for  purchase!  The  host  institution’s  user  profile  will  help 
to  formulate  more  specific  questions:  What  is  the  purpose  of  the 
reconstructed  views?  What  image  information  can  be  represented 
in  the  display  views?  How  must  the  image  data  be  organized?  Are 
semi-automated  or  script-processing  (programming)  tools  avail¬ 
able  for  preprocessing  and  can  the  procedures  be  adequately  tested 
and  tracked? 


WHY  DO  WE  WANT  TO  VISUALIZE 
MULTI-DIMENSIONAL  LASER-SCANNING 
MICROSCOPY  DATA? 

The  principle  uses  of  a  visualization  package  are  to  generate  sub- 
region  or  composite  reconstructions  from  multi-dimensional 
images  (Fig.  14.1).  To  collect  such  views  directly  from  the  micro¬ 
scope  is  a  time-consuming  and  inefficient  process. 

There  are  many  advantages  to  interactively  viewing  multi¬ 
dimensional  confocal  images  away  from  the  microscope: 

•  Damage  to  the  sample  by  the  illumination  is  reduced. 

•  Sample  throughput  on  a  heavily  used  system  is  improved. 

•  Optimal  equipment  for  data  presentation  and  analysis  can  be 
employed.  Serial  two-dimensional  (2D)  orthogonal  sections 
(e.g.,  xy,  xz,  yz,  xt,  etc.)  must  be  extracted  from  a  3D/four- 
dimensional  (4D)  image  interactively  at  speeds  adequate  for 
smooth  animation.  An  animation  of  confocal  sections  corre¬ 
sponds  to  a  digital  focal  series  without  contrast-degrading  blur. 
Oblique  sections  overcome  the  serial  section  bias  of  all  con¬ 
focal  instruments  and  their  smooth,  interactive  animation  is 
desirable. 

•  Reconstructed  views  are  essential  to  conveniently  display  a  3D 
(Drebin  et  ah,  1988;  Robb,  1990)  or  4D  image  (Kriete  and 
Pepping,  1992)  on  a  2D  display  device.  The  reconstructed 
volume  may  show  features  that  are  not  discernible  when  ani¬ 
mating  sequential  sections  (Cookson  et  ah,  1993;  Foley  et  al., 
1990).  Reconstructions  further  reduce  the  orientationally 
biased  view  obtained  from  serial  sections  alone. 

•  Multiple  views  are  a  useful  way  of  extending  the  dimensional 
limitations  of  the  2D  display  device.  Montages  of  views  make 
more  efficient  use  of  the  pixel  display  area.  Animations  make 
effective  use  of  the  display  bandwidth.  Intelligent  use  of  color 
can  further  add  to  our  understanding  of  the  information  in  a 
display  view  (e.g.,  Boyde,  1992;  Kriete  and  Pepping,  1992; 
Masters,  1992). 

•  For  multiple-channel  images,  flexibility  and  interactive  control 
are  essential.  Multiple  channels  may  require  complex  color 
merging  and  processing  in  order  to  independently  control  the 
visualization  of  each  component. 

Data  and  Dimensional  Reduction 

Simplification  of  a  complex  multi-dimensional  image  to  a  2D 
display  view  implies  an  important  side  effect  —  data  and  dimen¬ 


sional  reduction.  If  the  required  information  can  be  retained  in  a 
display  view,  substantial  improvements  in  storage  space  and  image 
transfer  times  are  possible.  This  is  increasingly  important  when 
presentation  results  must  be  disseminated  via  the  Internet  (now  a 
routine  option  for  medical  imaging  packages  such  as  those  from 
Cedara  and  Vital  Images  Inc.).  Data  reduction  is  most  obvious  in 
the  case  of  a  single  (2D)  view  of  a  3D  volume  or  multiple-channel 
image  but  is  actually  more  significant  when  4D  data  can  be  dis¬ 
tilled  into  a  series  of  2D  views  (Volocity,  Imaris,  and  Amira,  among 
other  packages,  can  now  seamlessly  visualize  multi-channel  4D 
images).  Significant  reduction  can  also  be  achieved  when  a  single 
3D  volume  of  voxels  can  be  represented  by  a  smaller  number  of 
geometric  objects.  To  combine  data  reduction  with  quantitative 
analyses,  a  precise  description  of  the  object  extraction  algorithm 
must  be  recorded  along  with  the  view;  only  then  can  the  user  deter¬ 
mine  the  significance  of  extracted  features. 

Objective  or  Subjective  Visualization? 

The  conflict  between  realistic  display  and  objective  reconstruction 
persists  throughout  the  visualization  process.  All  of  the  important 
information  must  be  retained  in  a  well-defined  framework,  defined 
by  the  chosen  visualization  model  or  algorithm.  Recording  of  the 
parameters  at  every  stage  in  the  visualization  is  essential.  This  can 
be  consistent  with  the  production  of  convincing  or  realistic 
displays,  provided  enhancement  parameters  are  clearly  described 
and  can  be  called  upon  during  the  interpretation  phase.  Multi¬ 
dimensional  image  editing  must  be  faithfully  logged  in  order  to 
relate  subregions,  even  those  with  expansion  or  zooming,  back  to 
their  original  context.  Object  extraction  is  a  one-way  operation  that 
discards  any  original  image  data  that  falls  outside  the  segmentation 
limits.  To  interpret  a  reconstruction  based  on  graphically  drawn  sur¬ 
faces  we  will  need  to  refer  back  to  the  corresponding  image  voxels. 
To  do  this,  we  must  either  (1)  superimpose  the  view  on  a  recon¬ 
struction  that  shows  the  original  voxels,  using  so-called  embedded 
geometries  (e.g..  Analyze,  Amira,  Imaris  VolVis  and  others),  or  (2) 
make  references  back  to  the  original  image.  Voxel  distribution  sta¬ 
tistics  defining  the  degree  to  which  a  particular  extracted  object  fits 
the  image  data  would  be  a  significant  improvement. 

Prefiltering 

Low-pass  or  median  filtering  aids  segmentation  by  reducing  noise. 
Ideally,  noise  filters  should  operate  over  all  image  dimensions,  and 
not  just  serially  on  2D  slices.  Imaris,  Voxblast,  and  FiRender/ 
LaserVox,  for  example,  have  true  3D  filters;  the  latter  are  particu¬ 
larly  useful  as  they  allow  filtering  of  a  subvolume  for  comparison 
within  the  original.  Because  Nyquist  sampling  will  ideally  have 
been  adhered  to  in  the  image  collection,  the  ideal  preprocessing 
stage  would  include  a  suitable  Gaussian  filter  or  (even  better)  a 
point-spread  function  (PSF)  deconvolution  (Agard  et  ah,  1989). 
This  step  can  effectively  remove  noise,  that  is,  frequency  compo¬ 
nents  outside  the  contrast  transfer  function  of  the  instrument. 

Identifying  Unknown  Structures 

The  first  thing  to  do  with  a  newly  acquired  3D  image  is  a  simple 
reconstruction  (usually  along  the  focus  or  z-axis).  Even  the  sim¬ 
plest  of  visualization  modules  in  2D  packages  (such  as  Metamorph 
and  the  basic  Scion  Image)  can  rapidly  project  even  modest  size 
data  sets.  The  aim  is  to  get  an  unbiased  impression  of  the  signifi¬ 
cant  structure(s).  For  the  reasons  discussed  above,  the  method  of 
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TABLE  14.1.  A  Representative  Collection  of  Visualization  Software  Packages  Available  at  the  Time  of  Writing 


System 

Source 

Supplier  type 

Platforms  supported 

Price  Guide 

Amira 

TGS  Inc. 

ind 

Win,  HP 

(B)-(C) 

5330  Carroll  Canyon  Road,  Suite  201,  San  Diego  CA  92121,  USA 

SGI,  Sun 

www.  amiravis  .com 

Linux 

Analyze 

Analyze  Direct 

acad 

Win 

(B) 

11425  Strang  Line  Road,  Lenexa,  KS  66215,  USA 
www.  analy  zedirect .  com 

ind 

Unix/Linux 

3D  for  LSM 

Carl  Zeiss  Microscopy 

LSM,  wf 

Win 

(A)-(B) 

&  LSM  Vis  Art 

D07740  Jena,  Germany 
www.zeiss.de/lsm 

AutoMontage 

Syncroscopy  Ltd. 

Ind 

Win 

(A)-(B) 

Beacon  House,  Nuffield  Road,  Cambridge,  CB4  1TF,  UK 
www.  syncroscopy.  com 

wf 

AVS 

Advanced  Visual  Systems  Inc. 

ind 

DEC,  SGI, 

(A)-(B) 

300  Fifth  Avenue,  Waltham,  MA  02451,  USA 

Sun,  Linux 

www.avs.com 

Cedara 

Cedara  Software  Corp. 

med 

Win 

(B)-(C) 

(formerly  ISG) 

6509  Airport  Road,  Mississauga,  Ontario,  L4V  1S7,  Canada 
www.cedara.com 

Deltavision 

Applied  Precision,  LLC 

wf 

Win 

(B) 

&  SoftWorx 

1040  12th  Avenue  Northwest,  Issaquah,  Washington  98027,  USA 
www.api.com 

FiRender 

Fairfield  Imaging  Ltd. 

1  Orchard  Place,  Nottingham  Business  Park,  Nottingham,  NE8  6PX,  UK 
www.fairimag.co.uk 

ind 

Win 

(B) 

Image  Pro  Plus  & 

Media  Cybernetics,  Inc. 

ind 

Win 

(B) 

3D  Constructor 

8484  Georgia  Avenue,  Suite  200,  Silver  Spring,  MD  20910-5611,  USA 
www.mediacy.com 

ImageJ 

National  Institutes  of  Health 

acad 

Win,  Mac 

(A)-(B) 

9000  Rockville  Pike,  Bethesda,  Maryland  20892,  USA 
http://rsb.info.nih.gov/ij 

Linux,  Unix 

Imaris 

Bitplane  AG 

Badenerstrasse  682,  CH-8048  Zurich,  Switzerland 
www.  bitplane  .com 

ind 

Win 

(A)-(C) 

Lasersharp 

Bio-Rad  Microscience  Ltd. 

LSM 

Win 

(A)-(B) 

&  Laser  Vox 

Bio-Rad  House,  Maylands  Avenue,  Hemel  Hempstead,  HP2  7TD,  UK 

&  LaserPix 

www.cellscience.bio-rad.com 

LCS 

Leica  Microsystems  AG 

LSM,  wf 

Win 

(A)-(B) 

&  LCS-3D 

Emst-Leitz-Strasse  17-37,  Wetzlar,  35578,  Germany 
www.leica-microsystems.com 

Metamorph 

Universal  Imaging  Corporation 

402  Boot  Road,  Downingtown,  PA  19335,  USA 
www.image  1  .com 

ind 

Win 

(A)-(B) 

Northern  Eclipse 

Empix  Imaging,  Inc. 

3075  Ridgeway  Drive,  Unit  #13,  Mississauga,  ON,  L5L  5M6,  CANADA 
www.empix.com 

ind 

Win 

(A)-(B) 

Stereo  Investigator 

MicroBrightField,  Inc. 

185  Allen  Brook  Lane,  Suite  201,  Williston,  VT  05495,  USA 
www.microbrightfield.com 

ind 

Win 

(A)-(B) 

Scion  Image 

Scion  Corp. 

ind 

Win 

(A) 

82  Worman’s  Mill  Court,  Suite  H,  Frederick,  Maryland  21701,  USA 
www.  scioncorp .  com 

Mac 

Visilog/Kheops 

Noesis 

ind 

Win 

(A)-(B) 

6-8,  Rue  de  la  Reunion,  ZA  Courtaboeuf,  91540  Les  Ulis  Cedex,  FR 

Unix 

www.noesisvision.com 

Vitrea2  (formerly 

ViTAL  Images,  Inc. 

med 

Win 

(B)-(C) 

VoxelView) 

5850  Opus  Parkway,  Suite  300,  Minnetonka,  Minnesota  55343,  USA 
www.vitalimages.com 

SGI 

Volocity 

Improvision  Inc 

ind 

Win 

(B)-(C) 

&  OpenLab 

1  Cranberry  Hill,  Lexington,  MA  02421,  USA 
www.improvision.com 

Mac 

VolumeJ  (plug-in 

Michael  Abramoff,  MD,  PhD 

acad 

Win,  Mac 

(A)-(B) 

for  ImageJ) 

University  of  Iowa  Hospitals  and  Clinics,  Iowa,  USA 
http://bij.isi.uu.nl 

Linux,  Unix 

VolVis 

Visualization  Lab 

ind 

Unix  source 

(A) 

Stony  Brook  University,  New  York,  USA 
www.cs.sunysb.edu/~vislab 

supplied 
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TABLE  14.1.  ( Continued ) 


System 

Source 

Supplier  type 

Platforms  supported 

Price  Guide 

VoxBlast 

VayTek,  Inc. 

ind 

Win 

(A)-(C) 

305  West  Lowe  Avenue,  Suite  109,  Fairfield,  I A  52556,  USA 

Mac 

www.vaytek.com 

SGI 

Voxx 

Indiana  Center  for  Biological  Microscopy 

acad 

Win 

(A)-(B) 

Indiana  University  Medical  Center 

Mac 

www.  nephrology .  iupui .  edu/imaging/ voxx 

Linux 

Ind  =  independent  supplier  (not  primarily  a  microscopy  system  supplier),  acad  =  system  developed  in,  and  supported  by,  academic  institution,  LSM  =  LSM  supplier,  wf  = 
widefield  microscopy  system  supplier,  med  =  medical  imaging  supplier.  Win  =  Microsoft  Windows,  Mac  =  Apple  Macintosh,  SGI  =  Silicon  Graphics  workstation,  HP  = 
Hewlett  Packard  workstation.  Price  guide  (very  approximate,  lowest  price  includes  entry  level  hardware  platform):  (A)  =  <  $5000,  (B)  =  $5000-$  15,000,  (C)  =  >  $15,000. 


choice  is  voxel  rendering,  as  it  avoids  potential  artifacts  of 
segmentation  at  this  early  stage.  This  catch-all  algorithm  could 
have  interactive  parameter  entry  in  order  to  explore  the  new  struc¬ 
ture  if  the  processing  were  fast  enough.  Contrast  control  and 
careful  data  thresholding  (to  remove  background  only)  would  nor¬ 
mally  be  used  with  this  quick-look  approach.  More  specific  voxel 
segmentation  (removing  data  values  outside  a  given  brightness 
band,  intensity  gradient,  or  other  parameter  range)  should  be  used 
with  caution  during  the  identification  of  a  new  structure.  Artifac- 
tual  boundaries  (surfaces)  or  apparently  connected  structures  (e.g., 
filaments)  can  always  be  found  with  the  right  segmentation  and 
contrast  settings. 

In  subsequent  refinement  stages,  a  case  can  usually  be  made 
for  a  more  specific  segmentation  model.  For  example,  maximum 


intensity  segmentation  can  be  used  to  visualize  a  topological 
reflection  image  of  a  surface.  The  prerequisites  for  such  a  choice 
can  only  be  confirmed  by  inspection  of  the  entire  image  data. 
Finally,  visualization  models  involving  more  complex  segmenta¬ 
tion.  absorption  and  lighting  effects,  whether  artificial  or  based  on 
a  priori  knowledge,  must  be  introduced  in  stages  after  the  basic 
distribution  of  image  intensities  has  been  established  (Fig.  14.2). 

Computer  graphics  research  is  beginning  to  offer  techniques 
for  automated  or  computer-assisted  refinement  of  the  visualization 
algorithm  to  automatically  tune  it  for  the  particular  supplied  data 
(He  et  al.,  1996;  Marks,  1997;  Kindlmann  and  Durkin,  1998). 
Some  useful  user-interface  tools,  such  the  Visual  Network  Editor 
of  AVS,  assist  in  the  design  stages  of  more  complex  multi-step  or 
interactive  visualization  procedures. 


FIGURE  14.1.  Viewing  multidimensional  LSM  data.  In  order  to  make  maximum  use  of  imaging  resources,  multidimensional  CLSM  images  are  routinely 
viewed  away  from  the  microscope.  ‘'Thick"  2D  oblique  sections  (A,  B)  can  be  extracted  at  moderate  rates  by  many  software  packages.  2D  orthogonal  sections 
(C-E)  can  be  viewed  at  video  rate.  Reconstructed  3D  views  (F,  G)  require  more  extensive  processing,  now  common  in  all  commercial  systems.  (A-D,  F,  G)  are 
reflection  images  of  Golgi-stained  nerve  cells.  (E)  Multiple  xy  views  (e.g.,  from  an  animation)  of  fluorescently  stained  nerve  cells. 
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FIGURE  14.2.  Identifying  unknown  structures.  It  is  important  to  make  as  few  assumptions  as  possible  about  the  imaged  structures  during  the  exploratory 
phase  of  3D  visualization.  Average  (or  summation)  projection  (A),  though  simple,  often  gives  low-contrast  views  due  to  the  low  weight  given  to  small  struc¬ 
tures.  Maximum  brightness  (B)  gives  higher  weight  to  small  bright  structures  but  when  used  in  isolation  provides  no  z-discrimination.  (C)  Background  thresh¬ 
olding  (setting  to  zero  below  a  base  line)  is  simple,  easy  to  interpret  and  increases  contrast  in  the  view.  (D)  Re-orientating  the  3D  volume  (even  by  a  few  degrees) 
can  show  details  not  seen  in  a  “top  down  view,”  and  coupled  with  animation  (see  text),  this  is  a  powerful  exploratory  visualization  tool.  (A-C)  processed  by 
simple  z-axis  projections,  (D)  “Maximum  intensity”  using  the  Lasersharp  software.  Lucifer  Yellow  stained  nerve  cell  supplied  by  S.  Roberts,  Zoology  Depart¬ 
ment,  Oxford  University. 


Highlighting  Previously  Elucidated  Structures 

Having  ascertained  the  importance  of  a  particular  feature,  the  next 
step  is  to  enhance  the  appearance  for  presentation  and  measurement 
(Fig.  14.3).  Connectivity  between  voxels  in,  for  example,  a  fila¬ 
ment  or  a  positively  stained  volume,  may  be  selectively  enhanced 
(the  extracted  structures  may  even  be  modeled  as  graphical  tubes 
or  solid  objects;  see  SoftWorx  from  API  and  Imaris  packages  for 
examples).  A  threshold  segmentation  band  can  be  interactively  set 
to  remove  intensities  outside  the  particular  structure.  3D  fill  rou¬ 
tines,  3D  gradient,  dilation,  and  other  rank  filters  are  the  basis  for 
structural  object  segmentation.  Opacity  (reciprocal  to  trans¬ 
parency)  is  possibly  the  most  used  visualization  parameter  to  high¬ 
light  structures  segmented  by  intensity  bands.  This  parameter 
controls  the  extent  to  which  an  object  in  the  foreground  obscures 
features  situated  behind  it.  Consequently,  it  artificially  opposes  the 
intrinsic  transparency  of  biological  specimens.  Artificial  lighting  is 


applied  during  the  final  stage.  Artificial  material  properties  (such 
as  opacity,  reflectivity,  shininess,  absorption,  color,  and  fluores¬ 
cence  emission)  are  all  used  to  simulate  real  or  macroscopic  objects 
with  shadows,  surface  shading,  and  hidden  features. 

Visualization  for  Multi-Dimensional 
Measurements 

Often,  the  final  requirement  of  objective  visualization  is  the  ability 
to  extract  quantitative  measurements.  These  can  be  made  on  the 
original  image,  using  the  reconstructed  views  as  aids,  or  made 
directly  on  the  display  views.  The  success  of  either  of  these 
methods  depends  on  the  choice  of  reconstruction  algorithm  and  the 
objective  control  of  the  rendering  parameters. 

Table  14.2  gives  an  overview  of  visualization  tools  that  might 
be  useful  for  objectively  exploring  the  image  data  (see  also 
Chapter  15,  this  volume). 
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FIGURE  14.3.  Enhancing  and  extracting  objects.  Having  elucidated  a  particular  structure  within  the  volume,  filtering  and  segmentation  permit  selective 
enhancement.  (A)  “Maximum  intensity”  view  [as  in  Fig.  14.2(B)]  after  two  cycles  of  alternate  high-pass  and  noise-reduction  filters  on  the  original  2D  xy  sec¬ 
tions  (using  3x3  high-pass  and  low  pass  Gaussian  filters).  (B)  More  extreme  threshold  segmentation  to  extract  the  enhanced  details  during  the  projection.  (C,D) 
two  rotated  and  tilted  views  (Lasersharp  software)  using  a  “local  average”  (see  text)  to  bring  some  “solidity”  to  the  view.  This  example  shows  the  principle 
danger  of  segmentation,  that  of  losing  fine  details  excluded  from  the  intensity  band. 


TABLE  14.2.  Overview  of  Visualization  Parameters  Desirable  for  Visualizing  Multi-Dimensional  Biological 

Microscopy  Data 


Processing  step 

Parameter 

Minimum  required 

Desirable  enhancements 

a3D  algorithms 

General  modes 

Xy,  xz,  yz  orthogonal  slices 

Z-weighted  “projections” 

Arbitrary  slices,  Voxel  a-blending 

Surface  rendering 

Quick  modes 

Fast  xy,  xz  slices 

Maximum  projection 

Hardware  acceleration,  Sub-sampling  data 

bControlling  the 

Visualization  parameters 

Projection  angles 

Viewing  angle,  z-fill,  Data  threshold, 

reconstruction 

Z-  stretch 

Voxels/surfaces,  Shading  control,  Lighting  controls, 

process 

Animation  controls 

Sequence  (movie)  mode 

Material  properties,  Opacity,  SFP/“special”  modes, 
Perspective,  Batch  processing.  Post-lighting 

Pre-processing  tools 

2D  &  3D  image  edit 

2D  image  rank  filter 

threshold/background  contrast 

n-D  image  edit,  n-D  filters,  image  restoration  z-correction, 
morphological  filters,  math  operations 

Tnteractive  controls 

Visualization  parameters 

Slice  positions  rotation  angles 
data  threshold  animation  controls 

All  render  parameters,  Data/view  angles,  View 

Zoom  &  pan,  “Real  time”  control 

Measurements  on  image 
Measurements  on  views 

2D  measures  on  slices 

3D  measures  on  slices,  n-D  measurements 

3D  measures  on  views,  n-D  measurements 

Simultaneous  measures 
on  image  &  views 

Multiple  measurements  on  screen 

Measures  auto,  Tracked  in  both  displays 

aThe  range  of  3D  algorithms  indicates  diversity  of  modes  for  tackling  various  kinds  of  image  data,  while  the  quick-look  modes  include  general  sectioning  and  the  fastest 
voxel  algorithms.  Simple  projections  and  section  movies  are  always  faster  than  more  sophisticated  reconstruction  modes.  b  Control  of  the  visualization  process  suggests  the 
range  of  parameters  that  the  user  can  modify  directly  to  influence  the  resulting  views.  More  controls  give  more  flexibility  but  also  more  complexity  in  use. 

CA  very  rough  idea  of  the  level  of  interactive  control  (i.e.,  rapid  changes  of  the  result  in  response  to  mouse  clicks,  etc.)  for  visualization  and  for  measurements  on  source 
image  data  and/or  reconstructed  views.  n-D  =  any  number  of  multiple  dimensions. 
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WHAT  CONFOCAL  LASER  SCANNING 
MICROSCOPY  IMAGES  CAN  THE 
VISUALIZATION  SYSTEM  HANDLE? 

Image  Data:  How  Are  Image  Values  Represented 
in  the  Program? 

Storing  the  Image  Values 

All  digital  microscopes  produce  image  data  values  from  analog- 
to-digital  (A/D)  converters.  These  will  be  packed  into  one  or  more 
8-bit  byte  values,  each  of  which  may  have  an  uncalibrated  inten¬ 
sity  range  of  0  to  255.  Greater  collection  precision  is  possible  by 
integration,  averaging,  or  other  high-dynamic-range  filtering  oper¬ 
ations.  However,  improvements  in  electronics  and  detectors 
now  make  possible  the  direct  acquisition  of  12-bit,  16-bit,  or  even 
higher  precision  digital  data.  Single-byte  storage  is  more  efficient, 
and  is  adequate,  for  the  majority  of  purposes,  particularly  for 
results  from  low  photon-flux  imaging  of  living  cells.  It  is  supported 
by  all  packages.  Some  instruments  allow  16-bit  storage  (a  range 
of  0  to  65,535).  Intensity  data  digitized  by  12-bit  A/D  converters 
(standard  in  most  current  LSMs)  is  usually  packed  and  unpacked 
into  16-bit  words  by  the  loss-less  conversion: 

/i6  =  / 12  x  16  +  15 

This  slightly  cumbersome  conversion  is  necessary  to  correctly 
rescale  values  in  the  range  0  to  (2"  -  1)  without  any  rounding 
errors.  The  16-  to  12-bit  operation  is  rapidly  achieved  by  bit-shift¬ 
ing  the  binary  12-bit  values  towards  the  high  byte  and  filling  the 
additional  4  bits  with  1  s.  This  operation  can  be  precisely  reversed 
for  any  integer  value  in  the  range.  Sixteen-bit  processing  of  orig¬ 
inal  8-bit  or  12-bit  data  may  also  be  desirable  for  derived  images 
such  as  those  from  some  fluorescence  ratio  experiments.  However, 
this  is  excessive  for  the  majority  of  confocal  fluorescence  images, 
which  seldom  record  more  than  a  few  hundred  photons/pixel  and 
therefore  have  10%  to  20%  shot  noise  (see  Chapter  2,  this  volume). 
Microscopy  image  pixel  values  and  views  are  economically  rep¬ 
resented  by  integer  values.  Permanent  floating-point  storage  is 
rarely  supported.  Floating-point  calibrations  of  integer  data  are 
discussed  in  the  following  section. 

A  distinction  should  be  made  between  storage  and  display  pre¬ 
cision.  Historically,  some  digital  microscopy  systems  have  used 
the  display  memory  as  both  a  recording  and  a  view  buffer  with  a 
built-in  image  or  signal  processor.  Current  approaches  use  a  virtual 
display  in  main  computer  memory,  which  is  copied  to  the  display 
view,  allowing  decoupling  of  data  and  display  view  and  greater 
storage  precision  than  video  memory  if  required.  This  is  necessary, 
for  example,  for  storing  and  displaying  intermediate  results  in 
image  restoration  programs. 

Image  processing  systems  developed  for  cross-platform 
compatibility  (see  Amira,  Image/volumeJ,  and  AVS  examples  of 
packages  running  over  four  platforms)  have  always  used  virtual 
displays  allowing  arbitrary  precision  images  and  views  to  be 
manipulated  with  as  little  as  5-  or  6-bits  of  display  depth  per 
primary  color.  The  price  of  this  flexibility  used  to  be  a  significant 
reduction  in  interactive  visualization  and  display  speed,  caused  by 
the  loss  of  direct  processor  access  to  the  video  memory.  One  solu¬ 
tion  is  to  isolate  platform-specific  accelerations  and  link  them  to 
core  routines  when  porting  to  high-performance  workstations  with 
non-standard  hardware.  Although  this  approach  allows  the  rapid 
introduction  of  new  proprietary  hardware,  it  has  now  been  almost 
universally  superseded  by  the  use  of  agreed  platform-independent 
hardware  standards  with  a  defined  software  interface.  Of  the 


several  contenders  for  a  universal  graphics  standard  the  clearly 
adopted  winner  is  the  OpenGL  scheme  [see  http://www. 
OpenGL.org  and  Woo  (1999)  for  details  of  the  OpenGL  software 
programming  interface].  This  evolving  scheme  adds  definitions  for 
the  handling  of  new  technologies  as  they  are  introduced  into  each 
newly  released  OpenGL  compatible  display  card  or  system. 

Calibrating  the  Image  Data  Values 

Multi-dimensional  microscopy  instruments  provide  the  means 
for  obtaining  accurate  and  repeatable  quantitative  measurements. 
All  parameters  including  calibration  must  be  linked  to  the  corre¬ 
sponding  image  for  inclusion  in  subsequent  processing  stages.  A 
discussion  of  file  formats  follows  the  section  on  image  dimensions. 
Software  packages  normally  use  their  own  internal  calibration 
structures  because  most  of  the  general  or  so-called  standard  image 
formats  do  not  support  all  the  parameters  necessary  to  fully 
describe  multi-dimensional  microscopy  data. 

It  might  be  thought  desirable  to  store  directly  calibrated  real 
number  data  values.  A  fixed-precision  mantissa  and  exponent 
would  certainly  provide  consistent  relative  accuracy,  regardless  of 
the  magnitude  of  the  data  values.  Constant  precision  could, 
however,  be  maintained  by  using  a  logarithmic  digitization  (or 
detector)  response.  This  is  consistent  with  the  fact  that  the  presence 
of  shot  noise  means  that,  if  gray  levels  are  separated  by  one  stan¬ 
dard  deviation  they  must  become  wider  as  the  signal  level 
increases.  More  bits  would  then  be  assigned  to  low  intensities  and 
less  to  brighter  values.  A  fixed  precision  (log)  calibration  could  then 
be  attached  to  the  8-  or  16-bit  integer  data  values.  The  minimum 
requirement  is  a  floating-point  offset  (the  real  value  of  a  0  pixel), 
an  increment  (the  real  increment/pixel  value),  and  at  least  a  text 
label  or  key  for  the  linear  parameter  represented  [e.g.,  log  (inten¬ 
sity),  concentration,  pH,  etc.].  Nonlinear  changes  require  a  look-up 
table  (LUT)  for  calibrations.  Multiple-channel  images  require  sep¬ 
arate  calibrations  for  each  component.  Ion  imaging  data  need  at 
least  a  fixed  precision  calibration  and  often  a  sigmoidal  scale 
(defined  by  l?min,  /fmax,  and.K';  e.g.,  Bolsoveref «/.,  1995).  Table  14.3 
summarizes  data  value  calibration,  arithmetic,  and  measurement 
requirements  for  a  multi-dimensional  visualization  system. 

What  Dimensions  Can  the  Images  and 
Views  Have? 

Programmable  scanning  capabilities  of  all  LSM  instruments, 
motorized  focus  and/or  xy-stage  control  of  any  microscope,  and 
spectral  or  time-lapse  capabilities  yield  images  with  a  number  of 
spatial,  temporal,  and  other  dimensions.  Point-scanning  LSM 
instruments  normally  acquire  a  temporal  (sequential)  and  spatial 
(line)  scan  in  the  x-axis,  repeated  at  further  time  points  and  option¬ 
ally  at  progressive  y-  and/or  z-axis  positions.  Hence,  spatial  and 
temporal  sampling  dimensions  are  simultaneously  generated.  In 
this  way,  xy,  xt,  xz,  etc.,  2D  sections  and  xyz,  zyt,  xzy,  xzt,  etc.,  3D 
volume  images  are  collected.  Time-lapsed  volumetric  (e.g.,  xyzt, 
etc.)  or  multi-channel  spectral  (e.g.,  xyzc,  xyct,  etc.)  are  examples 
of  4D  images.  Once  considered  no  more  than  a  curiosity  by  biolo¬ 
gists,  new  dimensions  of  data  are  becoming  routine.  The  possible 
five-dimensional  (5D)  (x,  y,  z,  t,  c)  imaging  space  can  now  be  aug¬ 
mented  with  xy-  (stage)  position  (5x,  5y,  5z),  spatial  rotation  (0,  ([), 
V),  lifetime  (x),  polarization  angle  (P),  polarization  anisotropy  (r). 
This  makes  3D  to  6D  data  (from  12  or  more  possible  dimensions 
available  on  a  given  system)  a  routine  target  for  data  management. 

Visualization  systems  need  to  support  multi-channel  images 
(Tables  14.3,  14.4,  14.6).  Although  ultimately,  each  channel  is 
processed  separately  and  the  results  merged  together  for  display, 
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TABLE  14.3.  Overview  of  Image  Data  Handling  Features  for  Visualizing  Multidimensional  Biological  Microscopy  Data 


Data  handling  feature 

Parameter 

Minimum  required 

Desirable  additional  enhancements 

“Data  storage 

Types 

byte 

Integer,  fp,  real 

Bits 

8,  24  (3  x  8) 

12/16,  24  (3  x  8),  36/48  (3  x  12/16),  n  x  8 

Channels 

R,G,B,  included,  Merge  function 

Arbitrary  no.  of  channels  n-channel  merge 

bCalibration  of  intensities 

Linear,  Offset,  Range 

Non-linear,  Log,  Sigmoidal,  arbitrary 

Intensity  measures  (distribution 

2D  Point,  2D  Line 

3D  point,  3D  line,  trace, 

of  pixel  values) 

2D  Arbitrary  area 

3D  area  (surface). 

Summed  area  volume 

Arbitrary  3D  volume, 

2D  histogram 

Results  histograms, 

ASCII  file  output 

DDE  to  Excel 

Math  operations 

+,—,/,*  logical,  Contrast/gamma  mapping, 

Trig  functions,  Log 

Manual  z-weighting 

Auto  z-weighting 

aAll  systems  support  8-bit  (byte)  data  types.  A  few  allow  higher  precision.  This  is  useful  for  high  dynamic  range  images.  The  use  of  8-bit  indexed  or  16-bit  “hi-color” 
modes  for  multi-channel  data  is  now  less  common  than  24-bit  RGB  support.  Most  scientific  CCD  cameras  and  LSMs  now  support  12-bit  data  (usually  packed  into  16-bit 
words)  but  few  packages  support  these  data  types  for  visualization. 

bIt  is  important  to  clearly  distinguish  calibration  of  the  intensity  data  values  from  the  image  dimensions  (Table  14.4).  Calibrated  intensities  also  allow  real  values  of  pH. 
Ca2+,  etc.  and  other  concentrations  to  be  visualized. 


visualization  packages  must  now  manage  these  parallel  operations 
seamlessly  in  order  to  show  multi-channel  changes  interactively. 
This  is  particularly  important  where  interaction  between  the  values 
across  channels  is  required  by  the  chosen  algorithm  (e.g.,  the 
Imaris  SFP  algorithm  allows  transparency  in  one  channel  to  alter 
the  simulated  light  emission  from  another  fluorochrome  channel). 

Image  editing  is  required  to  extract  (1)  subregions  of  a  large 
data  set  or  (2)  a  structure  from  the  complexity  of  surrounding  fea¬ 
tures.  Sub-region  editing  should  be  available  through  each  of  the 
many  possible  dimensions  of  the  data.  All  these  dimensions  must 
be  appropriately  treated,  for  example,  correctly  calibrated,  if  the 
results  are  to  have  quantitative  validity. 

Image  Size 

Maximum  image  dimensions  should  support  the  full  resolution  of 
the  instrument  (see  Table  14.4).  In  extreme  cases,  several  adjacent 
sections  or  even  volumes  may  be  co-aligned  (by  correlation  and 
warping)  and  tiled  together  to  form  a  single  giant  data  set  (e.g., 
Oldmixon  and  Carlsson,  1993).  Generally,  total  image  size  should 
be  limited  only  by  the  available  memory.  Virtual  memory  man¬ 
agement  provides  transparent  swapping  of  programs  and  data 
between  RAM  and  disk.  This  increases  run-time  significantly  but 
can  enable  very  large  data  sets  to  be  processed.  Many  software 
developers  prefer  to  implement  a  proprietary  mechanism  of  image 
caching  or  data  swapping  between  RAM  and  disk,  even  with  the 
built-in  capabilities  of  the  Windows  family  of  operating  systems. 
The  best  way  to  minimize  these  overheads  is  by  careful  crafting 
of  the  visualization  algorithm.  The  plummeting  price  of  RAM 
makes  the  use  of  ever  more  memory  irresistible  by  the  program¬ 
mer,  and  thus  inevitable  by  the  end  user. 

Anisotropic  Sampling 

Most  multi-dimensional  microscopes  are  operated  with  different 
sampling  steps  in  two  or  more  axes.  Visualization  software  must 
produce  views  with  correctly  proportioned  dimensions  and  prefer¬ 
ably  have  the  ability  to  expand  or  contract  each  individually  (Table 
14.4),  for  example,  artificially  expanding  the  ^-dimension  of  an 
image  through  a  thin  preparation  (such  as  a  biofilm  or  a  stratified 
tissue)  to  highlight  the  morphology  in  each  layer.  The  most  concise 
way  of  specifying  this  aspect  ratio  information  is  to  apply  a  cor¬ 


rection  factor  to  the  appropriate  axis  calibration.  This  should  be 
done  interactively  so  that  some  imaging  distortions  can  be  cor¬ 
rected  (e.g.,  for  a  specimen  such  as  skin  with  layers  of  different 
refractive  index).  This  does  not  change  the  data  values  in  any  way 
and  is  preferable  to  resampling  the  entire  data  volume,  which 
would  tend  to  use  up  precious  memory.  When  the  data  is  subse¬ 
quently  processed  or  displayed,  a  floating-point  z-stretch  parame¬ 
ter  (and  equivalents  for  x,  y,  etc.)  would  correctly  specify  the 
spacing  of  each  plane.  An  integer  z-fill  parameter  represents  the 
number  of  equally  spaced  sections  to  optionally  add  between  each 
of  the  repositioned  planes.  These  extra  data  values  are  derived  by 
interpolation,  by  pixel  replication  or  linear,  cubic,  or  higher  poly¬ 
nomial  spline.  An  obvious  question  arises  here:  How  real  are  the 
extra  data  points?  A  priori  knowledge  of  the  specimen  and  imaging 
system  is  required  for  an  informed  choice.  On-the-fly  data  expan¬ 
sion  during  processing  will  conserve  storage  space  but  requires 
more  computations.  Pre-expansion,  for  example,  during  the  image 
loading  cycle,  will  optimize  processing  speed  at  the  expense  of 
memory.  A  good  compromise  is  rapid  expansion  during  the  com¬ 
putation  using  precalculated  linear  geometric  LUTs. 

Calibrating  the  Image  Space 

To  make  measurements,  image  and  view  dimensions  must  have  the 
correct  calibrations  (Table  14.4).  These  must  be  updated  during 
any  resampling,  zooming,  and  image  editing.  Minimum  require¬ 
ments  for  each  dimension  are  again  floating-point  values  for  offset 
and  increment,  and  an  axis  label.  Warping  conveniently  handles 
nonlinear  dimensions  by  resampling  onto  a  rectilinear  grid.  Cor¬ 
rection  of  acquisition  errors  should  ideally  be  incorporated  into  a 
single  intensity  and  sampling  interpolation.  These  errors  include: 

•  Spherical  aberration  caused  by  mismatch  between  the  refrac¬ 
tive  index  of  the  immersion  medium,  the  imbibing  medium  and 
the  design  of  the  objective. 

•  Axial  chromatic  aberration  (a  focus  shift  seen  with  all 
objectives). 

•  Lateral  effects,  such  as  chromatic  magnification  error. 

•  Photometric  signal  attenuation  and  correction  of  geometric 
distortions  from  refraction  within  the  sample  (e.g.,  Carlsson, 
1991;  Visser  et  al,  1991,  1992;  Fricker  and  White,  1992;  Hell 
et  al.,  1993)  are  desirable  preprocessing  tools. 
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TABLE  14.4.  Overview  of  Desirable  Image  and  View  Dimension  Parameters  for  Visualizing  Multi-Dimensional  Biological 

Microscopy  Data 


Feature 

Parameter 

Minimum  required 

Desirable  additional  enhancements 

“Image  dimensions 

Single  plane 

Full  un-edited  image  (from  camera, 

Unlimited 

Total  image  size 

LSM  etc),  Held  in  RAM 

Fully  sampled  3D  image, 

Display  independent 

Multiple  images  in  RAM 

Unlimited,  Display  independent,  Multiple  3D  images  in  RAM, 

bSupported  dimensions 

3D  image  in  RAM 

2D,  3D 

n-D  images  in  RAM,  efficient  caching 

n-D,  View  angles,  Rotation  angles,  Stage  position,  Polarization/ 

Editing  the  dimensions 

Sub  regions  (ROI) 

x,y,z  time 

2D  sub-area,  3D  sub-volume,  edit 

anisotropy.  Lifetime 

3D  arbitrary  sub  volume,  Edit  in  view,  3D  cut-away,  n-D  ROI 

(geometric  operations) 

cData  corrections 

on  slices 

Background  normalization 

Non-linear  corrections,  Photobleaching,  Flat  field,  n-D 

Z-geometry 

z- stretch 

z-atttenuation 

Integer  value 

corrections,  Optical  corrections,  Image  restoration 

Real  value 

z-fill 

linear 

Integer  for  large  angles 

Non-linear  (e.g.,  cubic,  etc.) 

Adaptive  for  chosen  angles 

View  dimensions 

Single  view 

3D  diagonal  of  image 

Unlimited,  Display  independent 

Number  of  views 

View  movie  in  RAM 

JPEG  compression 

120  views  (360  x  3degs) 

Multiple  views  in  RAM 

Efficient  caching.  Efficient  compression 

Unlimited,  Display  independent 

Channels  in  view 

R.G.B 

Multiple  movies  in  RAM 

Arbitrary  no.  of  channels 

Calibration  of 

Image 

X,y,z,t 

All  dimensions 

dimensions 

View 

X,y,z,t,  angle 

All  dimensions 

Dimension  measures 

On  image 

2D  Point,  2D  Line,  2D  histogram, 

3D  point,  3D  line,  trace, 

On  view 

2D  Arbitrary  area,  ASCII  file  output, 
Summed  area  volume, 

3D  area  (surface),  Arbitrary  3D  volume, 

Results  histograms,  DDE  to  Excel 

3D  point,  3D  line,  Trace,  3D  area  (surface).  Arbitrary  3D  volume, 
Results  histograms  DDE  to  Excel 

aIn  most  cases  the  image  data  space  is  limited  only  by  the  amount  of  disk  and/or  memory  available.  The  operating  system  and/or  the  application  program  may  provide 
virtual  memory  management  and  disk  caching. 

bMost  packages  can  handle  time,  spectral,  etc.,  data  as  for  a  “z-stack”  but  few  can  directly  interpret  time  or  wavelength  calibrations  with  any  meaning. 
c  Complex  corrections  usually  involve  sample-specific  data  and  some  pre-processing. 

d  Calibration  of  dimensions  should  be  clearly  distinguished  from  those  of  the  data  intensity  values  (Table  14.3). 
n-D  =  any  number  of  multiple  dimensions. 


Standard  File  Formats  for  Calibration 
and  Interpretation 

While  there  are  many  standard  formats,  there  is  no  universal 
standard  currently  adopted  for  microscope  images.  However, 
there  are  established  imaging  formats  (e.g.,  DICOM,  see 
http://medical.nema.org/)  that  are  routinely  used  by  visualization 
packages  such  as  the  Cedara,  Vital  Images,  and  Analyze  software 
that  fully  describe  multi-dimensional  volume  data  from  medical 
scanners.  As  LSM  and  other  research  microscopes  become  more 
routinely  used  as  screening  instruments  and  for  clinical  applica¬ 
tions,  it  is  hoped  that  such  standards  will  become  routine  from 
these  suppliers  as  well.  A  catch-all  image  input  facility  such  as  the 
RAW  options  offered  by  many  programs  allows  any  packed  binary 
file  with  a  fixed-size  header  to  be  read  in.  Microscope  instrument 
manufacturers  have  taken  one  of  two  options:  (1)  developed  a 
completely  proprietary  structure  and  made  this  available  to  other 
developers  and  users,  or  (2)  taken  an  existing  extendable  format 
(such  as  the  Tagged  Image  File  Format  [or  TIFF])  and  added 
system-specific  components  (e.g.,  for  TIFF,  licensed  specific  tags) 
to  store  the  extra  acquisition  parameters.  A  problem  with  this 
second  approach  is  the  necessary  proliferation  of  a  number  of  vari¬ 
ants  or  compliance  classes  of  such  formats.  Any  third-party  reader 
program  must  recognize  (and  provide  updates  for)  several  differ¬ 
ent  versions.  A  widely  adopted  alternative  is  to  use  a  proprietary 


structure  and  to  provide  conversion  tools  to  import/export  data  via 
standard  formats.  Unsupported  parameters  are  transferred  into  the 
program  by  an  ad  hoc  semi-manual  process. 

A  flexible,  industry-standard  approach  to  image-related  details 
is  to  use  a  conventional  database  to  store  preprogrammed  fields  of 
information  (  sometimes  a  third-party  software  product  is  used  with 
the  visualization  tool  —  such  as  the  ImageAccess  database  used 
by  Imaris  —  which  can  manage  all  image  and  image-related  files). 
Two  types  of  information  must  be  stored  and  linked  with  each 
image:  (1)  instrument-specific  details  describing  the  instrument 
settings  used  for  the  collection  and  (2)  image-specific  information 
describing  the  dimensions,  calibration,  and  experimental  details. 
The  database  can  hold  both  sets  of  details,  together  with  a  pointer 
to  the  image  data  (or  even  the  image  data  itself  for  small  images). 
Alternatively,  the  database  may  hold  just  the  system  configurations 
used  as  stored  settings  or  methods  (e.g.,  the  Bio-Rad  Lasersharp 
program  stores  all  the  instrument  and  user  settings  in  a  Microsoft 
Access  database).  This  latter  approach  requires  a  pointer  to  the  rel¬ 
evant  settings  to  be  saved  with  the  image  data,  separately  from  the 
database.  Table  14.5  summarizes  some  important  image  and  view 
data  parameters. 

Processing  Image  Data 

Less  obvious  than  the  storage  representation  are  the  data  type  and 
precision  used  during  computations.  Floating-point  representations 
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TABLE  14.5.  Overview  of  Desirable  File  Format  and  Image  Information  Features  for  Visualizing  Multi-Dimensional 

Biological  Microscopy  Data 


Feature 

Parameter 

Minimum  required 

Desirable  additional  enhancements 

“Image  file  format 

Proprietary  standard 

Fully  defined  open  source 

Multi-file  TIFF,  AVI  (for  series) 

Fully  defined  open  source,  Full  range  of  conversions, 

A  Universal  standard! 

View  file  format 

Proprietary  standard 

Fully  defined  open  source 

Multi-file  TIFF/BMP,  JPEG,  AVI 

Fully  defined  open  source,  Full  range  of  conversions. 
Efficient  compression,  A  Universal  standard! 

bGeneral  params.  stored 

Size 

All  dimensions 

All  dimensions 

Calibration 

X,y,z,t  dimensions 

All  dimensions 

Annotation 

ROIs 

ROIs,  Text,  pointers 

Microscope 

Notes 

Data  specific  parameters 

Stored  in  image  format,  or  ASCII  file 

All  instrument  parameters.  Experimental  parameters. 
Stored  in  image  format  and/or  database 

User/exp  notes,  Informatics 

Special  view  parameters 
stored 

Source  image 

Orientation  etc. 
Visualization  parameters 

Image  filename,  Links  in  database  etc.,  Visualization 
history/log 

Rotation  angles,  etc. 

Algorithm  name,  Algorithm  parameters,  Display  options 

a  Proprietory  file  formats  are  used  by  most  systems.  “Standard  formats”  such  as  TIFF  may  also  cause  confusion  as  there  are  many  different  compliance  classes  of  TIFF,  so 
only  a  sub-set  of  the  TIFF  tags  in  a  particular  file  may  be  recognized  by  a  given  reader. 

bSome  parameters  may  be  stored  with  the  image  data  in  the  same  file,  in  a  separate  (e.g.,  ASCII)  file  or  in  a  database.  It  is  important  to  know  the  whereabouts  of  this  infor¬ 
mation  if  the  image  is  to  be  taken  to  another  program  with  the  associated  data  intact.  ROI  =  region  of  interest. 


reduce  rounding  errors  during  geometric  transformation  interpola¬ 
tions.  Even  this  requirement  for  floating-point  representation  can 
be  partly  avoided  by  either  (1)  combining  several  interpolation 
steps  into  a  single,  composite  geometric  and  photometric  trans¬ 
formation,  or  (2)  increasing  the  sampling  by  a  factor  of  at  least  2 
for  each  subsequent  interpolation.  This  second  approach  is  some¬ 
what  extravagant  in  terms  of  storage  and  will  not  help  if  the  sam¬ 
pling  is  already  at  the  Nyquist  limit.  The  processor  architecture  is 
an  important  factor  in  determining  the  processing  speed.  Fast 
multi-word  and  floating  point  arithmetic  is  now  standard  in  micro¬ 
processors.  Despite  this,  some  instruments,  notably  the  Zeiss  range 
of  LSMs,  use  specialized,  programmable  digital  signal  processors. 

Processor  Performance:  How  Fast  Will  My 
Computer  Process  Images? 

Personal  computer  (PC)  performance  for  image  manipulation  is 
constantly  improving,  making  the  specification  of  system  perfor¬ 
mance  in  a  text  such  as  this  somewhat  pointless.  However,  the 
principal  components  of  the  computer  system  required  can  be 
described  in  terms  of  their  relative  importance  to  performance. 

At  the  time  of  writing,  the  Pentium  PC  processors  are  the 
norm,  running  at  around  4  GHz  with  bus  speeds  around  1  GHz. 
These  are  very  approximately  30  times  faster  than  10  years  ago, 
representing  a  doubling  of  speed  every  2  years.  Non-Intel  proces¬ 
sors  with  alternative  combinations  of  price/performance  through 
low  power  consumption,  higher  capacity  of  on-chip  memory  for 
data  caching,  and  other  enhancements  appear  from  time  to  time 
with  advantages  in  different  applications.  Alternative  Intel  proces¬ 
sors,  such  as  the  Xeon,  also  compete  in  these  areas  and  offer 
improved  workstation  and  multi-processor  performance.  Provided 
the  software  is  correctly  designed,  transfer  bottlenecks  can  be 
reduced  with  a  processor  having  at  least  5 12  kB  of  level  2  memory 
cache.  Apple  Macintosh  machines  have  undergone  something  of  a 
renaissance  in  recent  years;  the  current  G5  is  broadly  equivalent 
to  the  latest  Pentium  devices,  and  still  have  competitive  and  equiv¬ 
alent  components  for  efficient  numerical  performance  and  a  highly 
optimized  bus  for  image  transfers.  The  current  Macintosh  OS  X 


operating  systems  have  been  significantly  updated  and  based  on 
Unix  technology  in  order  to  take  advantage  of  the  large  software 
developer  base.  Unix  workstations  are  still  a  costly  alternative  to 
ever-improving  PC  platforms.  Improving  processor  performance 
alone  is  still  reflected  in  the  voxel  rendering  performance  (in 
voxels/second  or  vps)  for  visualization  of  multi-dimensional 
microscopy  data. 

Improvements  in  other  areas  of  the  PC  have  been  either 
necessary  to  keep  pace  with  the  processor  speeds  or  provided 
enhanced  capabilities  directly.  Hard  disk  drive  data  transfer  speed 
can  limit  the  speed  of  animations  (movies)  and  3D  visualization 
when  applied  to  large  data  sets.  At  the  time  of  writing,  so-called 
ultra-fast,  ultra-wide  SCSI  interfaced  devices,  with  capacities  up 
to  250  Gb  per  disk  still  have  higher  performance  than  IDE  devices 
and  tend  to  be  more  robust  and  easier  to  upgrade,  although  plug- 
and-play  technology  makes  this  last  issue  less  important  in  the 
latest  PCs.  The  latest  PCs  and  Macintosh  computers  can  access 
4  to  8  Gb  of  RAM.  This  is  adequate  for  most  multi-dimensional 
data  sets  but  will  inevitably  still  limit  performance  if  many  large 
data  sets  are  opened  simultaneously,  especially  if  the  software  is 
designed  to  read  in  the  entire  data  set  and/or  the  system  caching 
or  swap  file  is  inefficiently  configured  for  the  ratio  of  disk  to  RAM. 

Computer  video  display  subsystems  have,  over  the  last  10 
years  or  so,  taken  on  more  and  more  graphics  processing  opera¬ 
tions,  allowing  greater  optimization  and  relinquishing  the  general 
purpose  CPU  for  other  tasks.  The  display  system  has  become  a 
dedicated  graphics  processing  unit  (GPU)  and  up  to  256  Mb  of 
dedicated  display  memory.  Some  entry-level  PCs  may  still  use  por¬ 
tions  of  main  memory,  set  aside  for  access  by  the  display,  but  these 
are  not  recommended  for  multi-dimensional  visualization.  Many 
functions  are  carried  out  by  dedicated  hardware  and/or  firmware 
running  on  the  GPU  and  associated  devices.  Depending  on  the 
sophistication  of  the  chosen  graphics  system,  supported  operations 
may  include: 

•  Rapid  display  of  2D  views  for  animations,  etc. 

•  Rendering  of  surfaces  composed  of  triangle  or  other  polygon 

primitives. 
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Voxel  Rendering  Speed 


(150MHz)  (150MHz)(150MHz)(200MHz)(200MHz)(200MHz)  (1.5GHz) 

FIGURE  14.4.  Voxel  rendering  speed:  dependence  on  hardware  perfor¬ 
mance.  When  assessing  a  visualization  system,  many  factors  need  to  be  taken 
into  account  (see  text).  Parameters  for  individual  performance  figures  should 
be  assessed  with  care.  On  standard  platforms,  processor  speed  is  still  an  impor¬ 
tant  factor.  From  a  simple  ratio  of  voxels  processed  per  second  per  MHz  proces¬ 
sor  speed  (with  a  basic  algorithm)  a  figure  of  merit  can  be  obtained.  Running 
the  same  a-blended.  voxel-rendering  software  (Voxblast)  on  multiple  Pentium 
processors  gives  broadly  the  same  figure  of  merit  as  single  processors 
when  normalized  for  the  total  processor  speed.  The  same  is  found  for  multi¬ 
processor  graphics  workstations.  Running  on  special  hardware  (this  example 
is  the  VolumePro  board)  sacrifices  portability  across  platforms  but  (as  has  his¬ 
torically  always  been  the  case)  gives  vastly  improved  performance. 


•  Geometric  manipulations  (e.g.,  warping  of  data  vectors). 

•  Rendering  of  voxel  objects. 

•  Transparency/opacity  of  graphics  and  voxel  objects. 

•  Artificial  lighting  and  shading. 

•  Rapid  manipulation  of  color  or  grayscale. 

•  Panning  and  zooming  of  the  display. 

•  Texture  mapping  (used  to  rapidly  render  image  layers  onto  a 
growing  display  view). 

Figure  14.4  shows  some  approximate  voxel  rendering  speeds 
that  might  be  expected  over  a  range  of  processing  platforms  (data 
corresponds  to  rendering  speeds  of  Voxblast  from  Vaytek  Inc.). 
Optimization  of  the  GPU  functions  is  controlled  largely  by  the  sup¬ 
plied  driver  software  and  contributes  to  the  major  differences 
between  various  hardware  configurations.  CPU  operations  are 
coded  by  the  application  programmer  and  this  is  also  reflected  in 
the  performance  of  the  software.  The  relative  efficiency  of  these 
two  aspects  can  have  an  important  influence  on  the  effectiveness 
of  the  package  as  a  whole.  An  effective  visualization  algorithm  for 
multi-dimensional  data  must  include  optimized  numerical  loops 
(particularly  nested  sequences)  and  use  fast  indexing  into  pre- 
processed  parameter  LUTs  for  frequently  used  values.  The  fol¬ 
lowing  section  describes  some  of  the  optimizations  that  are 
responsible  for  the  performance  range  seen  between  different 
programs. 


HOW  WILL  THE  SYSTEM  GENERATE  THE 
RECONSTRUCTED  VIEWS? 

Assessing  the  Four  Basic  Steps  in  the  Generation 
of  Reconstructed  Views 

(1)  The  image  (or  a  subregion)  is  loaded  into  the  data  space 
(an  area  of  computer  memory).  Preliminary  image  editing,  prepro¬ 
cessing,  and/or  analysis  is  used  to  define  calibrated  image  values 
with  known  dimensions.  This  constitutes  the  input  to  the  visual¬ 
ization  process.  Packages  such  as  Analyze  have  some  useful  pre¬ 
processing  capabilities  (Robb,  1990).  Alternatively,  a  more  general 
program  such  as  Metamorph,  ImageJ,  and  Image  Pro,  etc.,  can  be 
run  alongside  the  visualization  package  for  image  preprocessing 
(or  the  visualization  component  can  be  added  to  the  2D  package). 

(2)  A  view  must  be  chosen  (subject  to  the  available  recon¬ 
struction  algorithms)  that  will  produce  the  most  flexible  and  appro¬ 
priate  representation  of  the  image  data.  An  intelligent  choice  of 
view  can  minimize  the  number  of  reprocessing  cycles  (see  also  He 
et  al.,  1996;  Marks,  1997;  Kindlmann  and  Durkin,  1998,  for 
attempts  at  computer-assisted  choice  of  visualization  algorithm). 
The  visualization  step  consists  of  two  transformations:  First,  a  geo¬ 
metric  orientational  mapping  of  the  image  space  into  the  reduced 
dimensions  of  the  view,  and,  second,  a  photometric  mapping 
(sometimes  called  a  transfer  function)  whereby  the  image  intensi¬ 
ties  are  processed  to  determine  the  brightness  of  a  pixel  at  each 
position  within  that  view. 

(3)  The  display  step  consists  of  a  second  geometric  and 
photometric  mapping  that  together  constitute  output  or  matching 
of  the  multi-dimensional  view  into  the  available  physical  display. 
In  practice  this  may  consist  of  scaling  and  copying  or  a  more 
complex  operation  (e.g.,  animation  or  stereoscopic  presentation). 
These  presentation  or  output  options,  dictated  by  the  display  space 
capabilities,  will  determine  the  most  efficient  use  of  screen  reso¬ 
lution,  color,  animation,  etc.  In  turn,  these  will  influence  the  choice 
of  appropriate  reconstruction  algorithm. 

(4)  Dimensional  loss  during  the  visualization  processing  is 
partly  restored  by  the  display  step  and  partly  by  a  priori  knowl¬ 
edge  used  to  interpret  the  2D  display  view  (e.g.,  depth  cues,  ani¬ 
mation,  stereo,  etc.).  Inspection  and  analysis  of  the  display  view 
is  the  last  step  of  the  visualization  process. 

The  next  sections  describe  more  details  of  these  processes  that 
are  important  for  microscopy  images  using  techniques  found  in  the 
systems  listed  in  Table  14.1. 

Loading  the  Image  Subregion 

Any  image  processing  program  must  first  open  the  image  file,  read 
and  interpret  the  file  format  information,  and  then  load  some  or  all 
of  the  data  values.  Interpretation  of  the  file  format  largely  deter¬ 
mines  the  flexibility  of  subsequent  manipulations.  An  on-screen 
view  of  (1)  image  sizes  (in  pixels  and  calibrated  units),  (2)  inten¬ 
sity  data  calibrations,  (3)  associated  notes  and  (4)  a  quick-look 
reconstruction  will  aid  selecting  the  required  image  or  subregion 
(Analyze  and  Imaris,  e.g.,  show  image  volumes  as  thumbnail  pic¬ 
tures  with  a  grid  showing  the  image  dimensions,  etc.).  The  file 
format  information  will  determine  whether  the  data  needs  resam¬ 
pling  to  produce  correct  image  proportions,  and  interactive 
adjustment  is  essential  for  z-stretch/fill  (see  above).  Interactive 
photometric  rescaling  or  segmentation  (in  combination  with  a 
simple  volume  representation)  are  essential  to  remove  (e.g.,  set  to 
zero  or  ignore)  background  values  that  would  unnecessarily 
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slow  down  computations.  Multiple  passes  through  the  data  will 
be  time  consuming  for  large  images.  A  single  composite  geomet¬ 
ric  and  photometric  resampling  (using  fast  LUTs)  should  be  com¬ 
bined  with  loading  the  data  from  disk.  Data  expansion  options 
(e.g.,  z-fill)  may  be  temporarily  restricted  to  speed  up  computa¬ 
tions  during  the  exploratory  stages.  Flexibility  and  control  at  this 
stage  must  be  balanced  against  the  efficiency  of  a  single  process¬ 
ing  stage.  Some  image  preparation  options  are  detailed  in 
Table  14.2. 

Choosing  a  View:  The  5D  Image  Display  Space 

As  introduced  above,  the  efficient  use  of  all  the  available  display 
space  greatly  increases  the  flexibility  of  visualization  algorithms. 

The  2D  Pixel  Display  Space 

Pixel  resolution  must  accommodate  a  reconstructed  view  at  least 
as  big  as  the  longest  diagonal  through  the  3D  volume.  This  means 
that,  for  example,  to  reconstruct  a768x512x  16  frame  3D  image 
with  ^-stretch  of  4  and  arbitrary  rotation,  the  display  view  that  will 
be  generated  in  memory  could  be  up  to  950  pixels  square.  For  the 
same  computation  and  display  of  1024  X  1024  pixel  frames,  1450 
pixels  square  are  required  to  avoid  clipping.  As  it  is  advisable  to 
display  the  views  without  resampling,  these  values  represent  the 
minimum  display  window  in  pixels.  Processing  time  may  dictate 
that  only  a  subregion  can  be  processed  but,  in  general,  display  of 
a  single  full  resolution  frame  should  be  considered  a  minimum. 
Although  the  display  pixel  range  may  be  only  8  bits/channel,  the 
ability  to  generate  projected  views  with  a  higher  intensity  range 
(perhaps  16  bits)  means  that,  for  example,  average  projections  of 
images  with  dimensions  greater  that  256  pixels  can  be  made  with 
no  loss  of  detail.  It  is  now  standard  practice  to  produce  an  output 
view  that  is  independently  rescaled  according  to  the  size  of  the 
display  window  as  defined  by  the  user.  This  is  transparently 
handled  by  the  software  and  display  driver.  However,  rescaling  can 
always  give  rise  to  unwanted  aliasing  effects  so  it  is  wise  to  restrict 
the  display  zoom  to  integral  multiples  of  whole  pixels.  The  appli¬ 
cation  program,  perhaps  using  features  of  the  display  driver,  may 
allow  an  image  of  greater  size  than  the  physical  display  to  be 
rapidly  panned,  giving  access  to  display  views  of  almost  unlimited 
size.  High-resolution  non-interlaced  displays  are  now  standard  for 
all  computers.  Although  a  single  display  may  deliver  resolutions 
up  to  2650  pixels,  it  is  now  standard  practice  to  provide  multiple 
screen  outputs  from  the  best  display  systems.  In  this  way,  desk¬ 
tops  of  perhaps  2560  X  1024,  3200  x  1200  or  more  can  be  spread 
seamlessly  across  a  pair  of  similar  monitors  placed  together.  The 
display  drivers  will  automatically  handle  traversing  of  the  mouse 
and  program  windows  between  the  physical  screens  or  even  allow 
large  images  to  straddle  the  entire  display  space.  (A  commonly 
recommended  supplier  of  multi-view  compatible  OpenGL  graph¬ 
ics  cards  for  PCs  is  nVidia,  http://www.nvia.com,  while  on 
Macintosh's  they  are  built  in.) 

The  Color  Display  Space 

Color  is  a  valuable  resource  for  coding  multi-dimensional  infor¬ 
mation  on  a  2D  device.  It  is  important  to  ascertain  the  number  of 
different  colors  that  can  be  simultaneously  displayed,  as  distinct 
from  the  number  of  possible  colors  present  in  the  data  or  display 
view.  The  author  prefers  the  generic  term  palette  to  represent  the 
subset  of  simultaneously  displayable  colors  and  color  resolution 
to  indicate  the  full  set  of  possible  colors  present  in  the  data. 
Although  many  different  graphics  display  resolutions  have  been 
used  by  imaging  systems  in  the  past,  the  plummeting  cost  of  hard¬ 


ware  have  made  the  use  of  all  8-bit  displays  redundant.  Useful 
display  systems  will  be  encountered  that  have  either  16,  24,  or  32 
bits  per  pixel  of  display  depth.  It  should  be  carefully  noted  that  a 
display  system  may  have  additional  memory  associated  with  each 
pixel  for  storing  other  values  important  for  controlling  the  display 
process,  but  here  we  are  concerned  only  with  the  color  and  inten¬ 
sity  information.  A  standard  24-bit  display  has  8  bits  of  memory 
storing  1  of  256  possible  values  each  of  red,  green,  and  blue  inten¬ 
sity.  Additional  display  panes  are  available  in  32  bit  modes.  With 
224  possible  display  colors  (over  16  million),  a  4000  x  4000  pixel 
image  could  display  each  pixel  with  a  different  and  unique  color. 
There  is,  therefore,  significantly  more  contrast  available  in  a  color 
image  than  in  a  monochrome  (e.g.,  grayscale)  representation. 
However,  it  must  always  be  remembered  that  the  actual  red,  green, 
and  blue  colors  corresponding  to  each  component  are  fixed  by  the 
spectral  characteristics  of  the  physical  screen  material.  These  will, 
in  general,  never  coincide  with  filter  spectra  of  the  microscope,  or 
even  with  the  nominal  red,  green,  and  blue  characteristics  of  a 
color  camera. 

When  more  than  24  bits  of  data  are  stored  for  each  pixel  of  the 
display  view,  such  as  for  a  3-channel  16-bit  color  image  (a  48-bit 
data  set),  the  visualization  software  must  resample  this  down 
to  the  available  color  space  of  the  display  system.  An  extreme 
example  is  the  option  in  the  Imaris  program  of  having  color-space 
computations  carried  out  with  96-bit  precision  as  opposed  to  24- 
bit  precision,  improving  accuracy  at  a  4-fold  cost  in  extra  memory. 
However,  at  present  cathode-ray  tube  (CRT)  and  liquid-crystal  dis¬ 
plays  (LCD)  are  capable  of  displaying  little  more  than  8  or  9  bits 
of  data  in  each  color  channel. 

Pseudo  Color 

Pseudo  or  indexed  color  was  used  by  older  8-bit  displays.  It  is  now 
only  really  important  when  a  display  system  of  very  high  resolu¬ 
tion  is  implemented,  for  cost  purposes,  with  16  bits  of  high  color 
pixel  depth.  Each  of  65,536  entries  (for  a  16-bit  mode)  or  256 
entries  (for  8-bit  modes)  in  a  color  look-up  table  (CO-LUT)  is 
assigned  a  unique  composite  RGB  value.  This  CO-LUT  is  an 
indexed  palette  and  can  be  rapidly  updated  or  modified  to  change 
the  displayed  colors  without  altering  the  view  data  values.  It  is  the 
CO-LUT  value  pointed  to  by  each  data  value  that  determines  what 
RGB  intensities  appear  on  the  monitor.  Visual  perception  of  color 
is  far  more  acute  than  for  intensity  in  bright  VDU  displays  because 
cone  density  in  the  eye  is  highest  in  the  fovea  (Perry  and  Cowey, 
1985).  Pseudo  color  is,  therefore,  useful  for  presentation  of 
calibrated  16-bit  maps  of  image  intensities,  for  example,  from  ion 
indicators.  In  general,  it  adds  contrast  to  subtle  changes  in  (1) 
brightness,  (2)  temporal,  or  (3)  coordinate  (typically  z-relief) 
views. 

True  Color 

True  color  means  any  display  element  can  have  independent  values 
for  each  of  its  red,  green,  and  blue  components.  The  simplest 
way  of  representing  these  in  a  byte-structured  format  is  a  24-bit 
(3x8  bit)  RGB  voxel.  Other  color-coding  schemes  are  possible. 
Hue,  saturation,  and  brightness  (HSB)  are  useful  for  intensity- 
independent  color  transformations.  Process  color  space  (using 
cyan,  magenta,  yellow  and  black  —  CMYK)  is  the  form  used  for 
hard  copy  and  publication.  RGB  values  map  directly  into  a  24-bit 
display  with  no  intermediate  processing.  Color  manipulations  can 
be  carried  out  by  modifying  the  component  color  data  directly. 
Alternatively,  each  8-bit  channel  may  be  driven  by  a  monochrome 
(R,  G,  or  B)  8-bit  indexed  CO-LUT  (palette). 
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Multiple-Channel  Color  Display 

Either  color  or  brightness  resolution  (or  both)  can  be  traded  for 
extra  channels.  A  hue  (preferably  non-primary,  i.e.,  magenta,  cyan, 
yellow,  orange,  etc.)  is  defined  by  a  unique  RGB  ratio  for  each 
channel.  An  intensity  (brightness)  scale  in  each  hue  then  represents 
the  indexed  data  values  of  that  channel.  All  channels  are  combined 
into  one  true-color  RGB  view,  adjusted  to  fit  the  display  resolu¬ 
tion,  that  is,  each  channel  using  a  segment  of  the  available  palette. 
Figure  14.5  includes  examples  of  the  use  of  color  for  3D  multi¬ 
channel  and  stereo  display. 

Animations 

As  with  pixels  and  color,  the  temporal  display  space  can  also  be 
effectively  used  for  visualization.  The  simplest  temporal  mapping 
is  the  sequential  display  of  view  sequences.  Temporal  range  is 
determined  by  the  number  of  frames  that  can  be  stored  for  rapid 
animation.  Time  resolution  has  two  components:  the  frame  rate 
(the  time  between  successive  views)  and  the  refresh  rate  (how  fast 


a  single  view  is  updated)  (Table  14.6).  The  refresh  rate  contributes 
significantly  to  the  perception  of  smooth  animation.  Retinal  per¬ 
sistence  results  in  a  screen  refresh  rate  of  <1/1 8th  of  a  second 
being  perceived  as  virtually  instantaneous.  For  through-focus 
sequences,  fading  between  frames  is  advantageous.  A  long-persis¬ 
tence  display  phosphor  (such  as  on  older  video-rate  monitors) 
assists  this  fading  process  for  low  framing  rates  <4  Hz  but  con¬ 
tributes  degrading  blur  at  higher  speeds.  Perception  of  smooth 
motion  requires  high  lateral  resolution  and  visual  acuity.  There¬ 
fore,  smooth  rotation  animations  require  (1)  fine  rotation  steps,  (2) 
a  short  persistence/high  refresh  rate  display,  and  (3)  an  animation 
frame  rate  of  above  10  Hz  (with  <0.05  s  data  refresh).  Hardware 
and  software  compression/expansion  (see  Chapter  48,  this  volume) 
are  built  into  some  display  systems,  allowing  suitable  data  to  be 
animated  at  up  to  video  rate  with  reduced  storage  requirements. 
Both  RAM-based  and  hard-disk-based  systems  can  now  easily 
provide  full-color  video-rate  animation.  Screen  update  is  improved 
by  a  double-buffered  display  comprised  of  both  a  visible  and  a 


FIGURE  14.5.  Efficient  use  of  the  5D  display 
space.  The  upper  four  images  are  two  “maximum 
intensity”  stereo-pairs  (Lasersharp  software). 
They  were  generated  with  ~8deg  of  “rotation” 
around  the  y  (vertical  axis)  giving  binocular 
stereoscopy  through  the  x  display  dimension  and 
similar  “tilts”  around  x  to  give  motion  parallax 
and  temporal  interpolation  depth  cues  using  the  y 
display  axis.  In  these  static  views,  the  top  pair  tilt 
forwards  and  the  lower  pair  tilt  backwards.  With 
few  exceptions,  the  most  efficient  use  of  the  color 
space  is  for  the  display  of  multi-channel  views. 
Data  (courtesy  of  Bio-Rad  Microscience)  is  from 
a  large  confocal  series  from  lung  tissue.  (A) 
shows  a  triple-stained  fluorescently  labelled  pan¬ 
creatic  islet,  with  each  channel  “maximum  pro¬ 
jected”  and  combined  into  a  24-bit  Windows 
bit-map  view  (data  collected  by  T.J.  Brelje, 
University  of  Minnesota,  Minneapolis,  MN).  (B) 
shows  a  dividing  shrimp  embryo  stained  with 
rhodamine-conjugated  beta-tubulin  antibody  (27 
five-micron  optical  sections)  rendered  using 
VoxelView/Ultra.  Cell  bodies  and  membranes 
(blue)  are  assigned  low  opacity  so  that  micro¬ 
tubule  structures  (orange-red)  can  be  viewed 
distinctly  in  their  cellular  context.  Analytical 
geometric  data  (lines  and  surfaces)  can  also  be 
inserted,  not  overlaid,  using  the  VoxelView/Ultra 
Embedded  Geometry  tool.  Embedded  lines  high¬ 
light  the  axes  of  division  and  centers  of  polarity 
of  the  dividing  cells  and  help  indicate  directions 
of  movement  during  mitosis.  (Data  courtesy  of 
Dr.  W.  Clark,  Jr.,  University  of  California, 
Bodega  Marine  Laboratory,  CA).  Intensity  gray 
levels  can  also  code  for  z-depth  (see  text).  (C) 
shows  a  Lasersharp  “height”  view  of  a  living 
fibroblast  imaged  by  fluorescence  optical  sec¬ 
tioning  of  an  excluded  FITC-dextran-containing 
medium  (see  Shotton  and  White,  1989).  (D) 
Binary  (single-bit)  line  graphics  make  inefficient 
use  of  display  space.  However,  this  “YMOD”  or 
height  profile  of  BCECF-stained  living  chondro¬ 
cytes  from  pig  articular  cartilage  (data  by  R.J. 
Errington,  Physiology  Dept.,  Oxford  University) 
does  show  relief  more  clearly  than  would  an 
intensity  view  alone. 
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TABLE  14.6.  Overview  of  Image  and  View  Display  Options  Desirable  Visualizing  Multi-Dimensional 

Biological  Microscopy  Data 


Feature 

Parameter 

Minimum  Required 

Desirable  additional  enhancements 

“Display  2D 

To  show  full  image  1 : 1  without  clipping 

As  large  as  possible,  Multiple-screens,  Seamless  desktop 

pixel  size 

(typically  needs  1280  x  1024  or  more) 

Color 

Display  mode 

24-bit  RGB 

24-bit  of  higher  -1-  overlays 

bMultiple-channels 

Standard  RGB 

Arbitrary  number  of  channels 

Screen 

Display  refresh  (Hz) 

70  Hz 

-100  Hz 

cSize  (diagonal  inches) 

CRT:  19 

CRT:  multi  21,  Flat:  multi  19,  Wide-aspect  &  Specialist 

Flat:  17 

Mega-pixel  screens 

Movie 

dFPS  (Hz) 

~10  at  full  image  resolution 

Variable  fps 

Up  to  about  30  fps,  Motion  compression,  Variable  fps 

Recording 

Store  movies  as  digital  file 

Digital  video  store,  Digital  Compression,  DVD  recorder, 

DVD  RAM 

Hardware  assisted 

z-buffer  video  system,  Compression  AV  quality  disk 

system 

Stereoscopic 

Modes 

Manual  stereo  pairs 

Side-by-side,  LCD  viewers, 

Movie  pairs 

Shuttered  screen,  Projection, 

Anaglyph  display 

Non-viewers  (e.g.,  lenticular) 

Standard  hardware  (e.g.,  OpenGL) 

Animation 

2  color  anaglyph 

Full  color 

Auto  stereo  from  6  deg  sequence 

“Most  operating  systems  and  application  programs  decouple  the  image  size  from  the  display  size.  If  the  image  must  be  sub-sampled  by  the  system  to  display  in  the  chosen 
window,  aliasing  may  occur  (some  graphics  systems  have  anti-aliasing  devices  built  in).  Usually,  not  all  of  the  screen  area  is  available  for  the  image,  so  the  screen  needs 
to  be  bigger  than  the  largest  image.  Some  graphics  systems  allow  an  image  larger  than  the  screen  to  be  “displayed”  and  the  visible  part  is  panned  around  with  the  mouse. 
bWith  color-space  mixing,  it  is  possible  to  “merge”  an  arbitrary  number  of  channels  into  a  standard  3-color,  RGB  display. 
c  CRT  sizes  are  usually  given  as  the  tube  diagonal,  not  the  visible  screen  size,  flat  panel  displays  indicate  the  actual  viewing  area. 

dThe  animation  software  must  be  highly  optimized  to  deliver  the  fastest,  smoothest  framing  rates.  This  can  be  achieved  in  software  at  the  expense  of  standardization  and 
so  the  fastest  systems  may  only  work  with  a  limited  range  of  display  hardware. 

FPS  =  frames  per  second.  LCD  =  liquid  crystal  display. 


second  (non-displayed)  buffer.  The  second  buffer  is  updated  while 
the  first  is  read  out  to  the  monitor  and  pointers  to  the  two  buffers 
are  switched  between  successive  frames.  New  graphics  standards, 
especially  OpenGL,  now  fully  support  double  buffering  on  stan¬ 
dard  displays.  Double  buffering  obviously  needs  twice  the  memory 
on  the  display  card. 

Stereoscopic  Display 

Depth  perception  by  stereo  parallax  provides  a  third  spatial  display 
dimension.  Some  of  the  geometric  z-information  can  be  mapped 
into  this  stereo  space.  Like  the  color/intensity  range  compromise, 
the  stereoscopic  depth  requires  some  reduction  in  bandwidth  of 
another  component.  The  two  halves  of  a  stereo-pair  can  be  dis¬ 
played  using  screen  space  components  normally  reserved  for  (1) 
2D  pixel  area,  (2)  color,  or  (3)  sequences  (temporal  space).  Each 
of  these  methods  requires  an  appropriate  viewing  aid  to  ensure  that 
each  view  is  presented  only  to  the  appropriate  eye.  The  detailed 
implementation  of  stereo  presentations  are  described  in  a  later 
section.  Table  14.6  shows  some  image  and  view  display  options 
for  visualization  systems. 

Optimal  Use  of  the  5D  Display  Space 

Several  conflicting  factors  must  be  balanced:  (1)  visual  acuity 
in  a  particular  display  dimension,  (2)  efficient  use  of  display 
resources,  and  (3)  minimizing  processing  time.  Because  intensity 
variations  are  difficult  to  interpret  in  a  low-contrast  image,  it  is 
sometimes  tempting  to  use  y-rnod  geometric  plots  and  other 
display  tricks  to  represent,  for  example,  a  fluorescence  ion  con¬ 
centration.  Using  the  geometric  space  rather  inefficiently  in  this 
way  for  a  simple  intensity  display  may  allow  us  to  view  inherently 
planar  information  with  greater  accuracy.  A  contrasting  example 
might  be  when  a  depth  profile  surface  can  be  defined  from  a  3D 


object:  (1)  the  geometric  space  in  x,  y,  and  z  (e.g.,  stereo)  could  be 
used  to  show  the  object’s  surface  relief,  (2)  color  used  for  mater¬ 
ial  properties  (fluorescence,  reflectivity,  etc.),  and  (3)  intensity 
(grayscale)  employed  to  reinforce  z-cues  by  depth  weighting. 
Automontage  is  an  interesting  application  (from  Syncroscopy, 
Ltd.)  where  surfaces  are  extracted  by  ray-casting  projections  of 
widefield  z-focus  series.  The  resultant  3D  data  is  then  visualized 
using  z- profile  plots  or  stereoscopic  views. 

The  best  display  space  for  a  particular  component  will  depend 
on  the  available  resolution  and  the  range  of  the  data.  In  general, 
multi-channel  images  are  best  shown  as  different  colors,  in  which 
case  depth  information  must  be  coded  using  stereo  or  motion  par¬ 
allax  or  some  lighting/shading  mechanism.  Binocular  stereoscopy 
works  for  views  rotated  around  the  y-(vertical)  axis  when  looking 
at  the  screen,  that  is,  with  parallax  shifts  in  the  horizontal  x-direc- 
tion  (Frisby  and  Pollard,  1991;  Poggio  and  Poggio,  1984)  (for  an 
upright  observer!),  while  motion  parallax  is  perceivable  around 
any  axis  within  the  xy-plane.  The  perception  of  depth  by  so-called 
motion  parallax  is  actually  a  subconscious  interpolation  of  the 
images  between  each  view  to  fill  in  the  path  of  features  presented 
at  discrete  loci  along  a  simple  trajectory  (Nakayama,  1985;  Fahle 
and  de  Luca,  1994).  Therefore,  a  sequence  of  side-by-side  stereo¬ 
pairs  at  increasing  tilt  angles  only  will  give  stereo  depth  in  x  and 
animation  depth  in  y  with  minimum  processing.  Rotations  are 
interpreted  more  readily  by  most  observers  than  other  temporal 
sequences,  such  as  through  focus  z-series,  largely  because  of  our 
acute  perception  of  parallax.  Detailed  assessment  of  particular  pre¬ 
sentation  modes  requires  an  in-depth  knowledge  of  the  physics  and 
physiology  of  visual  perception  (e.g.,  Braddick  and  Sleigh,  1983; 
Murch,  1984;  Landy  and  Movshom,  1991).  Alternative  temporal 
coding  strategies,  such  as  color-coded  tixels  (Kriete  and  Pepping, 
1992)  thus  aid  the  presentation  of  non-rotated  time  series,  partic- 
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ularly  if  interactive  measurements  are  to  be  made.  Figure  14.5 
shows  examples  of  the  efficient  use  of  display  space  for  multi¬ 
dimensional  display. 

Mapping  the  Image  Space  into  the 
Display  Space 

Having  chosen  both  the  image  dimensions  and  display  space,  a 
suitable  mapping  of  image  space  to  the  output  view  dimensions 
must  be  found.  Choices  for  this  geometry  processing  are  intimately 
linked  to  the  implementation  of  the  transfer  algorithm  for  com¬ 
bining  the  data  intensities,  however,  it  is  more  useful  to  consider 
these  components  separately  in  order  to  make  the  most  of  avail¬ 
able  resources;  geometry  and  intensity  processing  software  and 
hardware  may  reside  in  different  subsystems  of  the  visualization 
workstation. 

For  a  general  multi-dimensional  image  I(Xj,  y„  Zi ,  k,  ci; .  .  .)  and 
view  V(xv,  yv,  zv,  tv,  cv,  .  .  .)  we  can  define  an  overall  reconstruc¬ 
tion  function  (R)  such  that 

V(xv,  yv,  zv,  tv,  cv . . .)  =  R  [Ifc,  Vi,  Zi,  h,  cj, .  . .)]  (1) 

where  x,  y,  z  are  the  spatial  coordinates,  t  is  time,  c  is  color  chan¬ 
nels,  etc. 

R  has  two  components:  a  geometric  transform  (G)  converting 
image  to  view  dimensions  and  a  compositing  (sometimes  called 
transfer  function)  or  projection  operation  (P)  performed  on  inten¬ 
sities  through  the  view  space  (Fig.  14.6).  These  components  of  R 
are  thus  related  by 

V  P(xv,  yv,  Zv,  tv,  cv, . .  .) 

where  xv  =  Gx(Xi,  yi,  Zi,  t,,  ci; .  .  .),  yv  =  Gy(Xi,  .  .  .)  etc.  (2) 

The  following  sections  describe  various  G  functions  used  in  visu¬ 
alization  systems  (listed  in  Table  14.7).  Projections  are  described 
later  (and  listed  in  Table  14.8). 


Simple  Visualization:  Reducing  the 
Geometric  Dimensions 

This  involves  discarding  all  but  two  of  the  voxel  coordinates 
and  mapping  the  remaining  dimensions  to  screen  xy-positions.  A 
non-rotated  orthoscopic  (non-perspective)  view  of  a  3D  (x,,  yi;  zj) 
volume  (viewed  along  the  zv-axis)  is  a  simple  geometric  mapping 
of  serial  sections  that  can  be  projected  (e.g.,  by  summing, 
maximum  intensity,  etc.)  onto  a  2D  orthogonal  (xy)v  display  (Fig. 
14.6).  The  G  function  is  defined  by 

xv  =  Xi,  yv  =  Vi,  zv  =  Zi,  he.,  V(xv,  yv)  =  P(xi;  yi,  zO  (3) 

For  a  3D  time  series  (x;,  yi5  t,)  to  be  viewed  edge-on,  (x{,  tj,  the 
transformation  is  equally  simple: 

xv  =  Xi,  yv  =  h,  zv  =  yii.e.  V(xv,  yv)  =  Pfe  k,  y;)  (4) 

These  are  the  basis  of  the  familiar  three-pane  orthogonal  section 
views  found  in  many  visualization  programs. 

Rotations 

A  single,  orthoscopic,  x-axis  rotation  (0)  requires  a  geometric 
mapping  given  by 

xv  =  Xi,  yv  =  ViCos0  +  Zjsin0,  zv  =  -ViSin0  +  Z;cos0  (5) 

then  V(xv,  yv)  =  P(xv,  yv,  zv) 

The  projection  is  then  performed  in  the  (re-oriented)  view 
coordinates.  Offset  coordinates  (i.e.,  Xi  -  xQ,  etc.)  are  used  for  rota¬ 
tions  around  a  point  (x0,  y0,  zG)  other  than  the  image  center.  Rota¬ 
tions  are  non-comnrutative,  that  is,  the  order  of  x-,  y-,  and  z-axes 
matters.  The  observer’s  coordinate  system  (xv,  yv,  zv)  is  static,  and 
all  orientations  are  given  in  this  frame  of  reference.  To  generate  a 
view  of  the  image  rotated  simultaneously  about  the  three  image 
axes  (i.e.,  a  tumbling  algorithm),  three  consecutive  transformations 
are  required.  The  first  [e.g.,  a  tilt  around  x^  Eq.  (5)]  is  obviously 


B  Multi-Dimensional  View  V(xv,yv,Zv,tv,cv ) 


FIGURE  1 4.6.  The  visualization  process.  Voxel  and  object 
visualization  algorithms  proceed  in  three  clear  steps:  (1) 
Multi-dimensional  image  space  (A)  is  transformed  by  a  geo¬ 
metric  function  (G)  to  the  (re-orientated)  view  space  (B).  (2) 
Selected  objects  in  the  view  space  are  combined  using  a  pro¬ 
jection  operation  (P)  to  produce  a  sequence  of  2D  views.  (3) 
The  2D  views  are  presented  on  the  display  device  (C). 
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TABLE  14.7.  Overview  of  Image-to-View  Space  (Geometric)  Transformations  for  Visualizing  Multi-Dimensional  Biological 

Microscopy  Data 


Feature 

Parameter 

Minimum  required 

Desirable  additional  enhancements 

a2D  sections 

Orthogonal 

Arbitrary  orientation,  Section  rotation, 

Sections  with  translation 

Thick  sections  >1  pixel,  Curved  “surface”  extraction 

Projection  geometries 

bIsometric 

Arbitrary  orientation,  Full  matrix  rotation. 

Pixel  shift  (limited  angles) 

Pan  &  zoom,  Combinations  (e.g.,  “fly-by”), 

Rotation  or  tilt 

With  cut-aways 

Z-coding 

“Focus”  animation 

Simple  z-movie 

Interactive  arbitrary  section  movie 

On-the-fly  section  movie 

Motion  parallax 

Simple  pre-calculated  movie 

Interactive  pre-calculated  movie,  Motion  compression, 
On-the-fly  movie  (with  interaction) 

Stereo  parallax 

Manual  stereo  pairs 

Automatic  stereo,  Stereo  with  perspective 

“Height”  views 

Maximum  int  z-coordinate 

Threshold  z-coordinate,  z-coordinate  of  high  gradient 

Perspective 

Orthoscopic 

bZ-perspective 

Time-coding 

Mnovie 

/-coordinate 

A,-coding 

A,-movie 

3color  channels 

^-coordinate,  Arbitrary  color  channels 

Special  geometry 

OpenGL  with  enhancement,  z-buffer  hardware  “geometry 

hardware 

engine” 

aXY  orthogonal  sections  are  the  basis  of  “thru-focus  animations.”  Many  systems  now  permit  arbitrary  (including  “oblique”)  sections  to  be  extracted  from  a  3D  volume 
quickly  enough  to  be  considered  interactive.  Curved  sections  (rarely  supported)  are  particularly  useful  for  cellular  structures.  They  can  sometimes  by  produced  by  pre¬ 
processing  using  line-segments  extracted  serially  from  a  3D  stack. 

6  Isometric  and  ^-perspective  mapping  (G)  functions  increase  the  perception  of  depth  in  static  views,  and  remove  the  ambiguity  of  depth  seen  in  some  rotating  projections. 


also  around  xv.  The  following  two  transforms  (around  y,  and  zd  are 
then  around  oblique  (i.e.,  general)  axes  in  the  view  space.  Our  rota¬ 
tion  is  thus  three  separate  twists  around  axes:  X;  =  (ax,  bx,  cx),  fol¬ 
lowed  by  v’i  =  (ay,  by,  cy)  and  then  Zi  =  ( az ,  bz,  cz).  The  G  function, 
Eq.  (2),  for  each  twist  is  a  formidable  computation  with  three  sub¬ 
components 


where  5  is  sin0,  C  is  cos0,  and  a  final  ¥  turn  (around  zv)  gives 

=  x9C  -  y9 5,  yT  =  x9 5  -  z9C,  z*  =  z9  (9) 

where  5  is  sintp  and  C  is  cos'T. 

The  projection  will  eventually  be  done  in  the  reoriented  view 
coordinates  V(xv,  yv)  =  P(x.y,  >V,  zT). 


xtwist  =  x[a 2  +  C(1  -  a2)]  +  y[ab{\  -  C)  +  5c]  + 
z[ac{\  -  Q  -  Sb] 


(6) 


Ttwist  =  x[ab{  1  -  C)  -  5c]  +  y[b2  +  C(1  -  b2  )]  + 
z[bc(  1  -  C)  v  5a] 

Ztwist  =  x[ac(l  -  C)  -  Sb]  +  y[bc(  1  -  C)  -  5a]  + 
z[a 2  +  C(  1  -  a2)] 

where  5  is  the  sin( twist  angle),  C  is  the  cos(twist  angle)  around  an 
axis  (a,  b,  c),  and  a  is  the  cos©,^,,  b  is  the  cos<Dtwist,  c  =  cos'Ptwist, 
0,  ®,  and  T  are  the  view  space  polar  coordinates  of  the  twist  axis 
(Fig.  14.7). 

This  whole  transform  must  be  repeated  for  each  of  the  three 
re-orientation  axes  (a,  b,  c)x,  (a,  b,  c)y,  and  (a,  b,  c)z.  A  viewing 
transform  should  ideally  be  added  to  observe  the  rotated  structure 
from  different  viewing  positions.  However,  by  fixing  the  view 
direction,  this  additional  step  is  avoided.  The  efficiency  (and  ulti¬ 
mately,  the  speed)  of  the  geometric  algorithm  is  determined  by  the 
degree  to  which  the  general  form  (6)  can  be  simplified.  If  all  rota¬ 
tions  are  specified  around  the  observer’s  axes  (xv,  yv,  zv)  the  direc¬ 
tion  cosines  (a,  b,  c)  become  either  zero  or  unity.  It  is  also  easier 
for  the  user  to  anticipate  the  final  orientation  when  the  rotation 
axes  do  not  change  between  each  component  twist.  For  a  fixed 
viewing  axis  along  zv,  a  tilt  0  (around  xv)  is  obtained  by  a,  b,  c  = 
1,  0,  0  giving 

xe  =  Xi,ye  =  y,C  +  ZiS,  ze  =  -y{S  +  z,C  (7) 

where  5  is  sin0  and  C  is  cos0.  When  combined  with  a  subsequent 
cp  rotate  (around  yv)  this  becomes 

x<p  =  xeC  -  ze5,  y9  =  ye,  z<p=  xe5  +  zeC  (8) 


B  3-D  Tilt  around  xy  =  x. 


C  3-D  ‘Rotate’  around  yv 


FIGURE  14.7.  Rotation  transformations.  3D  rotations  can  be  performed  in 
the  image-space  around  a  general  axis  of  rotation  (A).  General  re-orientation 
requires  rotation  around  three  axes  jq,  yi  and  z\ •  A  simpler  (and  thus  faster) 
scheme  is  to  use  a  “tilt”  (B)  followed  by  a  “rotate”  (C)  around  xw  and  yv  in  the 
view- space.  If  viewing  along  zv,  no  additional  information  is  obtained  by  a  zv 
“turn.” 
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If  the  twists  are  “tilt  first,  followed  by  rotate”  and  the  viewing 
axis  is  oriented  along  xv,  yv,  or  zv,  the  twist  around  that  axis  can 
(optionally)  be  omitted  because  no  more  structure  will  be  revealed 
(Fig.  14.7).  By  thus  omitting  the  turn  (9)  and  projecting  (and 
viewing)  along  z»,  the  resultant  (xv,  _yv,  zv)  vectors  only  change  as 
a  function  of  xv  and  zv  (i.e.,  dyv/dXj  =  0)  while  traversing  a  row  of 
image  data.  This  can  lead  to  extremely  rapid  projection  algorithms. 
The  above  scheme  is  used  for  the  3D  visualization  routines  in 
Lasersharp,  which  are  executed  entirely  on  the  CPU  without  any 
involvement  of  the  graphics  hardware. 

Other  tricks  can  be  used  to  optimize  the  rendering  geometry. 

Working  in  Image  or  Space  View 

One  implementation  approach  is  to  progress  through  the  image 
space  (I  space)  coordinates  (in  xn  yh  z;),  voxel  by  voxel,  trans¬ 
forming  each  to  view  space  (V  space)  coordinates  (in  xv,  yv,  zv) 
before  projecting  each  intensity  into  the  final  2D  view  at  (xv,  yv). 
For  an  image  of  irq  x  ny  x  nz,  voxels,  the  orthoscopic  transfor¬ 
mation  proceeds  via  N  serial  planes  or  sections  of  data,  normal  to 
Zi  in  forward  (n  =  1 .  nzd  or  reverse  (n  =  nzi,  1 )  order.  Alternatively 
the  same  G  function  can  be  implemented  in  V  space.  There  are 
now  N  planes  normal  to  zv  (and  thus  cut  obliquely  through  the 
image  volume).  For  each  V(xv,  yv,  zv)  the  contributing  Ife  Vi,  zd 
are  found  and  the  computation  can  again  proceed  in  forward  ( n  = 
1 ,  nzv)  or  reverse  (n  =  nzv,  1 )  order.  The  V  space  implementation 
makes  the  G  function  more  difficult  but  facilitates  straightforward 
projections  (P)  (Fig.  14.8).  The  I  space  implementation  is  the 
reverse.  The  V  space  method  has  many  more  advantages  for  geo¬ 
metric  polygon  data  than  for  rastered  voxels.  Hybrid  implementa¬ 
tions  are  also  possible,  combining  both  I  and  V  space  components. 
In  a  hardware-accelerated  OpenGl  implementation,  a  favorite 
approach  is  to  take  each  image  plane  (i.e.,  to  progress  through  I 
space)  and  warp  it  into  its  projected  (rotated)  aspect  in  view  space. 
The  warped  frame  is  then  mapped  onto  the  view  layer  by  layer. 
Sophisticated  graphics  cards  have  hardware  for  texture  mapping 
that  assists  the  painting  of  images  onto  objects  using  this  technique 
by  altering  the  view  pixels  according  to  the  color  of  the  image. 
This  texture  mapping  hardware  can  be  used  to  paint  the  warped 
frames  into  the  growing  view  modifying  the  output  by  opacity 
values  controlled  by  the  voxel  intensities.  Hardware  texture 
mapping  is  used  extensively  by  Volocity  and  Amira  packages. 

In  all  these  examples  the  reconstruction  process  has  resulted 
in  a  loss  of  zv  dimensional  information.  Some  of  this  may  be  auto¬ 
matically  retained  by  the  P  function  (discussed  below),  but  an  effi¬ 
cient  G  function  can  further  optimize  the  dimensional  content  of 
the  view. 


How  Do  3D  Visualizations  Retain  the 
z-lnformation? 

True  3D  visualization  requires  that  zv  depth  information  is  retained 
or  at  least  partially  restored  when  the  image  is  mapped  into  a  2D 
view. 

Stereoscopic  Views 

Some  impressive  methods  of  coding  zv  information  use  stereo¬ 
scopic  techniques  (Tables  14.6  and  14.7;  Figs.  14.9  and  14.10). 
These  require  that  an  observer  be  simultaneously  presented  with  a 
left-eye  and  right-eye  view  differing  by  a  small  cp  rotation.  The 
author  has  found  the  following  geometry  to  give  acceptable  results: 
for  views  of  width  x,  angular  difference  cps,  viewed  at  a  distance 
d  with  an  interoccular  spacing  D 


A  I-space  projection  via  serial  xy  sections 


B  V-space  projection  via  serial  xvyv  sections 

FIGURE  1 4.8.  V-  and  1-spaee  projections.  I-space  voxel  reconstructions  (A) 
proceed  via  serial  sections  that  are  oblique  to  the  viewing  axis.  This  is  the 
most  efficient  way  of  processing  voxels  in  “object  order”  and  is  used  by  most 
voxel  Tenderers.  V-space  algorithms  (B)  process  Jt„y»  sections  that  are  normal 
to  the  viewing  axis.  This  zv-ordered  reconstruction  is  more  useful  for  con¬ 
structing  polygon  objects  than  for  voxels. 


D  =  0.05  -  0.07  m,  d  =  0.25  *  x/D  m, 

cps  =  2  *  arctan[(0.5  *  D/d)]  (10) 

for  example,  if  D  =  0.06,  x  =  0.06,  then  d  =  0.25  and  cps  =  13.6°. 

A  simpler  alternative  to  the  computationally  intensive  rotations 
is  to  shift  each  section  horizontally  (in  xv)  by  a  constant  factor  of 
the  zv  coordinate  during  a  top-down  compositing  projection  (3). 
This  corresponds  to  a  stereo  pixel-shift  G  function 

Xv  =  Xi  +  5x*zv,  yv  =  yi;  Zv  =  Zi  (11) 

where  5  Xief,  =  tan(0.5  *(ps)  and  8xrigbt  =  -5xleft. 

From  Eq.  (11)  and  the  viewing  conditions  of  Eq.  (10),  5x  is  ± 
tan  (7.8°)  or  about  0.14  times  the  z-spacing  (Fig.  14.9).  A  slightly 
different  equation  is  described  by  Cheng  and  colleagues  (1992) 
and  its  derivation  is  attributed  to  Hudson  and  Makin  (1970).  It  can 
be  used  to  derive  an  optimal  pixel-shift.  Using  a  notation  consis¬ 
tent  with  the  above 

5p  =  2*nzcaiib  *  M  *  sin(arctan(5x*nzi/nzcaiib)  (12) 

where  5p  is  the  parallax  shift  between  equivalent  points  in  two 
views  (ideally  kept  around  3-5  mm);  nzcalib  is  the  the  calibrated  z- 
size  of  the  data  (thickness  of  the  original  specimen),  and  M  is  the 
magnification. 

The  pixel-shift  method  is  only  an  approximation.  In  particular, 
the  result  is  stretched  out  or  warped  in  the  x-direction  by  a  small 
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(i)  CCW  (+ve)  rotation  around  y¥  (ii)  Right  eye  view  with 

for  left  eye  view  'Tilt'  only  tarn  Lind  xv) 


A  Stereo  pairs  from  rotated  views 


FIGURE  14.9.  Pixel-shift  and  rotation  stereo.  Pairs  of  views  differing  by 
typically  around  ±7-8  deg  (depending  on  pixel  dimensions  and  inter-plane 
spacing)  of  yv  rotation  (ps  (A)  can  be  extracted  from  a  sequence  and  displayed 
using  anaglyph,  switched-view  or  other  stereo  viewers  (see  text).  The  pixel- 
shift  approximation  to  these  rotations  (B)  results  in  only  trivial  distortion  and 
is  much  faster  to  implement  for  small  angles  about  the  z,  axis.  See  Figures  14.5 
and  14.10  for  example  images  and  the  section  on  3D  depth  information  for 
details  of  stereo  generation  algorithms. 


factor  of  l/cos(0.5  *  (ps).  but  for  small  angles  this  can  be  neglected. 
This  shearing  and  warping  algorithm  is  the  basis  for  fast  texture- 
mapped  voxel  projections  (e.g.,  Cabral  et  ai,  1994;  Lacroute  and 
Levoy,  1994;  Elisa  et  al.,  2000). 

The  two  views,  or  stereo-pair,  must  be  fused  into  a  3D  rep¬ 
resentation  by  positioning  each  in  the  display  space  so  as  to  occupy 
the  entire  field  of  view  (or  at  least  the  central  region)  for  each  cor¬ 
responding  eye.  The  observer  then  perceives  the  combined  infor¬ 
mation  as  a  single  3D  view  located  near  the  viewing  distance  d, 
depending  on  the  origin  of  the  ^-coordinates  used  in  the  above 
equation.  For  Zi-coordinates  centered  in  the  middle  of  the  image 
volume,  the  view  depth  is  centered  about  d.  A  few  high-contrast 
features  must  be  present  in  the  field  in  order  to  successfully  fuse 
the  reconstruction.  This  makes  the  choice  of  algorithm  and  asso¬ 
ciated  parameters  critical  for  successful  stereo  presentations.  The 
stereo  scene  geometry  is  efficiently  accomplished  by  placing  the 
views  in  two  halves  of  a  display,  bisected  in  x  or  y,  and  using  a 
viewing  aid.  The  viewer  must  make  the  two  views  appear  to  be 
coming  from  the  center  of  the  field  while  keeping  the  observer’s 
ocular  convergence/divergence  near  to  parallel  (i.e.,  relaxed  as 


though  viewing  at  a  distance).  By  carefully  editing  a  vertical  sub- 
region  (e.g.,  384  X  512  pixels)  of  interest,  side-by-side  pairs  can 
be  viewed  in  full  color  on  a  horizontal  display  format  (e.g.,  768  X 
512,  1024  X  768,  etc.)  without  subsampling.  Side-by-side  stereo¬ 
pairs  are  easily  viewed  with  horizontal  prismatic  viewers.  An 
alternative  is  to  fix  one’s  binocular  convergence  point  at  infinity 
and  refocus  each  eye  onto  the  respective  view  (using  the 
lens  and  cornea  only).  The  left  and  right  views  may  also  be 
swapped  and  the  eyes  crossed  or  converged  to  a  point  between  the 
observer  and  screen.  Some  seasoned  stereoscopists  can  fuse  stereo¬ 
pairs  using  these  methods  without  additional  aids,  although 
prolonged  viewing  can  give  rise  to  eye  strain  and  headaches. 
Above-and-below  pairs  can  be  seen  through  vertical  prismatic 
viewers,  but  this  geometry  cannot  be  so  easily  fused  by  the  unaided 
observer! 

Partitioning  the  color  space  into  two  distinct  regions  allows 
full  size  anaglyph  stereo-pairs  to  be  observed  in  monochrome 
(gray  levels  only).  Both  views  occupy  the  entire  pixel  display  area 
and  are  transmitted  to  the  corresponding  eyes  by  RG  or  RB  spec¬ 
tacles.  Due  to  the  spectral  characteristics  of  some  monitor  phos¬ 
phors  and  the  availability  of  low-cost  filter  materials,  some 
bleed-through  of  signal  between  RG  channels  often  occurs. 
Red/cyan  viewers  often  have  improved  extinction.  The  optimal 
intensity  balance  between  the  component  views  must  be  individ¬ 
ually  determined.  Anaglyph  stereo-pairs  can  be  fused  by  observers 
irrespective  of  their  capacity  to  differentiate  colors.  Even  rare 
red/green  color  blindness  is  no  obstacle  provided  sufficient  inten¬ 
sity  levels  can  be  differentiated,  although  it  is  usually  found  that 
around  10%  of  observers  fail  to  perceive  3D  effects  from  stereo 
cues  alone  (Richards,  1970).  Because  the  anaglyph  views  occupy 
the  same  physical  area,  the  eyes  are  drawn  into  a  convergence  nat¬ 
urally.  The  monitor  should  be  as  large  as  possible  to  increase  the 
distance  from  the  viewer  and  decrease  eye  strain,  that  is,  so  the 
convergence  angle  is  as  small  as  possible  but  each  eye  is  focused 
at  a  distance.  Health  and  safety  recommendations  usually  specify 
at  least  18"  comfortable  viewing  distance  for  VDUs  (more  for 
extended  viewing)  with  alternative  work  breaks  every  hour. 

In  order  to  maximize  simultaneously  spatial  and  color  resolu¬ 
tion,  the  temporal  display  space  can  be  partitioned  to  display  the 
component  stereo-pairs.  This  requires  more  sophisticated  hard¬ 
ware  than  the  previous  methods  at  increased  cost.  The  left  and  right 
component  views  are  displayed  alternately  on  the  video  monitor 
in  rapid  succession.  Observers  watch  through  a  viewing  device  that 
synchronously  blanks  and  unblanks  the  visual  field  of  each  eye 
while  each  corresponding  view  is  displayed.  These  alternate  (or 
switched)  display  stereo  systems  are  characterized  by  differences 
in  the  format  of  the  video  signal,  the  switched  viewing  hardware, 
and  the  method  of  synchronization.  Older  video  stereo  systems 
display  images  in  two  alternate  and  interlaced  fields  of  each  video 
frame.  This  method  gives  half  the  temporal  and  y-axis  resolution 
of  a  normal  video  signal,  an  obtrusive  (25/30  Hz)  display  flicker, 
as  well  as  the  low  intensity  associated  with  interlaced  displays. 
Computer  displays  are  exclusively  non-interlaced  but  alternative 
frames  may  be  used  to  show  stereo  components  rapidly  in  suc¬ 
cession.  An  interlaced  implementation  will  require  the  two 
component  images  to  be  interleaved  line-by-line  in  the  display 
memory.  A  more  convenient  organization  is  to  have  sequential 
buffers  for  the  two  fields  which  can  be  updated  independently. 
Non-interlaced  computer  displays  give  a  brighter  view  with  a 
flicker-free  refresh  rate  of  60  to  90  Hz.  This  would  allow  alternate 
frame  non-interlaced  stereo,  but  would  still  exhibit  noticeable 
flickering  around  30  to  45  Hz.  A  continuous  stereo  display  requires 
at  least  120Hz  and  preferably  a  double-speed  scan  of  180Hz. 
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FIGURE  1 4.1 0.  Example  stereo  images.  (A)  Anaglyph  (two  color,  red/green)  stereo-pair  of  Golgi-stained  nerve  cells,  generated  by  the  pixel-shift  method  (see 
text).  The  alternative  method  of  extracting  two  images  from  a  rotation  series  [shown  immediately  below  (A)  as  a  side-by-side  pair]  produces  a  virtually  identi¬ 
cal  result  to  (A)  for  small  angles  about  the  z-axis.  (B)  Anaglyph  stereo-pair  of  fluorescent  Paramecium.  [Data  for  (A)  and  (B)  courtesy  of  Bio-Rad  Microscience.] 
The  third  pair  of  panels  shows  a  stereo-pair  of  voxel-rendered  images  representing  Feulgen-stained  chromosomes  from  the  plant  Milium.  (Original  data  from 
Bennet  et  al.,  1990.)  Voxel  gradient  lighting  is  used  with  a  high-opacity  a-blend  algorithm  to  give  an  effect  of  “solid”  structure.  The  final  panels  show  a  triple- 
stained  thick  section  of  skin  showing  extra  cellular  matrix  proteins  and  vascular  structures.  Simple  maximum  projections  of  each  channel  z-series  are  combined 
into  the  24-bit  RGB  view.  While  the  anaglyph  stereos  can  span  the  entire  display  window  but  require  the  color  space  for  the  stereo  effect,  these  full  color  side- 
by-side  stereo  pairs  (as  for  monochrome  pairs)  can  span  only  half  the  available  display  resolution.  These  alternative  stereo  display  methods  illustrate  the  way 
that  display  resources  can  be  “traded”  for  improved  (x,y)  resolution,  z-depth  or  multi-channel  (color)  rendition.  Side-by-side  pairs  in  this  figure  should  be  viewed 
by  divergent  eyes  or  viewing  aids  (i.e.,  with  eyes  relaxed  as  when  viewing  distant  objects  but  focused  on  the  page).  This  is  best  achieved  by  focusing  on  a  distant 
object  and  bringing  the  page  into  view  while  keeping  the  eyes  relaxed  but  refocused  on  the  page.  If  the  eyes  are  crossed  or  a  convergent  viewing  aid  is  used, 
the  3-D  effect  will  appear  “back-to-front.”  This  can  be  problematic  with  some  maximum  projections  (e.g.,  the  Golgi-stained  neurons  above)  where  some  bright¬ 
est  features  are  towards  the  “back”  of  the  rotated  object  in  some  views.  This  is  often  found  when  there  is  attenuation  with  depth  in  the  sample.  Maximum  inten¬ 
sity  can  incorrectly  bring  these  features  to  the  “front”  of  the  view.  Samples  with  more  isotropic  intensity  distributions  with  depth  (e.g.,  the  skin  images  above) 
tend  not  to  show  these  anomalies. 
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Some  high-resolution  stereo  displays  still  use  an  interlaced  storage 
mode  (at  half  the  y-resolution)  for  stereo  presentation.  All  video 
display  systems  suffer  to  some  extent  from  flicker  arising  from  an 
interference  with  mains-powered  room  lights  and  should  be  run  in 
reduced  ambient  lighting. 

Synchronizing  the  display  to  the  switched  viewing  system  can 
be  accomplished  in  a  number  of  ways.  A  bright  intensity  marker 
in  each  video  frame  can  identify  the  left  and  right  views,  and  this 
has  been  used  to  trigger  a  photo  switch  placed  over  the  corner  of 
the  video  monitor.  This  switch  controls  synchronous  LCD  shut¬ 
tered  viewers.  The  synchronizing  pulse  can  be  generated  directly 
from  the  field  or  frame  synch  of  the  display  signal  and  passed  to 
the  viewers  by  cable  or  optical  (infra-red)  link.  A  more  convenient 
alternative  to  switched  viewers  is  the  polarizing  LCD  shuttered 
screen.  This  is  a  large  LCD  shutter  (often  constructed  from  two 
parallel  units  for  faster  switching)  that  toggles  between  two  polar¬ 
ization  states.  The  left-  and  right-eye  views  are  thus  differently 
polarized  and  can  be  viewed  through  inexpensive  glasses.  Plane- 
polarized  shutters  (and  glasses)  give  the  best  extinction,  but  clock¬ 
wise  and  counterclockwise  rotary  polarization  allows  for  more 
latitude  in  the  orientation  of  the  observer’s  head  (important  when 
a  large  group  is  viewing  a  single  monitor!).  Example  stereo  views 
are  shown  in  Figure  14.10  (and  also  Figs.  14.5  and  14.24).  The 
Imaris  package  has  a  number  of  selectable  stereo  modes  de¬ 
pending  on  the  type  of  viewers  to  be  used  (these  include  Raw 
Stereo  —  using  the  OpenGL  functions,  alternate  image  (interlaced 
modes),  and  three  combinations  of  two-color  anaglyphs).  The 
3D-constructor  plug-in  for  ImagePro  Plus  (Media  Cybernetics 
Inc.)  also  supports  OpenGL  stereo. 


Non-Orthoscopic  Views 

Visual  perception  of  depth  makes  extensive  use  of  non- 
stereoscopic  cues,  and  these  can  be  coded  by  appropriate  algo¬ 
rithms  into  a  corresponding  part  of  the  display  space.  A  series  of 
G  functions  exist  that  code  depth  information  with  unmodified 
intensities.  These  are  the  non-orthoscopic  transformations  and 
include  both  perspective  and  non-perspective  geometries  (Fig. 
14.11).  The  most  straightforward  of  these  algorithms  require  that 
the  corresponding  view  coordinates  for  each  data  voxel  are  mod¬ 
ified  by  the  image  z-coordinates.  The  isometric  G  function 
involves  a  constant  shift  in  screen  xv  and/or  yv  coordinates  as  the 
I  space  Tenderer  traverses  each  dimension  of  the  data 

xv  =  Xi  +  XiCos'P,  yv  =  y,  +  ViSin'P  +  z„ 

Zv  =  Zi  and  usually  V =  60°  (13) 

This  geometry  (as  the  name  suggests)  gives  rotated  xv,  yv  and 
zv,  axes  in  the  same  proportions  as  the  original  image  axes  (for  ren¬ 
dered  examples,  see  Wilson.  1990).  Direct  xt,  y,,  Z\  measurements 
can  be  made  from  the  2D  view  screen. 

True  perspective  views  attempt  to  visualize  the  data  as  a  large 
real-world  or  macroscopic  object  with  xv  and  yv  converging  to  a 
vanishing  point  at  a  large  zv  distance.  This  point  is  on  the  horizon 
(usually  in  the  center  of  the  field).  The  perspective  G  function 
decreases  the  dimensions  as  a  linear  factor  of  the  z-coordinate.  So 
after  the  data  have  been  rotated,  the  projection  is  accomplished 
through  a  new  perspective  space  (xp,  yp).  True  perspective  can  be 
approximated  in  the  G  function  by 

zp  —  zm*,  xp  =  Z'^,  _vp  =  nvvp/ Z'[<  (14) 

where  a  is  a  factor  reflecting  the  object-to-observer  distance. 

Perspective  views  can  be  readily  interpreted  from  objects  with 
well-defined  structures.  This  is  because  we  assume  such  views 


(i)  Single  vanishing  point  (vp0)  (ii)  Two  vanishing  points  (vp0,vp,) 
for  view-space  for  image-space 

A  True  perspective  geometries 


nXj 


ny, 


XY  section 


XZ  section 

FIGURE  14.1 1.  Non-orthosopic  views.  True  perspective  views  (A)  give  the 
most  “realistic”  reconstruction  geometries.  A  simple  approximation  can  be 
implemented  in  the  view-space  (1)  with  one  “vanishing  point.”  xv,  yv  coordi¬ 
nates  are  reduced  by  a  linear  function  of  zv  between  z0  and  Zi(see  text).  A  general 
perspective  G-function  for  image-space  implementation  (2)  with  rotation 
around  yv  requires  two  vanishing  points.  Isometric  reconstructions  (B)  retain 
the  physical  dimensions  of  the  original  image  axes  (rather  than  the  trigono¬ 
metrically  projected  axis  lengths).  3D  distance  measurements  can  thus  be  made 
directly  on  the  views. 


arise  from  connected  features  such  as  real-world  surfaces  and 
edges.  Popular  optical  illusions  indicate  this  is  not  always  true! 
Connectivity  between  neighboring  voxels  in  a  confocal  laser  scan¬ 
ning  microscopy  (CLSM)  fluorescence  image  can  rarely  be 
assumed  without  substantial  a  priori  knowledge  and  then  only 
after  careful  control  of  noise.  Non-orthoscopic  depth  coding  is 
useful  for  removing  ambiguity  from  stereoscopic  views.  Volocity 
and  Imaris  produce  both  orthoscopic  and  perspective  3D  (and  4D) 
views. 

Temporal  Coding  and  z  Depth 

Time  axes  can  often  be  treated  in  exactly  the  same  manner  as  depth 
or  z-coordinates.  This  is  a  useful  observation  because  many  visu¬ 
alization  packages  do  not  specifically  recognize  time  points.  Thus 
t-coordinates  can  be  directly  mapped  by  animation,  coded  in  the 
intensity  or  color  space,  and  even  represented  in  non-orthoscopic 
or  stereoscopic  views.  The  importance  of  time  points  as  a  compo¬ 
nent  sampling  space  in  4D  imaging  is  encapsulated  within  the  so- 
called  4D  tixel  object.  Significant  efforts  have  been  invested  in  the 
efficient  visualization  of  tixel  images  within  the  5D  display  space 
(Kriete  and  Pepping,  1992).  z-Position  can  be  inferred  from  the 
display  temporal  space  by  animating  serial  sections  as  a  through- 
focus  sequence  or  by  motion  parallax  (Fig.  14.12)  from  a  rotation 
series  (Wallen  et  al.,  1992).  Since  xy-acuity  improves  smooth 
motion  perception,  large  rotated  views  require  reduced  angular 
increments  as  well  as  higher  refresh  and  framing  rates  compared 
to  those  from  smaller  volumes. 
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Z-Depth  perception  by  temporal 
interpolation  and  motion  parallax 


FIGURE  14.12.  Z-depth  by  temporal  interpolation  and  motion  parallax. 

If  a  similar  feature  is  seen  in  two  sequential  views  of  a  rotation  series  viewed 
along  Zv  (e.g.,  the  dark  shape  appearing  at  “1”  in  the  first  view  and  at  “2”  in 
the  second),  two  processes  contribute  to  the  perception  of  its  depth  within  the 
view.  (1)  The  direction  of  motion,  either  left  or  right  across  the  field  of  view 
will  determine  whether  it  is  perceived  to  be  behind  (A),  the  center  (C),  or  in 
front  (B)  of  the  screen.  (2)  provided  the  views  are  at  small  angles  apart  and 
shown  in  a  “smooth”  sequence  (which  may  need  up  to  18  fps),  the  details  are 
mentally  interpolated  in  time  (taking  into  account  the  perceived  motion)  to  “fill 
in”  the  missing  information  (dotted  features).  For  a  fully  transparent  view  with 
no  other  depth  cues,  front  and  back  may  be  arbitrarily  reversed  by  different 
observers. 

Mapping  the  Data  Values  into  the  Display 

As  with  the  G  function,  data  values  must  also  be  transformed  to 
the  display  by  a  well-defined  operation.  This  is  the  combination, 
compositing  or  projection  function  P,  described  in  Figure  14.6 
(examples  in  Table  14.8  and  Fig.  14.14). 

The  Visualization  Model 

It  is  useful  to  consider  even  the  simplest  P  function  in  terms  of 
a  lighting  model.  In  a  gross  simplification,  each  image  voxel  is 
considered  to  have  a  brightness  equivalent  to  the  amount  of  light 
emanating  from  a  corresponding  point  in  the  specimen  and  col¬ 


lected  by  the  microscope  objective.  Digital  processing  to  more 
closely  realize  this  goal  for  CLSM  imaging  requires  a  knowledge 
of  the  microscope  transfer  function  (e.g.,  Sheppard  and  Gu,  1992, 
1993)  and  the  use  of  digital  restoration  methods  (e.g.,  Holmes  and 
Liu,  1992;  Shaw  and  Rawlins,  1991;  see  also  Chapters  23  and  24, 
this  volume).  For  now,  the  3D  image  is  considered  as  the  physical 
or  macroscsopic  object,  itself  being  imaged,  as  if  by  a  lens  or  eye, 
to  a  2D  view.  Light  rays  from  each  point  voxel  in  the  image  would 
contribute  to  an  equivalent  (but  not  unique)  point  in  that  view.  This 
mapping  of  object  points  X  image  voxels  X  display  view  pixels  is 
the  crude  model.  This  is  refined  and  made  more  elaborate  by  (1) 
segmenting  the  multi-dimensional  image  data  into  a  set  of  objects, 
(2)  considering  other  objects  in  the  path  of  each  simulated  light 
ray,  and  (3)  adding  additional,  artificial  lighting  and  surface  effects. 
These  aims  are  met  using  physical  or  material  properties  attached 
to  the  objects. 

Choosing  the  Data  Objects 

So  far,  we  have  considered  the  image  as  an  array  of  voxel  (3D)  or 
tixel  (4D  x,  y,  z,  t)  objects  with  no  implied  connectivity.  Computer- 
aided  macroscopic  imaging  and  display  systems  have  fostered  the 
growth  of  visualization  models  based  on  the  grouping  of  voxel 
samples  into  geometric  objects.  The  consequence  is  a  growth  in 
special  hardware  and  software  for  efficient  treatment  of  vectorized 
geometries.  Microscope  systems  invariably  produce  rastered 
image  arrays.  The  array  dimensions  (tu,  ny,  nz,  etc.)  imply  the 
arrangement  of  rows,  columns,  sections,  etc.  Vector  objects  are 
non-ordered  lists  of  geometric  figures,  each  of  which  is  defined  by 
a  very  few  parameters.  Thus,  a  simple  voxel  might  be  five  values 
(x,  y,  z,  intensity,  opacity,  etc.)  but  a  triangle  comprising  many  tens 
or  hundreds  of  voxels  may  be  specified  with  only  10  to  15  param¬ 
eters  (three  sets  of  coordinates  plus  some  material  properties).  If 
the  material  properties  vary  across  the  polygon,  values  at  each 
vertex  only  need  be  stored.  Each  vectoral  component  will  need  at 
least  16-bit  integer  precision  and  preferably  floating-point.  A 
compact  form  is  to  store  all  vertices  in  one  list  or  table.  Each  geo¬ 
metric  object  is  then  a  set  of  polygons,  specified  by  indices  point¬ 
ing  into  the  vertex  table. 

While  early  workstation  programs  stored  voxels  as  vectors,  the 
availability  of  optimized  hardware  and  software  for  G  functions 
and  3D  rendering  of  rastered  byte  images  has  assisted  the  devel- 


TABLE  14.8.  Overview  of  Image-to-View  Data  Mapping  (Projection)  Options  for  Visualizing  Multi-Dimensional  Biological 

Microscopy  Data 


Feature 

Parameter 

Minimum  required 

Desirable  additional  enhancements 

“Data  objects 

types 

Voxels 

n-D  array  of  voxels.  Object  list.  Surface  (triangles).  Surface 
(polygon  net)  Objects  “embedded”  within  voxels 

storage 

3D  rastered  array 

n-D  rastered  array,  object  vector  list 

Selective  enhancement 

Basic  filters,  Volume  cut-away 

Gradient  segmentation,  Image  masking,  Opacity/transparency 

of  features 

Region  of  interest/edit, 

Intensity  segment.  Color  channels 

Material  properties. 

Morphological  filters,  “Seed/flood  fill” 

Object  modelling 

bProjection  algorithms 

Maximum 

Average 

Front,  Z-co-ordinate,  a-blend,  SFP,  Iso-intensity  surface, 

“Local”  projection 

cSpecial  graphics  resources 

VGA  graphics 

OpenGL  with  acceleration,  Texture  mapping,  Hardware  shading/ 
lighting  etc.,  a-buffer 

“Data  objects  are  usually  voxels  or  lists  of  vertices  defining  geometric  surfaces.  Other  surface  descriptions  are  possible.  AVS  allows  for  a  particularly  large  range  of  geome¬ 
tries,  while  in  most  object  reconstructions,  the  options  are  usually  restricted  to  triangulated  surfaces  —  primarily  to  make  use  of  efficient  accelerations  in  the  graphics 
system. 

bSome  geometric  object  Tenderers  use  Front  or  a-blend  P-functions  in  the  same  way  as  for  voxel  objects.  Specific  “surface”  algorithms  may  also  be  implemented  that  cap¬ 
italize  on  the  connectivity  of  the  geometries.  There  is  always  a  trade-off  between  the  portability  of  a  program  across  platforms  (or  over  new  versions  of  platforms)  and  the 
use  of  non-standard  “special”  hardware.  The  OpenGL  standard  largely  avoids  these  problems  by  setting  the  interface  requirements  and  providing  standard  support  for  new 
hardware.  SFP  =  simulated  fluorescence  process. 
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opment  of  voxel  visualization  algorithms.  The  most  efficient  way 
to  process  rastered  voxels  is  by  whole-image  operations.  I  space 
implementations  effectively  warp  successive  orthogonal  (xy,  xz,  or 
tz) ;  sections,  using  a  transformation  geometry  engine,  to  their  pro¬ 
jected  shape  in  the  final  view.  Rows  and  columns  of  data  are 
traversed  in  forward  or  reverse  order  to  preserve  the  forwards 
or  backwards  compositing  direction.  The  only  drawback  of  this 
method,  as  opposed  to  a  z-ordered  list  of  polygons,  is  the  through¬ 
put  to  the  display.  Because  the  entire  image  data  must  be  streamed 
into  the  display,  bandwidth  is  critical.  This  requires  highly  efficient 
pipelined  operations  to  achieve  a  display  rate  sufficient  to  service 
the  output  from  an  optimized  geometry  engine.  The  highest  spec¬ 
ification  systems  use  multiple  display  devices  (often  video  projec¬ 
tors)  to  simultaneously  render  multiple  views  for  immersive  reality 
installations.  So-called  multi-pipe  versions  of  these  visualization 
programs  are  used  to  massively  increase  the  data  throughput  to  the 
display.  The  vectoral  representation  is  more  efficient  for  triangles 
encompassing  more  than  about  20  equivalent  voxels,  provided  the 
segmentation  algorithm  is  justified  for  the  particular  data.  Com¬ 
puter  models  allow  entire  surfaces  and  bounded  volumes  to  be 
described  by  single  parametric  equations  using,  for  example, 
Bezier  coefficients  (see  Watt,  1989).  This  type  of  simple  object 
definition  is  generally  not  practical  for  confocal  fluorescence  data 
due  to  the  low  signal-to-noise  and  discontinuities  in  antibody  or 
other  stains.  As  a  result,  many  vertices  must  be  individually  stored. 

Vectored  object  lists  can  be  traversed  in  I  or  V  space  in  object 
order.  Thus,  complex  figures  can  be  rotated  and  projected  individ¬ 
ually.  Refinements  in  geometric  object  technologies  may  appear  to 
place  voxel  rendering  at  a  disadvantage  due  to  its  ordered  pro¬ 
cessing  of  many  more  data  points.  Choice  of  data  objects  is  largely 
determined  by  the  amount  of  information  known  about  the  speci¬ 
men,  the  sophistication  or  realism  required  in  each  view  and  the 
availability  of  appropriate  hardware  and  software  on  the  chosen 
platform.  It  should  now  be  apparent  that  there  is  no  fundamental 
distinction  between  any  of  the  data  objects  discussed.  A  trade-off 
between  processing  speed  and  transfer  rate  must  be  weighed 
against  the  problems  or  bias  associated  with  segmentation  of  the 
image  data  into  parametric  structures.  In  any  case,  voxel  render¬ 
ing  speeds  are  now  equivalent  to  the  speed  at  which  vector  graph¬ 
ics  objects  could  be  drawn  just  a  few  years  ago.  All  of  the 
compositing  or  P  function  rules  described  below  can  be  applied  to 
any  data  objects. 

Sophisticated  graphics  systems  are  now  relatively  inexpensive 
and  are  supplied  with  even  modest-specification  PCs.  Some 
include  hardware  accelerated  geometry-processing  engines,  with 
standardized  OpenGl  interfaces,  intimately  linked  with  the  GPU 
and  display  memory  together  with  additional  buffers,  LUTs,  etc. 
Thus  many  of  the  above  operations  are  now  readily  available  to 
any  software  developer  writing  for  a  standard  hardware  platform. 

Segmenting  the  Data  Objects 

Segmentation  is  a  process  by  which  objects  are  extracted  from  the 
rastered  voxel  image  data.  Voxels  are  selected  according  to  their 
brightness  and/or  by  some  property  linking  them  to  other  voxels. 
A  lower  and/or  upper  threshold  may  be  applied  producing  a  seg¬ 
mented  band  containing  all  the  voxels  of  a  particular  object.  Most 
systems  allow  this  intensity  segmentation  to  remove  background 
or  to  isolate  a  homogeneously  stained  structure.  Threshold  values 
are  readily  chosen  by  masking  a  histogram  and  observing  this 
simultaneously  with  an  interactive,  color-banded  screen  display  of 
the  segmented  voxels.  Histogram  peaks  indicate  subpopulations  of 
background  voxels,  possible  structural  features,  etc.  This  edited 
histogram  produces  a  photometric  look-up  table  (P-LUT)  through 


which  the  image  is  rapidly  mapped.  Morphological  segmentation 
requires  a  selection  based  on  some  connectivity  rule  such  as 
the  geometric  surface  objects  extracted  by  the  Marching  Cubes 
(c.f.  VoxBlast)  or  Delaunay  Triangulation  (c.f.  Visilog)  algorithms. 
Three-dimensional  intensity  gradient  filters  can  also  find  or 
enhance  boundaries  between  geometric  objects.  Cheng  and  co¬ 
workers  describe  the  use  of  such  a  filter  to  extract  a  gradient  image, 
which  is  then  blended  with  the  original  to  enhance  edges.  Using 
the  earlier  notation 

I„utG„  yi,  Zi,)  =  l„,(x„  yi;  Zj,)  [k  +  (1  -  k)  grad(arctan(Iin(Xj,  Zj,))] 

(15) 

where  the  arctan  is  an  (optional)  approximation  to  a  sigmoidal 
background  suppressor  and  grad  is  the  unsigned  3D  gradient 

gradfe  Vi,  Zi)  '=  l[(dl/dvi)2  +  (dl/dvi)2  +  (dl/dz;)2] 

This  3D  gradient  is  simply  the  resultant  of  the  unsigned  com¬ 
ponent  gradients  along  each  axis.  These  individual  components  are 
conveniently  approximated  (for  low  noise  data)  by  subtracting  the 
values  on  each  side  of  the  voxel  whose  gradient  is  required,  that 
is,  grad  =  (l((I(Xj  +  l,Vi,  Zi)  -  Mx,  -  1,  yi;  z;))2  +  (Ife  Vi  +  1,  Zi)  - 
I(*i,  Vi  -  Zi))2  +  (Ife  Vi>  Zi  +  1)  -  Ife  Vi,  Zi  -  l))2))/2. 

Gradient  filters  are  also  extensively  used  to  provide  data  for  the 
realistic  artificial  lighting  effects  (discussed  later).  Other  back¬ 
ground  filters  are  based  on  patch  convolutions  with  kernel  weights 
given  by  smoothing  functions.  Forsgren  and  colleagues  (1990)  use 
a  3D  Gaussian  filter  where  the  mask  weights  are  given  by: 

F(x,  y,  z)  =  l/(ct327tl(27t))  exp  (-  (x2  +  y2  +  z2  )/o2)  (16) 

different  strengths  are  specified  by  the  width  term  o.  This  filter  is 
separable  and  is  readily  implemented  as  a  sequence  of  ID  filters, 
allowing  for  asymmetric  sampling  in  the  multi-dimensional  image. 
A  general  approach  is  to  use  a  3D  filter  (or  ideally  a  PSF  decon¬ 
volution)  with  a  cut-off  at  the  Nyquist  frequency  (see  also  Chap¬ 
ters  4  and  25,  this  volume),  followed  by  a  threshold  segmentation 
of  the  filtered  output.  The  segmented  output  is  then  used  in  a 
logical  test  (or  mask)  to  segment  the  original  image.  Segmentation 
(Fig.  14.13)  may  simply  exclude  background  voxels  or  the 
included  voxels  can  then  be  grouped  into  polygon  objects.  Voxel 
objects  can  use  a  material  look-up  table  (M-LUT)  for  each  prop¬ 
erty.  It  is  possible  to  specify  all  material  properties  for  any  object 
types  as  indices  in  one  or  more  material  look-up  tables  (M-LUTs) 
and  to  use  these  to  separate  different  materials  within  the  volume. 
The  Analyze  package  has  a  good  example  of  a  dedicated  seg¬ 
mentation  menu  that  brings  together  image  edit,  morphology 
operations,  and  object/surface  extraction  functions.  The  Surpass 
optional  module  in  Imaris  groups  together  the  object  segmentation 
capabilities  of  that  package. 

Scan  Conversion 

After  geometric  rotation,  etc.,  the  data  objects  are  drawn  into  the 
final  view.  For  polygons,  surface  nets,  etc.,  this  requires  a  scan  con¬ 
version  whereby  the  vectors  are  turned  into  rastered  lines  of  pixels 
(Watt,  1989).  A  pixel  view  of  each  polygon  patch  is  then  compos¬ 
ited  into  the  final  view  as  for  rastered  voxels.  A  simple  approach 
is  to  draw  a  closed  outline  and  to  use  a  fast-fill  algorithm,  modi¬ 
fied  by  any  shading  required.  It  is  often  more  efficient  to  generate 
just  the  ends  of  each  scan  line  through  the  polygon,  filling  the  gaps 
by  simple  line  drawing.  For  two  vertices  S(xs,  ys)  and  E(xe,  ye) 
defining  the  start  and  end  of  a  polygon  edge,  there  are  (ye  -  ys) 
intermediate  scan  lines.  The  view  coordinates  for  the  start  of  the 
/th  scan  line  are 
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FIGURE  14.13.  Object  segmentation.  Voxels  may  be  seg¬ 
mented  from  the  image  volume  only  to  remove  background  and 
unwanted  features,  or  they  may  additionally  be  grouped  into 
larger  geometric  objects.  (A-I)  show  2D  slices  from  a  small  3D 
volume  before  and  after  segmenting  with  various  algorithms. 
High-pass  (B)  and  mid-intensity  band  (C)  are  the  simplest  oper¬ 
ations  and  are  often  used  on  their  own,  or  in  conjunction  with 
other  operators.  Surface  extractions  (D,E)  (e.g..  Marching  cubes, 
c.f.  VoxBlast  or  Delaughney  Triangulation,  c.f.  Visilog)  produce 
(c)  Mid-intensity  band  bounded  structures  obscuring  internal  details.  Manual  or  semi¬ 

automatic  image  editing  or  reconstruction  cut-aways  (F)  reveal 
internal  details  without  corrupting  the  data  values.  Gradient  mag¬ 
nitude  filters  (G)  highlight  voxel  boundaries  and  edges.  Thresh- 
olded  seed  fill  (H)  (c.f.  Visilog,  ImageSpace)  gives  a  solid  object 
for  simple  volume  estimates,  etc.  Complex  modes  such  as  the 
gradient  magnitude  blended  with  the  image  (I)  (see  text)  can  give 
reasonable  enhancement  of  edges  without  artifactual  “halos”  (c.f. 
Prism  II/DeltaVision). 


(d)  Mid  threshold  surface  extract  (e)  High  threshold  surface 


(f)  Image  edit  or  cut-away 


(g)  Gradient  magnitude 


(h)  High  pass  threshold 
and  seed  fill 


(i)  Product  of  gradient 
magnitude  and  original 


xv  =  (int)(xs  +  j8x),  yv  =  ys  +  j  (17) 

where  bx  =  (.re  -  xs)/(ve  -  vs).  The  ends  of  each  scan  line  are 
obtained  by  a  similar  calculation. 

Scan  conversion  or  rasterization  is  a  well-defined  component 
of  the  OpenGL  graphics  pipeline  and  this  affords  a  convenient 
point  in  the  reconstruction  process  to  combine  or  embed  graphical 
objects  into  a  voxel  or  previously  generated  pixel  view.  This  is  a 
powerful  technique  used  to  good  effect  by  the  major  rendering 
packages  to  show  high  contrast  segmented  objects  within  the 
context  of  a  general  volume  view  (Amira,  for  example,  allows  for 
an  arbitrary  number  of  datasets,  image  modalities  and/or  visual¬ 
ization  modes  within  the  same  display  view). 


Projection  Rules 

As  each  geometric  object  or  image  plane  is  composited  into  the 
view,  incoming  data  either  replaces  or  is  blended  with  the  accu¬ 
mulating  result.  The  algorithm  used  is  usually  a  function  of  the 
z-coordinate,  the  intensity  value,  and  the  associated  material 
properties  (Table  14.8).  This  P  function  has  two  components:  an 
arithmetic  or  compositing  (C)  function  (which  may  be  just  a  simple 
assignment)  and  an  optional  logical  test  (T).  For  the  nth  (of  N) 
frames  or  objects  composited  into  the  view  V,  a  general  form  is 

if  {T  [I(xv,  y„  zv)„,  V(xv,  yv,  zv)„  _  J}  then  V(xv,  yv)„ 

C  [V (xv,  yv,  zv)n>  hvv,  y v,  zv)„  [  (18) 

Simple  compositing  functions  (Fig.  14.14)  include: 

•  Maximum  Intensity 


Maximum  intensity  projection,  or  MIP  (e.g.,  Sakas  et  al.,  1995), 
is  now  the  most  widely  used  quick  visualization  function  (e.g.,  see 
Analyze,  Cedara,  Imaris,  FIRender,  Voxblast  etc)  and  can  be  effi¬ 
ciently  implemented  as  a  stand-alone  mode  or  in  combination  with 
other  algorithms. 

•  Average  Intensity 

(no  test)  V(xv,  yv)„  =  V(xv,  yv,  zv)„  +  I(xv,  yv,  zv)„/N 

(20) 

Less  common  than  MIP,  mean  intensity  is  found  in  the  LSM  pro¬ 
grams  and  Analyze.  It  is  useful  when  the  data  volume  is  very 
dense,  which  would  tend  to  give  a  very  saturated  view  with  MIP. 
Because  it  effectively  averages  along  the  projected  ray,  this  func¬ 
tion  reduces  noise  but  produces  lower  contrast.  This  can  be  partly 
overcome  with  an  appropriate  display  LUT. 


•  First  or  Front  Intensity  >  t 

if  { [I(xv,  yv,  zv)„  >  t]  &  [(zv,„_!)  <  (zv,„)] } 
then  V (xv,  yv)„  =  I(xv,  yv,  zv)„ 

(21) 


This  is  a  quick  way  of  exploring  a  surface  or  boundary  in  a  voxel 
representation,  particularly  as  an  aid  to  determining  parameters  for 
more  complex  modes  (e.g.,  see  Lasersharp  projection  modes). 


•  Alpha  Blend 


(no  test)  V(xv,  yv)„  =  (1  -  a) 

VUv,  Vv,  Zv)n-i  (ot)  1  (xy,  yv,  zv)„  (22) 


if  [I(xv,  yv,  zv)n  >  V(xv,  yv,  zv)„_ i]  then  V(xv,  yv)„  =  I(xv,  yv,  zv)„ 

(19) 


This  is  the  standard  mode  used  for  object  visualization  and  works 
particularly  well  for  voxel  objects  and  is  implemented  in  virtually 


FIGURE  14.14.  Projection  combination  or  compositing  rules.  3D  visualizations  of  a  stack  of  130  confocal  optical  sections  through  a  portion  of  Arabidop- 
sis  plant  root  using  Lasersharp  (A-C)  and  Imaris  (J-L).  Autofluorescence  (and  a  little  non-specific  fluorescent  thiol-staining)  shows  vascular  tissue,  cell  walls 
and  nuclei  and  lateral  root  buds.  A  small  attached  fungus  is  seen  attacking  the  plant  along  the  lower  edge  towards  the  center  of  this  portion  of  root.  (Maximum 
intensity  projections  as  rotations  at  —30 deg  (A),  Odeg  (B)  and  +30  deg  (C).  Equivalent  average  projections  (D-F)  show  the  characteristic  “fuzzy  X-ray”  char¬ 
acter  of  images  made  with  this  algorithm.  The  first  voxel  (nearest  the  viewer)  along  each  cast  ray  is  recorded  in  (G-I)  showing  a  more  solid  appearance  while 
retaining  original  intensity  values  but  at  the  expense  of  losing  many  details,  a-rendering  at  the  same  angles  (J-L)  clearly  represents  the  density  of  the  structure 
as  well  as  fine  details  but  introduces  some  spurious  high  intensities  where  the  structure  is  thin  (e.g.,  at  the  end  of  the  lateral  root  tips). 
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all  of  the  packages  in  Table  14.1.  Although  the  added  complexity 
of  having  to  set  the  a  parameter  sometimes  makes  it  harder  to  use 
than  MIP  for  LSM  fluorescence  data,  this  parameter  is  the  basis 
for  all  opacity/transparency  effects  (see  below). 

The  tests  in  Eqs.  19  and  21  may  be  reversed  to  obtain  minimum 
and  below-threshold  versions.  The  alpha  term  is  a  general  func¬ 
tion  that  can  partly  define  the  physical  properties  of  the  rendered 
object.  This  blending  factor  can  be  dynamically  modified  as  a  func¬ 
tion  of  zv  or  n  during  the  projection.  A  Kalman  version  of  Eq.  20 
may  be  implemented  in  this  way;  the  number  of  composited  frames 
N  need  not  be  known  in  advance,  and  the  process  can  be  stopped 
when  the  required  result  is  reached. 

•  Kalman  Average 

V(xv,  yX  =  (1  -  1/n) 

V(xv,  yy,  Zy)„-i  +  ( l/n)I(xv,  yv,  zv)„  (23) 

This  is  a  dynamically  modified  a  =  1//?,  and  the  projection  may 
proceed  in  positive  or  negative  zv  order.  These  define  the  front-to- 
back  and  back-to-front  view  space  rendering  geometries.  It  should 
be  readily  obvious  that  these  two  processes  afford  a  simple  means 
of  modeling  the  lighting  of  voxels  from  front-to-back  or  the  emis¬ 
sion  of  light  from  voxels  running  from  back-to-front  with  respect 
to  the  viewer.  The  latter  rendering  order  also  shows  the  view  build¬ 
ing  up  towards  the  viewer  that  can  reveal  internal  details  of  an 
object  during  the  visualization  process,  provided  the  screen  is 
updated  during  the  computation.  A  display  equipped  with  a-plane 
hardware  (with  associated  firmware  or  software)  automatically 
blends  the  incoming  data  using  factors  stored  and  updated  in  an 
alpha  memory  plane,  a-blending  may  also  be  implemented 
entirely  in  software.  This  is  a  standard  OpenGL  feature.  The  a 
value  is  an  eighth  parameter  for  vectors  (after  the  coordinates  and 
RGB  intensity)  or  a  fifth  value  for  voxels  (with  implied  coordi¬ 
nates)  specifying  blending  properties  for  the  intensities  of  that 
object.  Visibility  of  an  incoming  voxel  or  data  object  can  thus  be 
made  dependent  on  object  opacity  or  transparency  and  therefore 
depth  (zv)  in  the  rotated  view  (see  below).  Earlier  we  discussed 
ways  in  which  the  geometric  transformation  could  be  modified  to 
encode  additional  z-information  into  the  display  geometry.  In  addi¬ 
tion  to  z-related  blending  operations,  the  compositing  algorithm 
can  retain  z-information  within  the  displayed  intensities  using 
alternative  algorithms. 

How  Can  Intensities  Be  Used  to  Retain 
z-Information 

z-Coordinate  or  Height  Views 

The  height,  range,  topological,  relief  or  (the  author’s  preference) 
Z-coordinate  view  (e.g.,  Boyde,  1987;  Freire  and  Boyde,  1990; 
Forsgren  et  al.,  1990;  Odgaard  et  al.,  1990)  technique  has  been 
used  for  many  years  to  directly  record  the  zv-depth  in  the  intensity 
or  color  space  (or  both)  of  the  display.  The  rule  includes  a  test  that 
selects  a  particular  voxel  along  the  observer’s  line  of  sight.  The  zv_ 
coordinate  is  then  assigned  [after  an  offset  (zD)  and  scaling  (zf)]  to 
a  value  in  the  view.  A  range  of  z-coordinate  tests,  similar  to  the 
intensity  tests  (Eqs.  19,  20)  above,  are  found  (Fig.  14.15): 

•  Coordinate  of  Maximum  Intensity 

if  [  I(xv,  yv,  zv)„  >  V(xv,  y„  zv)„_i]  then  V(xv,  yv)„  =  z0  +  zfzv,„ 

(24) 

•  Coordinate  of  First  Intensity  >  (t)  (i.e.,  Maximum  Height 
or  Nearest) 


if  {[I(xv,  yv,  Zv)„_i  >  t]  &  [(Zv,„-i)  <  (zv,„)]}  then  V(xv,  yv)„  =  etc. 

(25) 

These  z-coordinate  modes  are  used  by  simple  3D  topology  pro¬ 
grams  (e.g..  Automontage)  as  well  as  some  Tenderers  including 
Analyze,  VolumeJ,  Lasersharp.  etc. 

•  Iso-Intensity  Surface 

if  [I(xv,  >y,  zv)„_i  =  t)]  then  (etc.)  (26) 

Although  related  to  the  previous  projections,  the  iso-surface 
routine  is  usually  implemented  by  a  recursive  3D  algorithm  such 
as  the  marching  cubes  (Lorensen  and  Cline,  1987)  and  is  the  basis 
of  many  surface  object  segmentation  algorithms  (e.g.,  Voxblast 
and  Analyze). 

Through-focus  images  from  widefield  microscopy  have  been 
processed  using  a  maximum  test  applied  to  a  prefiltered  version  of 
each  frame.  For  example,  the  maximum  local  variance  can  produce 
an  auto-focus  intensity  or  coordinate  view  (c.f.,  Automontage). 
The  depth  discrimination  of  the  confocal  microscope  allows  a 
simpler  auto-focus  routine  using  just  maximum  brightness  (for  a 
single-surface  object).  This  requires  that  the  z-response  for  the  par¬ 
ticular  specimen  (point,  plane,  etc.)  has  no  significant  side  lobes 
(the  confocal  PSF  is  investigated  by  Shaw  and  Rawlins.  1991). 
Instead  of  replacing  the  brightness  with  zv,  intensities  can  be  mod¬ 
ified  (weighted)  by  their  z-coordinate  giving  a  so-called  depth- 
weighted  projection  (Fig.  14.16).  The  simplest  form  is  a  linear 
weighting  from  1  (nearest  the  observer)  to  0  (furthest  away). 

•  Linear  Depth- Weighted  Projection 

V (xv,  >\.)  =  C[I(xv,  yv,  zv)z]  (27) 

where  z  =  (zback  -  zv,„)/(zback  -  Zfront),  that  is,  a  normalized  z- 
coordinate.  A  more  sophisticated  form  is 

•  Exponential  Depth  Weighting 

V(xv,  yv)  =  C[I(xv,  yv,  zv)  *f  ]  (28) 

where  f  is  a  constant  <1. 

This  algorithm  (which  can  be  implemented  in  any  z-order)  is 
often  described  as  an  attempt  to  model  the  absorption  of  light  from 
a  source  directed  along  the  z-axis  of  the  data  from  behind  (trans¬ 
mitted)  or  the  attenuation  of  emitted  fluorescence.  Not  surprisingly 
it  turns  out  that  this  result  is  identical  to  that  achieved  by  an 
ordered  recursive  algorithm  traversing  the  data  from  front  to  back 
using  a  constant  a-blending  factor  (<1): 

•  Recursive  Exponential  Weighting 

V(xv,  yv)  =  C{[(1  -  f)*  I(xv,  y„  zv)„],  [I(xv,  yv,  zv)„+if] }  (29) 

The  commonest  form  of  Eqs.  27  to  29  is  a  linear  average  or 
summation,  but  any  compositing  function  can,  in  principle,  have 
a  depth-weighted  component.  By  making  the  factor  equal  to 

f  =  zst/(Nn),  (30) 

where  N  is  the  number  of  serial  planes  in  the  projection,  a  non¬ 
linear  projection  of  strength  equal  to  zstr  is  obtained.  If  zstr  =  N,  the 
result  is  identical  to  a  Kalman  average.  For  zstr  =  1.  the  front  or 
maximum  height  voxels  completely  dominate  the  view.  Other 
values  give  intermediate  results  [Fig.  14. 16(A— C)]. 

Hidden-Object  Removal 
Z-Buffering 

z-Ordered  compositing  simplifies  the  implementation  of  z- 
algorithms.  Front-to-back  projections  using  the  first-object  test 
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FIGURE  14.15.  Z-eoordinate  views.  (A)  Maximum  intensity  projection  of  serial  confocal  optical  sections  through  pollen  grains  in  a  partially  dissected  plant 
anther.  Some  sections  are  missing  from  the  data  stack  (or  perhaps  the  specimen  is  incomplete)  but  it  cannot  be  elucidated  whether  the  loss  is  from  the  top  or 
bottom  of  the  set  from  this  projection,  which  carries  no  z-information.  (B)  shows  a  height  or  distance  projection  where  the  brightness  is  coded  by  axial  distance 
from  the  viewer  (dark  is  furthest  away)  recording  the  first  voxel  above  a  background  threshold.  It  can  now  be  seen  that  the  lost  sections  are  at  the  top  of  the 
stack  which  was  projected  top-to-bottom.  (C)  shows  the  height  of  each  voxel  chosen  by  maximum  projection  in  (A),  confirming  the  finding  from  (B).  (D)  shows 
the  same  algorithm  as  in  (B)  projecting  bottom-to-top  through  the  stack. 


merely  check  if  a  non-zero  intensity  has  been  encountered.  Hidden 
voxels  are  never  processed.  Conversely,  if  a  back-to-front  pass  is 
used,  the  last  voxel  will  be  the  one  displayed  and  no  test  is  required. 
The  entire  volume  is  now  traversed  and  all  voxels  are  processed. 
This  can  be  very  informative  if  the  rendering  process  is  visible  on 
an  interactive  display.  For  non-z-ordered  objects,  a  more  intensive 
logical  comparison  of  intensities  and/or  ^-coordinates  is  needed, 
such  as  a  front-object  z-test;  this  is  known  as  "-buffering  and  is  a 
fundamentally  important  OpenGL-controlled  process  in  modern 
graphics  systems.  Implementation  can  be  at  various  stages  of  the 
rendering  process.  Voxel  data  is  z-buffered  efficiently  as  a  whole 
section  operation  after  warping.  Polygon-ordered  rendering  uses 
standard  pixel  z-buffering  but  does  not  use  information  about  adja¬ 
cent  vertices  efficiently.  Scan  line  z-buffering  is  more  economical. 
It  is  intimately  associated  with  the  scan  conversion  of  polygon 
edges  to  pixels  line-by-line  without  needing  a  full  2D  z-buffer. 
Spanning  scan  line  z-buffering  is  highly  optimized  to  extract  visible 
object(s)  from  all  structures  that  intersect  the  screen  line  that  is 
currently  being  drawn.  Modern  graphics  hardware  can  encompass 
intensity,  z-buffer,  and  a-planes  to  program  all  of  the  computations 
described  above  into  the  display  logic,  releasing  the  processor  for 
other  computations. 


Local  Projections 

The  compositing  functions  described  above  can  be  used  in  com¬ 
bination  for  more  control  over  the  rendered  objects.  A  conflict 
exists  between  hidden-object  removal  and  significantly  modifying 
the  image  intensities  (by  excessive  use  of  a  factors  and  lighting 
terms).  A  novel  solution  is  implemented  in  the  Lasersharp  visual¬ 
ization  program:  the  trick  is  to  use  coordinate  or  z-buffer  algo¬ 
rithms  to  derive  a  segmentation  reference  (R;)  for  projected  pixels 
in  the  final  view.  R-  defines  the  z-coordinate  of  a  surface  [e.g.,  by 
maximum  intensity  (Eq.  24)]  or  boundary  [e.g.,  by  maximum 
height  above  threshold  (Eq.  25)],  etc.  R-  (for  each  view  pixel)  can 
then  become  the  center  for  a  z-banded  or  local  projection.  This 
technique  is  another  example  of  an  intelligent  z-buffer.  The  range 
of  the  local  projection  is  defined  by  z(ron,  and  zback  given  as  z-offsets 
from  the  reference.  Useful  local  projections  include: 

(l)Reference  Height  Above  Threshold  +  Local  Maximum 
Intensity,  e.g..  Maximum  Height  >  t  (Reference). 

if  ][I(xv,  >y,  zT)„_ i  >  t]  &  [(z„_!)  <  (z„)]}  then  R-(xv,  y,)  =  z„ 

followed  by 

Local  maximum  intensity 
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FIGURE  14.16.  Depth  weighting.  (A-C)  MPL  “non-linear  average”  depth-weighted  projections  (see  text)  with  “strengths”  of  1  (A),  3  (B)  and  9  (C)  (number 
of  sections  =  50).  Micro  Voxel  “depth-weighted”  views  (D)  summation  (average),  (E,F)  “first  mode”  renderings  of  nerve  cells. 


if  [I(*v,  yv,  zr)  >  V(xv,  yv,  zr-i)]  then  V(xv,  yv)  =  l(x„  yv,  zr) 

where  [R-(.rv,  yv)  +  zfront]  >  zr  <  [R-(*v,  yv)  +  zback],  (i-e.,  the  local 
range). 

Similarly  one  might  use 

(2)  Height  at  Maximum  Intensity  +  Local  Kalman  Average. 

(3)  Height  at  First  Intensity  >  tb  +  Offset  Local  Height  at  Inten¬ 
sity  >  t2 

e.g.,  with  Zfront  <  Zback  <  R;(*v,  yv). 

(4)  Height  at  Maximum  Intensity  +  Offset  Local  Maximum 
Intensity 

e.g.,  with  Zback  >  Zfront  >  R-C*v,  yv). 

Local  projections  3  and  4  use  a  range  that  is  offset  from,  that 
is,  does  not  span,  R-.  This  is  an  objective  way  to  segment  a  second 
object  or  surface  within  a  given  range  of  a  more  dominant  primary 
feature  (which  is  used  for  the  reference).  Thus,  a  plant  cell  wall  or 
animal  cell  membrane  may  be  found  by  a  reference  segmentation, 
and  then  structures  within  zfront  voxels  outside  or  zback  voxels  inside 
the  cell  can  then  be  projected  in  isolation.  Comparative  results 
of  some  local  projections  from  Lasersharp  are  shown  in  Figure 
14.17. 


Adding  Realism  to  the  View 

The  algorithms  discussed  so  far  use  test  and  compositing  rules  to 
project  multi-dimensional  images  into  the  view  space.  Views  of 
macroscopic  objects  contain  depth  cues  (similar  to  those  described 
above)  along  with  textural  cues  arising  from  the  position  and 
properties  of  light  sources.  These  can  be  used  to  add  realism  to 
reconstructed  views  in  microscopy  by  (1)  mimicking  artificial 
macroscopic  lighting  effects  or  (2)  developing  a  more  objective 
visualization  model  incorporating  a  priori  knowledge  concerning 
the  optical  properties  of  the  sample.  Advanced  algorithm  options 
are  listed  in  Table  14.9  and  described  in  Figures  14.18,  14.19, 
14.21,  and  14.22  with  example  results  in  Figures  14.17  and  14.20. 

Artificial  Lighting:  Reflection  Models 

The  ambient  lighting  of  the  model  assumed  above  has  no  direc¬ 
tional  components.  Artificial  lights  have  intensity  (photometric), 
directionality  (geometric),  and  color  (chromatic)  properties.  These 
characteristics  interact  with  the  material  properties  of  data  objects 
to  modulate  the  rendering  process.  Local  lights  are  near  or  within 
the  data  volume  and  infinity  sources  are  parallel  rays  coming  from 
infinity.  Ambient  lighting  is  a  general  level  diffused  by  multiple 
reflections  off  all  objects  within  the  volume,  as  distinct  from  light 
coming  direct  from  the  source  to  a  given  object.  Reflections  from 
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FIGURE  14.17.  Local  projections.  These  3D  views  are  made  from  the  same  data  set  shown  in  Figure  14.14.  (A.B)  show  local  projections  where  the  maximum 
intensity  is  found  through  about  one  third  of  the  depth  of  the  sample  above  and  below  each  voxel  found  by  a  previous  application  of  the  “above  threshold"  rule. 
(A)  is  from  above  and  (B)  below.  (C,F)  show  the  same  operations  applied  through  a  depth  of  about  one  tenth  of  the  sample  thickness  from  the  reference  voxel. 
More  structures  towards  the  outer  surface  of  the  root  are  apparent  compared  to  regular  maximum  projections,  masking  underlying  features.  (D.E)  are  local 
average  projections  corresponding  to  (A.B).  Normalizing  these  intensities  to  fit  the  dynamic  range  of  the  display  leads  to  lower  contrast  views  than  in  (A.B)  but 
shows  some  weak  thiol-staining  more  clearly  against  the  vascular  autofluorescence.  These  views  are  more  amenable  to  direct  quantification  than  z  or  b. 


an  object  (Fig.  14.18)  can  be  approximated  by  a  function  (L)  com¬ 
posed  of  (1)  ambient  reflections  (La),  (2)  diffuse  reflections  (Ld), 
and  (3)  specular  highlights  (Ls)  (strongest  in  the  source  direction). 

L  =  La  +  (Ld  +  Ls)/(zobj  +  constant)  (31) 

where  sobj  is  the  distance  (in  zw  of  the  object  from  the  observer). 


The  denominator  in  Eq.  19  is  an  approximation  to  the  zl 
denominator  from  the  diffuse  reflection  law  of  Lambert  (Born  and 
Wolf,  1991,  p.  183)  for  cases  where  zobs  is  large  compared  to  the 
object  size.  Each  component  Lm  is  the  product  of  the  light  source 
intensity  Sm,  reflectivity  Rm  (a  material  property),  and  a  direction 
term  Dm  (m  =  a,  d,  or  s).  The  color  of  a  voxel  is  determined  by  the 


TABLE  14.9.  Overview  of  "Realistic"  Visualization  Techniques  for  Multi-Dimensional  Biological  Microscopy  Data 


Feature 

Parameter 

Minimum  Required 

Desirable  additional  enhancements 

“Visualization  models 

Voxel  render 

Shaded  surface,  Voxel  gradient.  Lighting  models,  SFP,  Embedded  objects 

Material  properties 

color 

RGB  channels 

Arbitrary  colors/channels 

opacity 

a-channel,  channel  dependent 

reflection 

Diffuse,  Specular  (“shiny”),  Interactive  control 

Hardware  texture  mapping 

emission 

Simulated  fluor. 

Hidden-objects 

Software  z-buffer 

Hardware  z-buffer 

Surface  shading  models 

Flat,  Incremental,  Gourard,  Phong  model 

Artificial  lighting 

Color 

RGB 

Arbitrary  colors  &  sources 

Lightable  objects 

Voxels 

Voxels,  Surface  normal 

Lighting  models 

Ambient/diffuse 

Voxel  gradient,  Smoothed  voxel  “surface,” 

(brightness) 

Phong  surface  normal,  Fast  mode,  Precision  mode 

a  Realistic  visualization  modes  are  often  used  to  promote  particular  packages,  and  often  with  carefully  chosen  data!  Control  of  the  object  material  properties  and  a  clear 
understanding  of  each  parameter  are  essential.  The  final  reconstructed  view  should  always  be  studied  along  with  a  record  of  all  the  steps  and  variables  and  preferably  along¬ 
side  the  original  image  sections. 

b  Material  properties  and  artificial  lighting  should  be  standardized  for  each  view  if  intensity  information  is  to  be  reliably  compared  between  results  (particularly  for  SFP 
and  gradient-lit  voxel  modes).  Hardware  acceleration  has  made  possible  more  interactive  3D  views  and  “real-time”  processing  of  modest  data  sets  on  desktop  PCs. 
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FIGURE  14.18.  The  artificial  lighting  model.  Ambient  lighting  contributes 
only  to  the  overall  image  brightness  and  has  no  directional  components.  An 
artificial  light  source  is  directed  along  1  with  parallel  rays.  Diffuse  reflections 
from  the  object  surface  (solid  panel)  emanate  in  all  directions  (Ld).  Some  of  I.., 
may  be  seen  by  the  viewer  along  v.  Specular  reflections  (Ls)  occur  in  a  narrow 
cone  around  the  “reflection”  direction  r  (9nl  away  from  the  surface  normal  n). 
These  are  observed  along  v  if  <|)v[  is  small  enough.  The  width  of  the  cone 
depends  on  the  shininess  term  (see  text).  The  Phong  model  describes  this  geom¬ 
etry  in  terms  of  (|)vr  and  0nl  Blinn  showed  that  only  <|)nh  (between  the  surface 
normal  and  a  theoretical  plane,  shaded,  that  would  reflect  all  light  from  1  to  v) 
was  needed  for  parallel  lighting  and  a  stationary  observer. 


light  emanating  from  it.  In  general,  colors  are  differentially 
reflected,  absorbed,  and  fluorescently  emitted  (see  below).  The 
reflection  components  (Fig.  14.18)  are 

La  =  Sa  RaDa  (32) 

where  Sa  is  the  ambient  light  level  (constant),  Ra  is  the  diffuse 

reflectivity,  and  Da  equals  1. 

Ld  =  Sd  RdDd  (33) 

where  Sd  is  the  source  intensity,  Rd  is  the  diffuse  reflectivity,  and 
Dd  is  cos  0„i 

Ls  =  Ss  RSDS  (34) 

where  Ss  is  the  source  intensity,  Rs  is  the  specular  reflectivity,  and 
Ds  is  cossh  (pvr 

where  0nl  is  the  angle  between  the  local  object  surface  normal  and 
the  light  source  direction;  cpvr  is  the  angle  between  the  reflection 
angle  (center  of  the  highlight)  and  the  viewing  angle;  and  sh  is  the 
shininess  parameter,  which  for  a  perfect  mirror  is  infinite. 

This  full  form  is  the  Phong  model  (Phong,  1975)  and  ignores 
secondary  reflections  (i.e.,  is  a  local  approximation). 

A  global  version  requires  multiple  rays  to  be  followed  through 
many  reflections.  This  is  known  as  ray-tracing  (as  distinct  from 
ray-casting  often  used  for  voxel  projection  methods)  or  photoreal¬ 
istic  rendering  (Kriete  and  Pepping,  1992)  and  is  very  computa¬ 
tionally  intensive.  Ray-tracing  applies  reflection  rules  to  rays  from 
the  source  as  they  bounce  from  surface  to  surface  within  the 
volume.  In  practice,  tracing  is  limited  to  rays  seen  by  the  viewer 
and  the  depth  or  number  of  surfaces  considered  is  also  restricted. 
The  best  implementations  use  an  adaptive  depth  method  varied 
according  to  local  material  properties  (Watt.  1989).  In  the  Phong 
model,  as  the  shininess  factor  increases,  the  highlight  sharpens  and 
can  be  seen  through  a  smaller  angle  (cpvr)  around  the  reflection 
angle  (2*0nl  away  from  the  light  source,  i.e.,  on  the  other  side  of 


the  surface  normal).  The  Phong  model  is  thus  a  simple,  local 
approximation  in  terms  of  the  light  source  direction,  surface 
normal,  material  properties,  and  the  viewing  angle. 

The  computation  can  be  simplified.  For  a  light  source  at  infin¬ 
ity,  the  viewing  and  lighting  angles  are  constant  over  the  volume,  cp 
can  then  be  expressed  as  the  angle  (tpnh)  between  the  surface  normal 
(n)  and  the  normal  to  a  hypothetical  surface  (h)  that  would  reflect  all 
light  to  the  viewer  (Blinn,  1977).  Then  cp^  =  (pvr/2  so  larger  values 
of  sh  are  needed.  These  approximations  mean  that  only  the  surface 
normal  changes  during  rotations.  A  fast-lighting  look-up  table 
(L-LUT)  can  be  used  to  precalculate  the  reflection  terms  and  the 
surface  normal  used  to  index  the  L-LUT.  Moving  lights  are  also  easy 
to  implement  because  a  static  view  has  constant  surface  normals. 
Finally,  the  Phong  model  gives  rise  to  (1)  diffuse  reflections  that  are 
the  same  color  as  the  material,  (2)  specular  highlights  the  same  color 
as  the  light  source,  and  (3)  if  the  (zobj  +  constant)  term  is  ignored, 
flat  surfaces  (e.g.,  facets  of  a  polygonal  net)  exhibit  no  shading 
variations.  The  RGB  version  of  the  Phong/Blinn  model  uses 
separate  coefficients  for  each  primary  color 

L  (r,g,b)  =  Sa  Ra  (r.g.b)  +  Sd  (Rd  (r,g,b)  cos  0nl  +  Rs  cossh  (p^)  / 

fobj  +  constant)op,ionaI  (35) 

This  is  the  usual  tri-color  space  model.  More  subtle  effects  are 
obtained  by  using  more  than  three  channels.  Imaris  is  a  good 
example  of  a  package  with  flexible  lighting  and  materials  proper¬ 
ties  options  that  include  ambient,  diffuse,  and  specular  reflection 
colors  and  light  emission  characteristics  (see  also  discussion  of 
absorption  and  emission  below). 

Enhancing  the  Phong  and  Blinn  Models 

Criticisms  of  the  basic  reflection  scheme  include  rather  flat  sur¬ 
faces  and  an  artifact  at  the  polygon  edges  where  the  intensity 
changes  rapidly.  The  eye  overreacts  because  it  is  so  well  adapted 
to  detect  edges  by  the  brightness  second  derivative.  A  thin  bright 
Mach  band  is  seen  (Watt,  1989)  and  the  rendered  view  has  a  syn¬ 
thetic  appearance.  Incremental  surface  shading  is  then  required 
(Figs.  14.19,  14.20).  Techniques  are  available  to  minimize  these 
and  similar  artifacts,  seen  as  stripes  on  many  reconstructions 
(Amira  uses  surface  smoothing  and  simplification,  Cedara  uses 
intelligent  opacity  to  reduce  Mach  bands). 

Gourard  Shading 

Incremental  shading  gives  flat  surfaces  variable  intensities  by 
interpolation  between  the  material  properties  at  the  vertices. 
Gourard  shading  (Gourard,  1971)  applies  bilinear  in-plane  inter¬ 
polation  between  the  (lit)  vertex  intensities.  This  gives  a  gradual 
change  across  each  facet  with  a  reduction,  but  not  elimination,  of 
Mach  bands.  Often  only  diffuse  reflections  are  used  because  high¬ 
lights  not  seen  at  the  vertices  cannot  be  reconstructed  in  the  facet. 
Diffuse  reflection  is  found  in  Eq.  33  for  each  vertex.  The  average 
surface  normal  for  facets  sharing  each  vertex  is  used  to  get  0nl. 
Local  vertex  intensities  are  derived  from  Eq.  35.  Line-by-line 
bilinear  interpolation  is  then  used  (during  the  scan  conversion)  to 
vary  intensity  over  the  polygon.  Gourard  shading  thus  gives  some 
smoothing,  but  with  (1)  residual  Mach  banding,  (2)  some  artifac- 
tual  loss  of  fine  relief,  and  (3)  discontinuities  between  animations 
of  rotation  sequences.  Surface-shaded  views  are  shown  in  Figure 
14.20. 

Phong  Shading 

Instead  of  using  vertex  intensities,  Phong  introduced  the  bilinear 
interpolation  of  vertex  normals  (Phong  and  Crow,  1975).  Intensi¬ 
ties  are  generated  incrementally  as  before,  but  now  include  a  spec- 
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FIGURE  14.19.  Incremental  surface  shading  models.  Once  segmented  from 
the  voxel  array,  geometric  objects  (constructed  from  polygons)  must  be  ren¬ 
dered  to  a  “smooth”  surface.  Incremental  shading  is  used  to  “fill  in”  the  polygon 
areas.  Flat  shading  (A)  applies  a  single  value  (intensity,  color,  etc.)  to  the  whole 
facet.  This  gives  very  matte  views  with  Mach  banding  (see  text)  between  facets. 
Gourard  shading  (B)  interpolates  the  average  intensity  and  color  values  (In)  at 
the  apices  across  the  facet.  This  appears  to  “smooth”  the  surface,  but  still  gives 
bands,  disjointed  rotations  and  inaccurate  specular  highlights.  Phong  shading 
(C)  interpolates  the  surface  normal  (hn)  between  the  average  values  at  each 
vertex  across  the  facet.  This  can  introduce  highlights  in  the  facet  center  as  well 
as  appearing  to  “round  off”  the  abrupt  edges  and  restoring  the  impression  of  a 
smoothly  curved  surface. 


ular  term  at  each  point  (instead  of  the  optional  Gourard  single  term 
for  the  whole  facet).  Thus,  highlights  within  each  face  can  now  be 
generated,  even  if  none  was  apparent  at  any  vertex.  Light  sources 
are  still  at  infinity  so  only  the  surface  normal  varies,  but  now 
within  each  facet.  Thus,  Phong  interpolation  tends  to  restore  the 
curvature  lost  when  a  polygonal  surface  is  approximated  during 
segmentation  or  iso-surface  coding.  Phong  interpolation  can  be 
speeded  up  with  Gourard-type  averaging  in  subregions  of  each 
face.  A  better  efficiency  return  is  to  be  gained  by  using  the  H-test 
(Watt,  1989)  that  decides  which  facets  require  a  highlight  and  thus 
Phong  shading.  The  rest  are  Gourard  shaded  with  no  quality  loss. 
All  packages  that  render  graphical  objects  use  surface  shading  with 
lighting  and,  perhaps,  material  properties.  The  user  should  care¬ 
fully  determine  which  modes  are  being  used  for  any  given  ren¬ 
dering  in  order  to  correctly  interpret  the  sophisticated  views 
obtained. 


Advanced  Reflection  Models 

Although  the  Phong  model  attempts  to  mimic  diffuse  reflection 
by  Lambert’s  law,  the  cossh®  specular  terms  are  still  empirically 
derived.  Cook  and  Torrance  (1982)  modeled  reflections,  with 
advanced  specular  terms,  using  a  physical  model  of  incident  light 
energy.  The  specular  term  is  derived  for  micro  facets  within  a 
locally  averaged  surface.  Detailed  surface  models  were  also  devel¬ 
oped  by  Blinn  (1977)  and  Torrance  and  Sparrow  (1967).  Cook  and 
Torrance  resolved  the  specular  term  into  wavelength-dependent 
components  using  material  refraction  and  the  Fresnel  equations 
(see  Born  and  Wolf,  1991),  thus  modeling  dispersion.  Local  and 
extended  light  sources  have  been  simulated  by  Warn  (1983)  and 
others.  Such  advances  produce  smaller  and  smaller  returns  for  the 
microscopist  seeking  to  objectively  render  confocal  images.  The 
computational  expense  of  even  more  sophisticated  reflection 
models  is  rarely  justified  because  the  fine-tuning  of  relevant  mate¬ 
rial  properties  cannot  be  reliably  accomplished  for  biological 
specimens. 

Gradient  Lighting  Models  for  Voxel  Objects 

A  surface  normal  is  used  for  all  reflective  lighting  models.  Local 
topology  must,  therefore,  be  accurately  determined.  For  rastered 
voxel  data,  the  computations  are  greater  in  number  and  more  prone 
to  noise,  etc.  Fluorescence  CLSM  images  suffer  particularly  from 
low  signal-to-noise  ratio  (Sheppard  et  al.,  1992)  and  must  there¬ 
fore  always  be  carefully  filtered  and/or  PSF-deconvolved  before 
gradient  segmentation  or  lighting  algorithms  are  applied.  Gradient 
filters  for  edge  and  surface  segmentation  (Eq.  15)  are  often  also 
used  to  highlight  boundaries  for  voxel  gradient  lighting  models 
(Fig.  14.20;  see  also  Forsgren  et  al.,  1990;  Odgaard,  1990).  An 
alternative  to  these  expensive  3D  filters  is  to  use  a  height  or  zx- 
coordinate  view  zco(.xv,  yv,  zv)  (Eqs.  24-26).  Surface  normals  can 
be  derived  from  a  local  gradient  in  zco  in  2D  V  space  (Gordon  and 
Reynolds,  1985;  Aslund  et  al.,  1988).  This  is  the  basis  for  post¬ 
lighting  models  (Fig.  14.21),  which  apply  the  lighting  algorithm 
to  a  2D  z-buffer  after  the  data  projection  stage  (c.f.,  VoxBlast). 

grad(xv,  yv)  =  I  [(dzco/dxv)2  +  (dzco/dvv)2  +  1]  = 

I  {UcoCl  +  1)  -  zco(xv  _  l)]2  +  [zco(yv  +  1)  -  ZcoCVv  -  1)]2  }  /  2 

(36) 

The  quantization  of  depth  in  zco  can  give  rise  to  artifactual  edge 
effects,  so  at  least  10-  and  preferably  16-bits  of  z-buffer  are  desir¬ 
able.  In  both  this  and  the  3D  filters,  steep  gradients  cannot  be  dis¬ 
tinguished  easily  from  discontinuities. 

Artificial  Lighting:  Absorption  and 
Transparency 

Transparency  can  be  modeled  by  an  attenuation  coefficient  applied 
to  each  object.  This  can:  (1)  modify  the  light  emanating  towards 
the  viewer  (emission),  (2)  attenuate  artificial  light  as  it  passes  from 
the  source  through  the  data  volume  (illumination  or  excitation), 
and  (3)  modify  the  light  emanating  from  an  object  that  contributes 
to  the  lighting  of  another.  Ray-tracing  is  required  to  properly 
implement  (3).  The  first  process  (1)  is  the  most  often  encountered 
in  visualization  programs.  All  objects  are  evenly  lit  by  the  ambient 
light,  which  does  not  include  attenuation  by  nearby  objects. 
Each  object  brightness  is  seen  by  the  observer  according  to  its 
opacity/transparency.  This  is  achieved  by  a-blending  (Eq.  22)  with 
an  a  value  derived  by  using  the  object  intensity  as  an  index  into 
an  opacity/transparency  look-up  table  (OT-LUTs).  By  using  the 
back-to-front  projection,  the  combined  opacity  of  all  other  objects 
through  which  the  imaginary  rays  pass  is  taken  into  account. 
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Lit  and  shadowed 
voxels 

Z-buffer  of 
‘heights’ 

A  Voxel  render  B  Post  lighting 

FIGURE  1 4.21 .  Post  lighting  is  a  rapid  way  of  interactively  applying  lighting 
models  to  pre-rendered  views.  The  zv  information  is  retained  during  the  voxel 
intensity  render  in  the  z-buffer  as  a  “height”  view  (A).  If  an  iso-intensity  surface 
is  defined  or  a  constant  segmentation  threshold  used  the  intensity  view  may 
show  little  or  no  detail  [upper  part  of  (A)].  Local  relief  in  the  z-buffer  is  inter¬ 
rogated  to  produce  a  map  of  surface  normals  that  are  used  as  pointers  into  a 
fast  lighting  L-LUT  (see  text).  These  lighting  terms  are  used  to  highlight  and 
shadow  the  rendered  voxels  (B). 


Opaque  objects  at  the  front  of  the  volume  hide  those  at  the  back. 
The  OT-LUT  may  have  separate  partitions  for  different  structures 
within  a  single  volume.  Kay  and  Greenberg  (1979)  used  the  zv 
component  of  the  surface  normal  to  attenuate  rays  traversing  poly¬ 
gons.  For  voxel  objects,  the  average  opacity  at  each  voxel  is 
sufficient.  This  basic  attenuation  produces  no  lateral  shading  or 
shadowing. 

The  excitation  part  of  the  model  (Eq.  2)  is  used  to  simulate  the 
attenuation  of  illumination  between  a  source  at  infinity  and  data 
objects.  An  excitation  source  is  positioned  at  infinity  with  direc¬ 
tion  L  =  (L„  Ly,  1).  This  light  illuminates  the  rotated  image  I(xv, 
yv,  zv)  by  a  plane  wavefront  normal  to  L  passing  through  the 
volume.  This  wavefront  intersects  with  the  volume  in  serial 
(oblique)  planes.  Voxels  cut  by  the  nth  plane  during  the  excitation 
phase  are  given  by 

I(*ex,  Vex,  n)  =  I(xv  +  L Xn,  yv  +  L/i,  n)  (37) 

The  components  of  the  lighting  vector  can  be  made  integers  to 
speed  up  computations.  As  the  excitation  wave  propagates  through 
the  volume,  it  is  attenuated  by  preceding  layers  of  objects  accord¬ 
ing  to  an  excitation  extinction  coefficient  aex.  Usually  the  object 
(or  voxel,  etc.)  intensity  is  used  to  represent  the  amount  of  absorb¬ 
ing  material  at  each  position  (a  can  be  varied  by  an  OT-LUT).  The 
excitation  wavefront  at  the  nth  plane  is  now 


Lem(n) 


FIGURE  1 4.22.  The  simulated  fluorescence  process  (SFP)  algorithm  (see  also 
Fig.  14.20)  attempts  to  simulate  the  excitation  and  emission  from  fluorescent 
features.  Excitation  light  Lex  excites  voxels  in  successive  planes  (n,  n+1,  etc.). 
Each  illuminated  voxel  (Iex)  emits  fluorescence  (Lem)  in  all  directions.  Some  of 
Lem  is  seen  by  the  viewer.  Excitation  light  not  absorbed  by  the  voxels  in  plane 
n  (A)  excites  regions  in  plane  n+1  (B),  etc.  Similarly,  emitted  light  from  each 
plane  passes  through  other  voxels  (C)  as  it  travels  out  of  the  volume.  Absorp¬ 
tion  of  Lem  thus  follows,  but  re-excitation  by  Lem  is  usually  ignored.  Separate 
absorption  co-efficients  for  Lex  and  Lem  are  used  for  selective  control  of  trans¬ 
parency  (see  text). 

Lex(Aex>  ft)  Lex(Aex,  yex->  ft  1 )( 1  f^ex)  I(-^ex»  ^ex?  ft)  (38) 

A  lit  or  excited  image  volume  Iex  can  then  be  constructed 

lexC^-ex?  yexi  ft)  I(-^ex»  y^xi  ft)  LexC^ex*  3^ex>  ft)  (39) 

This  illuminated  volume  could  be  plugged  directly  into  the 
reflection  model.  Alternatively,  one  can  simulate  the  light  emitted 
by  each  object  as  if  it  was  self-luminous  or  fluorescent  (Fig.  14.22). 
The  simulated  fluorescence  process  (SFP)  has  been  refined  by  a 
number  of  workers  (e.g.,  van  der  Voort  et  al. ,  1989;  Brakenhoff 
et  al .,  1990;  Hallgren  and  Buchholz,  1992;  Messerli  et  al .,  1993). 
Excitation  and  emission  phases  are  implemented  as  before.  The 
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FIGURE  14.20.  “Realistic”  views  by  surface  shading,  gradient  voxel  and  other  lighting  effects.  (A)  shows  a  Gourard-shaded  geometric  object  surface  ren¬ 
dering  of  one  set  of  chromosomes  from  the  plant  Milium  (Bennet  et  al.,  1990),  using  the  Geometry  render  module  of  the  AVS  package.  (See  also  Levoy  1988; 
Rigaut  et  al,  1992,  for  Gourard-shaded  reconstructions  using  the  Visilog  software).  Since  the  segmented  geometric  polygons  are  very  numerous  but  small,  Mach 
band  effects  (see  text)  and  the  corruption  of  reflection  highlights  are  not  obvious,  though  the  views  have  a  certain  “plastic”  or  “synthetic”  look.  (B-J,  and  L)  are 
produced  by  the  Imaris  software.  (B,C)  show  the  same  confocal  data  as  (A)  visualized  using  a  simulated  fluorescence  process  (SFP)  voxel-based  algorithm. 
Directional  lighting  (and  resultant  shadows)  can  be  controlled  independently  from  the  ambient  lighting  (similar  to  a  fluorescence  emission  property)  to  achieve 
stronger  shadows  (C)  or  a  more  “luminous”  fluorescence  (B).  (D)  shows  the  use  of  segmented  objects  and  artificial  colors  to  highlight  individual  or  pairs  of  hair 
cells  in  this  reconstruction  of  confocal  data  (courtesy  of  Bio-Rad  Microscience)  from  a  mouse  inner  ear  preparation.  (E,  F)  show  voxel  gradient  lighting  applied 
from  the  left  and  right  respectively  to  a  top-down  reconstruction  of  the  same  data  as  (D).  Combining  object  segmentation,  surface  shading  and  a  Phong  lighting 
model  allows  a  material  property  defining  reflectance  to  be  attributed  to  structures.  (G)  shows  a  selected  portion  of  the  inner  ear  data  surface  rendered  with  a 
diffuse  lighting  model  only.  (H)  shows  a  specular  (highlight)  lighting  model  and  (J)  combines  the  specular  model  with  an  ambient  model  rather  like  the  SFP 
algorithms.  (J)  shows  a  two-channel  fluorescence  data  set  obtained  by  multi-photon  LSM  through  rat  intervertebral  disk  tissue  (in  consultation  with  Dr.  R.J. 
Errington,  Cardiff  University,  UK)  and  shows  CMFDA-stained  chondrocytes  and  autofluorescent  extracellular  matrix  (mostly  collagen)  (Metamorph  software). 
(K)  shows  the  same  data  by  a  combination  of  surface-shaded  object  rendering  (of  the  cells)  and  voxel-based  rendering  (of  the  ECM)  with  some  directional  light¬ 
ing  and  careful  application  of  transparency  to  render  the  tissue  transparent.  (L)  shows  the  same  data  visualized  by  the  SFP  algorithm,  exhibiting  the  character¬ 
istic  shadows  that  are  an  optional  feature  of  this  method. 
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emitted  light  propagates  to  the  viewer  in  a  direction  described  by 
the  vector  (v)  where 

v  =  (y„  vy,  1)  (40) 

The  emitted  light  wavefront  Lem  is  then  found  by: 

hemCty?  y'vi  tr  1)  Iem(xv,  y  v,  ti  1)3"  Lem(jcv,  yv,  n) 

1 1  -  aem)  I(xv,  yv,  n  -  1))  (41) 

where  Iem(xv,  yv,  ti)  Iex(.rv,  yv,  ti)  A  (xv,  y v -  ti). 

A  is  an  empirical  term  encompassing  the  quantum  efficiency 
and  emissivity  of  the  object.  (This  may  be  set  to  unity,  i.e.,  deter¬ 
mined  solely  by  aex.)  aem  is  the  opacity  (or  OT-LUT)  for  the  emis¬ 
sion  wavelength.  The  emission  computation  is  carried  out  in  the 
reverse  n  order  compared  to  excitation.  A  consequence  of  this 
absorption/emission  model  is  the  casting  of  shadows  by  the  two 
waves  as  they  pass  densely  absorbing  structures.  These  shadows 
may  fall  onto  the  background  or  onto  underlying  structures.  Coef¬ 
ficients  should  be  implemented  separately  for  each  channel  of  a 
multichannel  image.  Opacity  in  one  channel  can  then  be  used  to 
modulate  intensities  in  another,  as  in  the  Imaris  and  LCS  (Leica) 
SFP  mode.  AVS  uses  a  two-pass  transparency  operator  to  achieve 
a  similar  result. 


HOW  CAN  I  MAKE  MEASUREMENTS  USING 
THE  RECONSTRUCTED  VIEWS? 

Direct  measurements  from  image  data  can  sometimes  be  automated 
(Cohen  et  al.,  1994;  see  also  Chapter  15,  this  volume),  but  interac¬ 
tive  3D  analysis  often  requires  feedback  from  the  volume  recon¬ 
struction.  Views  are  used  in  several  ways  during  multi-dimensional 
measurements  (a  few  examples  are  shown  in  Fig.  14.23). 

(1)  One  or  more  sections  may  be  displayed  as  a  montage  and 
their  intersection  with  an  interactive  screen  cursor  used  to  pinpoint 
original  image  voxels  in  3D.  The  image  data  can  then  be  used  to 
obtain  a  3D  intensity  measurement.  This  gives  the  highest  spatial 
and  photometric  reliability  but  cannot  be  used  for  complex  struc¬ 
tures  that  are  not  discernible  from  a  few  intersecting  sections.  The 
exception  to  this  rule  involves  the  use  of  stereological  estimators 
to  probe  sections  randomly  oriented  within  the  volume  (e.g.,  see 
Analyze  volume  measuring  tool).  This  is  a  particularly  good 
method  of  estimating  complex  3D  measures  such  as  surface  area 
(e.g.,  Floward  and  Sandau,  1992)  or  orientation  (Mattfeldt  et  al., 
1994).  Other  grid-sampling  estimators  (e.g.,  Gundersen  et  al., 
1988)  have  been  implemented.  The  approach  works  best  when  an 
interactive  cursor  is  tracked  simultaneously  in  at  least  three  dif¬ 
ferent  orthogonal  cross-sections.  Through-focus  animations  in  all 
three  directions  are  also  useful  aids  in  identifying  structures 
passing  through  many  planes.  The  orthogonal  section  display  with 
cross-hair  cursor  is  a  standard  element  of  most  visualization  pro¬ 
grams.  Some  also  allow  oblique  (arbitrary)  sections  to  be  displayed 
for  measurements. 

(2)  A  reconstructed  view  may  be  displayed  to  identify  features 
to  be  measured  (with  animation,  etc.)  before  interrogating  the  orig¬ 
inal  image  data  for  the  precise  values  of  each  voxel.  The  problem 
remains  of  how  to  determine  exactly  which  image  data  voxels  are 
represented  by  a  particular  feature  in  the  reconstruction.  This  stems 
from  the  fact  that  the  third  spatial  (or  temporal,  etc.)  dimension 
has  been  significantly  reduced.  The  precise  method  of  coding  this 
reduced  dimensional  information  determines  how  accurately  orig¬ 
inal  voxels  may  be  retrieved.  The  method  is  most  successful  where 
high  contrast  objects  have  been  efficiently  segmented.  An  elegant 


solution  is  to  embed  a  voxel  section  into  a  3D  rendered  view  so 
as  to  show  both  the  reconstruction  and  the  original  data  (see  Amira, 
Analyze,  Voxblast,  Imaris,  Volvis,  and  others  for  examples). 

(3)  A  third  method  involves  the  direct  measurement  of  recon¬ 
structed  views.  This  is  possible  by  improving  the  coding  method 
used  in  (2).  Parallax  shifts  are  used  effectively  to  measure  depth 
in  stereo  views  (Fig.  14.24).  In  practice,  two  cursors  are  moved  in 
tandem  through  the  left  and  right  pairs  of  the  stereoscopic  space. 
Tandem  movements  of  the  two  cursors  sweep  out  xw,  yv- 
coordinates.  Zv-distances  are  swept  out  by  altering  the  separation 
of  the  two  cursor  components.  An  observer  using  the  appropriate 
stereoscopic  viewing  aid  will  perceive  the  cursor  to  track  in  and 
out  of  the  screen.  This  method  is  particularly  useful  for  very  trans¬ 
parent  structures.  Opaque  objects  are  measured  more  easily  by 
interrogating  a  z-coordinate  or  surface  view.  The  z-value  at  each 
(xv,  yv)  pixel  allows  full  3D  measurements  to  be  made  of  the 
object’s  surface.  Reconstructions  from  segmented  objects  defined 
by  surfaces  or  z-coordinates  almost  always  proceed  via  the  gener¬ 
ation  of  a  z-buffer  intermediate.  This  can  be  kept  in  memory  while 
an  intensity  reconstruction  is  built  up  on  the  monitor.  Visible  fea¬ 
tures  in  the  intensity  view  (which  may  be  artificially  lit,  etc.),  can 
return  their  z-position  by  interrogating  the  z-buffer  (this  technique 
is  used  efficiently  by  voxel  Tenderers  such  as  VoxBlast).  Ten-  to 
16-bits  of  z-buffer  are  desirable.  Line  transects  through  the  z-buffer 
return  profile  plots  through  a  structure.  Area  measurements  in  the 
z-buffer  return  integrated  heights  that  are  equivalent  to  the  volume 
under  the  surface.  All  these  measures  may  be  made  alongside 
equivalent  xv,  vv  and  intensity  plots  (Fig.  14.23). 

Geometric  objects  objectively  segmented  from  the  image 
volume  for  surface  visualization  can  also  report  their  total  surface 
area  (and  also  their  included  volume,  e.g.,  Guilak,  1993).  Because 
they  are  defined  by  polygons,  many  statistics  of  shape,  asymme¬ 
try,  etc.,  can  be  automatically  recorded.  Particle  analysis  software 
is  now  implemented  in  3D  (and  4D)  to  count  segmented  objects 
(most  object-based  systems)  and  even  track  them  over  time  (e.g., 
Imaris,  Volocity).  A  more  extensive  discussion  of  computerized 
measurements  and  multi-dimensional  data  can  be  found  in  the  fol¬ 
lowing  chapter. 

CONCLUSION 

Visualization  is  not  a  precise  science,  but  by  understanding  the 
functions  of  your  display  program,  the  user  can  derive  useful 
objective  information  from  views  of  multi-dimensional  images. 
Image  data  collected  from  biological  microscopes  is  necessarily 
complex  and  noisy,  and  it  contains  ill-defined  structures  with 
largely  unknown  photometric  and  geometric  statistics.  Step-by- 
step  visualization  algorithms  of  increasing  sophistication  must  be 
applied  in  a  controlled  manner,  with  adequate  parameter  tracking 
and  validation  in  order  to  have  confidence  that  the  final  results 
portray  real  features.  Quantitative  measurement  from  multi¬ 
dimensional  views  adds  additional  constraints  to  the  artificial  prop¬ 
erties  that  can  be  added  to  rendered  views.  Simple  algorithms 
produce  the  fastest  visualizations.  Object-based  reconstruction, 
though  supported  by  a  vast  range  of  affordable,  yet  high  specifi¬ 
cation  graphics  hardware,  is  critically  dependent  on  the  segmen¬ 
tation  used  to  extract  the  vertex  geometries  from  original  3D 
voxel  data.  Test  samples  and  control  data  should  always  be 
processed  in  order  to  understand  the  significance  of  complex  ren¬ 
dered  views.  Seeing  should  never  be  used  as  the  sole  criterion  for 
believing. 
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FIGURE  14.23.  3D  Measurements  on  multiple  sections  and  views.  (A)  shows  the  Micro  Voxel  “Caliper  Tool”  being  used  to  make  a  distance  measurement 
on  a  “First  mode”  rendered  view  of  a  pollen  grain.  (B)  is  a  representative  example  of  intensity  statistics  and  3D  volume  from  a  Micro  Voxel  volume  measure¬ 
ment.  (C-F)  show  Thru  View  PLUS  views  of  living  articular  cartilage  chondrocytes  (data  supplied  by  R.J.  Errington,  Physiology  Dept.,  Oxford  University)  used 
for  3D  measurements.  (C)  is  a  single  time  point  from  a  4D  series  of  “height  coded”  views.  Profiles  and  volumes  of  individual  chondrocytes  (here  labeled  with 
CMFDA)  are  derived  directly  from  each  reconstructed  view.  (D-F)  show  oblique  sections  through  another  data  set  with  automatic  serial  “area”  measurements 
being  taken  (using  the  MPL  “area”  verb  in  a  macro  program)  of  each  segmented  slice  through  a  single  cell.  The  corrected  z-step  applied  to  the  integrated  area 
sum  gives  a  direct  volume  estimate.  (G)  shows  the  neurone  of  Figures  14.2  and  14.3,  rendered  using  the  MPL  “maximum  height”  mode.  MPL  height  modes 
automatically  record  both  the  z-coordinate  (z-buffer)  and  the  corresponding  intensity  view.  Here  the  maximum  intensity  is  shown  in  the  inset  gray-scale  view 
with  a  cursor  track  along  a  prominent  process.  The  corresponding  intensity  trace  is  shown  in  the  upper  plot  and  the  z-depth  in  the  lower  trace. 
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FIGURE  1 4.24.  3D  Measurements  using  a  stereoscopic  cursor.  Stereo  pair  of  the  Lucifer  Yellow  stained  neuron  also  shown  in  Figures  14.2  and  14.3  showing 
the  use  of  a  pair  of  software-generated  cursors  with  variable  parallax  shift  between  images  to  mark  out  x,  y  and  z  positions  in  the  transparent  reconstruction. 
These  images  were  generated  using  MPL  pixel-shifted  maximum-intensity  projections  and  the  macro-programming  language  to  generate  the  stereo  cursors.  On 
a  fast  ’486  computer,  these  can  be  moved  in  and  out  along  the  ^-direction  interactively. 
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INTRODUCTION 

Image  analysis  is  the  process  of  making  quantitative  structural 
and  functional  measurements  from  an  image.  With  the  widespread 
availability  of  three-dimensional  (3D)  microscopy,  coupled  with  a 
growing  trend  towards  quantitative  studies,  there  is  an  increasing 
need  for  3D  image  analysis.  The  goal  of  this  chapter  is  to  describe 
3D  image  analysis  techniques,  with  an  emphasis  on  highly  auto¬ 
mated  methods. 

Figure  15.1  shows  two  examples  of  3D  image  analysis.  In  the 
first  (top  row),  confocal  microscopy  is  used  to  image  cell  nuclei 
from  the  rat  hippocampus  using  a  DNA  stain,  and  the  transcription 
products  (mRNA)  of  an  immediate  early  gene  (Arc)  are  labeled  by 
fluorescence  in  situ  hybridization  (FISH)  (Guzowski  et  al.,  1999; 
Guzowski  and  Worley,  2001 ;  Vazdarjanova  et  al.,  2002).  The  table 
on  the  right  is  a  partial  display  of  the  results  of  the  3D  image  analy¬ 
sis.  It  consists  of  a  table  listing  all  the  nuclei,  their  morphometric 
features,  and  the  amount  of  FISH  signal  associated  with  each 
nucleus.  The  second  example  (lower  row)  illustrates  3D  analysis 
of  a  dye-injected  neuron  imaged  by  confocal  microscopy  (So  et 
al.,  2000;  Brown  et  al.,  2001;  Al-Kofahi  et  al.,  2002;  Jain  et  al., 
2002).  Various  windows  in  this  screen  view  are  from  a  software 
system  (Neuroexplorer,  Microbrightfield,  Inc.)  that  indicates 
automatically  generated  traces,  branch  points,  and  morphological 
measurements. 

The  motivation  for  3D  image  analysis  is  to  achieve  correct 
morphometry  of  biological  structures  without  the  errors  associated 
with  projecting  onto  a  two-dimensional  (2D)  plane.  For  example, 
it  is  difficult  to  distinguish  overlapping  objects  from  2D  projec¬ 
tions,  whereas  3D  imaging  provides  richer  segmentation  cues. 
Many  structures,  such  as  neurons  and  vasculature,  are  often  thicker 
than  the  depth-of-field  of  the  microscope,  necessitating  3D 
imaging  and  analysis.  Often,  non-planar  regions  are  of  interest.  For 
such  cases,  optical  sectioning  via  confocal  microscopy  better  pre¬ 
serves  3D  spatial  relationships  compared  to  physical  sectioning, 
and  can  therefore  lead  to  more  accurate  morphometry,  and 
correct  topological  analysis.  Finally,  3D  imaging  can  often  result 
in  faster  and  more  convenient  sampling  of  tissue  for  large-scale 
studies. 


A  particular  emphasis  of  this  chapter  is  automated  3D  image 
analysis.  This  term  is  understood  to  imply  that  the  amount  of 
manual  image  analysis  is  minimal,  typically  in  the  0%  to  10% 
range.  These  methods  are  distinct  from  manual  methods,  whether 
or  not  computer  assistance  is  involved.  In  other  words,  computer- 
assisted  methods  are  not  considered  to  be  automated.  For  example, 
sophisticated  software/hardware  packages  are  available  for  tracing 
neuronal  processes  using  a  computer  pointing  device  such  as  a 
tablet  or  mouse  (Capowski,  1989;  Glaser  and  Glaser,  1990).  Often, 
these  packages  are  enhanced  with  visualization  tools  such  as  stereo 
projections  and  devices  such  as  the  camera  lucida  (Glaser  and 
Glaser,  1990;  Marko  and  Leith,  1992).  These  are  examples  of 
computer-assisted,  but  not  automated,  image  analysis.  Computer- 
assisted  methods  are  very  appropriate  when  a  small  number  of 
images  need  to  be  analyzed,  or  if  the  complexity  of  the  scene  is 
too  high  for  successful  automated  analysis.  On  the  other  hand, 
there  are  software  packages  available  that  accept  the  3D  digital 
image  file  as  input,  and  generate  as  output  another  computer  file 
representing  a  tracing  of  the  neuron  (Al-Kofahi  et  al.,  2002).  This 
output  may  require  modest  amounts  of  oversight  and/or  editing. 
This  latter  type  of  software  is  appropriately  termed  automated  and 
is  needed  when  a  large  number  of  images  must  be  analyzed. 

Stereology  is  an  important  class  of  computer-assisted  image 
analysis  methods  (Howard  and  Reed,  1998;  Russ  and  DeHoff, 
2000;  Mouiton,  2002).  In  these  methods,  the  manual  observer  per¬ 
forms  a  systematic  subsampling  of  the  image  data  using  statisti¬ 
cally  motivated  rules  and  assumptions.  For  example,  it  is  assumed 
that  the  tissue  of  interest  has  certain  uniformity  properties.  In  con¬ 
trast,  the  methods  of  interest  in  this  chapter  are  designed  to  analyze 
each  and  every  object  in  a  given  image,  without  assumptions. 

Although  manual  image  analysis  methods  are,  in  principle,  far 
more  powerful  than  automated  methods,  due  to  the  inherent  supe¬ 
riority  of  the  human  visual  system  over  any  algorithm,  there  are 
many  reasons  for  seeking  highly  automated  methods.  While  the 
human  visual  observer  excels  at  pattern  recognition/classification 
tasks,  s/he  is  often  poor  at  detailed  scoring  tasks.  It  is  common  for 
the  human  observer  to  miss  objects,  score  objects  twice,  and 
produce  unsteady  traces  (Jaeger,  2000;  Al-Kofahi  et  al.,  2002). 
Indeed,  it  is  common  for  the  same  observer  to  produce  different 
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FIGURE  1 5.1 .  Two  illustrations  of  3D  image 
analysis.  In  the  first  example  (top  row),  con- 
focal  microscopy  is  used  to  image  cell  nuclei 
from  the  rat  hippocampus  using  a  DNA  stain, 
and  fluorescence  in  situ  hybridization  (FISH) 
is  used  to  image  the  transcription  products 
(rnRNA)  of  an  immediate  early  gene  (Arc). 
The  cell  nuclei  are  segmented,  and  FISH 
activity  is  measured  relative  to  the  segmenta¬ 
tion,  generating  a  wealth  of  measurements. 
The  table  on  the  right  is  a  partial  display  of 
the  results  of  3D  image  analysis.  In  the  second 
example  (lower  row),  a  dye-injected  neuron  is 
traced  automatically,  generating  a  wealth  of 
topological  and  morphological  measurements 
displayed  in  tables  and  dendrograms  (image 
courtesy:  Microbrightfield,  Inc.,  USA). 
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markups  for  the  same  image  at  different  times.  Computer  automa¬ 
tion  makes  sense  when  a  large  number  of  images  must  be  ana¬ 
lyzed.  It  can  eliminate  the  tedium  and  labor  associated  with  manual 
methods.  It  is  ruthlessly  consistent  compared  to  manual  methods, 
eliminating  the  subjectivity  inherent  to  manual  analysis.  Faster 
computers  make  automated  methods  faster,  and  they  can  work 
round  the  clock. 

Automated  methods  are  free  from  many  limitations  of  manual 
analysis.  For  example,  manual  methods  are  limited  in  their  ability 


to  process  3D  data  because  the  manual  observer  is  limited  to  visual 
observation  of  stereo  pairs,  which  are  merely  a  projection  of  the 
3D  reality  of  interest.  Occasionally,  some  observers  have  visual 
handicaps,  and  most  observers  are  susceptible  to  optical  illusions. 
Finally,  if  a  study  needs  to  be  redone,  it  is  much  easier  to  run  mod¬ 
ified  software  rather  than  ask  manual  scorers  to  repeat  a  tedious 
task.  Overall,  manual  methods  are  limited  to  small  numbers  of 
images,  are  unavoidably  subjective,  and  are  limited  in  the  handling 
of  three-dimensionality. 
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FIGURE  15.2.  Illustrating  the  role  of  image  analysis  systems  in  hypothesis  testing  studies.  Test  and  normal  (control)  tissue  are  imaged  to  generate  batches  of 
3D  images.  Image  analysis  systems  process  this  volumetric  data  to  generate  concise  tables  of  measurements  that  are  subjected  to  statistical  analysis. 


TYPES  OF  AUTOMATED  IMAGE 
ANALYSIS  STUDIES 

Figure  15.2  illustrates  the  most  common  type  of  image 
analysis-based  hypothesis  testing  study  or  assay.  In  this  type  of 
study,  tissue  samples  are  subjected  to  a  condition  of  interest 
(e.g.,  exposed  to  a  biochemical).  As  a  reference,  normal  (untreated) 
tissue  samples  are  also  obtained,  maintaining  other  conditions 
the  same.  Confocal  microscopy  is  performed  to  generate  batches 
of  3D  images.  Image  analysis  on  these  batches  results  in 
numerical  tables  representing  cytometric  and/or  histometric 
data.  Statistical  hypothesis  testing  can  be  performed  on  these 
data.  We  are  interested  in  discovery  and  quantification  of  all  sta¬ 
tistically  significant  differences  between  the  treated  and  untreated 
samples.  The  result  of  hypothesis  testing  and  an  understanding 
of  the  nature  of  changes  is  a  powerful  source  of  biological 
insight. 


Figure  15.3  illustrates  a  type  of  study  that  is  now  feasible, 
thanks  to  advances  in  methods  and  apparatus  for  live  tissue 
imaging  (in  vitro  or  in  vivo),  as  exemplified  by  the  data  in  Figure 
15.4.  In  this  type  of  study,  one  collects  a  temporal  series  of  images 
from  the  same  region.  In  addition  to  statistical  analysis  of  mor¬ 
phometric  data  as  a  function  of  time,  it  is  possible  to  register  the 
3D  time-series  images,  and  identify  the  changes  in  a  direct  manner. 

The  types  of  changes  that  may  be  caused  by  the  tested  condi¬ 
tions  can  be  diverse,  and  can  occur  at  multiple  levels.  At  the  lowest 
level,  molecular  and  subcellular  effects  can  be  detected  by  fluo¬ 
rescent  tagging  (Valeur,  2002).  At  the  cellular  level,  changes  can 
take  the  form  of  increasing/decreasing  cell  counts,  and  changes  in 
morphometric  features  of  cells.  Finally,  subtle  high-level  and/or 
long-range  changes  in  tissue  architecture  can  be  analyzed  by  mon- 
taging  the  results  of  image  analysis  from  a  large  number  of  over¬ 
lapping  windows  across  large  specimens  (Becker  et  al.,  1996).  The 
diversity  and  complexity  of  these  changes  define  the  breadth  and 
scope  of  image  analysis  problems. 
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FIGURE  15.3.  Illustrating  the  methodology  for  hypothesis  testing  with  time-series  data.  In  this  case,  successive  images  are  registered  (aligned)  and  compared. 
The  registration  algorithms  usually,  but  not  always,  operate  on  the  output  of  the  image  analysis  results.  Registered  images  (or  summarized  image  analysis  results) 
are  compared  to  generate  tables  of  change  measurements  that  are  analyzed  statistically. 
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FIGURE  15.4.  Time  series  in  vivo 
images  (each  768  x  512  X  32,  8  bit/pixel) 
of  fluorescently  labeled  skin  vasculature 
altered  by  growth  of  a  nearby  tumor, 
shown  in  their  maximum  intensity  pro¬ 
jections  (x-y,  y-z,  and  x-z).  Automated 
segmentation  (tracing)  results  are  over¬ 
laid  in  green.  Volumes  highlighted  in  blue 
are  common  overlapping  regions  across 
all  images  in  this  set  of  time  series 
images,  where  measurements  only  from 
these  common  regions  are  used  to 
analyze  the  changes  in  vasculature 
between  time  points,  (image  courtesy: 
Microvascular  Research.) 
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COMMON  TYPES  OF  BIOLOGICAL 
IMAGE  OBJECTS 

Notwithstanding  the  diversity  of  changes  that  may  be  of  interest, 
it  is  advantageous  to  identify  and  study  certain  generic  types  of 
image  objects,  and  common  image  analysis  tasks  associated  with 
these  object  types.  Practically  speaking,  an  object  is  really  a  col¬ 
lection  of  voxels  representing  a  biological  structure  imaged  by  the 
microscope.  Three  of  the  most  common  object  types  in  the  context 
of  confocal  microscopy  are  listed  below. 

1.  Blob-like  objects:  The  labeled  cell  nuclei  in  Figure  15.1(A) 
are  examples  of  3D  blobs,  and  blobs  are  by  far  the  most  common 
type  of  object  of  interest  in  automated  image  analysis.  Geometri¬ 
cally,  they  can  be  thought  of  as  3D  ellipsoids  that  have  been 
deformed  irregularly.  These  apparently  simple  objects  nevertheless 
provide  a  rich  set  of  challenges  from  the  standpoint  of  automated 
image  analysis,  due  to  preparation  and  imaging  artifacts  (Lin  et  al., 
2003). 

2.  Tube-like  objects:  Vasculature  and  neurons  [Fig.  15.1(C)] 
are  examples  of  tube-like  objects.  Geometrically,  they  can  be 
thought  of  as  non-uniformly  deformed  3D  cylinders  (Al-Kofahi  et 
al.,  2002;  Abdul-Karim  et  al.,  2003).  Like  blobs,  these  objects  too 
provide  a  rich  set  of  challenges  for  automated  image  analysis 
systems.  Tube-like  objects  can  be  filled  (i.e.,  solid)  (Al-Kofahi  et 
al.,  2002;  Abdul-Karim  et  al.,  2003;  Fie  et  al.,  2003)  or  hollow 
(Weichert  et  al.,  2003)  in  appearance. 

3.  Irregular  cloud-like  distributions:  The  diffuse  FISH 
signals  in  Figure  15.1(A)  are  a  good  example  of  this  class  of  object 
(Lin  et  al.,  2003).  The  diversity  and  specificity  of  fluorophores, 
coupled  with  the  ability  of  modern  microscopes  to  image  multiple 
fluorophores,  makes  this  an  important  category  of  image  analysis 
targets.  Although  these  objects  do  not  have  distinctive  geometries, 
they  are  amenable  to  quantitation  by  spatial  association  with  bio¬ 
logically  related  blob-like  or  tube-like  objects.  For  the  example  in 
Figure  15.1(A),  segmentation  of  the  nuclei  provides  spatial  masks 
over  which  the  intra-nuclear  FISH  signals  are  integrated.  The 


extra-nuclear  FISH  signals  are  associated  with  the  nearest  nucleus 
by  means  of  a  Euclidean  distance  transformation  (Lin  et  al.,  2003). 


SPECIMEN  PREPARATION  AND  IMAGE 
PREPROCESSING  METHODS 

Data  Collection  Guidelines  for  Image 
Analysis  Purposes 

Specimen  preparation  and  microscopy  procedures  for  successful 
automated  image  analysis  are  stricter  than  for  manual  scoring. 
Unlike  humans,  computers  are  easily  misled  by  confounding 
objects,  artifacts,  variability,  and  clutter.  Therefore,  it  is  important 
to  make  every  effort  during  the  specimen  preparation  and  imaging 
procedures  to  ensure  that  the  objects  of  interest  are  delineated  with 
a  high  degree  of  contrast  against  the  uninteresting  structures  in  the 
tissue.  It  is  helpful  to  experiment  with  specimen  handling  steps, 
reagents,  fluorophores,  and  software  tools  such  as  those  for  spec¬ 
tral  unmixing.  Figure  15.5  illustrates  structure-specific  imaging 
using  multiple  fluorophores.  The  different  highlighted  structures 
(astrocytes,  microglia,  and  vasculature)  appear  in  separate  image 
channels,  eliminating  the  need  for  separating  structures  by  image 
analysis.  It  is  wise  to  make  sure  that  the  automated  scoring  soft¬ 
ware  is  well  behaved  when  used  on  a  small  pilot  set  of  images 
before  embarking  on  large-scale  data  collection.  It  is  important 
to  maintain  a  high  degree  of  uniformity  in  specimen  handling, 
reagents,  imaging  protocols,  and  instrument  settings  across  a  batch. 

Sampling  conditions  may  be  different  for  image  analysis.  For 
example,  lossy  image  compression  algorithms  are  best  avoided. 
While  image  analysis  algorithms  are  driven  by  the  desire  to  max¬ 
imize  the  accuracy  of  morphometry,  the  percentage  impact  of 
a  single-voxel  error  depends  heavily  on  the  number  of  voxels 
representing  the  object.  If  the  tissue  is  fairly  uniform,  it  is  often 
possible  to  sample  across  the  tissue  in  a  random  manner,  using 
standard  stereological  sampling  principles  (Howard  and  Reed, 


320  Chapter  15  •  B.  Roysam  et  al. 


FIGURE  15.5.  Neurobiology  example  illustrating  the  use  of  multiple  fluorophores  to  simplify  image  analysis  by  performing  selective  imaging  of  specific  struc¬ 
tures  in  the  same  region  of  tissue.  Maximum  projections  of  rat  cortices  altered  by  the  implantation  of  a  model  neuroprosthesis.  In  panel  (A),  the  green  regions 
represent  astrocytes,  the  blue  regions  represent  microglia.  In  panel  (B),  the  red  regions  additionally  represent  vasculature.  These  different  structures  are  avail¬ 
able  to  the  image  analysis  system  as  separate  images.  The  images  when  compared  to  controls  (not  shown)  demonstrate  increased  numbers  of  astrocytes,  microglia 
and  proliferator  of  vessels,  (image  courtesy:  Dr.  Beth  Sipple  at  Wadsworth.) 


1998;  Russ  and  DeHoff,  2000;  Mouton,  2002).  If  this  assumption 
is  not  valid,  it  makes  sense  to  resort  to  montaging.  The  border 
regions  in  3D  images  and  optical  slices  near  the  top  and  bottom  of 
a  stack  often  capture  partial  objects.  These  objects  should  be  elim¬ 
inated  in  a  systematic  manner  using  Howard’s  brick  rule  (Howard 
et  al.,  1985),  which  is  easily  implemented  on  the  spreadsheet  tables 
of  measurements  generated  by  the  software. 

If  live  tissue  is  being  imaged  in  a  time-lapse  series,  it  is  impor¬ 
tant  to  ensure  that  the  same  region  of  tissue  is  imaged  every  time 
(Brown  et  al.,  2001).  Additionally,  automatic  image  registration 
tools  can  be  used  to  refine  the  alignment  (Al-Kofahi  et  al.,  2003). 
Once  registered,  the  changes  can  be  quantified  over  the  overlap¬ 
ping  regions  (Addul-Karim  et  al.,  2003).  Figure  15.4  shows  a  time 
series  of  in  vivo  images  (each  768  x  512  X  32,  8  bit/pixel)  of 
fluorescently  labeled  vasculature  of  skin  altered  by  growth  of  a 
nearby  tumor. 

Confocal  microscopes  are  unavoidably  anisotropic  in  terms  of 
the  spatial  sampling  density  because  the  axial  resolution  of  confo¬ 
cal  microscopes  is  lower  than  the  lateral  resolution.  Image  resam¬ 
pling  by  techniques  such  as  shape-based  interpolation  often 
simplifies  3D  image  analysis  at  the  expense  of  generating  a  larger 
image  data  set  (Raya  and  Udupa,  1990). 

Finally,  the  image  analyst  must  be  aware  that  the  brightness  of 
a  voxel  in  a  confocal  image  is  not  necessarily  only  proportional  to 
the  fluorophore  concentration  (Pawley,  2000). 

Image  Preprocessing  Methods 

Common  imaging  artifacts  include  non-uniformity  and  the  pres¬ 
ence  of  uninteresting/nuisance  objects.  Their  effects  on  the  image 
analysis  results  can  sometimes  be  reduced  by  preprocessing. 
Figure  15.6  shows  several  imaging  artifacts  in  neuron  and  vascu¬ 
lature  images. 

Morphological  filters  (Serra  and  Soille,  1994;  Bovik  et  al., 
2001),  background  subtraction  (Russ,  1994),  and  signal 
attenuation-correction  (Adiga  and  Chaudhuri,  2001b;  Can  et  al., 
2003)  are  examples  of  common  preprocessing  methods  for  confo¬ 
cal  images.  Morphological  filters,  such  as  the  median  and  top-hat 


filter,  are  typically  utilized  to  reduce  image  noise,  such  as  blobs  of 
dye  in  the  background,  or  to  smooth  out  Poisson  noise  (Sard  et  al., 
2000)  and  non-uniformities  of  foreground  intensity.  However,  the 
optimal  method  for  removing  Poisson  noise  is  full  3D  deconvolu¬ 
tion  (see  Chapter  25,  this  volume).  Background  subtraction  is 
performed  to  correct  illumination  non-uniformity  across  the  entire 
image.  Signal  attenuation-correction  is  useful  to  rectify  depth- 


FIGURE  15.6.  Examples  of  imaging  artifacts  in  the  context  of  neuron 
imaging.  (A)  Blobs  of  dye.  (B)  Neurites  with  relatively  dim  and  discontinuous 
appearance.  (C)  Strip  of  noise,  resembling  neurites.  (image  courtesy:  Natalie 
Dowell  at  the  Wadsworth  Center.) 
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dependent  attenuation  of  signals  (Can  et  al.,  2003).  In  general, 
images  are  preprocessed  globally,  that  is,  without  localizing  the  pre¬ 
processing  operations  to  a  subset  of  the  image.  This  makes  prepro¬ 
cessing  computationally  expensive.  Furthermore,  the  preprocessor 
parameters  usually  need  to  be  fine-tuned  to  ensure  preservation  of 
objects  of  interest  while  reducing  unwanted  artifacts. 

Another  rationale  to  preprocessing  is  to  generate  standardized 
images  where  one  segmentation  method  is  used  for  images  coming 
from  different  sources.  Yet  another  is  to  highlight  particular  struc¬ 
tures,  for  example,  images  of  neuronal  processes  and  vasculature 
may  be  preprocessed  to  highlight  the  tube-like  morphology  of 
these  biological  objects  (Sato  et  al.,  1998). 


GENERAL  SEGMENTATION  METHODS 
APPLICABLE  TO  CONFOCAL  DATA 

Segmentation  is  the  process  of  labeling  some  or  all  of  the  voxels 
in  the  image  as  being  part  of  specific  objects  of  interest  (the  fore¬ 
ground)  and  not  part  of  the  background.  This  is  the  process  of 
transitioning  from  a  conceptually  simple  array  of  point  measure¬ 
ments  (voxels)  to  the  more  abstract  and  more  informative  notion 
of  image  objects,  or  parts  thereof  (Haralick  and  Shapiro,  1985). 
Following  segmentation,  it  is  straightforward  to  make  morpholog¬ 
ical  and  functional  measurements.  The  purpose  of  this  section  is 
to  summarize  the  main  segmentation  approaches.  The  reader  inter¬ 
ested  in  more  detailed  descriptions  is  referred  to  other  sources 
(Bhanu,  1982;  Wilson  and  Spann,  1988;  Bhanu  and  Lee,  1994; 
More  and  Solimini,  1995;  Rezaee,  1998;  Suri  et  al.,  2002). 

The  difficulty  of  segmenting  biological  images  is  largely  attrib¬ 
utable  to  their  sheer  complexity  and  variability.  Interestingly,  it  is 
simpler  to  segment  3D  images,  compared  to  2D  images  because, 
in  the  former,  some  overlapping  objects  can  be  distinguished  on 
the  basis  of  3D  depth  information  (Ancin  et  al.,  1996). 

The  prior  literature  on  segmentation  is  vast  (Zucker,  1977; 
Kanade,  1978;  Sklansky,  1978;  Rosenfeld  and  Davis,  1979; 
Ranade  and  Prewitt,  1980;  Fu  and  Mui,  1981;  di  Zenzo,  1983;  Har¬ 
alick  and  Shapiro,  1985;  Mitiche  and  Aggarwal,  1985;  Nevada, 
1986;  Cooper,  1998;  Kerfoot  and  Bresler,  1 999;  Pham  el  al..  2000). 
Generally,  techniques  for  image  segmentation  can  be  grouped  into 
three  categories:  bottom-up  methods,  top-down  methods,  and 
hybrid  methods.  Regions  and  boundaries  are  two  common  con¬ 
cepts  commonly  used  in  segmentation.  Intuitively,  an  object  is  out¬ 
lined  by  its  boundary  and  a  region  is  contained  by  a  boundary.  In 
addition,  segmentation  may  be  performed  either  image  wide,  or  in 
an  exploratory  manner.  Non-exploratory  methods  visit  each  voxel 
and  label  them.  Exploratory  methods  (Can  et  al.,  1999;  Al-Kofahi 
et  al.,  2002;  Streekstra  and  van  Pelt,  2002;  Abdul-Karim  et  al., 
2003)  begin  with  voxels  known  or  estimated  to  be  on  the  object, 
such  as  a  point  on  a  vessel,  and  proceed  to  segment  adjacent  voxels 
of  the  same  object. 

Bottom-Up  Segmentation  Methods 

Bottom-up  methods  start  with  an  over-segmentation  of  the  image 
and  iteratively  merge  object  fragments  based  on  some  measure  of 
similarity.  They  can  be  described  in  terms  of  three  steps:  identify¬ 
ing  the  set  of  initial  regions  representing  object  fragments  (Zucker, 
1976;  Besl  and  Jain,  1988;  Adams  and  Bischof,  1994;  Trucco  and 
Fisher,  1995;  Zhu  and  Yuille,  1996),  identifying  an  appropriate 
region-similarity  measure  (Blake  and  Zisserman,  1987;  Black  and 
Rangarajan,  1996),  and  merging  the  voxels  that  share  these  fea¬ 


tures  into  objects  (Guigues  et  al.,  2003).  Other  related  methods  are 
graph  based,  for  example,  the  work  of  Shi  and  colleagues  (Beck 
et  al.,  2000)  has  presented  a  global  criterion  termed  the  “normal¬ 
ized  cut”  for  segmenting  the  graph.  Ying  and  Uberbacher  (1997) 
proposed  a  2D  image  segmentation  using  a  minimum  spanning  tree 
partitioning  approach.  In  the  bottom-up  category,  there  are  four 
main  approaches:  intensity  thresholding,  region  based,  boundary 
based,  and  integrated  techniques.  These  are  detailed  in  the  fol¬ 
lowing  paragraphs. 

Intensity  Threshold-Based  Segmentation 
Methods 

In  this  approach  (Sahoo  et  al.,  1988),  all  voxels  whose  values  lie 
within  a  certain  range  belong  to  one  class.  Global  thresholding 
(Shapiro  and  Stockman,  2001;  Koh  et  al.,  2002)  uses  a  single 
threshold  value  for  an  entire  image.  When  the  threshold  value 
changes  for  different  regions  of  the  image,  it  is  called  adaptive 
thresholding  (Cohen  et  al.,  1994;  Kim  et  al.,  2001;  He  et  al.,  2003). 
Thresholding  may  be  sufficient  for  segmentation  when  the  back¬ 
ground  intensity  range  does  not  overlap  with  the  foreground  inten¬ 
sity  range.  To  overcome  the  limitations  of  thresholding,  such  as 
susceptibility  to  background  noise  and  intensity  non-uniformity, 
connected  components  analysis  (Dillencourt  et  al.,  1992;  Ancin  et 
al.,  1996)  is  commonly  used  to  link  voxels  belonging  to  an  object 
of  interest,  and  reject  overly  small  clusters  of  pixels. 

Region-Based  Segmentation  Methods 

Region-based  methods  (Zucker,  1996;  Hojjatoleslani  and  Kittler, 
1998)  rely  on  the  assumption  that  neighboring  voxels  within  one 
region  of  an  object  are  homogeneous  according  to  a  specified 
measure.  This  leads  to  a  class  of  algorithms  known  as  split-and- 
merge  (Horowitz  and  Pavlidis,  1974;  Haralick  and  Shapiro,  1985; 
Chang  and  Li,  1994),  seeded  region  growing  (Adams  and  Bischof, 
1994;  Hojjatoleslami  and  Kittler,  1998),  and  partitional  clustering 
(Mackin  et  al.,  1993;  Roysam  et  al.,  1994).  The  general  procedure 
is  to  compare  each  voxel  to  its  neighbors.  If  the  criterion  of  homo¬ 
geneity  is  satisfied,  the  voxel  is  said  to  belong  to  the  same  class. 
Neighboring  voxels  are  examined  and  added  to  the  region  if  they 
are  evaluated  to  have  similar  characteristics.  This  region  growing 
process  will  eventually  converge,  yielding  a  delineation  of  the 
object.  The  growing  volume  should  be  constrained  for  maximum 
success.  Multiple  regions  can  be  initialized  in  the  image,  and 
regions  can  be  merged  with  each  other.  Obviously,  the  perfor¬ 
mance  of  this  approach  largely  depends  on  the  selected  homo¬ 
geneity  criterion. 

In  the  split- and-merge  technique,  the  entire  image  is  initially 
considered  as  one  region.  In  each  step,  each  heterogeneous  region 
of  the  image  is  divided  into  four  rectangular  segments  and  the 
process  is  terminated  when  all  regions  are  homogeneous.  After 
the  splitting,  a  merging  process  unifies  the  similar  neighboring 
regions.  Quadtree-structured  split-and-merge  (Pavlidis,  1977)  is  a 
popular  approach  because  of  its  simplicity  and  computational  effi¬ 
ciency.  Seeded  region  growing  techniques  (Adams  and  Bischof, 
1994;  Hojjatoleslami  and  Kittler,  1998)  select  some  initial  image 
points,  and  then  segments  the  image  into  regions,  each  originating 
with  a  single  seed.  Clustering-based  methods  (Mackin  et  al.,  1993; 
Roysam  et  al.,  1993,  1994)  treat  the  segmentation  problem  as  a 
partitional  cluster  analysis  (Theodoridis  and  Koutroumbas,  1999) 
in  a  four-dimensional  space  [(x,  y,  z),  Kx,  y,  z)],  combining  the 
usual  spatial  coordinates  with  the  image  intensity.  This  approach 
is  most  valuable  when  the  available  geometric  information  about 
the  objects  is  weak  (Shain  et  al.,  1999). 
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Boundary-Based  Segmentation  Methods 

Boundary-based  methods  (Davis,  1975)  detect  image  edges 
(Ballard  and  Brown,  1982),  and  link  them  into  contours/surfaces 
representing  object  boundaries.  The  output  of  most  existing  edge 
detectors  can  only  provide  candidates  for  the  region  boundaries 
because  the  edges  obtained  are  normally  discontinuous  or  over¬ 
detected.  However,  the  actual  region  boundaries  should  be  closed 
curves.  Therefore,  some  postprocessing,  such  as  edge  tracking,  gap 
filling,  smoothing,  and  thinning  should  be  performed  to  obtain  the 
closed  region  boundaries.  Such  postprocessing  is  often  problem¬ 
atic,  and  this  is  an  inherent  limitation  of  boundary-based  methods 
(Shapiro  and  Stockman,  2001). 

Integrated  Segmentation  Methods 

Integrated  methods  combine  boundary  and  region  criteria.  In 
this  class  of  methods,  the  morphological  watershed  algorithm  is 
widely  studied  and  used  for  efficient  object  separation  (Beucher, 
1991;  Dougherty,  1993;  Vincent,  1993;  Serra  and  Soille,  1994; 
Shafarenko  et  al.,  1997).  Introduced  by  Digabel  and  Lantuejoul 
(1978),  extended  by  Beucher  (1982),  analyzed  theoretically  by 
Maisonneuve  (1982),  and  formally  defined  in  terms  of  flooding 
simulations  by  Vincent  and  Soille  (1991),  the  term  “watershed” 
comes  from  a  graphic  analogy  attributed  to  Vincent  and  Soille.  In 
this  analogy,  the  gray  level  image  is  treated  as  a  topographic  inten¬ 
sity  surface.  It  is  assumed  that  holes  have  been  punched  in  each 
valley  bottom.  If  this  surface  is  “flooded”  from  these  holes,  the 
water  will  progressively  flood  the  catchment  basins  (set  of  surface 
points  whose  steepest  slope  paths  reach  a  given  minimum)  of  the 
image.  At  the  end  of  this  flooding  procedure,  each  minimum  is 
completely  surrounded  by  “dams,”  which  delimit  its  associated 
“catchment  basins.”  The  set  of  dams  obtained  in  this  way  corre¬ 
sponds  to  watersheds  (called  the  watershed  surface  in  3D  case) 
from  a  geophysical  analogy,  and  provides  a  tessellation  of  the  input 
image  into  its  different  catchment  basins.  Unlike  the  boundary- 
based  methods  above,  the  watershed  is  guaranteed  to  produce 
closed  boundaries  even  if  the  transitions  between  regions  are  of 
variable  strength  or  sharpness.  Its  popularity  is  also  attributable  to 
its  computational  efficiency  and  extendability  to  3D  spaces 
(Higgins  and  Ojard,  1993;  Sijbers  et  al..  1997). 

The  second  method  in  the  class  is  variable-order  surface  fitting 
(Besl  and  Jain,  1988)  that  starts  with  a  coarse  segmentation  of  the 
image  into  several  surface-curvature-sign  primitives  (e.g.,  pit, 
peak,  ridge,  etc.)  which  are  refined  by  an  iterative  region-growing 
method  based  on  variable-order  surface  fitting. 

Another  method  in  the  class  is  to  integrate  region-growing  and 
edge-detection  (Pavlidis  and  Liow,  1990).  This  starts  with  a  split- 
and-merge  algorithm,  and  eliminates  or  modifies  region  bound¬ 
aries  based  on  some  criterion  using  boundary  smoothness  and  the 
variation  of  image  gradients.  Haddon  and  Boyce  (1993)  generate 
regions  by  partitioning  the  image  co-occurrence  matrix  and  then 
refining  them  by  relaxation  using  edge  information. 


Top-Down  Segmentation  Methods 

Unlike  bottom-up  algorithms,  such  as  those  described  in  the  pre¬ 
vious  section,  which  are  based  on  low-level  image-based  cues, 
such  as  coherence  of  brightness,  color,  texture  or  motion,  top-down 
methods  rely  on  prior  models  for  the  objects  in  the  image  to  carry 
out  the  segmentation.  Even  sophisticated  pre-  and  postprocessing 
techniques  cannot  overcome  the  inherent  limitations  of  purely 
intensity-based  methods.  Actually,  for  automatic  object  seg¬ 


mentation,  some  kind  of  prior  related  knowledge  can  and  must  be 
incorporated  into  the  algorithms.  The  mid-  and  high-level  knowl¬ 
edge  must  be  used  to  either  confirm  or  modify  the  initial  seg¬ 
mentation.  This  leads  to  the  top-down  approach,  where  the 
segmentation  is  guided  by  some  known  representation  or  prior 
knowledge  of  the  objects  under  study.  One  method  in  this  class  is 
model-based  segmentation,  which  optimizes  the  fit  between  a 
model  and  the  image  data.  Different  procedures,  such  as 
deformable  shape  models  (Cootes  et  al.,  1995;  Mclnerney  and  Ter- 
zopoulos,  1996;  Lorigo  et  al.,  2001;  Ghanei  and  Soltanian-Zadeh, 
2002)  and  statistical  models  (Vemuri  and  Radisavljevic,  1994; 
Staib  and  Duncan,  1996),  have  been  proposed.  Deformable 
models,  also  known  as  “ snakes ”  or  “ active  contours,"  have  been 
used  as  a  way  to  incorporate  application-specific  a  priori  knowl¬ 
edge.  For  example,  in  order  to  segment  a  bone  in  a  medical  image 
or  in  order  to  visually  track  a  person,  models  describing  the  pos¬ 
sible  contours  of  the  objects  of  interest  are  used.  The  parameters 
of  the  models  specify  object  properties  such  as  the  pose,  size,  and 
shape.  Sunil  Kumar  and  Desai  (1999)  presented  a  method  of  joint 
segmentation  and  image  interpretation.  Neumann  (2003)  proposed 
a  knowledge-guided  segmentation  in  3D  imagery.  Statistical  shape 
modeling  methods  depend  upon  the  availability  of  parametric 
models  to  describe  the  objects.  These  parameters  must  be  selected 
carefully  in  order  to  accurately  characterize  the  objects,  and 
discriminate  outliers  from  real  objects  in  an  effective  manner. 
The  set  of  parameters  must  be  rich  enough  to  describe  complex 
objects. 

Although  top-down  methods  are  appealing  because  they  seem 
similar  to  the  human  cognitive  process,  they  have  the  disadvan¬ 
tage  that  they  require  a  large  amount  of  a  priori  knowledge  (e.g., 
object  models,  number  of  regions)  and  this  tends  to  limit  the  scope 
of  applicability. 

Hybrid  Segmentation  Methods  Combining 
Bottom-Up  and  Top-Down  Processing 

Both  top-down  and  bottom-up  methods  have  their  advantages 
and  disadvantages.  The  latest  segmentation  methods  often 
combine  both  low-level  and  high-level  approaches  (Roysam  et  al., 
1992;  Roysam  and  Miller,  1992;  Bhattacharjya  and  Roysam, 
1994).  However,  as  these  approaches  tend  to  be  computationally 
intensive  and  overly  specialized,  they  are  not  described  in  further 
detail  here.  More  pragmatic  approaches  do  exist.  One  such  is 
described  in  the  following  section. 


EXAMPLE  ILLUSTRATING  BLOB 
SEGMENTATION 

Accurate,  reliable,  and  highly  automated  segmentation  of 
fluorescently  labeled  cell  nuclei  from  3D  confocal  images  is  an 
essential  first  step  for  quantification  by  association  of  various 
genomic  and  proteomic  products  in  nuclear  or  cytoplasmic  com¬ 
partments  [Fig.  15.1(A)].  Tightly  packed  cell  layers,  which  often 
result  in  the  appearance  of  “touching  objects”  in  image  stacks,  is 
a  common  segmentation  problem.  Precise  segmentation  of  nuclei 
is  an  absolute  requirement  for  correct  assignment  of  gene  tran¬ 
scription  products  to  nuclear  versus  cytoplasmic  locations  for 
achieving  optimal  temporal  resolution  (Lin  et  al.,  2003). 

Lin  and  colleagues  (2003,  2004)  described  an  algorithm  that 
uses  a  statistical  model-based  approach  to  combine  the  attractive 
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features  of  the  3D  watershed  algorithm,  with  algorithms  that  exploit 
available  intensity-gradient-based  cues,  and  the  knowledge  of  the 
expected  anatomic  shape  of  the  object.  Whenever  the  cell  shapes 
are  well  defined,  the  best-available  methods  for  separating  touch¬ 
ing  objects  are  based  on  the  watershed  algorithm  (Ancin  etal.,  1996; 
Lin  et  al.,  2003).  When  the  cell  shapes  are  poorly  defined, 
partitional  cluster-analysis  algorithms  work  better  (Roysam  et  al., 
1994). 

The  watershed  algorithm  is  widely  studied  and  used  for  effi¬ 
cient  object  separation  (Vincent  and  Soille,  1991;  Ancin  et  al., 
1996;  Malpica  et  al.,  1997;  Solorzano  and  Rodriguez,  1999). 
Notwithstanding  its  popularity,  watershed  algorithm  has  several 
limitations  arising  from  the  fact  that  it  relies  on  touching  objects 
exhibiting  a  narrow  “neck”  in  the  region  of  contact.  These  neck¬ 
lines  play  a  critical  early  role  in  estimating  the  number  of  objects 
in  a  given  cluster,  and  is  notoriously  error  prone.  Considerable 
effort  has  been  devoted  to  the  design  of  algorithms  for  generating 
the  correct  set  of  “geometric  markers”  to  guide  the  object  seg¬ 
mentation.  The  problem  of  determining  the  correct  number  of  such 
markers  is  inherently  a  difficult  one,  and  is  conceptually  similar  to 
the  problem  of  automatically  determining  the  number  of  groups  in 
multi-dimensional  statistical  data.  To  overcome  the  above  diffi¬ 
culties,  Lin  and  colleagues  (2003)  proposed  a  “ gradient-weighted 
distance  transform,”  that  combines  object  separation  hints  derived 
from  geometric  and  intensity  cues  in  the  image  data  to  improve 
watershed  segmentation.  Specifically,  the  geometric-distance 
transform  D  (Borgefors,  1986)  and  the  gradient  transform  G  (Lin 
et  al.,  2003)  are  combined  into  a  single  representation  that  cap¬ 
tures  the  object  separation  cues  available  in  the  data,  as  given  by 
the  following  formula. 

D'  =  Dx  exp  j  1 — G~Gmm  j, 

V  ^Anax  ^Amn  J 


where  and  Gmax  are  the  minimum  and  maximum  values  of  the 
gradient  G  needed  for  normalization.  It  is  easy  to  observe  that  the 
modified  distance  value  D',  has  a  high  value  at  positions  closer  to 
the  center  of  foreground  objects,  and  for  voxels  with  smaller 
gradient  values.  It  has  smaller  values  close  to  the  boundary  of  the 
foreground  objects,  or  where  the  gradient  is  relatively  large.  This 
expresses  the  object  separation  cue  that  the  voxels  with  bigger  gra¬ 
dient  values  tend  to  lie  on  the  boundary  of  an  object.  In  practice, 
the  watershed  algorithm  requires  the  inverse  of  this  distance  trans¬ 
formation.  This  inverse  is  denoted  T,  and  is  computed  as  follows: 

T  =  .S’„fmax(D'j  -  D), 

where  max(D')  is  the  global  maximum  within  the  distance  images, 
and  Sg  represents  a  Gaussian  smoothing  operator  (Castleman, 
1996).  The  smoothing  operation  is  needed  because  the  transformed 
image  may  contain  tiny  noise-caused  intensity  peaks,  usually  due 
to  uneven  cell  staining.  Figure  15.7  illustrates  the  effectiveness  of 
the  combined  measure.  Panel  (A)  shows  a  sample  image,  with  the 
nuclei  indicated  in  blue,  and  the  FISH  signal  displayed  in  red.  Panel 
(B)  is  a  surface  plot  of  the  geometric  distance  D  for  the  region  indi¬ 
cated  by  the  white  box  in  (A).  Panel  (C)  is  the  result  of  combining 
the  geometric  and  gradient  measures  D  and  G  above.  It  is  clear  that 
the  combined  transformation  in  Panel  (E)  is  effective  in  discrimi¬ 
nating  this  touching  nuclear  cluster  even  though  it  does  not  have 
the  characteristic  bottleneck-shaped  connection  pattern. 

Model-Based  Object  Merging 

Although  the  gradient-weighted  distance  transform  described 
above  essentially  eliminates  under-segmentation,  the  problem  of 
over-segmentation  remains.  To  correct  the  over-segmentation,  it  is 
necessary  to  detect  and  break  (eliminate)  false  watershed  surfaces 
and  thereby  merge  cell  objects  (Adiga  amd  Chaudhuri,  2001a;  Lin 


FIGURE  15.7.  Illustrating  the  combined  gradient-weighted  distance  transform  as  a  cue  of  separating  connected  objects.  (A)  A  small  portion  of  a  3-D  confocal 
image  stack,  showing  nuclei  in  the  CA1  region  of  a  rat  brain.  The  white  box  in  panel  (A)  indicates  the  region  of  interest,  which  includes  two  touching  nuclei. 
Panel  (B)  shows  a  surface  plot  showing  a  standard  geometric  distance  map  D  which  is  not  adequate  for  nuclei  separation.  The  combined  gradient- weighted  dis¬ 
tance  map  in  panel  (D)  results  in  the  correct  segmentation  shown  in  panel  (E).  (Courtesy:  Journal  of  Cytometry.) 
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FIGURE  15.8.  Illustrating  two  example  cases  encountered  by  the  watershed 
surface-breaking  algorithm  for  touching  objects.  Panel  (A)  shows  a  case  that 
leads  to  merging  of  the  two  objects.  Panel  (B)  shows  the  case  that  one  object 
has  multiple  watershed  surfaces  (two  in  this  example).  In  this  case,  there  are 
two  candidate  watershed  surfaces  to  choose  from  for  breaking.  The  watershed 
surface  vv  that  has  a  greater  merging  score  C„  is  given  the  higher  priority,  indi¬ 
cating  better  fit  to  the  object  model,  thus  the  higher  confidence  towards  its 
breaking.  (Courtesy:  Journal  of  Cytometry.) 


et  al.,  2003).  Currently,  the  best  methods  rely  on  a  quantitative 
score  based  on  a  mathematical  model  of  the  objects  of  interest  (i.e., 
the  nuclei).  The  object  model  is  described  by  a  vector  of  features 
that  are  denoted  as  Xc  for  object  (cell  nucleus)  c.  The  features 
include  measurements  such  as  volume,  texture,  convexity,  circu¬ 
larity,  and  shape.  The  merging  score  based  on  these  features  is 
given  below  assuming  that  they  are  Gaussian-distributed: 

T  =  5g(max(D/)  -  D'), 

where  X  and  are  the  mean  and  covariance  matrix  of  the  train¬ 
ing  feature  set  respectively,  which  are  obtained  from  some 
example  intact  nuclei  in  the  given  image  stack. 

Intuitively,  the  merging  decisions  are  based  on  the  following 
two  criteria:  (i)  the  merging  score  Scw,  that  is,  the  score  of  the  com¬ 
bined  object  by  nucleus  c J,,  c„,  and  watershed  surface  w  should  be 
higher  than  the  score  of  either  nucleus  Scl  or  Sc l  before  merging, 
(ii)  The  gradient  of  watershed  surface  w  should  be  relatively  large 
compared  with  the  gradient  of  the  nucleus  ci,  and  cl,.  It  is  assumed 
that  intranuclear  gradients  are  smaller  than  internuclear  gradients. 
With  these  observations  in  mind,  the  following  ratios  are 
calculated: 


R 


s 


R 


2x^ 

(^+YC;,) 

2><y» 


objects.  The  model-based  watershed  surface  breaker  effectively 
eliminates  almost  all  of  these  over-segmented  nuclei  during  the 
postprocessing.  One  example  is  shown  in  Figure  15.9,  where  the 
above  algorithm  achieves  a  97%  accuracy  compared  to  a  consen¬ 
sus  of  three  expert  observers. 


EXAMPLE  ILLUSTRATING  SEGMENTATION  OF 
TUBE-LIKE  OBJECTS 

Two  broad  types  of  algorithms  exist  for  segmenting  tube-like 
objects  (neurons,  vasculature):  (i)  skeletonization  and  (ii)  vector- 
ization.  Skeletonization  methods  work  by  systematically  eroding 
a  binarized  version  (the  results  of  adaptive  thresholding)  of  the 
image  until  only  the  innermost  voxels  (i.e.,  the  skeleton)  remains. 
They  are  attractive  for  applications  in  which  the  objects  are  irreg¬ 
ular,  for  example,  spiny  neurons  (Koh  et  al.,  2002).  On  the  other 
hand,  vectorization-based  methods  are  valuable  when  the  objects 
are  much  more  regular  in  appearance,  and  can  be  modeled  as  gen¬ 
eralized  cylinders  in  3D  space  (Al-Kofahi  et  al.,  2002;  Abdul- 
Karim  et  al.,  2003,  2005).  The  latter  method  embeds  more 
assumptions  about  the  objects  of  interest,  and  is  therefore  better  at 
rejecting  image  clutter  than  skeletonization-based  methods.  Figure 
15.10  shows  an  example  of  skeletonization  results  (Koh  et  al., 
2002;  He  et  al.,  2003)  and  Figure  15.11  exemplifies  vectorization 
results  (Al-Kofahi  et  al.,  2002),  both  on  dye-injected  neurons 
imaged  by  a  confocal  microscope. 


Skeletonization  Methods 

These  methods  work  by  performing  adaptive  segmentation  (pro¬ 
ducing  a  binary  image)  followed  by  skeletonization  and  graph 
extraction  (Koh  et  al.,  2002;  He  et  al.,  2003).  The  3D  skeleton  of 
an  object  is  a  connected  set  of  lines  (traces)  that  coincide  with  its 
medial  axis,  and  which  can  be  obtained  by  repeatedly  performing 
erosion  operations  and  connectivity  analysis  on  the  3D  binary 
volume,  until  only  the  medial  skeleton  of  the  object  remains.  During 
the  thinning  process,  a  voxel  is  considered  deletable  if  its  removal 
does  not  cause  any  other  voxels  to  become  disconnected.  Clearly, 
improved  detection  of  voxel  connectivity  allows  more  voxels  to  be 
deleted,  and  hence  a  more  concise  skeleton  can  be  obtained.  The 
traditional  methods  (Tsao  and  Fu,  1981;  Gong  and  Bertrand,  1990) 
for  3D  skeletonization  are  based  on  analyzing  a  3  x  3  X  3  window 
around  each  voxel  to  test  for  voxel  deletability.  One  drawback  of 
such  methods  is  their  inability  to  detect  many  cases  when  voxels  are 
connected  via  more  circuitous  paths,  especially  in  noisy  data  such 
as  that  from  confocal  microscopy.  Specifically,  these  algorithms  can 
magnify  each  of  the  however-small  noise-caused  surface  irregular¬ 
ities  into  full  skeletal  appendages,  resulting  in  an  excessively 
complex  skeleton.  Cohen  and  colleagues  (1994)  developed  a  thin¬ 
ning  algorithm  based  on  performing  voxel  detability  tests  over 
windows  much  larger  than  3x3x3,  and  using  long-range  bound¬ 
ary  detection  criteria  (He  et  al.,  2003). 


The  final  decision-making  criterion  is  the  combination  of  the  above 
two  ratios: 

Rn.  =  RSw  x  Rlt  >  |3, 

where  P  is  an  empirically  set  decision  threshold.  Further  details 
concerning  the  above  algorithms  are  provided  in  Lin  and  col¬ 
leagues  (2003).  Figure  15.8  illustrates  two  example  cases  encoun¬ 
tered  by  the  watershed  surface-breaking  algorithm  for  touching 


Vectorization  Methods 

In  contrast,  vectorization  methods  utilize  a  more  rigid  model  that 
embeds  tighter  assumptions  about  the  objects.  For  example,  it 
often  explicitly  models  tube-like  objects  geometrically,  say  using 
a  generalized  cylinder  constrained  by  parallel  boundaries 
(Al-Kofani  et  al.,  2002)  (Fig.  15.12).  Figure  15.4  illustrates  an 
example  of  tracing  tumor  microvasculature  using  a  semi-rigid 
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FIGURE  15.9.  Segmentation  result  generated  by  the  model-based  merging  procedure  (i.e.,  the  watershed  surface  breaker).  There  were  66  watershed  surfaces 
broken,  and  most  cases  of  over- segmentation  were  eliminated.  An  explicit  mathematical  model  for  the  anatomic  characteristics  of  cell  nuclei,  such  as  size,  texture, 
convexity  and  shape  measures,  is  incorporated  during  the  post-merging  procedure.  Panel  (A)  shows  the  segmentation  right  after  the  enhanced  watershed  algo¬ 
rithm.  As  can  be  seen,  clusters  of  nuclei  are  basically  eliminated,  and  the  over-segmentation  is  allowed,  as  indicated  by  yellow  arrows.  Panel  (B)  shows  the  final 
results  after  the  model-based  merging  procedure.  (Data  courtesy:  Carol  Barnes  Lab  at  the  University  of  Arizona.) 
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FIGURE  15.10.  Illustrating  neuron  tracing  by  skeletonization.  (A)  3-D  image  of  a  dye-injected  neuron,  shown  in  maximum-intensity  projection  (8-bit/voxel, 
383  x  328  x  150).  (B)  Segmented  image  by  adaptive  thresholding.  (C)  Skeletonized  neuronal  process  shown  with  the  detected  soma.  The  skeletonization  method 
is  very  general,  so  it  can  apply  to  spines,  for  example,  but  it  is  not  nearly  as  fast  and  selective  as  the  vectorization  method,  (image  courtesy:  Microscopy  and 
Microanalysis.) 


FIGURE  1 5.1 1 .  Illustrating  neuron  tracing  using  a  generalized  cylinder  model  and  vectorization.  (A)  A  3-D,  512  x  480  x  301  grayscale  image  of  a  dye-injected 
neuron  shown  by  the  x-y,  x-z,  and  y-z  maximum  intensity  projections.  (B)  The  segmentation  results  using  a  rigid,  generalized  cylinder  model.  Notice  that  since 
the  segmentation  is  guided  using  a  spatio-physical  model,  no  background  voxels  are  misclassified  since  they  do  not  fit  the  generalized  cylinder  model.  This 
approach  can  be  much  more  selective  than  the  skeletonization  method  and  substantially  faster.  However,  it  is  not  as  effective  at  segmenting  non-tube-like  struc¬ 
tures  such  as  spines. 


Automated  Three-Dimensional  Image  Analysis  Methods  for  Confocal  Microscopy  •  Chapter  15  327 


A  B 


FIGURE  15.12.  (A)  Modeling  a  tube-like  object  using  a  generalized  cylinder.  Irregularities  of  the  object  boundary  relative  to  the  straight-boundary  assumption 
are  incorporated  into  the  model.  (B)  Illustrates  the  recursive  exploratory  tracing  (vectorization)  of  a  tube-like  structure,  advancing  with  the  information  gathered 
from  best-fit  cylinder  at  each  point,  (pictures  courtesy:  Microvascular  Research,  IEEE  Press.) 


model.  The  following  paragraphs  describe  the  core  ideas  in  further 
detail. 

At  the  simplest  level,  tube-like  structures  can  be  modeled  as 
piecewise-linear  segments,  where  each  segment  is  a  generalized 
cylinder  (Al-Kofahi  et  al.,  2002).  The  key  geometric  property  is 
the  existence  of  anti-parallel  edges  located  a  small  distance  apart. 
Even  this  apparently  simplistic  modeling  leads  to  a  highly  suc¬ 
cessful  analysis  exemplified  by  Figure  15.11.  In  practice,  the  strict 
cylinder  model  is  relaxed  to  account  for  slight  irregularities  in  the 
vasculature  boundaries  [Fig.  15.12(A)]  by  incorporating  a  toler¬ 
ance  to  deviations  from  the  strict  cylinder  model  in  the  software 
implementation.  Often,  the  structures  of  interest  are  much  more 
irregular,  as  exemplified  by  the  tumor  microvasculature  in  Figure 
15.4.  This  irregularity  can  be  modeled  by  the  use  of  robust  statis¬ 
tics,  which  implies  the  use  of  a  median  instead  of  the  arithmetic 
mean  (Abdul-Karim  et  al.,  2003).  This  method  is  described  below, 
and  illustrated  diagrammatically  in  Figure  15.12(B). 

At  the  core  of  this  method  is  the  estimation  of  boundary 
locations  and  direction.  The  estimation  utilized  directional  edge 
detectors,  called  templates,  comprised  of  linearly  stacked  one¬ 
dimensional  (ID)  edge  detectors  (Sun  et  al.,  1995)  of  the  form 
[— 1,  -2,  0,  2,  l]r  along  a  direction.  Along  a  particular  direction, 
the  edge  strengths  from  each  ID  edge  detector  are  averaged  [using 
mean  (Al-Kofahi  et  al.,  2002)  or  median  (Abdul-Karim  et  al., 
2003)]  and  the  resulting  value  is  associated  with  the  correspond¬ 
ing  template,  referred  to  as  a  template  response.  This  can  be 
expressed  mathematically  as 

(b‘,  u',k')=  argmax  [/?(b,  u,  K)] , 

j( b,  u,  K)  | b=p'+mu± ,  m= 1, . . .  ^ .  u  e  U,  k  e  ^  j 

where  U  is  the  set  of  unit  vectors  along  directions  in  the  neigh¬ 
borhood  of  u'  and  K  is  the  set  of  all  template  lengths.  The  vector 
u  is  a  unit  vector  along  a  particular  3D  angle  9,  while  Uj.  is  the 
unit  vector  perpendicular  to  u.  The  parameter  M  is  the  user-defined 


diameter  of  the  widest  expected  vasculature.  Values  b'.  u',  k‘  are 
the  results  of  this  exhaustive  search  at  iteration  i,  each  represent¬ 
ing  the  [ x ,  y,  7]  location,  orientation,  and  length,  respectively,  of 
the  template  that  returns  the  maximum  response  R.  This  search  is 
performed  four  times  corresponding  to  the  four  templates  that 
make  up  the  generalized  cylinder  model  [Fig.  15.12(A)].  The  cor¬ 
responding  mean  and  median  template  response,  respectively,  are 
expressed  as 

R( b,  u,  K)  =  arg  max  {median(r( b  +  ;u,  ux))}, 

keK  j=l..K 

and 

R(b,  u,  K)  =  arg  max  1  y ^ (r(b  +  ju,  uj)i, 

keK  [k  j=l  j 

where  /-(b.iiO  is  the  response  of  a  single  ID  edge  detector  at  b  along 
the  direction  that  is  perpendicular  to  u.  Notice  that  a  template  of 
length  k  is  comprised  of  A:  ID  edge  detectors  stacked  together,  hence, 
r  is  essentially  a  template  of  length  1 .  The  simple  switch  from  mean 
to  median  averaging  deserves  a  closer  inspection.  By  using  the 
median,  the  response  function  is  robust  to  at  most  50%  of  outliers 
(Huber,  1981),  loosely  termed  as  bad  edges.  Figure  15.13 
further  demonstrates  the  difference  between  these  two  averaging 
methods. 

Notice  that  by  using  this  rigid  model  both  the  centerline  and 
boundary  are  readily  obtained  while  segmenting  the  image,  yield¬ 
ing  morphometries  such  as  length  (cumulative  distance  between 
center  points)  and  diameter  (distance  between  boundary  points)  on 
the  fly.  In  contrast,  the  skeletonization  approach  requires  much 
more  sophisticated  postprocessing  to  extract  these  morphometries. 
In  either  case,  additional  processing  must  be  performed  for  branch 
analysis  and  to  extract  any  other  morphometries  from  the  seg¬ 
mented  structure.  Details  of  this  method  are  described  elsewhere 
(Al-Kofahi  et  al.,  2002;  Abdul-Karim  et  al.,  2003). 


328  Chapter  15  •  B.  Roysam  et  al. 


A  B 

FIGURE  15.13.  Illustrates  the  results  of  using  mean  template  response  (A),  and  the  median  template  response  (B)  on  noisy  confocal  images.  Note  the  reduc¬ 
tion  in  terms  of  erroneous  false-positive  traces  when  using  the  median  template  response,  (image  courtesy:  IEEE  Press.) 


EXAMPLE  COMBINING  TUBE  AND 
BLOB  SEGMENTATION 

One  sometimes  encounters  images  containing  both  blob-like  and 
tube-like  objects.  From  an  image  analysis  standpoint,  it  is  best  to 
use  distinctive  fluorophores  to  highlight  these  two  types  of  struc¬ 
tures  (e.g..  Fig.  15.5),  and  apply  the  appropriate  analysis  method  to 
each  image  channel  (Lin  et  al.,  2005).  An  example  is  shown  in 
Figure  15.14(A),  which  shows  an  original  image  containing  both 
blob-like  objects  (nuclei)  in  red  and  the  tube-like  cerebral  vascula¬ 
ture  in  green.  Figure  15. 14(B)  shows  the  resulting  composite  image 
after  the  separate  segmentation  of  cell  nuclei  in  the  red  channel  and 
tracing  of  vasculature  in  the  green  channel.  Figure  15.14(C)  shows 
the  original  image  containing  blob-like  objects  (nuclei)  in  red  and 
the  cloud-like  cytoskeleton  in  green.  Figure  15.14(D)  shows  the 
resulting  composite  image  after  the  segmentation  of  cell  nuclei  and 
cytoskeleton  in  the  cytoplasm.  After  the  initial  image  segmentation, 
a  variety  of  postanalyses  can  be  performed,  such  as  morphometries 
quantification,  spatial  analysis  of  different  objects,  object  classifi¬ 
cation,  and  statistical  analysis.  Overall,  these  automated  methods 
provide  the  useful  tools  for  biologists  replacing  manual  procedures 
that  are  tedious,  subjective,  and  time  consuming. 


REGISTRATION  AND  MONTAGE 
SYNTHESIS  METHODS 

Registration  is  the  process  of  spatially  aligning  two  or  more  dif¬ 
ferent  images  of  the  same  region.  When  the  object  of  interest  is 
larger  than  the  field-of-view  of  the  microscope,  two  or  more  par¬ 
tially  overlapping  images  must  be  acquired  and  then  registered  to 
provide  an  extended  view  of  the  specimen.  Conversely,  a  sequence 
of  images,  taken  at  two  different  times,  say  before  and  after  treat¬ 
ment,  must  then  be  registered  to  yield  a  common  image  subset  to 
detect  corresponding  changes.  Broadly,  there  are  two  approaches: 
landmark-based  registration  (Can  et  al.,  2002;  Al-Kofahi  et  al., 
2003)  and  intensity-correlation  based  registration  (Capek  and 
Krekule,  1999;  Beck  et  al.,  2000).  Landmark-based  registration 
relies  on  matching  a  set  of  distinctive  points  (landmarks)  in  pairs 
of  the  images  in  sequence,  for  example,  spatially  or  temporally. 
Intensity-based  registration  methods  work  directly  with  the  volu¬ 
metric  image  data,  utilizing  all  of  the  voxels.  They  compute  a 
measure  of  correlation  between  the  image  pairs  to  estimate  their 
relative  spatial  disposition.  Finally,  hybrid  methods  that  combine 
ideas  from  landmark  and  intensity  correlation  methods  are  now 
emerging  (Abu-Tarif,  2002).  Most  of  these  methods  are  well 
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FIGURE  15.14.  Image  analysis  examples  illustrating  (A)  blob-like  objects  (nuclei)  in  red  channel  and  tube-like  objects  (cerebral  vasculature)  in  green  channel; 
(C)  blob-like  objects  (nuclei)  in  red  channel  and  cloud-like  objects  (cytoskeleton)  in  the  green  channel.  In  both  cases,  image  analysis  is  performed  one  spectral 
channel  at  a  time,  using  separate  methods  for  segmenting  the  blobs  and  tubes/clouds.  Once  this  is  done,  it  is  possible  to  quantify  the  spatial  relationships  between 
the  two  kinds  of  objects,  (data  courtesy:  Chris  Bjomsson  and  Karen  Smith  at  the  Wadsworth  Center,  and  Jan  Stegemann  Laboratory  at  Rensselaer.) 


implemented  in  toolkit  form,  for  example,  the  Insight  toolkit  from 
the  National  Library  of  Medicine  (2001). 

The  transformation  model  used  should  have  adequate  degrees 
of  freedom  to  account  for  motion  and  possible  distortion  between 
the  two  images  (Hartley  and  Zisserman,  2000).  For  example,  if 
only  rigid  body  motion  is  considered  between  the  two  images,  a 
rigid  body  transformation  (rotation  and  translation)  is  adequate. 
However,  if  the  anisotropic  nature  of  the  voxels  and  confocal 
system  artifacts  such  as  curvature  of  field  and  geometric  distor¬ 
tions  are  present,  a  higher  order  transformation  should  be  used. 
The  affine  model  was  proposed  as  a  good  compromise  between 
model  complexity  and  flexibility. 

Landmark-based  methods  are  generally  fast  and  can  scale  up 
easily  with  higher-order  transformation  models,  but  their  use  is 
limited  to  the  images  where  good  landmarks  can  be  extracted  first. 


Centerline  locations  and  branching  points  of  tube-like  objects  (Al- 
Kofahi  et  al.,  2003)  and  the  centroids  of  blob-like  objects  (Becker 
et  al.,  1996)  are  examples  of  landmarks  that  can  be  used  for  reg¬ 
istration.  Although  intensity-correlation-based  methods  do  not 
require  the  landmark  extraction  step  prior  to  registration,  they  are 
susceptible  to  imaging  artifacts,  such  as  non-uniform  illumination, 
and  do  not  exploit  specific  knowledge  of  the  application  context 
to  simplify  the  problem.  Furthermore,  correlation-based  algo¬ 
rithms  are  naturally  far  more  computationally  intensive  than  the 
landmark-based  methods. 

Montage  synthesis,  also  known  as  mosaicing,  is  the  process 
of  stitching  together  a  large  number  of  partial  local  views 
(“windows”)  of  the  biological  scene  of  interest  to  generate  a  more 
complete  or  extended  field  of  view.  This  is  especially  valuable 
when  the  object/scene  of  interest  is  significantly  larger  than  the 
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field  of  the  microscope.  The  simplest  type  of  montage  synthesis 
consists  of  directly  stitching  together  confocal  images.  Commonly, 
the  microscope  stage  can  be  stepped  in  a  series  of  overlapping 
two-dimensional  (2D)  or  3D  windows  that  can  then  be  merged 
into  a  single  composite  image  that  can  be  analyzed  all  at  once. 
Alternatively,  each  of  the  windows  can  be  subjected  to  automated 
image  analysis,  and  the  results  stitched  together  to  generate  a 
synthetic  montage-like  representation.  In  addition  to  the  common 
grayscale  montage,  it  is  possible  to  generate  artificial  representa¬ 
tions  such  as  segmentation  label  montages,  and  ball/stick  dia¬ 
grams,  also  known  as  object  montages  (Becker  et  al.,  1996). 

Another  application  for  registration/montage  synthesis  is 
multi- view  deconvolution  (Cogswell  et  al.,  1996;  Heintzmann  et 
al.,  2000)  and  multi-view  attenuation  correction  (Can  et  al.,  2003). 
In  the  former,  confocal  images  are  blurred  by  the  point  spread 
function  (PSF),  which  is  asymmetric  in  the  x-,  y-,  and  z- 
dimensions.  Multiple  views  can  sometimes  be  used  to  account  for 
the  missing  information  and  improve  the  axial  resolution;  ideally, 
we  need  tilted  views  separated  by  90°  angle.  In  the  latter,  the  exci- 
tation/fluorescent  light  attenuates  as  a  function  of  depth  according 
to  Beer's  law  (Weast,  1974)  although  spherical  aberration  is  a  more 
common  cause  of  signal  loss  with  depth.  To  correct  for  the  atten¬ 
uation,  two  views  separated  by  180°  are  used.  In  this  scenario,  the 


portions  of  one  image  with  low  signal-to-noise  ratio  (S/N)  (far 
from  the  objective)  are  closer  to  the  objective  in  the  other  image, 
with  higher  S/N,  and  vice  versa  (see  also  Chapter  37,  this  volume ). 
These  two  images  can  be  montaged  to  produce  a  more  faithful  rep¬ 
resentation  of  the  entire  specimen. 

Figures  15.15  and  15.16  illustrate  many  of  the  issues  described 
above.  The  neuron  shown  in  Figure  15.15  was  larger  than  the  micro¬ 
scope  field-of-view  at  the  recording  resolution.  Also,  the  specimen 
was  thick  (more  than  300pm)  which  resulted  in  a  significant  signal 
loss  for  the  deeper  parts  of  the  specimen.  Montage  synthesis  was 
used  to  overcome  these  limitations;  the  specimen  was  mounted 
between  two  coverslips  (Can  et  al.,  2003)  to  allow  imaging  from 
both  sides.  The  first  image  stack  was  captured  in  the  conventional 
way.  then  the  specimen  was  moved  using  the  x,y-stage,  and  the 
second  image  stack  was  taken  to  capture  the  adjacent  x,y-field.  To 
correct  for  attenuation  and  to  increase  the  imaging  depth,  the  spec¬ 
imen  was  then  flipped  top-to-bottom,  and  two  more  sets  of  images 
were  captured  as  before.  Figure  15.16  shows  the  result  of  montag- 
ing  the  four  image  stacks  with  attenuation  correction;  the  result  is 
clearly  a  more  faithful  representation  of  the  specimen.  It  captures 
the  whole  neuron,  with  high  resolution  and  better  S/N. 

Finally,  the  microscopist  is  often  faced  with  multi-channel 
data.  Techniques  for  registering  and  montaging  such  data  are 


FIGURE  1 5.15.  Sample  views  of  an  Alexa  stained  neuron  from  a  rat  brain.  A-v  maximum  intensity  projection.  Top  row,  before  flipping;  the  specimen  was  trans¬ 
lated  to  capture  the  whole  neuron  between  the  positions  in  (A)  and  (B)  respectively.  Bottom  row,  after  flipping;  again,  the  specimen  was  translated  to  capture 
the  whole  neuron  between  the  positions  in  (C)  and  (D)  respectively.  Each  image  is  512  X  480  X  261  voxels  in  size,  except  for  the  one  in  (C),  512  X  480  X  323 
voxels.  The  neuron  is  larger  than  the  microscope  field  of  view,  the  four  views  cover  the  whole  neuron  maintaining  high  resolution.  The  sampling  rates  are 
0.375  mm/voxel  for  x  and  y  dimensions  and  0.5  mm/voxel  for  the  z  dimension,  (courtesy:  Journal  of  Microscopy.) 
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FIGURE  15.16.  Mosaic  image  constructed  from  the 
four  views  in  Figure  15.15  (A)  top  view  ( x-y ),  and 
(B)  side  view  (j k-z),  maximum  intensity  projection. 
The  result  of  the  montaging  is  a  high-resolution 
image  for  the  whole  neuron  with  increased  effective 
imaging  depth,  (courtesy:  Journal  of  Microscopy.) 


analogous  as  long  as  corrections  for  chromatic  aberration  are 
performed.  Figure  15.17(A)  illustrates  a  two-channel  mosaic 
representing  one  hemisphere  of  a  rat  brain  section.  The  green 
regions  are  the  nuclei  and  red  color  indicates  the  FISH  signal.  The 
full  mosaic  consists  of  7911  x  6188  voxels  (140MB),  and  is  syn¬ 
thesized  from  70  imaging  windows  each  of  width  1024  x  1024 
pixels.  The  inset  is  an  enlarged  view  of  a  small  portion  of  the 
montage  at  higher  resolution.  Figure  15.17(B)  illustrates  the  mon¬ 
taging  based  on  the  traced  vasculature  structure. 

METHODS  FOR  QUANTITATIVE 
MORPHOMETRY 

Morphometric  data  are  either  the  end  goal  of  image  analysis  or 
an  intermediate  step  towards  testing  a  hypothesis.  Listed  below 
are  common  types  of  object  measurements  for  blob-like  and/or 
tube-like  objects.  Interested  readers  are  referred  to  other  books 
(Theodoridis  and  Koutroumbas,  1999;  Shapiro  and  Stockman, 
2001)  for  details  on  the  algorithms  needed  to  calculate  these 
measurements. 

•  Size  measures:  Diameter,  volume,  surface  area,  length,  width. 

•  Intensity  and  spectral  measurements:  Integrated,  mean,  and 
median  intensity  of  one  or  more  spectral  channels  over  an 
object. 

•  Shape  measures:  Eccentricity,  elongatedness,  compactness, 
convexity,  thickness,  shape  complexity,  tortuosity,  area- 
perimeter  ratio,  bending  energy. 

•  Texture  measures:  Intensity  variance,  clumping,  homogeneity. 

•  Location  and  pose:  Centroid,  major  and  minor  axes,  angle  of 
tilt. 

•  Interest  points:  Convex/concave  corners,  inflexion  points, 
bifurcations. 


•  Invariants:  Dimensionless  ratios  such  as  shape  factors,  affine 
invariant  moments. 

•  Topological  measures:  Branching  factors  and  angles. 

•  Group  properties:  Group  size  and  shape,  repetition  length. 

Morphometric  data  can  be  generated  from  the  segmented 
objects  during  segmentation  or  by  postprocessing  the  segmenta¬ 
tion  results  (Koh  et  al.,  2002;  Williams  et  al.,  2003).  In  some  cases, 
the  segmentation  algorithms  that  produce  just  a  segmented 
volume  may  need  postprocessing  steps  such  as  skeletonization 
(Lee  et  al.,  1994;  Maddah  et  al.,  2003)  to  obtain  the  measurements. 
In  the  analysis  of  cellular  compartment  temporal  activity  by 
fluorescence  in  situ  hybridization  (catFISH)  example,  a  set  of 
measurements  are  made  for  each  segmented  nucleus.  The  mea¬ 
surements  include  nuclear  features  such  as  volume,  intensity, 
texture,  shape,  and  FISH  measurements  such  as  FISH  spot 
volume,  z-depth  and  intensity  value,  intranuclear  and  cytoplasmic 
FISH  measurement,  etc.  Figure  15.1(B)  shows  a  part  of  a  spread¬ 
sheet  containing  such  data.  This  data  is  ready  for  the  statistical 
analysis  required  for  hypothesis  testing.  It  can  also  be  used 
for  manual  evaluation  to  verify  the  accuracy  of  the  automated 
results. 

In  the  vessel  tracing  example,  a  region  of  interest  is  defined 
for  each  image  of  a  time  series  data  set  by  the  intersection  of  the 
image  and  all  other  images  in  the  set.  Morphometric  data  collected 
within  these  regions  include  total  vasculature  length,  average  vas¬ 
culature  segment  length,  average  horizontal  width,  and  average 
vertical  width,  all  extracted  by  the  tracing  algorithm.  Statistics  and 
traces  generated  outside  these  regions  are  ignored.  The  generated 
statistics  are  entered  into  a  spreadsheet  and  plotted  to  highlight  the 
changes.  Actual  change  measurements  such  as  percentage  reduc¬ 
tion  in  the  total  number  of  vasculature  segments  or  percentage 
increase  in  the  total  vasculature  length  can  be  obtained  directly 
from  the  spreadsheet. 
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FIGURE  15.17.  Illustrating  the  feature-based  montaging  of  multi-spectral  confocal  microscope  images.  (A)  Each  individual  image  is  first  segmented,  and  a  set 
of  features  (cell  nuclei)  are  extracted  from  the  segmented  objects.  These  images  are  the  basis  of  pairwise  registration  to  montage  the  images.  (B)  Neurites  are 
traced  first,  which  become  the  basis  of  the  montaging. 
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METHODS  FOR  VALIDATING  THE 
SEGMENTATION  AND  MAKING  CORRECTIONS 

Validation  can  and  should  be  performed  at  various  stages  of  auto¬ 
matic  image  analysis.  The  main  objective  is  to  quantify  the  dis¬ 
agreement  between  automated  results  and  the  “ground  truth,” 
where  a  measurement  of  the  disagreements  is  interpreted  as  the 
error.  The  ground  truth  is  usually  obtained  manually.  Being  sub¬ 
jective,  it  is  prone  to  inter-  and  intraobserver  variability.  This  can 
be  reduced  by  using  multiple  and  independent  manual  segmenta¬ 
tion  results  that  are  combined  to  approximate  the  ground  truth.  An 
effective  method  for  combining  the  manual  segmentations  is  to 
establish  a  consensus.  For  this,  each  disagreement  between  manual 
observers  is  resolved  by  careful  discussion.  Furthermore,  the  type 
of  disagreement  between  the  manual  and  automated  results 
depends  upon  the  type  of  objects. 

For  blobs,  the  types  of  errors  include  false  detection  (false  pos¬ 
itives),  misses  (false  negatives),  errors  in  separating  connected 
objects,  and  errors  in  correctly  delineating  the  boundaries  of 
objects  (Chawla  et  al.,  2004).  For  tube-like  objects,  image  analy¬ 
sis  errors  include  falsely  traced  segments  and  branch  points  (false 
positives)  and  missed  segments  and  branch  points  (false  nega¬ 
tives),  accuracy  of  the  trace,  accuracy  of  width  measurements,  and 
accuracy  in  locating  branching/crossover  points  (Abu-Tarif  et  al., 
2002;  Al-Kofahi  et  al.,  2003).  When  considering  time-lapse  series 
of  live  specimens  (Abdul-Karim  et  al.,  2003),  it  is  also  useful  to 
quantify  registration  errors,  and  errors  in  detecting  change. 

As  an  example,  the  catFISH  automated  image  analysis  soft¬ 
ware  was  validated  with  respect  to  three  criteria:  (1)  accuracy  of 
nuclear  segmentation,  (2)  accuracy  in  classifying  intranuclear  foci; 
and  (3)  accuracy  in  classifying  cytoplasmic  FISH  signal.  The 
manual  image  analysis  was  performed  using  MetaMorph  software 
(Universal  Imaging  Corporation,  West  Chester,  PA).  Three  sepa¬ 
rate  experts  manually  rated  each  of  the  test  image  stacks  for 
numbers  of  nuclei,  nuclei  that  contained  FISH  signal,  and  cells  that 
contained  cytoplasmic  FISH  signal.  Statistical  analysis  was  done 
using  a  “consensus  count”  established  by  three  experts. 


Quantitative  validation  may  also  involve  voxel-to-voxel  com¬ 
parisons  of  the  segmented  volume,  or  comparisons  of  any  other 
entities  derived  from  the  segmented  volume  such  as  the  vascula¬ 
ture  centerline  or  the  cell  centroid.  The  comparison  can  be  per¬ 
formed  automatically  to  further  reduce  the  subjectivity  of 
validating  the  segmentation  results  (Al-Kofahi  et  al.,  2002).  In  the 
case  of  tube-like  objects,  the  centerline  locations  are  extremely 
important  because  almost  all  morphometries  depend  on  them.  Cen¬ 
terline  locations  are  typically  validated  for  the  deviation  from  a 
“true”  centerline  (reflects  the  accuracy  of  the  automated  method), 
and  the  centerline  coverage  of  the  objects  in  the  image  (reflects  how 
much  manual  editing  the  automated  result  needs).  To  obtain  the 
accuracy  metric,  a  distance  map  (Borgefors,  1986)  can  be  used  to 
calculate  the  average  Euclidean  distance  between  every  voxel  of 
manually  segmented  centerlines  and  every  voxel  of  automatically 
segmented  centerlines  that  are  within  a  certain  Euclidean  distance. 

Going  further,  validation  of  the  extracted  measurements  from 
a  segmented  volume  requires  manual  extraction  of  the  measure¬ 
ments  from  the  manually  segmented  volume.  This  is  a  prohibi¬ 
tively  time-consuming  procedure  in  3D  image  analysis,  and  as  the 
results  will  be  subjective,  it  may  not  be  useful.  Instead,  phantoms 
or  artificial  images  are  generated  using  ideal  models  (blobs,  tubes, 
or  clouds)  of  the  biological  objects  with  known  morphometries. 
Measurements  are  extracted  automatically  using  the  automated 
methods,  and  validated  with  the  known  values.  Noise  and  other 
irregularities  may  be  introduced  to  the  phantoms  to  simulate  actual 
images. 

Manual  editing  is  usually  assisted  by  some  software  tools. 
Figure  15.2  shows  the  catFISH  example  of  such  software  devel¬ 
oped  by  this  group.  After  manual  validation  obtained  by  consen¬ 
sus  of  several  independent  observers,  automated  segmentation 
results  are  further  corrected  using  the  convenient  tools,  such  as 
buttons  for  add/delete,  split/merge,  and  shrink/dilate  in  the 
catFISH  system,  as  shown  in  Figure  15.18.  Figure  15.19  illustrates 
the  common  editing  to  the  tube-like  object  segmentation.  Note 
that,  by  editing  the  automated  results,  the  subjectivity  factor  of 
manual  analysis  is  reduced  down  to  the  correspondence  between 
manual  and  automated  results. 


FIGURE  15.18.  Illustrating  common  image  analysis  errors  associated  with  blob  segmentation,  and  manual  editing  tools  to  correct  them:  (A)  Adding  a  missed 
object  using  a  region  growing  tool;  (B)  Splitting  connected  objects;  (C)  Correcting  a  boundary  location  error. 
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FIGURE  15.19.  Illustrating  the  manual  edits  to  the  tube-like  object  segmentation.  (A)  Original  image;  (B)  Automated  tracing  results;  (C)  Manually  modified 
traces  shown  in  light  blue;  (C)  Final  results. 


ANALYSIS  OF  MORPHOMETRIC  DATA 

The  statistical  analysis  of  morphometric  data  is  necessarily 
application-specific.  The  following  paragraphs  provide  two  exam¬ 
ples  illustrating  the  main  ideas. 

In  the  catFISH  system,  the  important  goal  is  to  classify  the 
nuclei  in  the  image  based  on  the  presence  of  the  intranuclear  and 
cytoplasmic  FISH  signal.  The  measurements  are  integrated  and 
associated  spatially  with  the  nuclear  segmentation  results  to  gen¬ 
erate  a  detailed  tabular/database  representing  the  location  and 
detailed  morphometric  parameters  for  each  nucleus,  and  the  asso¬ 
ciated  FISH  signals.  Nucleus  class  information  is  defined  by  the 
user  according  to  some  criteria  such  as  the  color  channels,  pres¬ 
ence  or  absence  of  signal,  and  location  of  the  signal.  The  software 
uses  this  information  to  automatically  classify  each  nucleus  as  neg¬ 
ative  or  as  containing  intranuclear  and/or  cytoplasmic  FISH  signal 
based  upon  its  morphometric  and  functional  (FISH)  measure¬ 


ments.  The  final  result  is  output  into  a  text  file  including  total 
nuclear  count  and  the  number  and  percentage  of  nuclei  in  each 
class. 

In  the  angiogenesis  example,  the  primary  intent  is  to  quanti¬ 
tate  temporal  vessel  changes  in  a  set  of  time-series  images.  There 
are  two  broad  methods  for  change  analysis.  One  method  is  to 
compute  morphometric  data  from  images  at  each  temporal  sam¬ 
pling  point,  and  to  perform  a  statistical  comparison  of  these  data. 
A  more  ambitious  approach  is  to  register  the  images  over  time  and 
extract  detailed  changes  on  a  vessel-segment  by  vessel-segment 
basis.  In  this  work,  the  less  ambitious  approach  was  adopted  as  a 
starting  point.  Vessel  lengths,  widths,  and  count  can  be  readily 
obtained  from  the  traces  generated  by  the  automatic  tracing  algo¬ 
rithm  described.  Naturally,  only  vasculature  segments  located  in 
the  volume  common  to  all  four  images  contributed  to  these  statis¬ 
tics.  An  overall  Change  Index  is  calculated  as  the  simple  ratio  of 
the  current  measurement  over  the  previous  measurement. 
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DISCUSSION,  CONCLUSION, 

AND  FUTURE  DIRECTIONS 

We  have  attempted  to  summarize  techniques  for  automated  3D 
image  analysis  by  organizing  the  methods  into  those  suitable  for 
a  small  number  of  categories  of  features  (blobs,  tubes,  clouds, 
mixed).  By  recognizing  image  objects  along  these  lines,  the  reader 
is  better  able  to  select  from  the  available  image  analysis  tools.  It 
has  also  served  as  the  basis  for  organizing  image  analysis  pack¬ 
ages  such  as  the  catFISH  system. 

The  life-sciences  researcher  is  often  able  to  perform  automated 
2D  image  analysis  using  off-the-shelf  software,  for  example,  NIH 
Image,  MetaMorph  by  Universal  Imaging,  and  VoxelView  by  Vital 
Images  (more  examples  discussed  in  Chapter  14,  this  volume).  The 
sheer  complexity  and  variability  of  biological  imagery  and  the 
current  state  of  evolution  of  automated  3D  image  analysis  tech¬ 
nology  imply  that  the  assistance  of  an  image  analysis  specialist  is 
often  called  for.  It  is  hoped  that  the  terminology  and  language  pre¬ 
sented  here  can  bridge  these  disciplines.  The  organization  of  image 
analysis  by  type  of  objects  also  forms  a  good  basis  for  facilitating 
productive  discussion  between  life-sciences  researchers  and  com¬ 
puter  scientists. 

Automated  image  analysis  is  a  dynamic  computer-intensive 
discipline.  It  has  benefited  tremendously  from  problems  posed  by 
life-science  researchers.  Much  work  remains  to  be  done.  For 
example,  the  algorithms  are  still  not  nearly  as  adaptive  as  the 
human  visual  system.  Given  a  novel  set  of  image  data,  the  human 
observer  can  be  trained  much  more  quickly  than  new  image  analy¬ 
sis  tools  can  be  designed  and  tested.  Nevertheless,  development  of 
new  tools  is  worthwhile  whenever  large  amounts  of  data  must 
be  processed,  for  instance,  in  the  development  of  high-content 
screening  assay  as  described  in  Chapter  46,  this  volume.  This 
chapter  has  accordingly  devoted  attention  to  studies  that  require 
this  type  of  sophisticated  automation. 
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INTRODUCTION 

Fluorescence  is  probably  the  most  important  optical  readout  mode 
in  biological  confocal  microscopy  because  it  can  be  much  more 
sensitive  and  specific  than  absorbance  or  reflectance,  and  because 
it  works  well  with  epi-illumination,  which  greatly  simplifies 
scanner  design.  These  advantages  of  fluorescence  are  critically 
dependent  on  suitable  fluorophores  that  can  be  tagged  onto  bio¬ 
logical  macromolecules  to  show  their  location,  or  whose  optical 
properties  are  sensitive  to  the  local  environment.  Despite  the 
pivotal  importance  of  good  fluorophores,  little  is  known  about  how 
rationally  to  design  good  ones.  Whereas  the  concept  of  confocal 
microscopy  is  only  a  few  decades  old  and  nearly  all  the  optical, 
electronic,  and  computer  components  to  support  it  have  been 
developed  or  redesigned  in  the  last  few  years,  the  most  popular 
fluorophores  were  developed  more  than  a  century  ago  (in  the  case 
of  fluoresceins  or  rhodamines)  or  several  billion  years  ago  [in  the 
case  of  phycobiliproteins  and  green  fluorescent  proteins  (GFPs)]. 
Moreover,  whereas  competition  between  commercial  makers  of 
confocal  microscopes  stimulates  ardent  efforts  to  refine  the  instru¬ 
mentation,  relatively  few  companies  or  academic  scientists  are 
interested  in  improving  fluorophores. 

PHOTOPHYSICAL  PROBLEMS  RELATED  TO 
HIGH  INTENSITY  EXCITATION 

Singlet  State  Saturation 

The  properties  of  current  fluorescent  probes  relevant  to  conven¬ 
tional  fluorescence  microscopy  have  been  reviewed  recently  (e.g., 
Waggoner  et  al.,  1989;  Tsien,  1989a,b;  Chen  and  Scott,  1985;  Sun 
et  al,  2004;  Johnson,  Chapter  17,  this  volume)  and  are  listed  in 
Table  16.1,  see  p.  344.  Absorption  spectra  of  several  representa¬ 
tive  fluorescent  probes  in  relation  to  the  common  laser  line  wave¬ 
lengths  available  for  confocal  laser  scanning  microscopes  are 
presented  in  Figure  16.1.  Confocal  microscopy  using  multiple 
apertures  scanned  across  an  image  plane,  that  is,  disk-scanning 
confocal  microscopy,  is  essentially  similar  to  conventional 
microscopy  in  its  requirements  on  fluorophores.  By  contrast,  con¬ 
tinuous  beam  (not  pulsed  as  in  multi-photon  excitation)  laser-scan¬ 
ning  microscopy  subjects  each  fluorescent  molecule  to  brief  but 
extremely  intense  bursts  of  excitation  as  the  focused  laser  beam 
sweeps  past.  If  the  laser-scanned  image  consists  of  n  pixels  (typi¬ 
cally  n  >  105),  any  one  pixel  is  illuminated  for  1  In  of  the  total  time, 


or  even  somewhat  less  if  some  of  the  cycle  time  must  be  devoted 
to  scan  retrace;  therefore,  the  peak  instantaneous  intensity  must 
equal  or  exceed  n  times  the  long-term  average  excitation  intensity. 

Some  idea  of  the  quantitative  magnitude  may  be  gathered  from 
the  following  example,  analogous  to  that  discussed  by  White  and 
Stryer  (1987).  If  just  1  mW  of  power  at  the  popular  488  nm  line  of 
the  argon-ion  laser  is  focused  to  a  Gaussian  spot  whose  radius  w  at 
1/e2  intensity  is  0.25  pm,  as  is  achieved  by  a  microscope  objective 
of  1.25  numerical  aperture  (Schneider  and  Webb,  1981),  the  peak 
excitation  intensity  I  at  the  center  will  be  10-3  W/[p  •  (0.25  X  10-4 
cm)2]  =  5.1  x  105W/cm2,  or  about  1.25  x  1024  photons/(cm2  -  s). 
Such  intensities  are  well  able  to  excite  fluorophores  so  rapidly  that 
few  molecules  are  left  in  the  ground  state  and  the  population  is 
emitting  photons  nearly  as  fast  as  the  limit  set  by  the  excited-state 
lifetime.  For  example,  if  fluorescein  is  the  fluorophore,  its  decadic 
extinction  coefficient  e  at  488  nm  is  about  80,000 L-mole_Icm_1  at 
pH  >  7.  To  convert  this  to  the  optical  cross-section  per  molecule, 
one  must  multiply  e  by  (1000cm3/L)  •  (In  10)/(6.023  x  1023 
molecules/mole)  =  3.82  X  10-21  cm3  ■  mole  •  L-1  ■  molecule-1,  giving 
a  cross-section  s  of  3.06  x  10-16cm2/molecule.  In  a  beam  of  1.25  X 
1024  photons  ■  cm'Y1,  each  ground  state  molecule  will  be  excited 
with  a  rate  constant  kd  =  al,  or  3.8  X  108s-1  in  this  example.  The 
excited  state  lifetime  xf  of  fluorescein  in  free  aqueous  solution  is 
known  to  be  about  4.5  ns  (Bailey  and  Rollefson,  1953),  which 
means  that  molecules  in  the  excited  state  return  to  the  ground  state 
with  a  rate  constant  kf  of  2.2  x  10s  s-1.  Note  that  k{  is  defined  here 
as  the  composite  rate  constant  for  all  means  of  depopulating  the 
singlet  excited  state,  the  sum  of  the  rate  constants  for  fluorescence 
emission,  radiationless  internal  conversion,  intersystem  crossing  to 
the  triplet,  etc. 

Because  of  the  Stokes  shift  between  excitation  and  emission 
As,  kf  is  not  significantly  enhanced  by  stimulated  emission  effects. 
If  x  is  the  fraction  of  molecules  in  the  excited  state  and  (1  -  x)  is 
the  fraction  in  the  ground  state,  at  steady-state  kfx  =  kj[l  —  x). 
Solving  for  x  yields  the  equation  x  =  kj(ka  +  kf),  which  shows  that 
in  this  example  63%  of  the  fluorescein  molecules  would  be  in  the 
excited  state  and  only  37%  in  the  ground  state.  Obviously  the  emis¬ 
sion  is  nearly  saturated,  and  further  increase  in  excitation  intensity 
could  hardly  increase  the  output.  The  actual  rate  of  photon  output 
per  molecule  is  QJqx  =  Qcktk.J(kd  +  kf),  where  Qc  is  the  emission 
quantum  efficiency,  about  0.9  for  free  fluorescein  dianion  (but 
usually  less  for  fluorescein  bound  to  proteins,  see  below). 

In  this  example  each  molecule  would  be  emitting  at  an  average 
rate  of  1.3  X  10s  photons/s,  close  to  the  absolute  maximum  of  Qck{ 
of  about  2  X  108  photons/s.  In  current  typical  scanning  confocal 
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FIGURE  1 6.1 .  Absorption  spectra  from  left  to  right  of  Hoechst  33342+DNA, 
and  Fluorescein,  CY3,  TRITC,  Lissamine  Rhodamine  B,  Texas  Red,  and  CY5 
conjugated  to  antibodies.  Extinction  coefficients  are  given  on  a  per-dye  basis. 
Common  laser  emission  A.S  are  presented  at  the  top  of  the  figure,  /.s  in  bold  are 
for  the  lower  power,  less  expensive  lasers,  which  provide  sufficient  excitation 
intensity  for  most  fluorochromes  imaged  with  laser-scanning  microscopes  with 
higher  power  objectives. 


microscopes,  the  beam  dwells  on  each  pixel  for  1  to  I  Ops,  so  one 
would  expect  each  molecule  to  produce  several  hundred  photons. 
However,  only  a  minority  enter  the  microscope  objective,  only  a 
fraction  of  these  manage  to  pass  all  the  way  through  the  scanner 
optics,  and  only  10%  to  20%  of  these  create  photoelectrons  in 
the  photomultiplier  cathode,  so  that  each  molecule  probably  con¬ 
tributes  on  the  order  of  only  one  photoelectron/pixel/sweep. 
Because  of  fluorescence  saturation,  increasing  the  laser  power  will 
not  significantly  increase  the  signal  amplitude.  In  reality,  it  is  dif¬ 
ficult  to  accurately  predict  the  laser  power  at  which  a  fluorophore 
will  saturate.  Accurate  knowledge  of  the  extinction  coefficient  at 
the  A,  of  excitation  and  the  excited  state  lifetime  of  the  fluorophore 
is  essential  but  not  always  easy  to  obtain.  Coumarins,  for  example 
(Table  16.1,  see  p.  344),  have  extinction  coefficients  approximately 
2  to  5  times  smaller  than  the  fluoresceins,  rhodamines,  and  cya¬ 
nines,  and  would  be  expected  on  this  basis  alone  to  require  a  2-  to 
5-fold  increase  in  laser  power  before  saturation.  However,  the 
emission  lifetimes  of  coumarins  are  intrinsically  longer  than  those 
of  the  fluoresceins,  rhodamines,  and  cyanines,  so  that  the  factor  of 
2  to  5  is  not  realized  in  practice.  To  make  calculations  more  diffi¬ 
cult.  the  extinction  coefficients  and  the  excited  state  lifetimes  of 
most  probes  depend  on  the  environment  of  the  fluorophore.  For 
example,  increasing  the  fluorochrome-to-protein  ratio  of  a  labeled 
antibody  from  2  to  5  can  reduce  the  average  excited  state  lifetime 
of  the  bound  fluorochromes  several-fold.  For  these  reasons,  the 
power  saturation  values  for  the  probes  listed  in  Table  16.1  are  not 
given.  However,  as  a  general  rule,  fluorochromes  with  extinction 
coefficients  and  quantum  efficiencies  similar  to  fluorescein  will 
also  saturate  under  similar  conditions. 


Triplet  State  Saturation 

The  above  calculation  considers  only  the  ground  state  and  lowest 
excited  singlet  state.  Saturation  of  emission  could  occur  at  even 
lower  excitation  intensities  if  a  significant  population  of  fluo¬ 
rophores  becomes  trapped  in  a  relatively  long-lived  triplet  state. 
This  would  take  place  if  a  significant  quantum  yield  exists  for 
singlet-to-triplet  conversion,  or  intersystem  crossing.  For  example, 
if  ground-state  fluorescein  molecules  each  absorb  3.8  X  10s 
photons  ■  s-1,  have  an  excited  singlet  lifetime  of  4.5  ns,  cross  to  the 
triplet  state  with  a  quantum  efficiency  Qisc  of  0.03  (Gandin  et  al. , 
1983)  and  reside  in  the  triplet  state  for  a  mean  time  xT  of  10-6s, 
the  triplet  state  would  contain  81%  of  the  fluorophore  population 
at  steady  state,  which  would  be  attained  with  a  time  constant  of 
about  [xT-1  +  QisckfkJikz  +  fcf)]-1  or  about  190ns  in  this  case.  Only 
12%  and  7%  would  be  left  in  the  first  excited  singlet  and  ground 
states,  respectively,  at  steady  state,  so  that  the  fluorescence  emis¬ 
sion  would  be  weakened  about  5-fold  compared  to  its  initial  value 
just  after  the  illumination  began  but  before  significant  triplet  occu¬ 
pancy  had  built  up.  Therefore,  if  the  dwell  time/pixel  is  compara¬ 
ble  to  or  greater  than  the  triplet  lifetime,  then  a  severe  reduction 
in  output  intensity  may  be  expected  beyond  that  due  simply  to  the 
finite  rate  of  emission  from  the  singlet  state. 

In  the  above  calculation,  the  most  uncertain  figure  is  that  for 
the  triplet  lifetime  xT;  in  very  thoroughly  deoxygenated  solution, 
XT  for  the  fluorescein  dianion  is  20  ms  (Lindqvist,  1960),  but 
oxygen  is  expected  to  shorten  xT  down  to  the  0.1  to  1  ps  range.  The 
rate  at  which  different  fluorophore  environments  in  a  sample 
quench  triplet  states  and  reduce  xT  will  of  course  affect  the  extent 
of  triplet-state  saturation  and  the  apparent  brightness  of  each  pixel 
at  these  high  illumination  levels.  When  xT  is  long,  triplet-state  sat¬ 
uration  is  easily  attained  even  without  laser  illumination  (Lewis 
et  al.,  1941). 


Contaminating  Background  Signals 
Rayleigh  and  Raman  Scattering 

Meanwhile,  there  may  be  unwanted  signals,  such  as  Rayleigh  scat¬ 
tering,  due  either  to  excitation  A,s  leaking  through  the  dichroic 
mirror  and  barrier  filter  or  to  imperfect  monochromaticity  of  the 
excitation  source,  for  example,  if  the  laser  is  being  run  in  multi-line 
mode  with  only  an  interference  filter  to  select  one  line.  Even  if 
all  the  A,  filtering  is  perfect,  Raman  scattering  will  contribute  a  flu¬ 
orescence-like  signal,  for  example,  at  a  A,  of  [A»exc-1  -  3380  cm-1]-1 
due  to  the  characteristic  3380cm-1  Raman  band  of  water.  For  an 
exciting  A,exc  of  488  nm,  the  Raman  peak  would  appear  at  584 nm. 
At  high  concentrations  of  protein  or  embedding  media,  additional 
Raman  bands  closer  to  the  excitation  wavelength  might  appear. 
Both  Rayleigh  and  Raman  scattering  are  directly  proportional  to  the 
laser  power  and  will  not  saturate  as  the  desired  fluorescence  does, 
so  that  excessive  laser  power  diminishes  the  contrast  between  flu¬ 
orescence  and  such  scattering  signals. 

Autofluorescence  from  Endogenous 
Fluorophores 

Another  major  source  of  unwanted  background  is  autofluorescence 
from  endogenous  fluorophores.  Flavins  and  flavoproteins  absorb 
strongly  at  488  nm  and  emit  in  the  same  spectral  region  as  fluores¬ 
cein.  Reduced  pyridine  nucleotides  (NADH,  NADPH)  and  lipo- 
fuscin  pigments  absorb  light  from  ultraviolet  (UV)  laser  lines. 
These  fluorophores  usually  have  lower  extinction  coefficients  or 
shorter  fluorescence  lifetimes  than  most  exogenous  fluorophores. 
For  example,  FMN  (flavin  mononucleotide)  and  FAD  (flavin 
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adenine  dinucleotide)  have  the  extinction  coefficients  of  1.1  to  1.2 
X  104M_1cm_1  at  445  to  450nm  (Koziol,  1971)  and  fluorescence 
lifetimes  of  about  4.9  and  3.4/0. 12ns,  respectively  (Lakowicz, 
1989),  whereas  NADH  has  an  extinction  coefficient  of  6.2  x  103 
1VT1  cm4  at  its  340  nm  peak  (Kaplan,  1960)  and  a  lifetime  of  about 
0.4ns  (Lakowicz,  1983).  Therefore  autofluorescence  from  these 
molecules  will  be  more  difficult  to  saturate  than  the  fluorescence 
from  most  of  the  common  probes.  See  Chapter  27,  this  volume,  for 
more  on  confocal  lifetime. 

What  Is  the  Optimal  Intensity? 

The  above  discussion  shows  that  if  laser  power  is  increased  to 
nearly  saturate  the  desired  fluorophores,  autofluorescence  as  well 
as  Rayleigh  and  Raman  scattering  will  increase  background  levels 
and  decrease  the  overall  signal-to-noise  ratio  (S/N).  What  is  the 
optimal  intensity?  Assuming  the  system  is  limited  by  photon¬ 
counting  statistics,  the  irreducible  noise  level  N  is  proportional  to 
the  square  root  of  the  background  signal  level  B.  Both  B  and  the 
absorption  rate  constant  k.d  of  the  desired  probe  are  directly  pro¬ 
portional  to  the  illumination  intensity  I  and  to  each  other.  There¬ 
fore  N  is  directly  proportional  to  kd2.  If  triplet  state  saturation  can 
be  ignored,  for  example,  if  the  dwell  time/pixel  is  short  compared 
to  the  time  for  the  triplet  state  to  build  up,  then  the  desired  signal 
is  QekfkJ(ka  +  kf )  as  derived  above,  so  that  S/N  is  proportional  to 
kd2/(kd  +  kf).  Regardless  of  the  proportionality  constant,  this  expres¬ 
sion  is  maximal  when  kd  =  k{.  This  is  a  remarkably  simple  but 
important  result,  for  which  we  are  grateful  to  Prof.  R.  Mathies 
(University  of  California-Berkeley). 

At  the  other  extreme,  once  the  ground  state  and  excited  singlet 
and  triplet  have  all  come  to  an  equilibrium  steady  state,  the  desired 
signal  is  readily  calculated  to  be 

+  kd  (kf  1  +  QiscU)] 
so  that  the  S/N  is  proportional  to 

Qskf/[1  +  kd  (kj  +  eIScXT)]. 

This  reaches  its  maximum  when  ka(Tf  +  giscT-r)  =  1,  where  Tf 
=  kf1  is  the  lifetime  of  the  excited  singlet.  In  this  approximation, 
appropriate  for  slow  scans  in  which  the  dwell  time/pixel  is  long 
compared  to  the  time  for  triplet  state  equilibration,  the  laser  power 
P  in  photons/s  should  be  optimal  at  about  jw2/[(3.82  x  10_21cm3  • 
M)  e  (Tf  +  QiscTt)]-  For  the  present  values  of  w  =  2.5  x  10_5cm,  e 
=  8  x  104M_1cm_1,  Tf  =  4.5  x  10_9sec,  gisc  =  0.03,  and  xT  =  10_6s, 
this  expression  gives  an  optimal  P  of  1 .86  x  1014  photons/s,  or  about 
76|TW  at  488  nm.  If  triplet  formation  could  be  neglected,  the 
optimal  P  would  be  590  |4W,  slightly  less  than  the  1  mW  initially 
postulated  to  be  the  input. 

PHOTODESTRUCTION  OF  FLUOROPHORES 
AND  BIOLOGICAL  SPECIMENS 

One  obvious  way  to  increase  the  total  signal  is  to  integrate  for  a 
longer  time,  either  by  slowing  the  scan,  or  by  averaging  for  many 
scans.  Given  that  image  processors  are  now  relatively  cheap,  the 
latter  alternative  is  likely  to  be  the  easier  to  implement,  and  it  has 
at  least  two  major  advantages:  repetitive  scans  give  time  for  triplet 
states  to  decay  between  each  scan,  and  the  user  can  watch  the  S/N 
gradually  improve  and  choose  when  to  stop  accumulating. 
However,  irreversible  photochemical  side  effects  such  as  bleach¬ 
ing  of  the  fluorophore  or  damage  to  the  specimen  set  limits  on 
the  useful  duration  of  observation  or  total  photon  dose  allowable. 


Photochemical  damage  is  one  of  the  most  important  yet  least 
understood  aspects  of  the  use  of  fluorescence  in  biology;  in  this 
discussion  we  can  do  little  more  than  define  our  ignorance. 

At  intensities  of  up  to  3  x  1022  (Hirschfeld,  1976)  or  4.4  x 
1023  photons  •  cm4  •  s_1  (Mathies  and  Stryer,  1986),  fluorescein  is 
known  to  bleach  with  a  quantum  efficiency  Qb  of  about  3  x  10-5. 
If  this  value  continues  to  hold  at  the  somewhat  higher  intensity  of 
the  above  example,  the  molecules  would  be  bleaching  with  a  rate 
constant  of  QbktkJ(kd  +  kf),  or  about  4.2  x  103  s_1.  This  would  mean 
that  1/e  or  37%  of  the  molecules  would  be  left  after  240  |is  of  actual 
irradiation.  The  corresponding  number  of  scans  would  be  240  |is 
divided  by  the  dwell  time  that  the  beam  actually  spends  on  each 
pixel.  The  average  number  of  photons  emitted  by  a  fluorophore 
before  it  bleaches  is  the  ratio  of  emission  quantum  efficiency  to 
bleaching  quantum  efficiency,  or  QJQb,  this  is  generally  true 
regardless  of  whether  the  illumination  is  steady  or  pulsed. 
For  fluorescein  under  ideal  conditions,  the  above  QJQb  works  out 
to  30,000  to  40,000  photons  ultimately  emitted/dye  molecule 
(Hirschfeld,  1976;  Mathies  and  Stryer,  1986).  The  number  of 
photons  detected  from  each  molecule  will,  of  course,  be  consider¬ 
ably  less. 

Thus,  in  obtaining  an  image  at  a  single  plane  by  averaging  say 
16  scans,  one  would  expect  from  6%  to  50%  bleaching  of  a  fluo¬ 
rescein  signal  (in  the  absence  of  antifade  reagents),  resulting  from 
exp[-(16  sweeps)  X  (1-lOp.s  dwell  time/sweep)/(240p,s  lifetime)]. 
In  an  optical  sectioning  experiment,  the  cone  of  light  illuminating 
the  sample  above  and  below  each  plane  of  data  being  acquired  is 
causing  photobleaching  even  though  under  confocal  conditions  no 
signal  is  being  recorded  from  these  regions.  This  means  that  if  16 
optical  sections  are  obtained  with  only  one  sweep  each,  the  last 
image  will  have  been  bleached  10%  to  50%  by  the  preceding 
sweeps.  If  the  signals  are  large,  it  may  be  possible  for  software  to 
adjust  the  intensities  of  sequential  images  to  compensate  for 
bleaching.  Of  course,  for  quantitative  fluorescence  measurements 
it  would  be  most  desirable  to  use  the  most  bleach-resistant  fluo¬ 
rophores  available. 

Dependency  on  Intensity  or  Its  Time  Integral? 
Theory 

One  major  uncertainty  is  whether  the  bleaching  quantum  efficiency 
Qb  really  does  remain  constant  even  at  such  high  instantaneous 
excitation  intensities.  Theoretically,  Qb  could  rise  at  high  intensi¬ 
ties  due  to  multi-photon  absorptions.  For  example,  the  normal 
excited  singlet  or  triplet  state  could  itself  absorb  one  or  more  addi¬ 
tional  photons  to  extra-high  energy  states,  which  will  probably 
have  picosecond  lifetimes.  If  their  main  reaction  pathway  were 
back  to  the  lowest  excited  state,  then  such  higher-order  states 
would  be  innocuous,  but  if  bond  dissociation  competes  with  decay 
to  the  lowest  excited  state,  then  photodestruction  will  rise  steeply 
as  intensities  reach  levels  that  significantly  deplete  the  ground  state 
(as  in  the  previous  example). 

One  can  also  imagine  the  opposite  dependency:  bleaching 
mechanisms  whose  quantum  efficiency  might  decrease  when  the 
excitation  was  bunched  into  brief  intense  pulses.  For  example, 
suppose  the  dye  is  bleached  by  a  1 : 1  reaction  of  its  excited  state 
with  a  molecule  of  oxygen,  and  that  the  overall  dye  concentration 
exceeds  the  oxygen  concentration.  Then  within  the  zone  of  intense 
illumination,  the  first  few  excited  dye  molecules  might  react  with 
all  the  locally  available  02,  and  the  resulting  anoxic  environment 
would  protect  the  rest  of  the  excited  molecules.  Oxygen  would  be 
diffusing  in  from  the  surrounding  non-illuminated  environment,  but 
the  time  required  would  be  on  the  order  of  the  spot  radius  squared 
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divided  by  the  diffusion  constant,  that  is,  (0.25  x  10^cm)2/(3  X 
10-5cm2  ■  s_1)  or  20|is,  that  is,  considerably  longer  than  the  time  for 
the  beam  to  move  to  the  next  pixel.  By  contrast,  with  low-intensity 
illumination,  no  local  anoxia  would  develop,  and  each  fluorophore 
would  take  its  chances  with  the  full  ambient  oxygen  concentration. 
Such  a  mechanism  would  be  the  photochemical  equivalent  of 
predator-prey  interactions  in  ecology,  where  it  is  often  advanta¬ 
geous  for  the  prey  to  school  together  or  breed  in  synchrony  in  order 
to  minimize  losses  to  predation  (Wilson,  1975). 

Experiment 

Theory  is  all  well  and  good,  but  empirically  how  does  the  photo¬ 
destruction  quantum  yield  depend  on  intensity?  White  and  Stryer 
(1987)  tested  R-phycoerythrin  in  a  flow  system  and  found  that  the 
rate  of  photodestruction  was  indeed  directly  proportional  to  laser 
power  so  that  the  quantum  yield  was  constant.  Unfortunately,  the 
range  of  intensities  tested  only  went  up  to  about  1020  photons  ■ 
cm_2-s_1,  so  that  they  were  still  3  orders  of  magnitude  below 
saturation  of  the  phycoerythrin.  Peck  and  colleagues  (1989)  exam¬ 
ined  B-phycoerythrin  and  found  that  photodestruction  saturated 
with  increasing  input  intensity  in  just  the  same  way  as  fluorescence 
emission,  so  that  the  photodestruction  quantum  yield  was  roughly 
constant  even  when  the  phycobiliproteins  were  heavily  driven  into 
saturation.  However,  there  was  some  indirect  evidence  that  simpler 
fluorophores  may  undergo  a  nonlinear  acceleration  of  bleaching 
under  such  conditions.  White  et  al.  (1987)  reported  that  bleaching 
by  their  scanning  confocal  microscope  seemed  to  be  greatest  at  the 
plane  of  focus.  Because  the  plane  of  focus  receives  about  the  same 
time-averaged  photon  flux  but  much  higher  peak  intensities  than 
out-of-focus  planes  do,  preferential  bleaching  at  the  plane  of  focus 
would  imply  that  a  given  number  of  photons  are  more  injurious 
when  they  are  bunched,  that  is,  that  the  photodestruction  quantum 
yield  increases  at  high  intensities.  Obviously,  further  testing  of  this 
possibility  and  improvement  in  photon  collection  efficiency  will 
be  of  great  importance  in  confocal  microscopy. 

A  number  of  workers  have  reported  that  damage  to  biological 
structures  (as  distinct  from  bleaching  of  the  fluorophore)  can  some¬ 
times  be  reduced  if  the  given  total  number  of  photons  is  delivered 
with  high  intensities  for  short  times  rather  than  low  intensities  for 
long  times.  An  early  report  of  the  advantage  of  pulsed  illumina¬ 
tion  was  by  Sheetz  and  Koppel  (1979)  studying  the  crosslinkage 
of  spectrin  by  illumination  of  fluoresceinated  concanavalin  A  on 
erythrocyte  membranes.  Bloom  and  Webb  (1984)  found  similar 
results  for  lysis  of  XRITC-labeled  air-equilibrated  erythrocytes, 
whereas  well-deoxygenated  cells  were  much  more  resistant  but 
lysed  after  a  constant  total  photon  dose  regardless  of  whether 
delivered  quickly  or  slowly.  Recently  Vigers  and  colleagues  (1988) 
showed  that  increasing  the  illumination  intensity  up  to  about 
103W/cm2  decreased  the  time  required  for  fluorescein-labeled 
microtubules  to  dissolve,  as  one  might  expect.  Surprisingly,  inten¬ 
sities  above  this  threshold  actually  stabilized  the  microtubules 
against  dissolution,  so  that  the  dissolution  time  became  a  linearly 
increasing  function  of  intensity.  This  paradoxical  stabilization  at 
high  intensities  was  attributed  to  local  heating  based  on  the 
assumption  that  diffusion  of  heat  was  negligible.  Because  this 
assumption  needs  to  be  checked  (see  Axelrod,  1977;  Bloom  and 
Webb,  1984)  and  because  microtubule  stabilization  and  dissolu¬ 
tion  are  not  well-defined  molecular  events,  the  local  heating 
hypothesis  should  be  viewed  with  caution.  Bonhoeffer  and  Staiger 
(1988)  have  reported  that  photodynamic  damage  to  rat  hippocam¬ 
pal  cells  was  reduced  if  the  light  is  delivered  at  100  exposures 
each  200  ms  long  separated  by  30  s  dark  periods  rather  than  con¬ 
tinuously  for  20  s.  They  speculated  that  intermittent  illumination 


was  better  because  it  allowed  repair  mechanisms  to  operate  during 
the  dark  intervals.  It  should  be  noted  that  in  all  the  above  biolog¬ 
ical  examples,  the  illumination  intensity  was  well  below  that 
expected  to  be  necessary  to  reach  saturation  of  excited  state  dye 
populations  (see  also  Chapter  38,  this  volume). 

STRATEGIES  FOR  SIGNAL  OPTIMIZATION  IN 
THE  FACE  OF  PHOTOBLEACHING 

Light  Collection  Efficiency 

The  above  discussion  has  shown  that  to  increase  signal  amplitude 
and  S/N  in  laser-scanning  confocal  microscopy,  increasing  laser 
power  helps  only  until  the  onset  of  saturation,  and  increasing 
observation  time  is  limited  by  photodestruction.  What  other  mea¬ 
sures  can  be  tried?  Obviously  any  increase  in  light-collection 
efficiency  (i.e.,  higher  numerical  aperture  of  the  objective), 
transmission  efficiency  through  the  scanner  and  A  filters,  and 
quantum  efficiency  of  photodetection  is  extremely  valuable. 
Despite  the  importance  of  these  factors,  newcomers  to  low-light- 
level  microscopy  often  use  low  numerical  aperture  (NA)  objec¬ 
tives,  excessively  narrow  emission  bandpass  filters,  inefficient 
optical  couplings,  and  photomultipliers  of  less-than-optimal 
quantum  efficiencies.  Nearly  all  the  fluorophores  that  fluoresce 
strongly  in  aqueous  solution  with  visible  As  of  excitation  are  char¬ 
acterized  by  small  Stokes  shifts,  or  difference  between  absorption 
and  emission  peak  As.  It  may  then  be  difficult  to  find  or  fabricate 
filters  and  dichroic  mirrors  that  efficiently  separate  the  two  A 
bands.  In  that  case,  it  would  usually  be  preferable  to  displace  the 
excitation  A  to  shorter  As  away  from  the  peak  of  the  excitation 
spectrum,  so  that  the  emission  filters  can  accept  as  much  of  the 
entire  output  as  possible.  Although  the  excitation  is  less  efficient, 
this  can  be  made  up  by  increased  laser  power  as  long  as  the  pho- 
tobleaching  is  reduced  by  the  same  factor.  By  contrast,  if  the  exci¬ 
tation  is  at  the  peak  A  and  the  emission  acceptance  band  is  pushed 
to  longer  As  that  exclude  much  of  the  emission  spectrum,  then 
emitted  photons  are  wasted,  while  excess  scattered  photons  are 
collected.  The  S/N  ratio  is  thus  lowered.  This  is  often  a  severe 
problem  with  rhodamine  excited  using  the  514nm  line  of  the 
argon-ion  laser. 

Spatial  Resolution 

Another  tactic  to  increase  signal  is  to  increase  the  effective  size  of 
the  confocal  apertures,  that  is,  decrease  the  spatial  resolution.  If 
the  illuminating  and  detecting  aperture  diameters  are  doubled,  the 
pixel  area  quadruples  and  the  volume  sampled  will  increase  8-fold. 
Assuming  the  total  laser  power  is  increased  to  maintain  the  same 
intensity  in  photons  -cnT'-s-1  and  that  the  fluorophore  concentra¬ 
tion  is  uniform  in  the  increased  volume  (as  might  be  true  for  an 
ion  indicator  distributed  in  the  cytosol),  the  signal  should  increase 
8-fold,  though  at  the  price  of  degraded  spatial  resolution. 

Protective  Agents 

As  mentioned  above,  light-induced  damage  to  both  the  fluorophore 
and  to  the  biological  specimen  is  often  dependent  on  the  presence 
of  molecular  oxygen,  which  reacts  with  the  triplet  excited  states 
of  many  dyes  to  produce  highly  reactive  singlet  oxygen.  Reduc¬ 
tion  of  the  partial  pressure  or  concentration  of  oxygen  often  greatly 
increases  the  longevity  of  both  the  fluorophore  and  the  specimen. 
In  dead,  fixed  samples,  it  has  become  common  to  add  antioxidants 
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such  as  propyl  gallate  (Giloh  and  Sedat,  1982),  hydroquinone,  p- 
phenylenediamine,  etc.,  to  the  mounting  medium.  The  preserva¬ 
tive  effects  of  these  agents  may  go  beyond  removing  oxygen 
because  White  and  Stryer  (1987)  found  propyl  gallate  to  be  more 
effective  than  thorough  deoxygenation  at  protecting  phycoerythrin 
in  vitro.  One  might  speculate  that  polyphenols  like  propyl  gallate 
might  quench  dye  triplet  states  and  other  free  radicals,  which  could 
prevent  forms  of  photodegradation  other  than  singlet  oxygen  for¬ 
mation.  Protection  of  GFP  and  fluorophores  from  photobleaching 
in  fixed  cells  has  been  discussed  recently  by  Bernas  and  colleagues 
(2004). 

The  problem  of  protecting  living  cells  from  oxygen-dependent 
photodynamic  damage  is  more  difficult.  The  above  antioxidants 
would  not  be  attractive  because  they  would  be  expected  to  have 
strong  pharmacological  effects  at  the  high  concentrations  gener¬ 
ally  employed  on  fixed  tissue.  If  the  tissue  can  tolerate  hypoxia  or 
anoxia,  one  would  probably  prefer  to  remove  02  by  bubbling  the 
medium  with  N2  or  Ar  rather  than  using  chemical  reductants.  Bio¬ 
logical  oxygen-scavenging  systems  such  as  submitochondrial 
particles  or  glucose  oxidase  +  glucose  are  often  helpful  (Bloom 
and  Webb,  1984). 

If  one  cannot  reduce  the  02  concentration,  the  next  best  tactic 
may  be  to  use  singlet  oxygen  quenchers.  The  most  attractive  here 
are  those  already  chosen  by  natural  selection,  namely  carotenoids. 
Their  effectiveness  is  shown  by  classic  experiments  in  which 
carotenoid  biosynthesis  was  blocked  by  mutation;  the  resulting 
mutants  were  rapidly  killed  by  normal  illumination  levels  at  which 
the  wild  type  thrived  (Matthews  and  Sistrom,  1959).  A  water- 
soluble  carotenoid  would  be  easier  to  administer  acutely  than  the 
usual  extremely  hydrophobic  carotenoids  such  as  carotene  itself. 
The  most  accessible  and  promising  candidate  is  crocetin,  which  is 
the  chromophore  that  gives  saffron  its  color,  and  which  consists  of 
seven  conjugated  C  =  C  units  with  a  carboxylate  at  each  end.  Cro¬ 
cetin  quenches  aqueous  singlet  oxygen  with  a  bimolecular  rate 
constant  of  5.5  x  109M_I-s_1,  which  is  almost  diffusion  controlled; 
of  this  rate,  about  95%  represents  catalytic  quenching  and  about 
5%  represents  bleaching  or  consumption  of  the  crocetin  (Manitto 
et  al.,  1987;  somewhat  more  pessimistic  rate  constants  are  reported 
by  Matheson  and  Rodgers,  1982).  Longer-chain  carotenoids  are 
supposed  to  be  even  more  efficient  at  destroying  singlet  oxygen 
without  damage  to  themselves,  but  have  not  yet  been  tested  in 
aqueous  media.  There  is  some  evidence  that  50  p.M  of  either 
crocetin  or  etretinate  (a  synthetic  aromatic  retinoid)  can  protect 
cultured  cells  (L  cells  and  WI-38  fibroblasts)  from  hematopor- 
phyrin-induced  photodynamic  damage  (Reyftmann  et  al.,  1986). 
Other  water-soluble  agents  that  might  be  considered  as  sacrificial 
reactants  with  reactive  oxygen  metabolites  include  ascorbate  (e.g., 
Vigers  et  al,  1988),  imidazole,  histidine,  cysteamine,  reduced 
glutathione  (Sheetz  and  Koppel,  1979),  uric  acid,  and  Trolox,  a 
vitamin  E  analog  (Glazer,  1988);  these  would  have  the  advantage 
over  carotenoids  of  being  colorless  and  non-fluorescent,  but  would 
have  to  be  used  in  much  higher  concentrations  (probably  many 
millimolar)  because  their  bimolecular  reaction  rates  with  oxygen 
metabolites  are  not  as  high  and  they  are  consumed  by  the  reaction 
rather  than  being  catalytic.  It  should  also  be  remembered  that  thor¬ 
ough  deoxygenation  may  increase  the  triplet  state  lifetime,  xT  and 
worsen  the  problem  of  excited-state  saturation. 

Fluorophore  Concentration 

Increasing  the  concentration  of  fluorophore  molecules  will  only 
increase  the  signal  as  long  as  they  do  not  get  too  close  together. 
When  multiple  fluorophores  are  attached  within  a  few  nanometers 


of  each  other  on  a  macromolecule,  they  usually  begin  to  quench 
each  other.  For  example,  the  relatively  high  quantum  yield  for  free 
fluorescein  in  aqueous  solution  at  pH  7  is  reduced  to  near  0.25 
when  an  average  of  5  fluorophores  are  bound  to  each  IgG  antibody 
(South wick  et  al.,  1990).  Charge-transfer  interactions  with  tryp¬ 
tophanes  are  yet  another  mechanism  for  quenching  fluoresceins 
bound  to  a  protein,  for  example,  anti-fluorescein  antibody  (Watt 
and  Voss,  1977).  Most  rhodamines  and  certain  cyanines  also  show 
a  striking  reduction  in  average  quantum  yield  on  conjugation, 
which  appears  to  arise  from  dye  interactions  on  the  protein  surface. 
Absorption  spectra  of  labeled  antibodies  clearly  show  evidence  of 
dimers,  and  fluorescence  excitation  spectra  demonstrate  that  these 
dimers  are  not  fluorescent.  The  propensity  of  relatively  nonpolar, 
planar  rhodamines  to  interact  with  one  another  is  not  surprising. 
Even  if  the  fluorophores  do  not  form  ground-state  dimers,  they  can 
also  rapidly  transfer  energy  from  one  to  another  until  a  quencher 
such  as  02  is  encountered.  In  other  words,  proximity-induced 
energy  transfer  between  fluorophores  multiplies  the  efficacy  of 
quenchers.  Perhaps  fortunately  it  also  shortens  the  excited-state 
lifetime,  so  that  a  higher  intensity  of  laser  excitation  can  be  applied 
for  a  given  degree  of  saturation  (Hirschfeld,  1976).  If  such 
increased  intensity  is  available,  much  of  the  emission  intensity  lost 
by  fluorophore  proximity  can  be  regained,  but  at  the  cost  of 
increased  background  signal  from  Rayleigh  and  Raman  scattering 
and  other  non-saturated  fluorophores. 

Choice  of  Fluorophore 

Perhaps  the  most  drastic  alteration  is  to  change  to  a  different  flu¬ 
orophore  altogether.  Unfortunately,  there  is  not  a  wide  selection  of 
fluorescent  physiological  indicator  probes  that  respond  selectively 
to  a  cellular  parameter  and  can  be  excited  at  appropriate  A.s. 
However,  with  fluorescent  labeling  reagents,  it  is  sometimes  pos¬ 
sible  to  choose  an  optimal  fluorophore  within  a  defined  A  range. 
A  selection  of  fluorescent  labels  for  confocal  microscopy  is 
described  below. 


FLUORESCENT  LABELS  FOR  ANTIBODIES, 
OTHER  PROTEINS,  AND  DNA  PROBES 

Fluorescent  Organic  Dyes 

Certain  fluorescent  reagents  carry  reactive  groups  for  covalent 
attachment  to  target  biomolecules  and  show  minimal  spectral  sen¬ 
sitivity  to  environmental  changes  (Table  16.1).  These  reagents  are 
primarily  used  to  quantify  the  presence  and  distribution  of  probes 
and  targets.  The  most  well-known  dyes  of  this  type  are  derivatives 
of  fluorescein  and  rhodamine  (e.g.,  tetramethyl  rhodamine,  lis- 
samine  rhodamine,  and  sulforhodamine  101).  A  variety  of  reactive 
groups  have  been  incorporated  into  these  dyes  permitting  coupling 
of  the  dyes  to  different  functionalities.  Isothiocyanates,  succin- 
imidyl  esters,  and  pentafluorophenyl  esters  couple  with  amino 
groups  of  target  molecules,  while  haloacetamides,  maleimides,  and 
vinyl  sulfones  react  with  sulfhydryl  groups.  Dichlorotriazinyl 
(DCT)  groups  can  couple  effectively  with  both  amines  and  alco¬ 
hols,  depending  upon  reaction  pH  and  temperature.  Reactivity  and 
selectivity  of  the  conjugation  reactions  can  be  manipulated  by  the 
reactive  groups  chosen  and  the  conditions  of  the  coupling  reaction 
(e.g.,  reaction  pH,  solvent  polarity,  and  temperature). 

The  cyanine  dyes  (Mujumdar  et  al.,  1993),  the  borate- 
dipyrromethene  (BODIPY)  complexes  (Wories  et  al,  1985; 
Kang  et  al.,  1988),  and  the  AlexaFluor  dyes  (Panchuk- Voloshina 
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et  al.,  1999)  have  been  developed  to  complement  the  traditional 
fluorescein  and  rhodamine  reagents.  The  sensitivity  of  detecting 
fluorescent  conjugates  is  determined  by  the  spectral  properties  of 
the  fluorescent  dye  (its  molar  extinction  coefficient  and  fluores¬ 
cence  quantum  yield)  and  the  quality  of  the  dye-target  conjugate 
(i.e.,  its  tendency  to  precipitate  and  its  degree  of  fluorescence 
quenching).  The  cyanine  reagents,  and  later  the  AlexaFluor  dyes, 
were  engineered  to  have  very  high  water  solubility  as  well  as 
excellent  spectral  properties.  Galbraith  and  colleagues  (1989)  and 
Mujumdar  and  colleagues  (1993)  have  shown  that,  at  least  in  the 
case  of  cyanine  dye  labeling  agents,  appropriate  placement  of 
charged  sulfonate  groups  on  the  fluorophore  can  reduce  dye  inter¬ 
actions  and  increase  the  brightness  of  relatively  heavily  labeled 
antibodies.  Particularly  useful  in  this  regard  are  the  indopentame- 
thine-cyanines,  CY5  dyes  (excitation,  630-650  nm;  emission 
670  nm)  and  indotrimethine-cyanines,  CY3  dyes  (excitation, 
530-550nm;  emission  575  nm).  Antibodies  labeled  with  these  dyes 
have  a  brightness  comparable  to  or  brighter  than  fluorescein- 
labeled  antibodies  and  have  little  tendency  to  precipitate  from  solu¬ 
tion  even  when  labeled  with  as  many  as  10  dye  molecules/antibody 
(Wessendorf  and  Brelje,  1992).  CY5  is  somewhat  more  photo¬ 
stable  than  fluorescein,  and  CY3  is  significantly  more  stable.  CY5 
can  be  optimally  excited  with  a  red  He/Ne  laser,  whereas  CY3  can 
be  excited  fairly  efficiently  with  the  514nm  line  and  marginally 
well  with  the  488  nm  line  of  the  argon-ion  laser.  These  fluo¬ 
rophores  are  useful  for  nucleic  acid  labeling  and  have  found  wide 
use  in  in  situ  hybridization  and  gene  expression  assays.  Fluores¬ 
cent  reagents,  structurally  related  to  the  cyanines,  with  a  squaric 
acid  group  replacing  part  of  the  polymethine  chain  have  been 
described  (Oswald  et  al.,  1999).  Also,  novel  polymethine  dyes 
developed  by  Czerney’s  group  were  introduced  recently  as  fluo¬ 
rescent  reagents  (Czerney  et  al.,  2001).  These  new  fluorescent 
reagents  in  combination  with  fluorescein,  the  phycobiliproteins,  or 
other  fluorophores  make  it  possible  to  do  multi-color  fluorescence 
imaging  with  laser-scanner  microscopes  equipped  with  an  argon 
and  a  He/Ne  laser  or  with  the  argon/krypton  “white  light”  laser, 
which  has  lines  at  488,  568,  and  647  nm.  Brelje  and  colleagues 
(1993)  have  described  the  use  of  the  Ar/Kr  laser  to  excite  samples 
stained  with  fluorescein  (488  excitation);  Texas  Red,  Lissamine, 
or  CY3  (568 nm);  and  CY5  (647).  Fluorescent  labeling  reagents 
available  commercially  are  described  on  the  following  Web  sites: 
http://www.amershambiosciences.com;  http://www.probes.com; 
http://www.dyomics.com;  and  http://www.mobitec.de. 

Phycobiliproteins 

Phycobiliproteins  (Oi  et  al.,  1982)  currently  hold  the  record  for  the 
highest  extinction  coefficients  and  largest  number  of  photons 
emitted  before  bleaching  ( QJQb ;  Mathies  and  Stryer,  1986),  partly 
because  each  macromolecule  simply  contains  a  large  number  of 
component  fluorophores,  partly  because  the  proteins  have  been 
engineered  by  natural  selection  to  protect  the  tetrapyrrole  fluo¬ 
rophores  from  quenching  processes  (Glazer,  1989;  Sun  et  al., 
2003).  Suitable  optimization  of  laser  power,  optics,  flow  rate,  and 
detection  permits  the  detection  of  fluorescence  pulses  from  single 
phycobiliprotein  molecules  in  flowing  systems  (Peck  et  al. ,  1989; 
see  also  Nguyen  et  al,  1987).  Phycoerythrin  (PE)  in  combination 
with  fluorescein  has  been  valuable  for  immunofluorescence  deter¬ 
mination  of  cell-surface  markers  by  single  laser  (488  nm  excitation) 
flow  cytometry.  This  approach  can  also  be  useful  in  confocal 
microscopy,  provided  that  the  emission  signal  is  split  into  a  530 nm 
(fluorescein)  and  a  575  nm  component  (PE)  and  two  photomulti¬ 
pliers  are  used  for  detection.  Detection  of  a  third  color  is  possible 


using  another  photosynthetic  protein,  PerCP  (Rechtenwald,  1989), 
or  the  PE  tandem  conjugates  formed  by  coupling  energy  acceptor 
fluorophores  that  emit  at  long  Ks  to  PE.  PE  conjugates  with  Texas 
Red  or  to  CY5  (Waggoner  et  al,  1993)  have  become  popular  for 
cell  surface  measurements  with  flow  cytometers  using  a  488  nm 
laser.  Phycobiliproteins  have  also  been  coupled  with  CY7  dyes 
providing  additional  choices  of  fluorescence  colors  (Gerstner  et  al., 
2002;  Roederer  et  al.,  1996;  Beavis  and  Pennline,  1996).  The  use 
of  phycobiliproteins  is  likely  to  prove  less  useful  for  intracellular 
antigens  because  the  size  of  a  PE-antibody  conjugate,  around 
410kD,  restricts  penetration  into  denser  regions  of  fixed  cells 
and  tissues.  Efforts  to  develop  low-molecular- weight  analogs  of  PE 
that  have  similarly  large  Stokes  shifts  and  excitation  Xs  have  not 
yet  been  successful,  but  work  in  this  area  continues.  If  fluorophores 
with  large  Stokes  shifts  could  be  found  for  both  488  nm  and 
633  nm  excitation,  a  two-laser  microscope  could  obtain  four- 
color  immunofluorescence  images.  Other  probes  are  listed  in 
Table  16.1. 

DNA  Probes 

An  application  of  fluorescent  labels  that  has  attracted  a  number  of 
investigators  is  fluorescence  in  situ  hybridization,  or  FISH  (Trask, 
1991).  Initially,  DNA  to  be  used  to  probe  genetic  sequences  in  chro¬ 
mosomes  and  interphase  nuclei  was  labeled  by  nick  translation 
using  biotin  or  digoxigenin-tagged  deoxynucleotide  triphosphates 
(dNTPs).  Fluorescent  secondary  reagents  were  applied  after 
hybridization  to  detect  the  binding  of  the  DNA  probe.  The  use  of 
dNTPs  attached  by  linker  arms  to  fluorophores  to  form  directly 
labeled  fluorescent  DNA  probes  is  replacing  methods  involving 
fluorescent  secondary  reagents.  Directly  labeled  DNA  probes  are 
simpler  to  use,  often  give  less  background  and  provide  easier  access 
to  multi-color-multi-sequence  detection  (Ballard  and  Ward,  1993). 
However,  for  detection  of  short  genetic  sequences,  use  of  multiple 
fluorescent  secondary  reagents  may  be  required  for  sufficient 
sensitivity.  Directly  labeled  oligonucleotides  provide  alternatives 
when  there  are  numerous  copies  of  the  target  sequence,  as  in  the 
case  of  histone  mRNA  (Yu  etal.,  1992).  For  non-specific,  but  stable 
labeling  of  DNA,  the  bis-intercalating  reagents,  TOTO,  YOYO, 
DRAQ5,  etc.,  provide  an  attractive  solution,  and  the  reagents  are 
available  in  several  fluorescent  colors  (Rye  et  al.,  1992).  The 
anthraquinone  derivative,  DRAQ5,  is  membrane  permeant,  is 
highly  selective  for  nuclear  DNA,  and  can  be  used  with  two-photon 
excitation  from  800  nm  to  beyond  lOOOnm.  Studies  of  living  cells 
with  DRAQ5  must  be  done  with  care  because  of  its  high  cytotoxic¬ 
ity  (Smith  et  al.,  2000;  Errington  et  al.,  2005).  Snyder  (2003)  sug¬ 
gests  that  some  caution  may  be  needed  when  combining  DRAQ5 
with  other  probes.  There  appears  to  be  decreased  uptake  of  bodipy- 
labelled  compounds  in  the  presence  of  the  nuclear  stain,  DRAQ5. 

Luminescent  Nanocrystals 

Fluorescent  nanocrystals,  or  quantum  dots,  exhibit  interesting 
properties  for  biological  labeling  reagents.  These  nanometer-sized 
inorganic  crystalloid  structures  have  broad  absorption  (and  excita¬ 
tion)  spectra  with  molar  absorptivities  of  more  than  six  million  at 
450  nm.  The  emission  band,  determined  by  particle  composition  and 
dimensions,  can  be  very  narrow  (ca.  25  nm  FWHM)  with  quantum 
yields  approaching  unity.  Also,  quantum  dot  luminescence  is  very 
resistant  to  photobleaching.  Selection  of  uniformly  sized  quantum 
dots  gives  preparations  with  very  sharp  fluorescence  bands.  The 
challenge  for  biological  applications  is  to  provide  biocompatible 
surfaces  for  the  nanocrystals  that  maintain  their  fluorescence  in 
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TABLE  16.1.  Spectroscopic  Properties  of  Selected  Probes 

Absorption 

Extinction 

Emission 

Quantum 

Measurement 

Parameter 

Probe0 

Maximum^ 

Maximum* 

Maximum*’ 

Yield 

Conditions 

References'* 

Covalent 

Fluorescein-amines,  sulfhydryl 

490 

67 

520 

0.71 

pH  7,  PBS 

Haugland  (1983),  W,  MP 

labeling 

Tetramethylrhodamine-amines 

554 

85 

573 

0.28 

pH  7,  PBS 

Haugland  (1983),  MP 

reagents 

X-rhodamine-amines 

582 

79 

601 

0.26 

pH  7,  PBS 

W 

Texas  Red®-amines 

596 

85 

620 

0.51 

pH  7,  PBS 

Titus  et  al.  (1982),  W,  MP 

CY3 

554 

130 

568 

0.14/ 

pH  7,  PBS 

Mujumdar  et  al.  (1993) 

CY5 

652 

200 

672 

0.18f 

pH  7,  PBS 

Mujumdar  et  al.  (1993) 

CY7 

755 

200 

778 

0.0/ 

pH  7,  PBS 

Mujumdar  et  al.  (1993) 

BODIPY®  FL 

502 

80 

510 

MeOH 

MP 

BODIPY  581/591 

581 

136 

591 

MeOH 

MP 

BODIPY  630/650 

625 

101 

640 

MeOH 

MP 

Cascade  Blue® 

378,  399 

26 

423 

Water 

MP 

AlexaFluor®  430 

430 

15 

545 

pH  7 

MP 

AlexaFluor  488 

494 

73 

517 

pH  7 

MP 

AlexaFluor  532 

530 

81 

555 

pH  7 

MP 

AlexaFluor  594 

590 

92 

617 

pH  7 

MP 

NBD-amine 

478 

24.6 

520-550 

0.36/0.21 

EtOH/MeOH 

Kenner  &  Aboderin  (1971), 

Allen  &  Lowe  (1973), 
Bratcher  (1979) 

NBD-S-CH2CH20H 

425 

12.1 

531 

0.002 

pH  7.5.  10% 

glycerol 

Coumarin-phalloidin 

387 

470 

Water 

MP 

DY-555 

555 

100 

580 

MoBiTec  GmbH 

(http://www.mobitec.com) 

DY-631 

637 

185 

658 

MoBiTec  GmbH 

(http://www.mobitec.com) 

Phycoerythrin-R 

480-565 

1960 

578 

0.68 

pH  7,  PBS 

Oi  et  al.  (1982) 

Allophycocyanine 

650 

700 

660 

0.68 

pH  7,  PBS 

Oi  et  al.  (1982) 

PerCP 

488 

680 

pH  7,  PBS 

Rechtenwald  (1989) 

Nanocrystals 

(quantum  dots ) 

365  &  higher 

~6000@400nm 

400-850+ 

PBS 

Lanthanide 

Europium  chelate 

340 

Determined 

615 

See  FN  i 

Water 

Paul  (2002) 

chelates 

Terbium  chelate 

340 

by  ligand 

545 

See  FN  j 

Paul  (2002) 

Expressible 

EBFP 

383 

31 

445 

0.25 

pH  7,  PBS 

Patterson  et  al.  (2001) 

labels 

ECFP 

434 

26 

477 

0.40 

pH  7,  PBS 

Patterson  et  al.  (2001) 

EGFP 

489 

55 

508 

0.60 

pH  7,  PBS 

Patterson  et  al.  (2001) 

EYFP 

514 

84 

527 

0.61 

pH  7,  PBS 

Patterson  et  al.  (2001) 

DsRed 

558 

72.5 

583 

0.68 

pH  7,  PBS 

Patterson  et  al.  (2001) 

DNA-RNA 

Hoechst  33342 

340 

120 

450 

0.83 

+DNA  (excess) 

W 

content* 

DAPI 

350 

470 

+DNA  (excess) 

W 

DRAQ5 

646 

21 

681 

PBS 

Smith,  PJ  et  al.  (2000) 

Ethidium  Bromide 

510 

3.2 

595 

+DNA  (excess) 

Pohl  et  al.  (1972) 

Propidium  Iodide 

536 

6.4 

623 

0.09 

+DNA  (excess) 

W 

Acridine  Orange 

480 

520 

+DNA 

Kapuscinski  et  al. 

(1982),  Shapiro  (1985) 

440^170 

650 

+RNA 

Pyronine  Y 

549-561 

67-84 

567-574 

0.04-0.26 

+ds  DNA; 

Darzynkiewicz  et  al. 

(1987),  Kapuscinski  & 
Darzynkriwicz  (1987) 

560-562 

70-90 

565-574 

0.05-0.21 

+ds  RNA/ 

497 

42 

563 

Low 

+ss  RNA 

Thiazole  Orange 

453 

26 

480 

0.08 

RNA 

Lee  et  al.  (1986) 

TOTO-1 

514 

112 

533 

MeOH 

MP,  Rye  et  al.  (1992) 

YOYO-3 

612 

115 

631 

MeOH 

MP,  Rye  et  al.  (1992) 

Membrane 

diO-Cn-(3) 

485 

149 

505 

0.05 

MeOH 

Sims  et  al.  (1974),  W 

potential 

diI-Cn-(5) 

646 

200 

668 

0.4 

MeOH 

Sims  et  al.  (1974),  W 

diBA-Isopr-(3) 

493 

130 

517 

0.03 

MeOH 

Sims  et  al.  (1974),  W 

diBA-C4-(5) 

590 

176 

620 

EtOH 

W 

Rhodamine  123 

511 

85 

534 

0.9 

EtOH 

EK,  Kubin  & 

Fletcher  (1983) 
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TABLE  16.1.  ( Continued ) 


Absorption 

Extinction 

Emission 

Quantum 

Measurement 

Parameter 

Probe“ 

Maximum^ 

Maximum0 

Maximum'" 

Yield 

Conditions 

References'' 

pH 

BCECF 

505 

530 

High  pH 

MP 

460 

Low  pH 

MP 

SNARF-1  (pKa  =  7.5) 

518-548 

587 

pH  5.5 

MP 

574 

636 

pH  10.0 

MP 

DCDHB 

340-360 

500-580 

High  pH 

Valet  et  al.  (1981),  MP 

340-360 

420^140 

Low  pH 

Membrane 

Diphenylhexatriene  (DPH) 

330,  351,  370 

77  (351  nm) 

430 

Hexane 

MP 

location 

diI-C18-(3) 

546 

126 

565 

0.07 

MeOH 

W 

and  fluidity 

DiO 

484 

149 

501 

MeOH 

MP 

DiA 

491 

52 

613 

MeOH 

MP 

NBD  phosphatidylethanolamine 

450 

24s 

530 

Lipid 

Struck  et  al.  (1981) 

Anthroyl  stearate 

361,  381 

8.4,  7.5 

446 

MeOH 

Waggoner  &  Stryer  (1970) 

Pyrene-  sulfonamidoalky  Is 

350 

30 

380^100 

MP 

Calcium'' 

Fura  2 

335 

33 

512-518 

0.23 

Low  calcium 

Grynkiewicz  et  al.  (1985) 

360 

27 

505-510 

0.49 

High  calcium 

Indo  1 

330 

34 

390-410 

0.56 

High  calcium 

Grynkiewicz  et  al.  (1985) 

350 

34 

482-485 

0.38 

Low  calcium 

Fluo-3 

506 

83 

526 

0.183 

High  calcium 

Minata  et  al.  (1989) 

&  Mukkala  et  al. 

(1993),  MP 

506 

78 

526 

0.0051 

Low  calcium 

02  sensor 

Ru[dpp(S03Na)2]3 

365/450 

590 

x  =  3.7  gs 

No  oxygen 

Castellano  & 

Lakowicz  (1981) 

x  =  0.93  |ts 

Air 

cAMP 

FIERhR 

490 

70 

520,  573 

— 

pH  7,  cAMP 

Adams  et  al.  (1991) 

Enzyme 

Rhodamine-di-arg-CBZ 

495 

— 

532 

0.09 

Hepes  pH  7.5 

Leytus  et  al.  (1983) 

substrates 

Product  of  rxn.  (rhodamine) 

495 

67 

523 

0.91 

+15%  EtOH 

Coumarin-glucoside  substr. 

316 

13 

395 

pH  5.5  +  1% 

W 

Lubrol 

Rxn.  product  (hydroxy 

370 

17 

450 

pHIO  +  1% 

W 

coumarin) 

Lubrol 

Fluorescein  digalactosidase 

490 

67 

520 

0.71 

pH  7,  PBS 

MP 

product 

Resorufin  galactosidase  product 

571 

58 

585 

pH  9 

MP 

ELF97  phosphatase  product 

345 

Precipitate 

530 

pH  8 

MP 

“Abbreviations:  NBD,  7-nitrobenz-2-oxa- 1,3-diazole;  DAPI,  4',6-diamidino-2-phenylindole;  DCDHB,  dicyano-dihydroxybenzene. 
b  Measured  in  nanometers. 

‘  Multiply  value  listed  by  1000  to  get  liters/molcm. 

^EK,  Eastman  Kodak  Chemical  Catalog;  MP,  Molecular  Probes,  Inc  catalog;  W,  Waggoner  laboratory  determination. 

‘See  Table  III  in  Arndt-Jovin  &  Jovin  (1989)  for  additional  DNA  content  probes. 

^Base-pair  dependent. 

8  Value  for  NBD-ethanolamine  in  MeOH  which  has  an  abs.max  at  470  nm  and  an  emission  max  at  550  nm  [Barak  &  Yocum  (1981)]. 

''See  Tsien  (1989)  for  additional  details  and  other  ion  indicators. 

'See  corporate  Web  sites:  Quanturm  Dot  Corp.  (www.qdots.com),  Evident  Technologies  (www.evidenttech.com),  BioCrystal,  Ltd.  (www.biocrystal.com)  and  Crystalplex 
Corp.  (www.crystalplex.com). 

j  Time  resolved  detection.  Extinction  times  quantum  yield  approx.  2100. 


aqueous  media.  Promising  results  have  been  obtained  using 
quantum  dots  (Jaiswal  et  al. ,  2003;  Hoshino  et  al.,  2004;  Voura 
et  al.,  2004).  However,  there  is  ample  opportunity  for  significant 
improvements  before  these  reagents  can  be  used  routinely  for  fluo¬ 
rescent  labeling.  Their  relatively  large  size  and  high  mass  limit  their 
use  in  applications  requiring  high  diffusional  mobility. 

Fluorescent  Lanthanide  Chelates 

Lanthanide  chelates  are  another  group  of  fluorescent  reagents  with 
special  spectral  properties.  These  reagents  have  microsecond 
fluorescence  lifetimes  that  are  readily  distinguished  from  typical 
nanosecond  autofluorescence  background  (Soini  et  al.,  1988; 
Seveus  et  al.,  1994;  Vereb  et  al.,  1998).  Temporal  separation  of 


probe  fluorescence  from  background  signal  can  give  a  very  high 
S/N  and  highly  sensitive  probe  detection  even  though  the  bright¬ 
ness  of  these  reagents  is  only  modest.  The  main  advantage  of  these 
compounds  is  that,  as  their  fluorescent  properties  are  dependent  on 
electron  energy  levels  in  an  atom  rather  than  those  of  a  molecule, 
they  are  very  resistant  to  photodamage  (and  perhaps  also  to  pho¬ 
totoxicity).  On  the  other  hand,  it  is  possible  that  the  excitation  light 
may  break  the  bond  with  the  chelator  causing  the  “dye  atom”  to 
become  a  non-specific  stain. 

The  current  versions  of  these  reagents  require  enhanced  anten- 
nary  ligands  for  more  efficient  lanthanide  excitation  and  improved 
biological  stability  and  compatibility.  On  the  other  hand,  the  long 
decay  times  give  rise  to  problems  when  they  are  used  in  scanning 
microscopes.  When  the  decay  time  is  equal  to  the  pixel  dwell  time. 
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each  molecule  can  be  excited  no  more  than  one  time  and  signal 
levels  are  very  low.  As  a  result,  these  dyes  will  probably  only  be 
used  with  widefield  imaging  where  the  exposure  time  seen  by  each 
molecule  is  long  compared  to  its  fluorescent  lifetime  and  discrim¬ 
ination  from  the  fast  decay  of  the  autofluorescence  is  still  useful. 


FLUORESCENT  INDICATORS  FOR  DYNAMIC 
INTRACELLULAR  PARAMETERS 

Membrane  Potentials 

The  use  of  confocal  microscopy  to  measure  dynamic  properties  of 
living  cells  such  as  membrane  potentials  (Gross  and  Loew,  1989) 
or  ion  concentrations  (Tsien,  1988,  1989a, b)  deserves  some  special 
comment.  Preliminary  attempts  to  use  fast-responding,  non¬ 
redistributive  voltage-sensitive  dyes  in  neuronal  tissues  were 
unsuccessful  (Fine  et  al.,  1988);  the  dye  could  be  seen,  but  the  S/N 
was  inadequate  to  observe  voltage-dependent  changes,  which 
would  have  been  at  most  only  a  few  percent  of  the  resting  inten¬ 
sity.  Lasers  are  inherently  noisy  light  sources;  even  with  optical 
negative  feedback,  their  fluctuations  are  greater  than  the  stabilized 
tungsten  filament  lamps  conventionally  used  to  see  the  small 
changes  in  fluorescent  output  that  characterize  fast  voltage- 
sensitive  dyes  (Cohen  and  Lesher,  1986).  “Slow”  redistributive 
dyes,  which  accumulate  in  cells  according  to  the  Nernst  equilib¬ 
rium  (Ehrenberg  et  al.,  1988),  would  seem  to  be  more  suitable  for 
present-day  confocal  microscopes  because  their  signals  are  much 
bigger  and  the  slowness  of  their  response  (seconds  to  minutes)  is 
actually  a  better  match  to  the  rather  slow  scan  times  of  the  current 
instrumentation.  Confocal  optical  sectioning  should  work  well 
using  such  accumulative  dyes  because  in  principle  one  could 
directly  compare  the  internal  concentrations  of  dye  accumulated 
without  having  to  correct  for  the  greater  path  length  of  a  thicker 
cell  or  for  extracellular  dye  above  and  below  the  plane  of  focus. 
Freely  diffusing  anionic  oxonol  dyes  have  been  paired  with  dyes 
anchored  at  the  cell  surface  permitting  fast  ratiometric  detection 
of  membrane  potential  changes  in  single  cells  by  fluorescence 
resonance  energy  transfer  (Gonzalez  and  Tsien,  1995,  1997; 
Gonzalez  and  Maher,  2002)  (see  also  Figure  8.45,  this  volume). 

Ion  Concentrations 
Wavelength  Ratioing 

Some  indicators  of  ion  concentrations  respond  not  just  with 
changes  in  fluorescence  amplitude  but  also  with  X  shifts  of  the 
excitation  or  emission  spectrum  or  both.  Such  shifts  permit  ratio¬ 
ing  between  signals  obtained  at  two  or  more  Xs  (Tsien,  1989a,b). 
Ratioing  is  highly  valuable  because  it  cancels  out  differences  in 
dye  concentration  and  path  length  as  well  as  fluctuations  in  overall 
illumination  intensity  (Tsien  and  Poenie,  1986;  Bright  et  al.,  1987). 
Emission  ratioing  is  the  most  valuable  because,  with  a  single  exci¬ 
tation  X,  the  emission  can  be  passed  through  a  dichroic  mirror  to 
split  it  into  two  bands  that  can  be  monitored  absolutely  simulta¬ 
neously.  Such  ratioing  would  give  the  best  possible  cancellation  of 
laser  noise  or  specimen  movement.  Emission  ratioing  is  particu¬ 
larly  easy  to  do  with  a  laser-scanning  system,  because  one  can 
simply  add  a  dichroic  mirror  and  an  extra  photodetector  after  the 
scanning  system.  Whereas  geometrical  registration  of  all  the  cor¬ 
responding  pixels  in  two  separate  low-light-level  video  cameras 
is  quite  difficult  (Jericevic  et  al.,  1989),  the  registration  problem 
is  trivial  in  a  laser-scanning  system  assuming  that  the  deflection  is 


achromatic,  which  it  must  be  in  order  to  get  excitation  and  even 
one  emission  in  register.  Disk-scanning  confocal  microscopes  use 
charge-coupled  device  (CCD)  or  electron  multiplying  CCD  (EM- 
CCD)  cameras  as  detectors  and  often  lack  this  elegant  compati¬ 
bility  with  emission  ratio  scanning. 

Excitation  ratioing  of  images  requires  sequential  illumination 
with  the  two  excitation  Xs.  Intensity  fluctuations  of  the  source  and 
movement  of  the  specimen  are  canceled  out  only  if  they  are  much 
slower  than  the  rate  of  alternation.  Excitation  ratioing  is  most 
applicable  to  tandem-scanning  systems  where  conventional 
systems  for  alternating  two  grating  monochromators  or  interfer¬ 
ence  filters  could  be  used.  Alternating  between  two  laser  lines  is 
more  convenient  now  that  acousto-optical  deflectors  are  common, 
but  it  is  still  less  flexible  in  choice  of  X  pairs. 

PH  Indicators 

A  number  of  ratiometric  pH  indicators  were  reviewed  by  Tsien 
(1989b)  and  more  recently  by  Yip  and  Kurtz  (2002).  The  most 
popular  excitation-ratioing  indicator  is  probably  the  modified 
fluorescein,  BCECF,  whose  pH-sensitive  and  insensitive  Xs  are 
around  490  nm  and  439  nm,  respectively.  Several  emission- 
shifting  probes,  3,6-dihydroxyphthalonitrile  (also  known  as 
2,3-dicyanohydroquinone;  Kurtz  and  Balaban,  1985;  Kurtz  and 
Emmons,  1993),  and  various  naphthofluorescein  derivatives 
(SNAFs  and  SNARFs;  Haugland,  1989)  are  also  available. 

Ca2+  Indicators 

Three  currently  available  Ca2+  indicators  have  different  sets  of 
advantages  and  disadvantages  for  confocal  microscopy  (Tsien, 
1988,  1989a,b).  Fura-2,  the  dye  most  used  in  conventional  micro¬ 
scopic  imaging,  shows  a  good  excitation  shift  with  Ca2+,  typically 
ratioed  between  340  to  350 nm  and  380  to  385  nm,  but  hardly  any 
emission  shift,  so  it  would  be  most  effectively  used  with  a  UV- 
enhanced  disk-scanning  instrument.  Considerable  re-engineering 
would  be  necessary  for  those  early  designs  of  tandem-scanning 
confocal  microscope  designed  mainly  for  reflectance  rather  than 
fluorescence.  The  beam-splitting  pellicles  used  are  inefficient 
because  they  are  partially  reflective  but  not  dichroic;  also,  they  are 
sometimes  made  from  UV-blocking  material  in  which  the  excita¬ 
tion  has  to  pass  through  the  pellicle  whereas  the  emission  would 
have  to  reflect  off  the  pellicle.  Even  if  the  pellicle  were  replaced 
by  a  dichroic,  this  choice  of  beam  geometry  is  unfortunate,  as  it  is 
much  easier  to  make  good  broadband  dichroics  in  which  the 
shorter  X  reflects  and  the  longer  X  transmits  than  vice  versa. 
Disk  systems  in  which  the  same  area  of  the  disk  is  used  for  both 
source  and  detector  may  be  more  flexible  (see  Chapter  10,  this 
volume). 

Indo-1,  the  dye  most  used  in  laser  flow  cytometry  for  [Ca^] 
determination,  shows  a  fine  emission  shift  from  485  to  405  nm  with 
increasing  Ca2+  and  is  preferred  for  ratiometric  laser  scanning. 
However,  either  a  UV  laser  (e.g.,  a  high-power  argon-ion  or 
krypton-ion  system)  or  a  titanium-sapphire  two-photon  system 
is  required  for  excitation  in  the  350  to  365  nm  region.  Also, 
Indo-1  fluorescence  has  Xs  similar  to  those  of  reduced  pyridine 
nucleotides,  so  autofluorescence  could  be  a  problem,  and  Indo-1 
also  bleaches  much  more  quickly  than  Fura-2. 

Fluo-3  and  its  less-tested  rhodamine  analogs  are  the  only  Ca2+ 
indicators  currently  available  with  visible  Xs  suitable  for  low- 
power  visible  lasers  (Minta  et  al.,  1989;  Kao  et  al.,  1989).  There¬ 
fore,  it  has  been  the  first  to  be  exploited  in  confocal  microscopy 
(e.g.,  Hernandez-Cruz  et  al.,  1989),  even  though  it  lacks  either  an 
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excitation  or  emission  shift  and  is  restricted  to  simple  intensity 
measurements  that  are  relatively  difficult  to  calibrate  in  terms  of 
absolute  [Ca2+]i  units. 

Of  course,  the  ideal  would  be  an  indicator  excitable  at  488  nm 
with  a  large  emission  shift,  high  quantum  efficiency,  and  strong 
resistance  to  bleaching,  but  this  goal  is  a  difficult  challenge  in  mo¬ 
lecular  engineering.  In  general,  strong  fluorescence  in  aqueous 
media  is  much  easier  to  obtain  using  shorter  excitation  Xs  because 
fluorescence  demands  planarity  and  molecular  rigidity,  which  is 
obviously  easier  to  achieve  in  small  molecules  that  absorb  short 
Xs  than  in  the  larger  molecules  with  longer  chromophores.  Most 
of  the  known  chromophores  that  combine  large  size,  long  Xs,  and 
rigidity  are  essentially  insoluble  in  water.  Even  if  solubilizing 
groups  are  added  on  the  periphery,  the  huge  expanse  of  hydropho¬ 
bic  surface  still  promotes  the  formation  of  non-fluorescent 
aggregates.  Finally,  the  quantum  mechanics  of  absorption  and 
fluorescence  predict  that  the  intrinsic  radiative  lifetime  of  a  chro- 
mophore  is  proportional  to  the  cube  of  the  X  if  other  factors  remain 
constant  (Strickler  and  Berg,  1962).  Short  radiative  lifetimes  mean 
that  fluorescence  emission  competes  more  successfully  with  non¬ 
productive  forms  of  deactivation  and,  therefore,  correlates  with 
high  quantum  yields  of  fluorescence. 

Oxygen  Sensor 

Much  effort  has  been  spent  developing  ruthenium  chelates  as  lumi¬ 
nescent  reagents.  Common  ligands  for  the  central  ruthenium  ion 
include  substituted  bipyridines  and  1,10-phenanthrolines.  Cur¬ 
rently,  the  most  promising  applications  for  these  complexes  are 
for  monitoring  the  oxygen  tension  of  solutions  (Li  et  al.,  1997; 
Castellano  and  Lakowicz,  1998;  Ji  et  al .,  2002).  As  with  the  lan¬ 
thanide  complexes,  the  modest  excitation  efficiencies  are  depen¬ 
dent  on  energy  transfer  from  the  ligands  to  the  metal  ion.  The 
emission  spectra  are  relatively  weak  and  broad.  However,  time- 
resolved  detection  of  the  long-lived  ruthenium  fluorescence  yields 
excellent  S/N  even  from  these  low  intensities.  The  presence  of 
oxygen,  even  at  the  levels  found  in  air,  decreases  the  fluorescence 
lifetimes  of  ruthenium  chelates  by  a  factor  of  4,  a  factor  that  should 
make  them  suitable  for  fluorescence  lifetime  imaging  (FLIM;  see 
Chapter  27,  this  volume),  now  that  the  equipment  is  commercially 
available.  Improved  ligands  with  enhanced  absorbance  at  visible 
wavelengths,  more  efficient  energy  transfer  to  the  metal,  and  better 
biocompatibility  are  needed  before  these  reagents  find  broader 
applications. 

cAMP  Indicators 

The  important  intracellular  messenger  cAMP  (cyclic  adenosine 
3,5-monophosphate)  can  now  be  imaged  with  a  fluorescent  indi¬ 
cator  made  from  cAMP-dependent  protein  kinase  labeled  on  its  C 
and  R  subunits  with  fluorescein  and  tetramethylrhodamine  respec¬ 
tively  (Adams  et  al.,  1991).  In  the  holoenzyme  complex,  the  fluo¬ 
rescein  and  rhodamine  are  close  enough  for  moderately  efficient 
fluorescence  resonance  energy  transfer,  so  that  excitation  of  the 
fluorescein  with  blue-green  light  gives  a  significant  amount  of 
orange  emission  from  the  rhodamine.  Binding  of  cAMP  dissoci¬ 
ates  the  subunits  and  eliminates  energy  transfer,  increasing  the 
emission  of  green  light  directly  from  the  fluorescein  and  decreas¬ 
ing  the  amplitude  in  the  rhodamine  band.  This  change  in  emission 
ratio  and  the  Xs  employed  are  ideal  for  dual-channel  detection  by 
confocal  microscopy  (Bacskai  et  al.,  1993).  The  strategy  of  label¬ 
ing  an  important  endogenous  sensor  protein  gives  both  advantages 
and  potential  problems.  Because  careful  derivatization  of  the 


kinase  does  not  change  its  cAMP  affinity  and  phosphorylating 
activity,  the  indicator  is  inherently  tuned  to  the  physiologically  rel¬ 
evant  concentration  range  (a  few  nanomoles  to  a  few  examoles), 
and  molecules  of  cAMP  that  bind  to  the  indicator  can  still  have  a 
biological  effect.  An  indicator  that  was  not  a  physiological  effec¬ 
tor  molecule  would  have  a  greater  tendency  to  competitively 
inhibit  or  buffer  the  pathway  under  study.  Furthermore,  after  ele¬ 
vation  of  cAMP,  the  interesting  trafficking  of  the  R  and  C  subunits 
can  be  separately  observed  by  standard  dual-label  imaging  (e.g. 
Harootunian  et  al,  1993).  However,  scrambling  of  subunits  with 
unlabeled  endogenous  kinase  is  a  potential  problem,  so  far  rarely 
serious  (for  discussion  of  this  point  and  a  more  extensive  review 
of  the  entire  technique,  see  Adams  et  al.,  1993). 

Fatty  Acid  Indicator 

Fatty  acids  are  of  considerable  importance  in  nutrition,  membrane 
structure,  protein  modification,  eicosanoid  formation,  and  modu¬ 
lation  of  cell  signaling.  Recently  a  group  led  by  Alan  Kleinfeld  has 
developed  an  emission-ratioing  fluorescent  probe  for  free  fatty 
acid  levels  (Richieri  et  al.,  1992)  by  labeling  recombinant  intesti¬ 
nal  fatty-acid-binding  protein  with  acrylodan,  an  environmentally 
sensitive  fluorophore.  The  acrylodan  reacts  with  surprising  speci¬ 
ficity  for  Lys27  of  the  protein  and  probably  resides  in  the  fatty- 
acid  binding  pocket.  Binding  of  fatty  acids  shifts  the  emission  peak 
from  432  nm  to  505  nm,  probably  by  displacing  the  acrylodan 
into  an  aqueous  environment.  The  505nm/432nm  ratio  thereby 
increases  by  up  to  25-fold,  which  would  be  an  ideal  signal  for  con¬ 
focal  microscopy  if  the  necessary  excitation  at  386 nm  or  400 nm 
were  available.  The  labeled  protein  (dubbed  ADIFAB)  binds  all 
common  long-chain  fatty  acids  with  approximately  micromolar 
dissociation  constants.  In  a  cuvet,  ADIFAB  can  detect  free  fatty 
acid  concentrations  as  low  as  a  few  nanomolar.  It  has  already 
proven  highly  useful  in  measuring  the  release  of  free  fatty  acids 
from  stimulated  rat  basophilic  leukemia  (RBL)  cells  (Richieri 
et  al,  1992)  and  the  binding  constants  of  fatty  acids  to  albumin 
(Richieri  et  al.,  1993)  and  to  cells  (Anel  et  al.,  1993). 

Other  Forms  of  Ratioing 

Because  ratioing  is  so  desirable  for  quantitative  measurements,  but 
appropriate  X  shifts  are  often  unavailable,  several  alternatives  to 
X  ratioing  have  been  proposed.  The  easiest  is  simply  to  ratio 
poststimulus  image  intensities  against  a  prestimulus  image.  An 
example  is  shown  by  Smith  and  Augustine  (1988).  This  method 
has  the  advantage  of  minimal  hardware  requirements  and  high  time 
resolution,  though  it  only  cancels  out  variations  in  dye  loading  and 
path  length,  not  shape  change  or  dye  bleaching,  and  by  itself 
cannot  yield  an  absolute  calibration  of  the  analyte,  for  example, 
[Ca2+]j.  Another  approach  would  be  to  link  the  fluorescent  indica¬ 
tor  covalently  to  a  separate  reference  fluorophore.  This  approach 
would  ideally  generate  a  composite  molecule  in  which  the  ratio  of 
the  indicator  fluorescence  to  the  reference  fluorescence  would 
signal  the  analyte  concentration.  Potential  disadvantages  would  be 
the  requirement  for  significant  skill  in  organic  synthesis,  the  like¬ 
lihood  that  the  conjugate  would  be  too  large  for  loading  by  ester 
hydrolysis,  and  the  possibility  that  the  two  fluorophores  would 
bleach  at  different  rates,  so  that  the  operation  of  ratioing  would  fail 
to  correct  for  bleaching.  Yet  a  third  mode  of  ratioing  could  be  based 
on  temporal  dissection  of  excited-state  lifetimes,  as  first  shown  for 
quin-2  by  Wages  and  colleagues  (1987).  If  the  free  and  bound 
forms  of  the  indicator  have  sufficiently  different  fluorescence  life¬ 
times,  their  relative  contributions  to  the  (ideally)  biexponential 
decay  might  be  separated  by  nanosecond  or  high  frequency  mod- 
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ulation  techniques  (see  Chapter  27,  this  volume).  However,  even 
when  the  instrumentation  challenge  of  combining  lifetime  kinet¬ 
ics  with  imaging  has  been  solved,  the  problem  remains  that  probes 
like  fura-2  and  Indo-1,  which  are  fairly  strongly  fluorescent  both 
when  free  and  when  bound  to  Ca2+,  have  almost  the  same  lifetimes 
in  those  two  states.  For  example,  fura-2  with  and  without  Ca2+  has 
lifetimes  of  1.8  ns  and  1.3  ns,  respectively,  at  25°C;  for  Indo-1  the 
corresponding  numbers  are  1.7  ns  and  1.3  ns  at  20°C  (Wages  et  al., 
1987).  In  order  to  have  a  significant  difference  in  lifetimes  between 
Ca2+-bound  and  free  indicator,  as  in  quin-2  (10.1ns  and  1.3ns, 
respectively,  at  25°C),  one  of  the  species  has  to  be  much  more 
dimly  fluorescent  than  the  other.  As  a  result,  the  weaker  and  faster 
component  will  be  hard  to  measure  accurately  and  to  distinguish 
from  autofluorescence  background. 

Ratiometric  measurements  can  also  be  applied  to  other  param¬ 
eters  such  as  probe  polarization  and  local  viscosity  (Tinoco  et 
al.,  1987;  Axelrod,  1989;  Dix  and  Verkman,  1989),  proximity 
between  macromolecules  by  fluorescence  energy  transfer  (Uster 
and  Pagano,  1986;  Herman,  1989),  and  even  water  permeability 
(Kuwahara  and  Verkman,  1988;  Kuwahara  et  al.,  1988). 


GENETICALLY  EXPRESSED  INTRACELLULAR 
FLUORESCENT  INDICATORS 

Green  Fluorescent  Protein 

Fluorescent  proteins  from  jellyfish  and  corals  have  revolutionized 
biological  optical  microscopy  because  they  provide  genetic  encod¬ 
ing  of  strong  visible  fluorescence  of  a  wide  range  of  colors.  Entire 
books  (Chalfie  and  Kain,  1998;  Sullivan  and  Kay,  1999;  Hicks, 
2002)  have  been  devoted  to  many  aspects  of  the  prototypical 
fluorescent  protein,  the  green  fluorescent  protein  (GFP)  from  the 
jellyfish  Aequorea  victoria.  A  shorter,  relatively  self-contained 
introduction  to  GFP  may  be  found  in  Tsien  (1998).  Other  general 
reviews  on  applications  of  GFP  and  other  members  of  the  fluores¬ 
cent  protein  superfamily  include  (Cubitt  et  al.,  1995,  1999; 
Hassler,  1995;  Niswender  et  al,,  1995;  Rizzuto  et  al.,  1995; 
Stearns,  1995;  Kahana  and  Silver,  1996;  Misteli  and  Spector,  1997; 
Patterson  et  al.,  1997;  Tsien  and  Miyawaki,  1998;  Ellenberg  et  al., 
1999;  Heim,  1999;  Lippincott-Schwartz  et  al.,  1999;  Phillips, 
1999;  Piston  et  al.,  1999;  Chamberlain  and  Hahn,  2000;  Miyawaki 
and  Tsien,  2000;  Sacchetti  et  al.,  2000;  Zaccolo  and  Pozzan,  2000; 
Zacharias  et  al.,  2000;  Blab  et  al.,  2001;  Chiesa  et  al.,  2001;  Harms 
et  al.,  2001;  Lippincott-Schwartz  et  al.,  2001;  Patterson  et  al., 
2001;  Reits  and  Neefjes,  2001;  Wahlfors  et  al.,  2001;  Labas  et  al., 
2002;  Matz  et  al.,  2002;  Miyawaki,  2002;  van  Roessel  and  Brand, 
2002;  Zacharias,  2002;  Zhang  et  al.,  2002;  Zimmer,  2002;  Choy 
et  al.,  2003;  Ehrhardt,  2003;  Hadjantonakis  et  al,  2003; 
Lippincott-Schwartz  and  Patterson,  2003;  Lippincott-Schwartz 
et  al.,  2003;  March  et  al.,  2003;  Meyer  and  Teruel,  2003; 
Miyawaki,  2003;  Tsien,  2003;  Viallet  and  Vo-Dinh,  2003;  Weijer, 
2003;  Verkhusha  and  Lukyanov,  2004).  Space  does  not  permit 
listing  the  huge  number  of  reviews  and  primary  papers  describing 
more  specialized  uses  of  fluorescent  proteins. 

Ligand-Binding  Modules 

Genetic  manipulations  have  also  been  used  to  incorporate  into  cells 
expressible  modules  that  bind  fluorescent  ligands.  One  type  of 
module  contains  a  tetracysteine  motif,  which  binds  biarsenical 
ligands  (Griffin  et  al.,  1998;  Adams  et  al.,  2002;  Nakanishi  et  al., 
2004).  These  small,  membrane-permeant  ligands  bind  with  high 


affinity  and  specificity  to  the  four  sulfhydryls  arranged  in  an  alpha 
helix  domain.  Biarsenical  fluorescein  (FlAsH),  tetramethyl  rho- 
damine  (TrAsH),  and  a  few  additional  dyes  have  been  reported.  In 
a  different  approach,  cells  transfected  with  the  sequence  encoding 
a  single-chain  antibody  (scFv)  expressed  a  module  that  tightly 
bound  cell-permeant  hapten-fluorophore  conjugates  (Farinas  and 
Verkman,  1999).  Incorporation  of  several  scFvs  would  enable  mul¬ 
tiple  ligands  to  be  detected  simultaneously.  Like  the  expressible 
GFPs,  these  modules  can  be  directed  to  specific  intracellular  com¬ 
partments  by  including  the  appropriate  localization  sequences. 

Ion  Indicators 

Mutants  of  GFP  have  been  identified  that  show  pH-dependent 
fluorescence  properties  (Kneen  et  al.,  1998;  Llopis  et  al.,  1998). 
Reversible  absorbance  and  fluorescence  emission  changes  were 
observed  with  apparent  pKa  values  ranging  from  4.8  to  7.1.  Com¬ 
bining  the  GFP  sequences  with  specific  targeting  signals  permit¬ 
ted  the  acidification  of  specific  organelles  to  be  followed 
noninvasively.  Genetically  encoded  Ca2+  indicators,  dubbed 
“chameleons,”  utilize  fluorescence  resonance  energy  transfer 
(FRET)  between  different  emitting  GFPs  attached  to  calmodulin 
and  a  calmodulin-binding  peptide,  M13  (Miyawaki  et  al.,  1997, 
1999).  Binding  of  Ca2+  to  calmodulin  increases  the  interaction 
between  the  GFPs.  Optimization  of  the  relative  orientation  of  the 
two  GFPs  in  the  chimeras  has  expanded  the  dynamic  range  of  Ca2+ 
detection  (Nagai  et  al,  2001,  2004). 

FUTURE  DEVELOPMENTS 

Speculation  on  future  directions  in  fluorophore  designs  is  difficult 
because  the  small  number  of  laboratories  working  on  fluorophore 
chemistry  makes  progress  a  much  noisier  function  of  time  than 
advances  in  instrumentation  or  computers.  One  major  advance 
seen  since  publication  of  the  previous  edition  of  this  book  is  a 
recognition  of  the  optimal  characteristics  for  fluorescent  reagents 
used  in  biology.  A  thought  prompted  by  preparation  of  this  review 
is  that,  for  present  purposes,  the  excited  triplet  state  of  the  fluo¬ 
rophore  is  a  major  villain  without  any  redeeming  virtues.  It  is 
responsible  for  a  pernicious  form  of  output  saturation,  for  singlet 
oxygen  production,  and  for  nearly  all  covalent  photochemistry 
such  as  bleaching.  Similar  problems  have  been  encountered  in 
laser  dyes;  a  proposed  solution  (Liphardt  et  al,  1982,  1983; 
Schafer,  1983)  is  to  attach  triplet-state  quenchers  to  each  fluo¬ 
rophore.  Such  a  construction  is  reminiscent  of  the  way  that  evolu¬ 
tion  has  assembled  photosynthetic  complexes  and  may  be  an  area 
where  biology  can  repay  its  debt  to  synthetic  chemistry.  Little 
progress  has  been  reported  in  this  area  during  the  two  decades 
since  these  approaches  were  proposed,  but  the  potential  gains  from 
triplet-state  relaxation  maintain  this  as  an  attractive  area  for  study. 
The  development  of  fluorescent  inorganic  nanocrystals  and 
chelates  may  provide  the  needed  photostable  biological  tagging 
reagents,  however. 

Another  area  ripe  for  development  is  signal  amplification 
schemes  for  detecting  low  copy  numbers  of  cell  receptors  and 
genetic  sequences.  Use  of  enzymes  that  produce  fluorescent  pre¬ 
cipitates  in  a  localized  area  offers  promise  (Haugland,  1989);  again 
reported  progress  is  limited. 

Finally,  there  is  always  need  for  additional  sensitivity.  Time- 
resolved  fluorescence  detection,  where  the  fluorescence  signal  is 
collected  against  a  dark  background  after  pulse  excitation,  offers 
a  method  to  circumvent  autofluorescence  and  Raman  light  scat- 
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tering  that  is  becoming  much  more  widely  available  (Lakowicz 
et  al.,  1983;  Marriott  et  al.,  1991;  Chapters  27  and  31,  this 
volume).  This  approach  is  now  realized  with  sophisticated  excita¬ 
tion/detection  components  and  will  work  with  any  fluorophore 
with  an  excited  state  lifetime  in  the  nanosecond  time  range. 
Quantum  dots,  with  fluorescence  lifetimes  in  the  range  of  20  ns, 
may  prove  particularly  useful,  but  their  early-stage  development 
as  biological  labeling  reagents  has  not  provided  sufficient  incen¬ 
tives  for  the  required  detection  system  modifications.  Another 
ingenious  approach  would  involve  excitation  of  an  extended  life¬ 
time  fluorophore  with  a  scanning  laser  followed  by  detection  of 
the  signal  with  an  array  detector,  such  as  a  cooled  EM-CCD 
camera,  instead  of  a  single  photomultiplier  tube.  The  signal  could 
be  integrated  on  the  cleared  pixels  of  the  CCD  chip,  even  mil¬ 
liseconds  after  the  excitation  beam  has  passed  the  corresponding 
region  of  the  sample.  The  extended  lifetime  labels  could  be  lan¬ 
thanide  complexes  with  millisecond  lifetimes  that  have  been 
developed  as  protein-labeling  reagents  by  Hemmila  and  others 
(Soini  et  al. ,  1988;  Mukkala,  1993).  In  situations  with  heavy 
autofluorescence,  the  lanthanide  complex  labels  have  demon¬ 
strated  sensitivity  over  fluorescein  labels  by  factors  of  hundreds 
(Seveus  et  al .,  1994).  Phosphor  particles  developed  by  Beverloo 
and  colleagues  (1992)  could  be  used  in  a  similar  fashion.  Surface 
chemistries  used  to  improve  the  biocompatibility  of  quantum  dots 
may  also  be  effective  in  enhancing  the  fluorescence  properties  of 
these  phosphors  in  biological  applications.  Chemistry  will  con¬ 
tinue  to  play  a  major  role  in  furthering  the  power  of  confocal 
microscopy. 
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Practical  Considerations  in  the  Selection  and  Application 
of  Fluorescent  Probes 

lain  D.  Johnson 


INTRODUCTION 

Due  to  its  sensitivity,  multiplexing  capacity,  and  applicability  to 
live  specimens,  fluorescence  is  the  dominant  contrast  mechanism 
used  in  three-dimensional  (3D)  biological  microscopy.  Use  of  flu¬ 
orescence  detection  generally  requires  specimens  to  be  labeled 
with  extrinsic  probes.  This  is  because  most  biological  molecules 
and  structures  of  interest  are  not  intrinsically  fluorescent  in  spec¬ 
tral  ranges  that  are  useful  for  detection,  and  even  those  that  are 
cannot  usually  be  discriminated  from  each  other  on  the  basis  of 
their  intrinsic  fluorescence.  Extrinsic  labeling  circumvents  these 
problems  at  the  expense  of  introducing  others.  Extrinsic  probes 
must  be  delivered  to  the  region  of  interest  and  remain  there  for 
long  enough  to  acquire  the  experimental  data.  Once  in  situ,  the 
probe  should  ideally  be  a  passive  reporter  that  does  not  induce  sig¬ 
nificant  perturbations  of  the  biological  structure  or  function  that 
we  wish  to  study.  Furthermore,  the  detection  process  itself  may 
have  deleterious  side  effects  in  the  form  of  photobleaching  and 
photoxicity,  resulting  from  the  interaction  of  excitation  light  with 
the  probe  and  the  specimen. 

Based  on  the  premise  that  understanding  probe  behavior  is  a 
key  component  in  evaluating  the  information  content  of 
images  obtained  using  fluorescence  microscopy,  this  chapter 
reviews  the  practical  considerations  involved  in  probe  selection 
and  use.  In  Table  17.1,  fourteen  key  characteristics  of  probes  and 
specimens  are  listed  in  relation  to  their  impact  on  labeling  and 
detection  processes.  The  ordering  of  topics  in  Table  17.1  reflects 
the  sequence  of  discussion  in  subsequent  sections  of  this  chapter. 


SELECTION  CRITERIA  FOR  DYES  AND  PROBES 

A  fluorescent  dye  (or,  synonymously,  a  fluorophore)  is  a  fluores¬ 
cent  molecule  that  does  not  associate  with  any  particular  biologi¬ 
cal  target.  A  fluorescent  probe  is  a  dye  that  has  been  modified  in 
some  way  to  detect  specific  biological  targets  (Fig.  17.1).  Targets 
include  specific  groups  of  cells  in  a  tissue,  organelles,  proteins, 
nucleic  acids,  ions  (Ca2+,  Mg2+,  H+,  Na+,  etc.)  and  enzymes.  From 
this  perspective,  fluorescein  is  a  dye  whereas  fluorescein-labeled 
proteins  and  peptides  are  probes.  Similarly,  the  green  fluorescent 
protein  (GFP)  can  be  considered  to  be  a  “dye”  and  GFP  fusion  pro¬ 
teins  are  probes.  In  some  cases,  the  structural  characteristics  of  the 
dye  itself  are  sufficient  to  confer  biospecificity.  For  example, 
cationic  dyes  such  as  MitoTracker  Red  CMXRos,  JC-1,  and  rho- 
damine  123  stain  mitochondria  driven  by  the  internally  negative 


membrane  potential.  Dyes  and  probes  have  both  biochemical  and 
spectroscopic  properties.  Biochemical  properties  determine  the 
molecular  association,  transport,  and  metabolic  fate  of  the  probe. 
Examples  include  water  solubility,  membrane  permeability,  recep¬ 
tor  binding  affinity,  and  enzymatic  conversion  rates.  Spectroscopic 
properties  primarily  determine  the  number  and  energy  distribution 
of  photons  available  for  detection.  They  include  excitation  and 
emission  spectra,  molar  absorptivity  (extinction  coefficient),  fluo¬ 
rescence  quantum  yield,  and  photobleaching  rate. 


Organic  Dyes 

Fluorescein  and  its  derivatives  (Fig.  17.1)  have  been  the  most 
widely  used  class  of  organic  dyes  used  as  fluorescent  probes.  Their 
utility  is  derived  in  part  from  the  fact  that  fluorescein  is  efficiently 
excited  by  the  488  nm  argon-ion  laser  line.  Coumarins  and 
rhodamines  have  been  the  primary  blue  (~450nm)  and  orange 
(-580 nm)  emitting  dyes  used  alongside  fluorescein  (green  emis¬ 
sion,  -520  nm)  in  two-color  labeling  applications.  Each  class  has 
some  significant  disadvantages.  Fluoresceins  are  pH  sensitive  and 
highly  susceptible  to  photobleaching.  Rhodamines  have  a  tendency 
to  aggregate  in  aqueous  solutions  resulting  in  self-quenching  of 
fluorescence  (see  below).  Coumarins  have  relatively  low 
excitation  efficiencies  —  emax  -  20,000  M"1  cm"1  compared  to  emax 

-  100,000  M"1  cm"1  for  fluoresceins  and  rhodamines.  Coumarins 
also  have  other  drawbacks  associated  with  ultraviolet  excitation, 
namely,  phototoxicity  and  a  requirement  for  expensive  quartz 
optical  components.  These  latter  problems  can  be  circumvented  by 
use  of  two-photon  excitation. 

The  deficiencies  of  fluorescein,  rhodamine,  and  coumarin  dyes 
have  spurred  the  development  of  new  dye  classes  that  have  been 
deliberately  optimized  for  biomolecular  detection  applications. 
Three  of  these  classes  will  be  discussed  here  (in  alphabetical  order) 

—  AlexaFluor  dyes,  BOPIDY  dyes,  and  cyanine  (Cy)  dyes  (Table 
17.2).  The  AlexaFluor  dye  series  has  19  members  with  excitation 
maxima  matched  to  principal  laser  output  lines  between  350  and 
750 nm  (Fig.  17.2).  The  AlexaFluor  dyes  are  more  water-soluble 
than  their  fluorescein  and  rhodamine  counterparts,  resulting 
directly  or  indirectly  in  reduced  levels  of  self-quenching  upon  cou¬ 
pling  to  proteins  and  improved  photostability  (Panchuk- Voloshina 
et  al.,  1999;  Berber  et  al.,  2003).  The  Cy  dye  series  (Mujumdar 
et  al.,  1993)  has  fewer  excitation  wavelength  variants  but  contains 
similar  design  elements  to  the  AlexaFluor  dyes  series  —  sulfonic 
acid  substituents  to  increase  aqueous  solubility  and  the  use  of 
V-hydroxysuccinimidyl  (NHS)  ester  reactive  chemistry  for  cou- 


lain  D.  Johnson  •  Molecular  Probes,  Inc.,  Eugene,  Oregon  97402 


Handbook  of  Biological  Confocal  Microscopy,  Third  Edition,  edited  by  James  B.  Pawley,  Springer  Science+Business  Media,  LLC,  New  York,  2006. 


353 


354  Chapter  17  •  I.D.  Johnson 


TABLE  17.1.  Key  Characteristics  of  Probes  and  Specimens 

Property  Significance 


1 .  Excitation  spectrum 

2.  Extinction  coefficient  (£;  units  M-1  cm-1) 

3.  Emission  spectrum 

4.  Fluorescence  quantum  yield  (QY) 

5.  Environment  sensitivity 

6.  Probe  size/permeability/solubility 

7.  Specimen  type  (single  cell,  cell  population,  tissue) 

8.  Target  abundance 

9.  Autofluorescence 

10.  Probe  localization 

1 1 .  Probe  metabolism  and  retention 

12.  Probe-mediated  cytotoxicity 

13.  Photobleaching 

14.  Phototoxicity 


Should  be  wavelength-matched  with  instrument  source  output  for  optimum  fluorescence  excitation 
efficiency  (see  Fig.  17.2). 

Overlap  with  donor  emission  spectrum  required  for  FRET. 

Determining  factor  in  fluorescence  output  per  dye. 

Ability  to  resolve  probe  signal  from  autofluorescence. 

Selection  of  probes  for  simultaneous  imaging  of  multiple  targets." 

Overlap  with  acceptor  excitation  spectrum  required  for  FRET. 

Determining  factor  in  fluorescence  output  per  dye.  Often  environment  dependent. 

Impacts  the  proportionality  of  dye  fluorescence  to  concentration. 

Impacts  choice  of  loading  method. 

Impacts  choice  of  loading  method. 

Determines  total  fluorescence  output.  May  compel  use  of  signal  amplification  techniques^  or  GFP 
overexpression. 

Impacts  selection  of  excitation/emission  wavelength  ranges  and  level  of  labeling  required. 
Localized  accumulations  are  more  readily  detectable  than  the  diffuse  distributions. 

Determines  stability  of  labeling. 

Impacts  specimen  viability. 

May  require  reduced  loading  concentration  and/or  selecting  a  different  probe. 

May  necessitate  attenuation  of  excitation  power. 

Determines  ability  to  conduct  time-lapse  experiments. 

Impacts  specimen  viability. 


"Dyes  with  narrow  emission  bandwidths  are  preferred  for  minimizing  spillover  of  the  signal  into  adjacent  detection  channels  (see  Fig.  17.2). 

Probes  that  exhibit  environment-dependent  spectral  shifts,  such  as  the  JC-1  and  BOPIDY  FL  ceramide  are  difficult  to  employ  in  these  applications  for  this  reason. 
fcSee  Wang  and  colleagues  (1999)  for  example. 


pling  to  free  amine  groups  on  proteins  and  other  biomolecules 
[Fig.  17.3(A)].  In  contrast  to  the  AlexaFluor  and  Cy  dye  series, 
BOPIDY  dyes  are  non-polar  and  relatively  insoluble  in  water  [Fig. 
17.3(B)].  Instead  of  protein  labeling,  they  are  primarily  utilized  in 
fluorescent  lipid  analogs  (Pagano  et  al.,  2000;  Farber  et  al .,  2001) 
and  analogs  of  receptor  ligands  such  as  nucleotides,  steroids,  alka¬ 
loids,  and  peptides  (Daly  and  McGrath,  2003). 


Environmental  factors  including  pH,  solvent  polarity,  binding 
to  proteins,  and  dye-dye  interactions  can  exert  strong  influences 
on  dye  fluorescence.  Susceptibility  to  environment  varies  widely 
among  dye  classes.  Dyes  that  are  designed  primarily  for  covalent 
labeling  of  proteins  and  nucleic  acids  such  as  AlexaFluor  dyes  and 
Cy  dyes  are  highly  fluorescent  in  water  and  retain  similar  levels 
of  fluorescence  after  coupling.  In  histochemical  and  cytochemical 
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FIGURE  17.1.  Derivatization  of  a  fluorescent  dye  (fluorescein)  to  obtain  target- specific  response  or  localization  characteristics.  Clockwise  from  top  left:  The 
dodecanoyl  (Ci2)  substituent  of  CJ2  fluorescein  results  in  membrane  localization.  The  intracellular  Ca2+  indicator  Calcium  Green- 1  consists  of  dichlorofluores- 
cein  coupled  to  the  Ca2+-selective  chelator  BAPTA.  Fluorescein  digalactoside  (FDG)  is  a  p-galactosidase  activity  sensor  that  generates  an  enzyme-dependent 
fluorescence  intensity  increase  upon  cleavage  of  the  3'-  and  6'-galactose  residues.  Fluorescein-labeled  substance  P  is  a  probe  for  neuropeptide  receptors  prepared 
by  coupling  5 -carboxy fluorescein,  NHS -ester  to  the  £-amino  group  of  lysine  via  the  reaction  chemistry  outlined  in  Figure  17.3(D). 
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TABLE  17.2.  Major  Dye  Classes 


Dye  Class 

Examples 

Brightness" 

Environment 

Sensitivity^ 

Photostability 

Biocompatabilityc 

Two-Photon 

Coumarin 

AlexaFluor  350 

+ 

+  +  +  + 

+ 

+  +  + 

+ 

Fluorescein 

FITC,  calcein  AM,  CFSE'' 

++  + 

+  +  +  + 

+ 

+  +  + 

+  + 

BOPIDY 

BOPIDY  FL,  BOPIDY  TR 

++  + 

+  +  + 

+  + 

+  +  +  + 

+  + 

Phycobiliproteins 

R-phycoerythrin  (R-PE) 

++++ 

+  + 

+  + 

+  + 

+  +  +  + 

AlexaFluor 

AlexaFluor  488.  AlexaFluor  568, 
AlexaFluor  594,  AlexaFluor  647 

+  +  +  + 

+  + 

+  +  +  + 

+  +  +  + 

+  +  + 

Quantum  dots 

Qdot  605.  Qdot  655 

+++++ 

+ 

+  +  +  +  + 

+ 

+  +  +  +  + 

Cyanines 

Cy3,  Cy5,  Dil  (DiIC18(3)),  DiOC6(3) 

++  + 

+  + 

+  +  +  + 

+  +  + 

+ 

Styryl  dyes 

FM  1-43,  FM  4-64,  Di-8-ANEPPS 

+ + 

+  +  +  +  + 

+  + 

+  +  + 

+  +  + 

Rhodamines 

TRITC,  TMRE',  Rhodamine  123 

++  + 

+  +  + 

+  +  +  + 

+  + 

+  +  + 

Fluorescent  proteins 

ECFP,  EGFP,  EYFP,  dsRed 

+  + 

+  + 

+  + 

+  +  +  +  + 

+  + 

"Approximate  representation  of  the  product  of  molar  extinction  coefficient  and  fluorescence  quantum  yield.  In  cellular  imaging  applications,  this  will  be  heavily  weighted 
by  concentration  (target  abundance,  expression  level)  and  localization  factors  that  are  discussed  in  the  text  but  are  not  included  in  this  consideration. 
b  Assessment  of  the  extent  to  which  brightness  is  modulated  by  factors  such  as  pH,  aqueous  exposure,  dye  interactions  etc. 

CA  compound  assessment  of  factors  such  as  cytotoxicity,  ease  of  delivery,  and  extent  to  which  the  dye  modifies  the  structure  and  behavior  of  molecules  to  which  it  is 
attached.  GFP  has  the  highest  rating  because  it  is  the  only  dye  class  of  entirely  biosynthetic  origin. 

^CFSE,  carboxyfluorescein  diacetate,  succinimidyl  ester. 
eTMRE,  tetramethylrhodamine  ethyl  ester. 


applications,  the  lack  of  environmental  sensitivity  of  these  dyes  is 
advantageous,  as  fluorescence  images  of  their  distribution  are  not 
biased  by  environment-dependent  factors.  Other  classes  such  as 
styryl  dyes  (e.g.,  FM  1-43,  FM  4-64)  and  many  non-covalent 
nucleic  acid  stains  (e.g.,  DAPI,  TO-PRO-3,  propidium  iodide)  are 
essentially  non-fluorescent  in  water  and  only  fluoresce  when 
bound  to  their  respective  targets  (membrane  surfaces  and  DNA). 
In  this  case,  unbound  dyes  are  undetectable  and  the  need  to 
perform  wash  steps  to  remove  them  prior  to  imaging  is  therefore 
eliminated. 

Within  the  confines  of  a  cell,  dyes  frequently  come  into  close 
spatial  proximity  with  each  other,  resulting  in  dye-dye  interac¬ 
tions.  Attachment  of  multiple  copies  of  fluorescein  or  rhodamine 
dyes  to  a  single  protein  molecule  usually  results  in  the  formation 
of  dimers  or  higher  aggregates  of  the  dye  that  absorb  but  do  not 
fluoresce.  The  process  is  commonly  referred  to  as  self-quenching. 


Dye  aggregation  is  signified  by  peaks  or  shoulders  in  the  absorp¬ 
tion  spectrum  of  the  labeled  protein  that  are  not  replicated  in  the 
fluorescence  excitation  spectrum,  and  also  by  changes  in  the  cir¬ 
cular  dichroism  spectrum  (Mercola  et  al.,  1972).  Self-quenching 
produces  the  counterintuitive  but  sometimes  useful  result  of  fluo¬ 
rescence  output  decreasing  as  dye  concentration  increases.  This  is 
clearly  detrimental  if  one  is  attempting  to  increase  the  fluorescent 
brightness  of  a  protein  by  labeling  it  more  heavily.  There¬ 
fore,  AlexaFluor  dyes  and  Cy  dyes  are  designed  to  avoid 
self-quenching  by  incorporating  negatively  charged  sulfonic  acid 
substituent  groups  that  produce  mutual  electrostatic  repulsion  and 
increased  water  solubility  (Fig.  17.3).  Self-quenching  can  be 
exploited  to  advantage  for  detection  of  proteolytic  activity  by 
preparation  of  protein  conjugates  that  are  deliberately  over¬ 
labeled.  Probes  of  this  type  allow  in  situ  localization  of  enzymatic 
activity  in  tissues  (Mook  et  al.,  2003)  and  organisms  (Farber  et  al.. 


FIGURE  17.2.  (A)  Matching  microscope  excitation  source  outputs  to  dye  spectra.  Fluorescence  excitation  spectra  of  AlexaFluor  488  (1),  AlexaFluor  568  (2), 
and  FM  1-43  (3)  dyes  relative  to  the  488  nm  argon-ion  laser  line  (shown  by  the  vertical  dashed  line).  Due  to  the  relatively  inefficient  excitation  of  the 
AlexaFluor  568  dye  at  488  nm,  it  is  not  suitable  for  multiplex  labeling  in  combination  with  FM  1-43  or  AlexaFluor  488  nm  dyes  unless  the  microscope  is  also 
equipped  with  a  568  nm  krypton-ion  laser  or  561  nm  diode-pumped  solid  state  laser.  (B)  Emission  overspill  considerations  in  multicolor  detection.  The  wide 
emission  spectral  bandwidth  of  FM  1-43  (3)  results  in  substantial  overlap  with  both  AlexaFluor  488  (1)  and  AlexaFluor  568  (2)  dyes,  making  discrimination  by 
means  of  wavelength  filtering  difficult  to  achieve.  The  AlexaFluor  488  and  AlexaFluor  568  dyes  are  spectrally  well  resolved.  Note,  however,  that  the  relative 
amplitudes  of  the  spectra  shown  in  the  figure  would  only  pertain  in  a  real  experimental  situation  if  the  dyes  were  excited  with  equal  efficiency  and  were  present 
at  equal  concentrations  within  the  field  of  view. 
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FIGURE  17.3.  (A)  Structures  of  amine-reactive  dyes.  (A)  AlexaFluor  488,  NHS  ester;  (B)  BOPIDY  FL,  NFIS  ester;  (C)  Cy5,  NHS  ester  (NFIS  = 
/V- h yd ro x y s uc c i n i rn  i d y ] ) .  (D)  Reaction  of  a  dye-NHS  ester  with  protein  amino  groups  yielding  a  stable  carboxamide-linked  conjugate. 


2001).  Fluorescent  proteins  and  quantum  dots  have  much  less 
intrinsic  environmental  sensitivity  than  organic  dyes  because  in 
both  cases  the  fluorophore  is  encapsulated  within  one  or  more  insu¬ 
lating  layers.  They  are  therefore  more  difficult  to  adapt  for  local 
environment  sensing  applications  of  the  type  described  above.  To 
do  so,  it  is  necessary  to  make  use  of  fluorescence  resonance  energy 
transfer  (FRET;  Jales-Erijman  and  Jovin,  2003),  a  type  of  dye-dye 
interaction  with  a  longer  effective  range  than  those  responsible  for 
self-quenching. 

Stable  coupling  of  organic  dyes  to  proteins,  nucleic  acids,  and 
other  biomolecules  can  be  accomplished  using  several  well- 
characterized  reaction  chemistries.  Currently,  the  most  widely  used 
method  is  the  reaction  of  NFIS  ester  dye  derivatives  with  amines 
(e.g.,  £-NH2  of  lysine)  to  form  stable  carboxamide  linkages  [Fig. 
17.3(D)].  The  principal  alternative  coupling  method  is  reaction 
of  iodoacetamide  or  maleimide  derivatives  of  dyes  with  cysteine 
sulfhydryl  groups  to  form  thioether  linkages.  Textbook-level 
descriptions  of  these  and  other  coupling  chemistries  may  be  con¬ 
sulted  for  further  details  (Hermanson,  1996;  Haugland,  2002). 
Labeling  protocols  generally  require  some  degree  of  protein- 
specific  optimization  with  respect  to  the  dye:protein  ratio.  This  is 
merely  a  reflection  of  the  fact  that  proteins  vary  widely  in  size  and 
in  the  number  and  distribution  of  reactive  targets.  Typically,  no 
more  than  about  three  to  five  dyes  can  be  attached  per  protein 


without  self-quenching  of  fluorescence  and/or  inactivating  the 
protein.  Note  that  degree-of-labeling  values,  as  conventionally 
determined  by  absorption  measurements,  are  sample  averages.  A 
preparation  with  an  average  of  n  dyes  per  protein  will  contain  a 
range  of  labeling  stoichiometries  (.  .  .  n  -  2,  n  —  1,  n,  n  +  1,  n  +  2 
.  .  .  etc.)  that  can  be  resolved  by  capillary  electrophoresis  or 
MALDI-TOF  mass  spectrometry  (Lu  and  Zenobi,  2000).  Further¬ 
more,  because  the  labeling  reactions  are  indiscriminate  with 
respect  to  the  location  of  the  target  amino  acid  residue,  labeling  in 
a  site  that  is  critical  for  activity  or  binding  may  result  in  an  in¬ 
active  conjugate  (Adamczyk  et  al.,  1999).  In  these  cases,  activity 
can  be  preserved  by  performing  the  labeling  reaction  with  the 
active  site  blocked  by  addition  of  its  cognate  substrate  or  ligand 
(Ramjeesingh  et  al.,  1990). 

Fluorescent  Proteins:  Green  Fluorescent  Protein 
and  Phycobiliproteins 

The  cloning  and  heterologous  expression  of  the  Aequoria  victoria 
green  fluorescent  protein  in  1994  has  had  an  immense  impact  on 
fluorescence  microscopy  of  living  cells  in  the  past  decade.  There 
are  many  excellent  review  articles  (Tsien,  1998;  Zhang  et  al.,  2002; 
Lippincott-Schwartz  and  Patterson,  2003)  and  entire  books 
(Chalfie  and  Kain,  1998;  Sullivan  and  Kay,  1999)  devoted  to  the 
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properties  and  applications  of  GFP.  Only  selected  points  setting 
GFP  in  relation  to  other  fluorescent  labeling  technologies  will 
be  considered  here.  The  spectroscopic  properties  of  GFP  and  its 
variants  are  unexceptional  (emax  ~  53,000 M_1cm_1,  QY  ~  0.6  for 
the  most  frequently  used  variant,  EGFP  (Patterson  et  al.,  1997). 
Flowever,  the  extraordinary  utility  of  GFP  resides  in  its  biochem¬ 
ical  properties  —  more  specifically,  in  the  ability  to  construct 
fusion  proteins  at  the  DNA  level  and  express  these  in  an  almost 
unlimited  range  of  cell  types,  tissues,  and  organisms.  In  principal, 
information  on  intracellular  protein  localization  that  has  tradition¬ 
ally  been  obtained  using  dye-labeled  antibodies  in  fixed  cells 
can  now  be  obtained  in  living  cells,  with  the  added  dimension  of 
following  translocation  processes  in  real  time  as  opposed  to 
observing  only  their  end-points.  Accurate  localization  of  GFP 
fusion  proteins  relative  to  their  native  counterparts  is  clearly  an 
essential  prerequisite  for  these  applications.  GFP  is  a  small  protein 
(molecular  weight  ~27  kDa)  consisting  of  1 1  P-strands  forming  a 
hollow  cylinder  approximately  4.2  nm  in  length  by  2.4 nm  in  diam¬ 
eter  that  encapsulate  the  p-hydroxybenzylideneimidazolidinone 
fluorophore.  These  dimensions  are  relatively  large  compared  to 
organic  dyes  that  typically  have  molecular  weights  <1000  and  are 
represented  by  a  flat  disk  of  ~1  nm  diameter.  Intracellular  GFP  con¬ 
centrations  are  determined  by  a  combination  of  factors  including 
levels  of  gene  expression  and  rates  of  posttranslational  folding  and 
proteolysis.  The  spontaneous  cyclization  reaction  responsible  for 
formation  of  the  fluorophore  from  residues  65  to  67  (Ser-Tyr-Gly 
in  the  wild-type  A.  victoria  protein)  requires  molecular  oxygen. 
Thus,  only  16%  of  wild-type  GFP  synthesized  in  vitro  under  con¬ 
ditions  of  limited  oxygen  availability  is  fluorescent,  compared  to 
88%  in  the  presence  of  abundant  oxygen  (Nemetz  et  al.,  2001). 
Therefore,  GFP  fluorescence  in  hypoxic  cells  and  tissues  such  as 
tumors  may  not  accurately  reflect  the  level  of  gene  expression 
(Coralli  et  al..  2001).  Site-directed  mutagenesis  of  A.  victoria  GFP 
has  generated  variants  with  shifted  excitation  and  emission  spectra. 
The  cyan  (CFP)  and  yellow  (YFP)  variants  are  particularly  useful 
for  co-localization  of  proteins  via  FRET  imaging  (Siegel  et  al., 
2000).  Spectral  coverage  has  been  extended  to  the  red  range  by 
cloning  and  expression  of  a  fluorescent  protein  known  as  DsRed 
from  Discosoma  coral  (Matz  et  al.,  1999).  Major  re-engineering 
of  DsRed  has  been  undertaken  to  eliminate  some  undesirable  char¬ 
acteristics  including  slow  maturation  and  obligate  tetramerization 
(Campbell  et  al.,  2002). 

Phycobiliproteins  are  a  family  of  photosynthetic  accessory  pro¬ 
teins  from  cyanobacteria  and  eukaryotic  algae.  The  most  commonly 
used  phycobiliprotein  for  biomolecular  labeling  applications,  R- 
phycoerythrin  is  a  large  cylindrical  protein  with  a  molecular  weight 
of  240  kDa  and  dimensions  of  6nm  in  length  by  12nm  in  diameter. 
Each  molecule  contains  34  tetrapyrrole  chromophores  attached  to 
the  polypetide  backbone  via  thioether  linkages.  To  fulfill  their  bio¬ 
logical  role  of  transferring  energy  from  absorbed  light  to  chloro¬ 
phyll  in  the  photosynthetic  reaction  center,  the  positioning  of  the 
chromophores  within  the  protein  structure  has  evolved  to  maximize 
the  efficiency  of  both  absorption  and  fluorescence.  Due  to  its  excep¬ 
tional  light  harvesting  capacity,  R-phycoerythrin  can  be  efficiently 
excited  at  488  nm  (e488  nm  ~  1 ,100,000  M-1  cm-1)  even  though  this  is 
far  from  its  absorption  maximum  of  565  nm.  It  is  therefore  well 
suited  for  single  excitation/dual  emission  detection  in  combination 
with  fluorescein,  AlexaFluor  488,  GFP  and  other  dyes  with  emission 
peaks  around  520  nm.  Due  to  their  large  size,  phycobiliproteins  are 
difficult  to  deliver  to  intracellular  targets  and  are  primarily  of  value 
for  detection  of  low-abundance  cell-surface  antigens.  Heterologous 
expression  of  C-phycocyanin  has  been  accomplished  in  E.  coli 
(Tooley  et  al.,  2001).  This  is  a  much  more  technically  demanding 


endeavor  than  GFP  expression  because  it  requires  co-expression  of 
enzymes  required  for  the  biosynthesis  of  the  tetrapyrrole  chro¬ 
mophores  from  heme. 

Techniques  for  targeting  dyes  to  specific  binding  motifs  on 
intracellular  proteins  that  combine  elements  of  entirely  intracellular 
expression  of  intrinsically  fluorescent  proteins  and  extracellular 
labeling  of  purified  proteins  by  amine-reactive  NHS  chemistry 
are  now  emerging.  These  techniques  include  tetracysteine  motif¬ 
binding  biarsenical  ligands  (FlAsH;  Adams  et  al.,  2002)  and 
engineered  intracellular  receptor  proteins  that  recognize  dyes  or 
dye-coupled  haptens  (Beste  et  al.;  1999;  Farinas  and  Verkman, 
1999). 

Quantum  Dots 

In  complete  contrast  to  fluorescent  proteins,  quantum  dots  have 
outstanding  spectroscopic  properties,  whereas  their  biochemical 
properties  of  large  size  and  inorganic  composition  represent  a 
considerable  challenge  in  biomolecular  labeling  applications. 
Quantum  dots  typically  consist  of  a  spherical  core  of  the  semi¬ 
conductor  cadmium  selenide  (CdSe)  surrounded  by  a  zinc  sulfide 
(ZnS)  shell  which  is  in  turn  surrounded  by  a  hydrophilic  polymer 
surface  coating  (Bruchez  et  al.,  1998;  Chan  and  Nie,  1998).  As 
well  as  stabilizing  the  electronic  excited  state  of  the  semiconduc¬ 
tor,  the  ZnS  shell  also  prevents  release  of  cytotoxic  cadmium  from 
the  core  (Derfus  et  al.,  2004).  The  hydrophilic  coating  confers 
water  solubility  and  incorporates  functional  groups  for  crosslink¬ 
ing  to  antibodies,  streptavidin,  and  other  targeting  groups.  Overall, 
the  trilaminar  particles  are  about  4  to  10  nm  in  diameter,  compa¬ 
rable  in  size  to  the  phycobiliproteins.  The  fluorescence  output  of 
a  single  quantum  dot  has  been  estimated  to  be  approximately 
equivalent  to  20  rhodamine  dye  molecules,  similar  to  that  of  R- 
phycoerythrin  (Chan  and  Nie,  1998).  Due  to  their  predominantly 
inorganic  composition,  quantum  dots  are  largely  invulnerable  to 
the  oxidative  photobleaching  reactions  that  affect  organic  dyes  and 
fluorescent  proteins.  The  diameter  of  the  semiconductor  core  deter¬ 
mines  the  fluorescence  emission  range  of  the  particle  (wavelength 
increases  with  increasing  size).  The  excitation  spectra  of  quantum 
dots  are  essentially  continuous  whereas  the  emission  spectra  are 
narrow  and  symmetrical  (Fig.  17.4).  Quantum  dots  exhibit 
exceptionally  high  molar  absorptivity  —  for  example,  e6oonm  = 
650,000 M_1cm_1  and  e4oonm  =  3,500,000 M^'cnT1  (Watson  et  al., 
2003).  Furthermore,  their  two-photon  excitation  cross-sections 
exceed  those  of  conventional  organic  dyes  by  factors  of  100  to 
1000  (Larson  et  al.,  2003).  Because  quantum  dots  are  extremely 
flexible  with  respect  to  excitation  wavelength,  they  are  well  suited 
for  simultaneous  immunofluorescence  detection  of  multiple  targets 
(Wu  et  al.,  2003).  Careful  optimization  of  staining  conditions 
(buffer  composition,  etc.)  is  necessary  to  minimize  non-specific 
labeling.  The  size  and  composition  of  quantum  dots  generally 
limits  the  range  of  accessible  targets  to  cell-surface  antigens  and 
receptors  and  targets  that  can  be  accessed  via  endocytosis  or  injec¬ 
tion  (Jaiswal  et  al.,  2003;  Wu  et  al.,  2003;  Ballou  et  al.,  2004). 
Development  of  intracellular  delivery  techniques  using  peptide 
vectors  has  recently  been  reported  (Mattheakis  et  al.,  2004). 

Multi-Photon  Excitation 

The  advent  of  two-photon  excitation  microsopy  (Denk  et  al.,  1990) 
has  added  a  new  spectral  dimension  to  fluorescence  microscopy, 
in  addition  to  the  benefits  of  increased  imaging  depth  and  reduced 
phototoxicity  that  derive  from  the  transparency  of  tissues  to 
infrared  excitation  light  and  confinement  of  excitation  to  the  focal 
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FIGURE  1 7.4.  Fluorescence  excitation  and  emission  spectra  of  QD605-strep- 
tavidin  and  QD655-streptavidin  (QD  =  quantum  dot).  The  continuous  and 
almost  identical  excitation  spectra  and  the  sharp,  well-differentiated  emission 
spectra  contrast  markedly  with  the  corresponding  characteristics  of  organic 
dyes  [Fig.  17.2(A,B)]. 


plane  (Zipfel  et  al.,  2003a).  Probes  that  were  previously  of  limited 
utility  in  confocal  microscopy  due  to  their  requirements  for  ultra¬ 
violet  excitation,  such  as  fura-2  (Ca2+),  SBFI  (Na+),  monochloro- 
bimane  (glutathione),  and  DAPI  (nuclear  DNA),  have  acquired  a 
new  lease  of  life  (Rose  et  al.,  1999;  Fricker  and  Meyer,  2001; 
Rubart  et  al.,  2003).  Furthermore,  in  situ  imaging  of  small  endog¬ 
enous  fluorophores  such  as  serotonin  and  NADFI,  that  are  almost 
inaccessible  to  one-photon  excitation,  has  now  become  practica¬ 
ble  (Zipfel  et  al.,  2003b;  Rocheleau  et  al.,  2004).  With  these 
exceptions,  the  collection  of  dyes  and  probes  currently  in  use  for 
two-photon  excitation  microscopy  is  largely  the  same  as  that 
employed  in  confocal  and  wideheld  fluorescence  imaging.  There 
are  several  published  collections  of  two-photon  excitation  spectra 
and  cross-sections  that  provide  guidance  on  compatibility  of  dyes 
and  probes  with  excitation  sources  (Xu  and  Webb,  1996;  Xu  et  al., 
1996;  Bestvater  et  al.,  2002;  Dickinson  et  al.,  2003).  If  two-photon 
spectral  data  is  not  available,  the  corresponding  one-photon  spec¬ 
trum  plotted  on  a  doubled  wavelength  axis  can  be  used  as  a  first 
approximation.  However  two-photon  excitation  spectra  differ  from 
their  one-photon  counterparts  to  an  extent  that  depends  on  the  mo¬ 
lecular  orbital  symmetry  of  the  fluorophore  (greater  difference  for 
higher  symmetry  fluorophores;  Zipfel  et  al.. 2003a).  Consequently, 
most  two-photon  excitation  spectra  are  blue  shifted  and  broader 
compared  to  the  corresponding  one-photon  spectra.  Simply  stated, 
a  dye  with  a  one-photon  excitation  peak  at  500  nm  will  probably 
have  a  two-photon  excitation  maximum  at  clOOOnm  (Ruthazer 
and  Cline,  2002).  In  practice,  most  dyes  with  single-photon  exci¬ 
tation  maxima  <600  nm  can  be  effectively  excited  within  the 
700  nm  to  lOOOnm  output  range  of  the  mode-locked  titanium- 
sapphire  laser  sources  used  in  the  majority  of  two-photon  excita¬ 
tion  microscopes.  In  general,  optimizing  the  excitation  wavelength 
in  relation  to  the  peak  of  the  dye  excitation  spectrum  appears  to 
be  less  critical  than  in  the  one-photon  case. 

INTRODUCING  THE  PROBE  TO  THE  SPECIMEN 
Loading  Methods 

After  selecting  a  probe  with  appropriate  biochemical  and  spectro¬ 
scopic  properties  with  regard  to  the  experimental  objectives  and 
the  microscope  hardware  to  be  used,  the  next  task  is  to  label  the 


features  of  interest  within  the  specimen.  This  is  a  relatively 
straightforward  process  in  the  fixed  and  permeabilized  specimens 
g  used  for  immunofluorescence,  for  which  there  are  well-established 
standard  protocols  (Brelje  et  al.,  2002).  The  major  practical  con- 
£  cerns  for  immunolabeling  of  fixed  specimens  include  antigen 
®  preservation  and  accessibility,  non-specific  binding  and  cross- 
§  reactivity  of  antibodies,  and  the  potential  for  increased  autofluo- 
«  rescence  resulting  from  the  chemical  reactions  used  to  achieve 
fixation  (see  also  Chapter  18,  this  volume). 

The  permeability  barriers  and  active  metabolic  processes  of 
living  cells  and  tissues  present  additional  challenges  for  probe 
delivery  and  localization  (Stephens  and  Pepperkok,  2001).  Gener¬ 
ally  speaking,  the  severity  of  the  intracellular  delivery  problem 
increases  in  proportion  to  the  size  of  the  probe  and  the  complex¬ 
ity  of  the  specimen  (cell  <  tissue  <  whole  organism).  Ions  are  quite 
easy  to  deliver  non-invasively  via  endogenous  ion  channels 
or  using  ionophores.  Small  organic  dye-based  probes  are  easy  to 
moderately  difficult,  depending  on  their  structural  characteristics. 
For  example,  the  rate  of  loading  of  the  acetoxymethyl  (AM)  ester 
forms  of  intracellular  Ca2+  indicators  into  intact  frog  muscle  fibers 
decreases  dramatically  when  their  molecular  weight  exceeds  about 
850  (Zhao  et  al.,  1997).  In  contrast,  DAPI  and  lucifer  yellow  CH 
have  similar  molecular  weights  but  can  exhibit  markedly  different 
permeability  through  gap  junctions  due  to  their  opposite  electro¬ 
static  charges  (Cao  et  al.,  1998).  Intracellular  delivery  of  large 
probes  such  as  phycobiliproteins,  labeled  antibodies,  or  quantum 
dots  almost  invariably  requires  invasive  methods.  An  exception  to 
the  latter  stipulation  is  GFP,  which  is  synthesized  and  folded  in 
situ,  thereby  eliminating  the  need  for  intracellular  delivery  except 
at  the  initial  DNA  transfection  stage.  Methods  for  transfection  and 
expression  of  GFP  will  not  be  discussed  here;  detailed  descriptions 
are  available  elsewhere  (Sullivan  and  Kay,  1999;  and  in  Chapter 
45,  this  volume). 

Several  factors  can  be  identified  that  influence  loading  method 
choices: 

•  Is  the  experimental  specimen  a  pure  or  mixed  cell  culture 
or  a  tissue? 

•  If  the  specimen  is  multi-cellular,  is  disseminated  (all  cells) 
or  selective  (one  cell  or  a  subgroup  of  cells)  loading  desired? 

•  What  is  the  size  of  the  cells  to  be  labeled? 

•  What  is  the  size  of  the  probe? 

•  Impact  of  loading  on  cell  viability  and  function? 

•  Precision  of  amount  delivered  and  location  of  delivery 
(“focal  application’’)? 

Before  describing  specific  methods,  a  generally  applicable 
guideline  should  be  noted.  Label  the  specimen  to  the  minimum 
extent  required  to  obtain  the  biological  information  that  is 
sought.  It  is  almost  never  desirable  to  increase  fluorescence 
signals  by  increased  dye  loading  or  fluorescent  protein  expression. 
The  following  deleterious  effects  are  all  positively  correlated 
with  increased  label  concentration:  phototoxicity,  cytotoxicity, 
non-specific  localization,  and  physiological  or  structural 
perturbation. 

1.  Direct  permeability:  This  is  the  simplest  of  all  techniques, 
involving  nothing  more  than  dispersing  the  probe  in  the  extracel¬ 
lular  medium  and  incubating  for  5  to  60  minutes.  It  is  generally 
applicable  to  neutral,  monoanionic,  and  monocationic  molecules 
with  molecular  weight  <1000.  Examples  include  JC- 1 ,  MitoTracker 
Red  CMXRos,  LysoTracker  Red,  BOPIDY  FL  ceramide,  and 
DiICi8(3).  Loading  of  lipophilic  probes  such  as  fluorescent  lipid 
analogs  (Tanhuanpaa  and  Somerharju,  1999)  and  acetoxymethyl 
(AM)  esters  (see  item  3,  below)  is  enhanced  by  the  addition  of 
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carriers  and  dispersing  agents  such  as  cyclodextrins,  nonionic 
detergents  (Pluronic  F-127,  Cremophor  EL),  and  proteins  such  as 
bovine  serum  albumin  (BSA).  Also  included  in  this  general  cate¬ 
gory  are  larger  probe  molecules  that  are  taken  up  by  endocytic 
processes  such  as  dye-labeled  transferrin  conjugates. 

2.  ATP-gated  cation  channels:  Provide  a  direct  loading 
conduit  for  dyes  with  molecular  weight  <1000.  Applicability 
restricted  to  cell  types  in  which  these  channels  are  found  (neurons 
and  other  sensory  cells,  dendritic  cells,  macrophages).  Dyes  to 
which  this  technique  has  been  applied  include  FM  1-43  (Meyers 
et  al.,  2003),  fura-2,  and  the  pH  indicator  HPTS  (pyranine;  Gan 
et  al.,  1998). 

3.  Membrane  permeant  esters:  An  AM  or  acetate  ester 
derivative  of  a  polyanionic  dye  diffuses  across  the  plasma  mem¬ 
brane  followed  by  intracellular  release  of  free  dye  mediated 
by  endogenous  esterase  activity  (Fig.  17.5).  The  requirement  for 
esterase  activity  imposes  limitations  on  the  cell  types  to  which  the 
technique  can  be  applied,  but  provides  a  rapid  and  convenient 
method  for  uniform  loading  of  adherent  or  suspended  cells.  Intra¬ 
cellular  distribution  of  the  dye  can  be  manipulated  to  some  extent 
by  variations  of  temperature,  dye  structure,  and  incubation  time 
(Lemasters  et  al.,  1999),  but  cannot  be  precisely  controlled 
(Fig.  17.5). 

4.  Peptide-mediated  uptake:  This  relatively  new  technique 
is  based  on  attachment  of  peptide  vectors  to  molecular  cargoes, 
which  may  be  anything  from  small  dyes  to  quantum  dots 
(Mattheakis  et  al.,  2004)  or  large  (>100 kDa)  proteins.  The  cellu¬ 


lar  uptake  mechanism  is  not  fully  understood  and  remains  under 
active  investigation  (Potocky  et  al.,  2003).  Efficacious  peptide 
sequences  (Fischer  et  al.,  2001)  consist  of  8  to  30  amino  acids  with 
a  preponderance  of  basic  residues,  and  are  largely  derived  from 
the  proteins  of  infectious  microorganisms.  At  present,  this  tech¬ 
nique  is  far  from  being  universally  applicable  and  the  attainable 
intracellular  dye  concentrations  are  rather  low  (10  to  lOOnM; 
Waizenegger  et  al.,  2002). 

5.  Transient  permeabilization:  Brief  treatments  of  cells  with 
low  doses  (<0.1mg/mL)  of  bacterial  toxins  (e.g.,  streptolysin  O, 
staphylococcal  a-toxin)  or  plant  glycosides  (P-escin,  saponin) 
allows  delivery  of  probes  and  other  molecules.  Although  these 
treatments  are  easy  to  implement,  the  physiological  integrity  and 
viability  of  the  cells  is  inevitably  compromised  to  some  extent. 
Walev  and  colleagues  (2001)  have  provided  a  detailed  analysis  of 
the  efficacy  and  collateral  effects  of  streptolysin-O  permeabiliza¬ 
tion  of  adherent  and  non-adherent  cell  types.  The  molecular  weight 
cutoff  for  loading  was  found  to  be  -100,000  with  intracellular  con¬ 
centrations  of  105  to  106  molecules  per  cell. 

6.  Osmotic  permeabilization:  Similar  to  bacterial  toxin 
permeabilization  in  applicability  (non-selective  loading  of  cell 
populations)  and  potential  for  compromising  cellular  integrity. 
Applicable  to  large  proteins  such  as  antibodies  (Chakrabati  et  al., 
1989). 

7.  Electroporation:  Perhaps  the  most  versatile  technique  cur¬ 
rently  available.  Applicable  to  single  cells  (Haas  et  al.,  2001),  bulk 
loading  of  cell  populations  (Hashimoto  et  al.,  1989),  and  focal 
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FIGURE  1 7.5.  Cell  loading  with  acetoxymethyl  (AM)  esters.  The  example  shown  represents  the  calcium  indicator  fluo-4.  The  intended  objective  of  most  mea¬ 
surements  using  fluo-4  is  imaging  of  cytosolic  Ca2+  dynamics.  Loading  of  the  dye  via  process  (1),  consisting  of  spontaneous  permeation  of  the  AM  ester  deriv¬ 
ative  of  fluo-4  (shown  in  red  and  blue)  across  the  plasma  membrane  followed  by  cleavage  of  the  acetoxymethyl  groups  (red)  by  intracellular  esterases  is  therefore 
the  desired  outcome.  However,  other  dye  transport  processes  may  occur  simultaneously.  These  include  extracellular  hydrolysis  of  the  AM  ester  (2),  extrusion  of 
intracellular  fluo-4  via  organic  anion  transporters  (3),  and  compartmentalization  in  membrane-enclosed  organelles  and  vesicles  (4).  Processes  (2-4)  all  result  in 
fluorescence  signals  that  are  not  responsive  to  cytosolic  Ca2+  (see  Fig.  17.6).  The  actual  cellular  labeling  pattern  obtained  is  determined  by  the  relative  rates  of 
the  permeation,  transport,  enzymatic,  and  chemical  hydrolysis  that  contribute  to  pathways  1—4.  These  in  turn  are  dependent  on  experimental  variables  such  as 
cell  type,  cell  viability,  incubation  medium  composition,  and  temperature. 
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(spatially  localized)  delivery  within  tissues  and  organisms  (Yasuda 
et  a /.,  2000).  Applicable  to  molecules  large  and  small,  from  dyes 
and  peptides  up  to  antibodies  (Marrero  et  al.,  1995). 

8.  Ballistic  microprojectile  (gene  gun)  delivery:  Primarily 
useful  for  labeling  groups  of  cells  deep  (up  to  300  pm)  inside 
tissues.  Widely  used  for  transfection  of  GFP  vectors  in  brain  slices 
(O’Brien  et  al.,  2001).  Applications  for  loading  neuronal  tracers 
and  intracellular  calcium  indicators  have  also  been  reported 
(Grutzendler  et  al.,  2003). 

9.  Microinjection:  Invasive,  sequential  (one  cell  at  a  time) 
but  extremely  precise.  Provides  exquisite  control  over  the  amount 
of  probe  delivered  and  in  the  selection  of  individual  cells  for  label¬ 
ing.  Microinjection  is  quite  demanding  in  terms  of  technical  skill 
and  instrumentation,  and  is  difficult  to  apply  to  small  cells  (e.g., 
hepatocytes).  Either  pneumatic  pressure  or  electrical  current  (ion¬ 
tophoresis)  may  be  used  as  the  driving  force  for  injection.  These 
two  methods  may  produce  significantly  different  cellular  labeling 
results,  even  when  they  are  applied  to  the  same  dye  (Gerhardt  and 
Palmer,  1987;  Storms  et  al.,  1998).  Iontophoretic  microinjection  is 
restricted  to  electrostatically  charged  dyes  such  as  lucifer  yellow 
CH  and  AlexaFluor  594  hydrazide  and  is  difficult  to  apply  to  large 
molecules  (molecular  weight  >1000). 

10.  Whole-cell  patch  pipette  delivery:  Similar  to  microin¬ 
jection  in  applicability  with  the  additional  capacity  for  simultane¬ 
ous  imaging  and  electrophysiological  measurements  (Eilers  and 
Konnerth,  2000).  Inward  diffusion  of  solutes  from  the  patch  pipette 
allows  precise  control  of  the  intracellular  environment  but  outward 
diffusion  of  cytoplasmic  contents  may  be  deleterious. 

A  selection  of  probes  with  proven  utility  for  general  charac¬ 
terization  of  living  specimens  using  simple  and  rapid  labeling  pro¬ 
tocols  is  listed  in  Table  17.3. 

Tissues 

Scientific  imperatives  allied  to  technical  developments  such  as 
two-photon  excitation  and  confocal  endoscopy  (Helmchen,  2002) 
mean  that  fluorescence  microscopy  is  increasingly  being  applied 
to  tissues  and  entire  organisms.  Most  of  the  techniques  described 
above  can  be  adapted  for  application  to  tissue  specimens,  while 
GFP  expression  can  be  confined  to  specific  cell  types  by  coupling 
to  tissue-specific  promoters  (Hara  et  al.,  2003).  Water-soluble  dyes 


such  as  FM  1-43  and  lucifer  yellow  CH  generally  show  deeper 
penetration  into  tissues  than  non-polar  molecules  such  as  AM 
esters,  which  tend  to  accumulate  in  the  superficial  cell  layers 
(Takahashi  et  al.,  2002).  A  modification  of  the  AM  ester  technique, 
referred  to  as  multi-cell  bolus  loading  (MCBL),  utilizes  localized 
ejection  of  small  volumes  (~0.4  nL)  of  dye  loading  solution  from 
a  micropipette  to  label  populations  of  neurons  in  brain  tissue 
(Stosiek  et  al.,  2003).  For  imaging  tissues  in  situ,  probe  delivery 
is  typically  accomplished  via  the  internal  pathways  of  the  diges¬ 
tive  (Farber  et  al.,  2001),  respiratory  (Lombry  et  al.,  2002),  circu¬ 
latory  (Ballou  et  al.,  2004),  or  nervous  (Grutzendler  et  al.,  2003) 
systems. 

Target  Abundance  and  Autofluorescence 
Considerations 

We  have  considered  spectroscopic  properties  relating  to  the  fluo¬ 
rescence  output  of  labels  such  as  excitation  wavelength,  exctinc- 
tion  coefficent,  photobleaching,  and  fluorescence  quantum  yield. 
However,  factors  such  as  the  abundance  and  spatial  distribution  of 
the  target  and  the  levels  of  background  autofluorescence  often  have 
more  impact  on  the  contrast  and  resolution  of  the  final  image.  The 
abundance  and  degree  of  localization  of  the  target  within  the  spec¬ 
imen  are  critical  determinants  in  probe  selection.  For  example,  it 
is  much  easier  to  image  DNA  localized  in  the  nucleus  than  recep¬ 
tors  distributed  on  the  plasma  membrane  surface.  The  DNA 
content  of  a  typical  mammalian  cell  is  about  7  pg,  corresponding 
to  about  6  X  109  base  pairs.  This  amount  of  DNA  can  accommo¬ 
date  the  binding  of  up  to  1.2  X  109  intercalating  dyes  (1  dye:  5  base 
pairs).  Consequently,  nuclear  stains  such  as  propidium  iodide  (PI) 
are  easily  detectable  despite  the  fact  that  the  fluorescence  intensity 
per  dye  is  relatively  modest  (emax  -  SOOOM^cm-1  and  QY  -  0.1). 
In  contrast,  detection  of  cell-surface  EGF  receptors,  present 
at  -10,000  copies/cell,  by  confocal  microscopy  may  require 
the  use  of  fluorophores  such  as  R-phycoerythrin  (emax  - 
1 ,960,000  M-1  cm-1  and  QY  -  0.82)  to  generate  sufficient  signal 
(Good  et  al.,  1992).  Similar  considerations  apply  when  simulta¬ 
neously  imaging  two  targets  using  a  single  excitation  wavelength. 
Typically  it  is  quite  difficult  to  find  two  dyes  that  can  be  excited 
with  equal  efficiency  at  the  selected  wavelength  and  also  have 


TABLE  17.3.  Dyes  for  Rapid  Assessment  of  Living  Cells  by  Fluorescence  Microscopy 


Labeling  Target 

Incubation 

Concentration" 

Ex/Em4 

Laser  Lines" 

GFP 

Compatible'7 

Hoechst  33342 

Nucleus 

lpM* 

350/460 

35 1-364  nm  or  405  nm 

Y 

Calcein  AM 

Cytosol* 

2pM* 

494/515 

488  nm 

N 

FM  4-64 

Plasma  membrane, 
endosomes 

5pM* 

506/751/ 

488  nm,  514nm,  or  568  nm 

Y 

Propidium  iodide 

Nucleus  (dead  cells)g 

5pM* 

535/620 

488  nm  or  514nm 

Y 

LysoTracker 

Lysosomes 

50  nM 

575/590 

568  nm 

Y 

Red  DND-99 

MitoTracker  Red 

Mitochondria 

50nM 

578/600 

568  nm 

Y 

CMXRos 

AlexaFluor 

Water  (fluid  phase  tracer) 

lOmM* 

588/615 

568  nm 

Y 

594  hydrazide 

"These  dyes  can  generally  be  used  to  stain  live  eukaryotic  cells  by  incubation  for  15  to  30 min  at  the  indicated  concentration.  Dyes  marked  with  an  asterisk  (*)  can  be 
imaged  directly  in  the  dye  incubation  medium,  without  a  subsequent  wash  step. 
b  Fluorescence  excitation/emission  maxima  in  nanometers. 

‘  Laser  lines  suitable  for  excitation. 

dY,  fluorescence  emission  is  spectrally  well  resolved  from  that  of  GFP. 

‘  Fluorescence  is  dependent  on  cytosolic  esterase  activity  and  is  therefore  positively  correlated  with  cell  viability. 

^The  fluorescence  emission  of  FM  4-64  has  a  wide  spectral  bandwidth  and  can  be  detected  anywhere  from  625  to  800  nm. 
g Propidium  iodide  is  impermeant  to  live  cells;  fluorescence  is  therefore  inversely  correlated  with  cell  viability. 

^Used  to  fill  cells  (typically  neurons)  via  microinjection  of  a  10 mM  aqueous  solution.  Lucifer  yellow  CH  excited  at  405  nm  is  a  widely  used  alternative  but  is  not  spec¬ 
trally  well  resolved  from  GFP. 
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emission  spectra  that  are  sufficiently  well  separated  to  be  discrim¬ 
inated  without  resorting  to  spectral  unmixing  (Fig.  17.2)  or  the  use 
of  quantum  dot  labels.  In  this  situation,  the  less  efficiently  excited 
dye  should  be  used  to  detect  the  more  abundant  target,  thereby 
equalizing  the  two  emission  signals. 

Bulk  loading  procedures,  such  as  AM  ester  loading,  generate 
intracellular  concentrations  of  up  to  100pM,  corresponding  to 
about  1  x  10s  molecules  in  the  cytoplasm  of  a  typical  mammalian 
culture  cell  with  a  total  volume  of  4000  |im3  (of  which  about  50% 
is  occupied  by  organelles).  Perhaps  of  more  immediate  concern  to 
the  experimentalist  is  the  cell-to-cell  concentration  uniformity.  A 
confocal  imaging  study  of  neuroblastoma  cells  loaded  with  fura- 
2,  AM  by  Fink  and  colleagues  (1998)  found  a  mean  intracellular 
dye  concentration  of  3  8  \xM  in  1 23  cells,  with  individual  cell  values 
ranging  from  10  to  90  \xM.  Self-referencing,  ratiometric  mea¬ 
surements  are  often  used  to  correct  for  variability  of  dye  concen¬ 
tration  when  making  cell-to-cell  comparisons  of  fluorescence 
intensity.  Similar  intracellular  concentrations  are  achieved  in 
typical  microinjection  protocols  in  which  lOmM  dye  solution 
equivalent  to  about  1%  of  cell  volume  is  injected.  Much  higher 
concentrations  can  be  attained  in  situations  where  probes  are 
sequestered  in  subcellular  compartments.  For  example,  potential- 
driven  uptake  of  cationic  dyes  in  mitochondria  can  result  in  con¬ 
centrations  that  are  up  to  1000-fold  higher  than  in  the  cytosol 
(Nicholls  and  Ward,  2000).  These  probes  should  therefore  be 
applied  at  very  low  external  concentrations  (<10nM)  to  avoid 
fluorescence  self-quenching  and  respiratory  inhibition  effects 
(Rottenberg  and  Wu,  1998). 

The  minimum  concentration  of  cytoplasmic  EGFP  required  to 
be  detectable  above  autofluorescence  is  about  0.2  pM  (Patterson  et 
al.,  1997),  equivalent  to  about  200,000  copies  per  cell.  This  is 
considerably  above  the  native  expression  levels  of  many  cellular 
proteins  (range,  ~50  -  107  copies  per  cell),  making  it  necessary  to 
overexpress  GFP  fusion  proteins  to  obtain  detectable  fluorescence 
levels.  Much  lower  copy  numbers  can  be  detected  in  restricted 
volumes.  For  example,  viral  particles  (~100nm  diameter)  con¬ 
taining  120  GFP  fusion  proteins  each  are  detectable  both  in  isola¬ 
tion  and  after  uptake  into  cells  (Charpilienne  et  al.,  2001).  Because 
the  number  of  GFP  molecules  in  these  particles  is  genetically  pre¬ 
scribed,  they  provide  useful  standards  for  quantitation  of  intracel¬ 
lular  GFP  expression  levels  (Dundr  et  al.,  2002). 

Autofluorescence  is  often  the  limiting  factor  that  determines 
how  many  probe  molecules  must  be  introduced  into  a  cell  in  order 
to  be  detectable  above  background.  The  autofluorescence  of  a 
single  3T3  fibroblast  cell  has  been  estimated  to  be  equivalent  to 
about  34,000  fluorescein  molecules  (Roederer  and  Murphy,  1986). 
Surveys  of  spectral  data  (Billinton  and  Knight,  2001;  DaCosta  et 
al.,  2003)  show  that  autofluorescence  originates  from  a  wide 
variety  of  molecular  sources  and  generally  decreases  at  longer 
wavelengths.  Autofluorescence  levels  also  depend  on  cell  or  tissue 
type  and  physiological  status  (Croce  et  al.,  1999).  For  example, 
lipofuscin,  a  heterogeneous  complex  of  lipids  and  proteins  found 
in  brain  tissues,  increases  in  prevalence  with  age  and  the  extent  of 
oxidative  metabolism  (Haralampus-Grynaviski  et  al.,  2003).  The 
fluorescence  excitation  (400-550nm)  and  emission  (550-750nm) 
spectra  of  lipofuscin  are  broad,  overlapping  with  many  common 
probes  including  GFP  (Doyle  et  al.,  2003).  Other  significant 
sources  of  autofluorescence  are  NADH/NADPH,  flavins,  and 
flavoproteins  in  mammalian  cells,  chlorophylls  and  flavonoids  in 
plants,  and  collagen  and  elastin  in  connective  tissues. 

Components  of  culture  media  such  as  riboflavin  (excitation, 
450-490 nm;  emission,  500-560 nm)  are  also  significant  sources 
of  autofluorescence.  The  use  of  minimal  media  such  as  Hank’s  bal¬ 
anced  salt  solution  (HBSS)  or  media  with  low  riboflavin  content 


such  as  Ham’s  F12  is  often  preferable  for  live-cell  fluorescence 
imaging.  Phenol  Red,  included  as  a  pH  indicator  in  common 
culture  media,  sometimes  causes  significant  quenching  of  fluores¬ 
cence.  Phenol  Red-free  media  are  available  from  most  commercial 
suppliers  of  culture  media.  To  provide  pH  monitoring  for  cells  cul¬ 
tured  in  Phenol  Red-free  medium,  it  is  useful  to  prepare  parallel 
control  cultures  in  conventional  Phenol  Red-containing  medium. 
A  significant  technical  advance  in  dealing  with  autofluorescence 
problems  is  the  advent  of  microscopes  with  spectral  analysis  and 
linear  unmixing  capabilities,  which  allow  probe  and  autofluores¬ 
cence  background  signals  to  be  resolved  based  on  their  different 
spectral  fingerprints  (Dickinson  et  al.,  2001).  Likewise,  fluores¬ 
cence  lifetime  analysis  (FLIM)  can  also  sometimes  be  of  use  for 
this  purpose.  Multi-photon  excitation  can  produce  an  additional 
type  of  background  signal  in  the  form  of  scattered  second  and  third 
harmonic  generation  signals  (SHG,  THG;  see  Chapter  40,  this 
volume).  SHG  and  THG  signals  are  in  the  visible  range  at  exactly 
1/2  or  1/3  of  the  excitation  wavelength.  Their  production  does  not 
depend  on  the  absorption  and  re-emission  of  photons  but  purely 
on  structural  features:  usually  ordered  structures  such  as  collagen 
and  cellulose  for  SHG  and  surface  features  for  THG.  Although  the 
emission  of  SHG  and  THG  is  very  strongly  oriented  in  the  direc¬ 
tion  of  the  beam,  in  scattering  specimens  some  photons  can  still 
be  deflected  towards  the  detector. 


INTERACTIONS  OF  PROBES  AND  SPECIMENS 

Once  labeling  of  the  specimen  is  complete,  it  is  imperative  that  the 
probe  should  display  appropriate  localization  for  the  duration  of 
the  planned  imaging  experiments.  Furthermore,  the  presence  of  the 
probe  should  perturb  the  normal  physiology  and  structure  of  the 
cell  or  tissue  as  little  as  possible. 

Localization  and  Metabolism 

As  previously  discussed,  the  association  between  a  probe  and  its 
target  is  critical  to  the  accurate  representation  of  the  biological 
properties  of  the  specimen  in  a  fluorescence  image.  This  associa¬ 
tion  is  obtained  by  a  variety  of  mechanisms,  some  more  target- 
specific  than  others.  The  majority  of  fluorescent  probes  are  labeled 
analogs  of  proteins,  peptides,  lipids,  nucleotides,  and  other  bio¬ 
molecules.  In  these  cases,  there  is  the  potential  for  perturbations 
to  native  distribution  and  activity  induced  by  the  attachment  of  the 
fluorescent  label.  However,  despite  the  fact  that  fluorescent  dyes 
and  proteins  are  not  insignificantly  small  relative  to  the  molecules 
to  which  they  are  attached,  their  presence  is  usually  tolerated  sur¬ 
prisingly  well.  For  example,  there  are  very  few  examples  reported 
in  the  literature  of  aberrant  localization  of  GFP  fusion  proteins 
(Katz  et  al.,  1998;  Unkila  et  al.,  2001;  Lim  et  al.,  2002). 

Many  dye-based  probes,  such  as  nucleic  acid  stains,  ion  indi¬ 
cators,  and  mitochondrial  markers  are  not  direct  biomolecular 
analogs.  The  behavior  of  these  probes  is  better  represented  in  terms 
of  molecular  pharmacology  than  cell  biology.  Like  therapeutic 
drugs,  their  distribution  is  determined  by  a  combination  of  several 
factors  including  the  applied  external  concentration  and  rates  of 
transmembrane  transport  and  intracellular  metabolism  (Fig.  17.5). 
The  staining  pattern  obtained  is  usually  quite  time-dependent  and 
sensitive  to  factors  such  as  temperature  and  removal  of  extracel¬ 
lular  dye  by  washing  (Lemasters  et  al.,  1999).  This  can  lead  to  sit¬ 
uations  such  as  that  depicted  in  Figure  17.6,  where  most  of  the 
fluorescent  labeling  is  insensitive  to  the  physiological  process  that 
it  was  intended  to  detect.  Another  example  is  provided  by  non- 
fluorescent  dihydrorhodamine  123,  which  is  oxidized  intracellu- 
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larly  by  reactive  oxygen  species  (ROS)  such  as  superoxide,  hydro¬ 
gen  peroxide,  and  hydroxyl  radicals.  The  product  of  this  reaction  is 
the  cationic  fluorescent  dye  rhodamine  123.  It  could  be  concluded 
from  the  resulting  mitochondrially  localized  fluorescence  that  mito- 
chondrially  localized  ROS  generation  is  being  detected  when  in  fact 
that  localization  reflects  the  redistribution  of  rhodamine  123  from 
the  cytosol  to  mitochondria  (Diaz  et  al.,  2003).  Many  dyes  are  sus¬ 
ceptible  to  extrusion  by  active  transporters  such  as  the  ATP  binding 
cassette  (ABC)  protein  superfamily.  Transport  rates  are  governed  by 
transporter  substrate  specificity  and  are  therefore  quite  dependent 
on  the  molecular  structure  of  the  dye  (Passamonti  and  Sottocasa, 
1988;  Wadkins  and  Houghton,  1995;  Loetchutinat  et  al.,  2003). 
Inhibitors  such  as  sulfinpyrazone,  probenecid,  MK571,  and  vera¬ 
pamil  can  be  used  to  stabilize  intracellular  dye  retention  (Di  Virgilio 
et  al.,  1990).  Dye  extrusion  can  be  advantageously  exploited  to 
monitor  transporter  activity  and  distribution  (weak  cellular  staining 
corresponds  to  high  transporter  activity)  (Hollo  et  al.,  1994;  Martin 
et  al.,  2003;  Breen  et  al.,  2004).  In  contrast,  experiments  designed 
to  track  the  distribution  of  cell  populations  in  tissues  and  organisms 
during  processes  such  as  embryological  development  demand 
stable  and  innocuous  incorporation  of  dyes.  Dyes  useful  in  these 
applications  are  irreversibly  coupled  to  proteins  [e.g.,  carboxyfluo- 
resceindiacetate,  succinimidyl  ester  (CFSE);  Parish,  1999]  embry- 


FIGURE  17.6.  (Top  panel)  HL-1  cardiomyocytes  (White  et  al.,  2004)  loaded 
by  incubation  with  the  fluo-4,  AM  as  represented  schematically  in  Figure  17.5. 
The  bright,  punctate  labeling  is  due  to  compartmentalized  dye.  Cytosolic  label¬ 
ing  is  faint,  diffuse,  and  only  marginally  visible  in  this  image.  The  image  was 
acquired  using  a  PerkinElmer  UltraView  laser-scanning  confocal  microscope 
with  a  60x,  1.2  NA,  water  immersion  objective  (pixel  size  =  0.4  pm).  (Bottom 
panel)  Photometric  traces  representing  the  average  fluorescence  intensities  in 
the  regions  of  interest  A-D  (A)  as  a  function  of  time.  The  cytosolic  regions  of 
two  different  cells  (A  and  B)  exhibit  synchronized  Ca2+  spikes  associated  with 
the  spontaneous  contractile  activity  of  the  cells.  No  systematic  variation  of  Ca2+ 
is  detected  by  the  compartmentalized  dye  (C  and  D). 


ological  or  inert  dextran  carriers,  or  are  incorporated  into  membrane 
lipids  (e.g.,  Dil). 


Perturbation  and  Cytotoxicity 

Compared  to  electron  microscopy,  specimen  preparation  for  fluo¬ 
rescence  microscopy  is  relatively  non-invasive.  Nevertheless, 
the  concentrations  of  extrinsic  probes  required  for  practical 
applications  are  not  insignificant  in  relation  to  physiological 
concentrations.  Consequently,  control  experiments  to  verify  that 
observations  are  independent  of  the  applied  probe  concentration 
are  always  worthwhile.  Severe  dye-induced  structural  and  physi¬ 
ological  perturbations  result  in  cytotoxicity  and  cell  death.  As  one 
might  intuitively  expect,  the  most  pronounced  cytotoxic  and  per¬ 
turbative  effects  tend  to  be  exerted  by  dyes  that  accumulate  in  the 
organelles  most  essential  to  respiration  and  replication  —  the  mito¬ 
chondrion  and  the  nucleus.  Concentrations  of  cyanine  dyes  such 
as  DiOC6(3)  as  low  as  lOnM  cause  significant  inhibition  of  mito¬ 
chondrial  respiration  (Rottenberg  and  Wu,  1998;  Scorrano  et  al., 
1999).  Subtoxic  doses  (l-10p,M)  of  the  DNA-binding  dye  Hoechst 
33342  cause  increased  luciferase  gene  expression  in  transfected 
myocytes  (Zhang  and  Kiechle,  2003).  Conversely,  probes  that  are 
diffusely  distributed  in  the  cytosol  or  the  plasma  membrane  are 
relatively  non-cytotoxic.  Cells  labeled  with  CFSE  continue  to 
divide  and  the  ensuing  bipartition  of  cellular  fluorescence  inten¬ 
sity  provides  a  direct  indication  of  proliferation  (Parish,  1999).  The 
utility  of  calcein  AM  labeling  for  monitoring  cell  adhesion  and 
migration  derives  from  its  benign  effects  on  these  functional  prop¬ 
erties  (De  Clerck  et  al.,  1994). 

A  significant  concern  in  the  application  of  GFP  chimeras  is 
the  need,  in  some  cases,  to  overexpress  a  fusion  protein  in  order 
to  achieve  detectability  over  the  autofluorescence  background. 
Although  overexpression  is  not  usually  cytotoxic,  it  may  induce 
abnormal  growth  or  physiology  (Gunjan  et  al.,  1999;  Wendland 
and  Bumann,  2002).  In  some  cases,  cytotoxicty  of  GFP  has 
been  found  to  be  due  to  products  of  adjacent  expression  vector 
sequences  (Endemann  et  al.,  2003).  Despite  their  disruptive  poten¬ 
tial,  cytotoxic  and  perturbative  effects  of  probes  are  almost  always 
manageable.  Even  notoriously  cytotoxic  probes  such  as  dye- 
labeled  phallotoxins  can  still  be  used  effectively  in  living  cells  if 
the  applied  concentration  is  sufficiently  low  (Pu  et  al.,  2000). 


UNDER  THE  MICROSCOPE 
Photobleaching 

Photobleaching  and  phototoxicity  are  two  aspects  of  the  same 
underlying  process  —  generation  of  reactive  oxygen  species 
(ROS)  in  the  specimen  as  a  by-product  of  fluorescence  excitation. 
Both  are  of  particular  concern  in  prolonged  imaging  experiments 
such  as  long  time-lapse  sequences.  Although  photobleaching  is 
usually  problematic,  resulting  in  an  effective  time-dependent 
decrease  in  fluorophore  concentration,  it  can  be  turned  to  advan¬ 
tage  in  some  cases.  Examples  include  the  use  of  FRAP  (fluores¬ 
cence  recovery  after  photobleaching)  to  measure  diffusion  rates 
and  acceptor  photobleaching  to  establish  no-transfer  reference 
points  in  FRET  microscopy  (Jales-Erijman  and  Jovin,  2003).  With 
single-photon  excitation,  the  rate  of  photobleaching  and  the  rate  of 
fluorescence  emission  are  usually  both  linear  functions  of  excita¬ 
tion  intensity.  The  primary  causative  mechanism  appears  to  be 
photosensitization  of  singlet  oxygen  ('02)  generation  by  the  dye 
triplet  excited  state  (Gandin  et  al.,  1983;  Song  et  al.,  1996; 
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Eggeling  et  al.,  1999;  Stracke  et  al.,  1999;  Gaigalas  et  al.,  2002). 
Singlet  oxygen  is  a  highly  reactive  transient  species  with  a  life¬ 
time  of  about  4|is  in  water.  In  cells,  the  lifetime  of  '02  is  reduced 
to  <0.5  |is  due  to  the  increased  number  of  reactive  targets  presented 
by  proteins  and  nucleic  acids.  Therefore,  the  radius  of  action  of 
'02  is  limited  to  <50  nm  from  its  point  of  origin.  The  short  range 
of  this  effect  may  be  important  when  “acceptor  photobleaching” 
is  used  as  a  rough  measure  of  FRET  (see  also  Chapter  45,  this 
volume ),  because  *02  produced  by  excitation  of  the  acceptor  is  just 
as  likely  to  bleach  any  nearby  donor.  Reaction  of  '02  with  dyes 
generally  results  in  irreversible  severance  of  the  conjugated  7t-elec- 
tron  system  responsible  for  fluorescence. 

The  extent  of  photobleaching  can  be  lessened  by  adjusting  any 
of  the  three  participants  in  the  reaction  —  excitation  light,  dye, 
and  oxygen.  Decreasing  excitation  light  levels,  in  both  intensity 
and  duration,  offers  the  most  direct  and  consistently  effective 
approach.  The  intrinsic  photostability  of  organic  dyes  varies 
widely  depending  on  their  molecular  structure  (Eggeling  et  al., 
1999;  Kanofsky  and  Sima,  2000).  The  photobleaching  quantum 
yield  (the  number  of  photobleaching  events  per  photon  absorbed) 
provides  a  quantitative  expression  of  propensity  for  photobleach¬ 
ing.  The  ratio  of  the  fluorescence  quantum  yield  to  the  photo¬ 
bleaching  quantum  yield  represents  the  average  number  of 
fluorescence  photons  generated  by  a  dye  before  it  is  photo- 
bleached.  For  fluorescein,  this  ratio  is  about  30,000  whereas  for 
rhodamine  dyes  it  is  >100,000.  Although  tabulations  of  photo¬ 
bleaching  quantum  yields  (Eggeling  et  al.,  1999)  provide  useful 
guidance,  photobleaching  rates  are  also  dependent  on  environ¬ 
mental  factors.  For  example,  dye-dye  interactions,  such  as  those 
induced  by  attaching  multiple  dyes  to  a  protein,  result  in  increased 
rates  of  photobleaching  relative  to  the  free  monomeric  dye  (Byers 
et  al.,  1976;  Song  et  al.,  1995,  1997).  This  may  account  at  least 
partly  for  the  photostability  of  protein  conjugates  labeled  with 
AlexaFluor  dyes,  which  are  designed  to  minimize  dye-dye  inter¬ 
actions  via  electrostatic  repulsion.  The  availability  of  reactive 
oxygen  species  (primarily  *02)  contributing  to  photobleaching  can 
be  reduced  by  addition  of  scavengers,  commonly  referred  to  as 
antifade  reagents,  to  the  specimen.  Several  effective  antifade  for¬ 
mulations  are  commercially  available;  unfortunately  these  are  only 


applicable  to  fixed  specimens  (Berrios  et  al.,  1999).  Furthermore, 
antifade  reagents  may  cause  fluorescence  quenching  in  addition  to 
attenuation  of  photobleaching,  yielding  only  a  small  net  increase 
in  the  number  of  photons  available  for  detection  (Eggeling  et  al., 
1999;  see  also  Chapter  39,  this  volume).  Some  live  cell- 
compatible  antifade  reagents  have  been  reported  in  the  literature 
including  Trolox  (an  analog  of  vitamin  E),  ascorbic  acid  and 
enzymatic  deoxygenation  systems  (e.g.,  Oxyrase;  Oxyrase  Inc., 
Mansfield,  OH;  Adler,  1990).  However  their  usage  appears  to  be 
rather  sporadic,  suggesting  that  they  are  not  consistently  effective. 

Photobleaching  in  two-photon  microscopy  is  fundamentally  a 
rather  different  problem  than  in  the  one-photon  excitation  case.  The 
extraordinarily  high  spatial  and  temporal  confinement  of  excitation 
photons  required  to  achieve  two-photon  excitation  (Zipfel  et  al., 
2003a)  potentially  opens  photobleaching  reaction  pathways  that  are 
not  accessible  through  one-photon  excitation.  Whereas  the  rate  of 
two-photon  excited  fluorescence  emission  increases  as  the  square  of 
excitation  intensity  (I2),  photobleaching  rates  show  higher  order 
(I>2)  increases  (Patterson  and  Piston,  2000;  Dittrich  and  Schwille, 
2001).  Analysis  by  fluorescence  correlation  spectroscopy  (FCS) 
indicates  that  the  reaction  pathways  are  distinct  from  photobleach¬ 
ing  arising  from  one-photon  excitation  in  that  they  do  not  involve 
dye  triplet  excited  states  (Dittrich  and  Schwille,  2001). 

Phototoxicity 

To  a  large  extent,  phototoxicity  and  photobleaching  go  hand-in- 
hand.  Phototoxicity  results  from  damage  exerted  by  photogener¬ 
ated  reactive  oxygen  species  on  proteins,  nucleic  acids,  and  other 
cellular  components.  Photodamage  is  usually  restricted  to  the  near 
vicinity  of  the  sensitizer  due  to  the  limited  diffusional  range  of  '02 
(Greenbaum  et  al.,  2000).  Effects  farther  afield  can  occur  through 
the  action  of  secondary  ROS  initially  generated  from  '02 
(Ouedraogo  and  Redmond,  2003).  Cell  death  is  only  the  most 
extreme  manifestation  of  phototoxicity.  Other  effects  may  be  much 
more  subtle,  such  as  photoinduction  of  intracellular  Ca2+  release 
(Knight  et  al.,  2003)  or  failure  to  divide. 

Although  phototoxicity  has  many  determining  factors  (Table 
17.4),  controlling  light  exposure  is  the  most  effective  limitation 


TABLE  17.4.  Factors  Influencing  Phototoxic  Effects 

Factor47  Trend  References 


Excitation  wavelength 
Excitation  intensity  and  duration 

Cell  type/status 
Culture  medium 
Dye  concentration 
Dye  type 

Dye  localization 
Antioxidant  additives 


Longer  wavelength  produce  less  photodamage. 

Higher  power  and  longer  exposure  result  in  more  photodamage.^ 


Larger  cells  can  sustain  a  higher  phototoxic  burden. 

Mitotic  cells  are  particularly  susceptible  to  photodamage. 
Riboflavin  and  tryptophan  induce  phototoxicity. 

Higher  concentrations  produce  more  photodamage. 

Cyanines  and  halogenated  xanthenes  are  particularly  phototoxic. 


Calcein  and  GFP  phototoxicity  is  decreased  upon  compartmentalization  in  mitochondria 
and  endoplasmic  reticulum,  respectively. 

Ascorbic  acid  and  enzymatic  deoxygenation  systems  reduce  photoxicity. 


Bloom  &  Webb  (1984) 
Manders  et  al.  (1999) 
Vigers  et  al.  (1988) 

Oh  et  al.  (1999) 

Schafer  &  Buettner  (1999) 
Manders  et  al.  (1999) 
Schafer  &  Buettner  (1999) 
Manders  et  al.,  (1999) 
Stoien  &  Wang  (1974) 
Grzelak  et  al.,  (2001) 
Vigers  et  al.  (1988) 

Oh  et  al.  (1999) 

Bunting  (1992) 

Gandin  et  al.  (1983) 

Lee  et  al.  (1995) 

Beghetto  et  al.  (2000) 
Haseloff  et  al.  (1997) 
Bloom  &  Webb  (1984) 
Vigers  et  al.  (1988). 


Note  that  some  factors  are  strongly  interdependent,  for  example,  dye  localization  and  dye  concentration.  *These  dependences  may  by  highly  nonlinear. 
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strategy  (Manders  et  al 2004).  Reduced  photodamage  and 
concomitantly  increased  specimen  viability  is  one  of  the  principal 
benefits  of  two-photon  excitation.  However,  as  in  the  case  of 
photobleaching,  although  the  overall  phototoxic  burden  is  vastly 
reduced  compared  to  one-photon  excitation,  the  severity  of  photo¬ 
damage  in  the  focal  volume  is  substantially  higher  (Koester  et  al ., 
1999;  Tirlapur  et  al .,  2001;  see  also  Chapter  38,  this  volume). 

SUMMARY 

An  extensive  range  of  dyes  and  probes,  and  techniques  for  incor¬ 
porating  them  into  living  and  fixed  specimens,  has  been  developed 
over  the  past  30  years.  This  development,  allied  to  complementary 
advances  in  optical  engineering  and  image  processing,  has  pro¬ 
pelled  fluorescence  to  become  the  dominant  contrast  mechanism 
used  in  biological  microscopy.  Fluorescent  labeling  technology  is 
a  much  less  exact  science  than  physical  optics.  This  situation  is  a 
reflection  of  the  fact  that  labeling  technology  must  interface 
directly  with  the  morphological  and  physiological  diversity  of 
biological  specimens,  rather  than  any  intrinsic  shortcomings  in  the 
technology  itself.  Nevertheless,  understanding  the  design  and 
implementation  of  labeling  techniques  is  an  essential  element  in 
the  ultimate  objective  of  deriving  biological  information  from 
image  data.  This  chapter  has  therefore  endeavored  to  assemble  a 
fraction  of  the  extensive  and  diverse  knowledge  base  on  fluores¬ 
cent  probe  techniques  in  one  place,  with  the  aim  of  facilitating 
critical  evaluations  of  labeling  strategies  by  experimenters. 
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INTRODUCTION 

Traditionally,  biologists  have  been  confined  to  transmission  elec¬ 
tron  microscopy  (TEM)  and  light  microscopy  (LM)  in  order  to  cor¬ 
relate  biochemical  and  molecular  data  with  morphology.  Electron 
microscopy  (EM)  provides  fine  ultrastructural  detail  but  is  limited 
to  the  study  of  cellular  structures  that  react  with  electron  dense 
stains  deposited  in  fixed  specimens.  Immunogold  labeling  permits 
the  study  of  non-electron-dense  material,  but  EM  sections  must 
still  be  very  thin  to  avoid  problems  with  the  penetration  of  the 
labeled  antibodies  and  to  reduce  scattering  of  the  electron  beam. 

The  electron  microscopist  faces  the  same  problem  in  re¬ 
constructing  three-dimensional  (3D)  cellular  structures  from 
two-dimensional  (2D)  projections.  In  practice,  the  better  the 
ultramicrotome,  the  more  accurate  the  3D  reconstruction.  In  any 
event,  reconstruction  from  serial  EM  sections  is  a  laborious 
process. 

The  use  of  traditional  LM  methods,  such  as  Nomarski  optics 
and  phase-contrast  microscopy,  confines  one  to  specimens  observ¬ 
able  by  transmitted  light,  such  as  in  vitro  studies  involving 
cytoskeletal  elements  (Abbott,  1884;  Ito,  1962;  Scheetz  and 
Spudich,  1983;  Inoue,  1986;  Dabora  and  Sheetz,  1988;  Lee  and 
Chen,  1988).  The  use  of  vital  dyes  and  fluorescent  labeling  has 
allowed  researchers  to  correlate  functional  biochemical  data  with 
structural  data.  The  introduction  of  the  confocal  microscope  has 
finally  allowed  the  present  generation  of  structural  biologists  to 
escape  from  Flatland.  However,  this  instrument  requires  not  only 
an  imaging  system,  but  also  an  object  worth  viewing. 

Although  it  would  probably  be  ideal  if  all  biological  confocal 
studies  could  be  carried  out  on  living  specimens  (Chapter  19,  this 
volume),  this  is  not  always  possible.  Apart  from  the  problem  of 
cell  movement  and  the  nuisance  of  keeping  cells  physiologically 
“happy”  on  the  microscope  stage,  there  are  additional  complica¬ 
tions  in  exposing  such  cells  to  antibody  labeling  or  embedding 
them  in  antibleaching  agents.  As  a  result,  the  vast  majority  of  con¬ 
focal  images  are  made  from  material  that  has  been  fixed,  stained, 
and  in  most  cases,  dehydrated.  However,  because  most  LM  spec¬ 
imen  preparation  techniques  were  developed  to  produce  only 
pleasing  2D  images,  they  are  often  inadequate  to  the  task  of  pro¬ 
ducing  specimens  that  retain  their  3D  structure. 

Indeed,  because  of  the  difficulty  in  obtaining  high  contrast 
images  from  non-planar  objects  using  conventional  microscopic 
techniques,  immunofluorescence-staining  protocols  that  tended  to 
flatten  the  specimens  under  study  have  often  gained  acceptance 
specifically  because  they  tend  to  reduce  the  effect  of  out-of-focus 
light  on  the  final  image.  This  tendency  has  pushed  the  biologist  to 


study  either  thin  cells  or  cells  grown  under  conditions  that  are  not 
optimal  for  the  expression  of  the  full  phenotype  of  the  cell. 

Confocal  fluorescence  microscopy  extends  the  value  of  these 
fluorescence-labeling  techniques  because  its  ability  to  exclude  out- 
of-focus  information  from  the  image  data  permits  the  acquisition 
of  3D  intensity  data  sets  that  can  be  viewed  as  3D  images.  The 
purpose  of  this  chapter  is  to  discuss  methods  for  ensuring  that  the 
specimens  from  which  such  data  are  acquired  maintain  the  3D 
structure  they  had  in  vivo. 

We  have  studied  the  Madin-Darby  canine  kidney  (MDCK) 
cell  line  grown  on  Costar  polycarbonate  membrane  filter  supports. 
The  filter  supports  have  the  disadvantage  of  being  opaque,  but  they 
do  allow  the  cells  to  be  supplied  with  nutrients  in  a  more  physio¬ 
logical  way.  Under  these  growth  conditions,  the  basal  membrane 
has  access  to  nutrients  at  all  times,  somewhat  reproducing  growth 
conditions  in  vivo.  MDCK  cells  form  a  more  completely  polarized 
monolayer  when  grown  on  the  membrane  filters  (Bomsel  et  al., 
1989). 

These  studies  have  asked  specific  questions  about  cellular 
organization  both  in  vitro  and  in  vivo  during  the  formation  of  an 
epithelial  monolayer.  This  chapter  reflects  the  lessons  learned 
while  attempting  to  study  these  cells  under  growth  conditions 
that  are  not  very  amenable  to  study  by  immunofluorescence 
methods. 

We  will  begin  with  a  description  of  fixatives  and  fixation 
methods.  This  will  be  followed  by  a  section  that  explains  how  to 
prepare  and  use  the  two  fixatives  that  we  have  found  most  useful 
(glutaraldehyde  and  pH  shift/formaldehyde)  and  that  also  de¬ 
scribes  mounting  and  staining  procedures.  We  will  then  describe 
how  these  procedures  were  evaluated  and  conclude  with  some 
general  comments  on  the  subject. 

This  chapter  will  discuss  many  of  these  methods  and  address 
some  of  the  potential  pitfalls  of  specimen  preparation. 

CHARACTERISTICS  OF  FIXATIVES 

The  first  problem  about  fixatives  is  their  name,  as  it  may  give  the 
impression  that  treatment  with  fixatives  really  does  “fix”  or  render 
immovable  the  macromolecular  components  of  the  cell.  In  fact, 
most  fixation  protocols  are  really  just  chemical  or  physical 
processes  that  cause  metabolism  to  stop  and  that  preserve 
microstructures  down  to  some  size  level  by  destroying  other  struc¬ 
tures,  such  as  enzymes,  which  have  sizes  below  that  level.  The  best 
fixation  for  a  given  experiment  is  that  which  does  the  best  job  of 
preserving  structure  down  to  the  level  required. 
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No  fixation  protocol  is  perfect,  but  its  imperfection  is  often 
masked  by  the  fact  that  the  specimen  dimensions  in  the  plane  of 
the  image  are  maintained  by  the  adherence  of  the  specimen  to  the 
glass  substrate.  As  a  result,  shrinkage  in  the  z-direction  is  often 
pronounced.  Such  shrinkage  is  particularly  serious  when  it  occurs 
on  specimens  to  be  viewed  in  the  confocal  microscope,  which 
allows  one  to  obtain  information  in  the  z-direction. 

There  are  three  general  types  of  fixation  protocol  available  for 
biological  specimen  preparation.  Chemical  fixatives  fix  tissue 
either  by  coagulating  proteins  or  by  chemically  crosslinking 
them.  The  other  two  methods  of  fixation,  freeze  substitution 
(Steinbrecht  and  Zierold,  1987)  and  microwave  fixation  (Jackson, 
1991)  will  not  be  considered  here. 

The  coagulating  fixatives,  such  as  ethanol,  methanol,  or 
acetone,  fix  the  specimen  by  rapidly  changing  the  hydration  state 
of  the  cellular  component.  Proteins  either  coagulate  or  are  ex¬ 
tracted  during  this  process.  Coagulating  fixatives  are  popular,  as 
they  are  easy  to  apply  reproducibly  and  tend  to  preserve  the  antigen 
recognition  sites  for  immunolabeling  very  well.  The  major 
problem  with  the  use  of  coagulating  fixatives  in  confocal  micro¬ 
scopy  is  the  profound  shrinkage  of  the  specimen  that  these  fixa¬ 
tives  produce.  We  have  found  that  methanol  will  shrink  cell  height 
by  50%  during  the  fixation.  This  artifact  renders  the  spatial  infor¬ 
mation  obtained  by  confocal  microscopy  inaccurate  and  should  not 
be  depended  on,  especially  when  the  data  sets  are  to  be  used  for 
3D  reconstruction.  It  is  also  important  to  note  that  commercial 
preparations  of  formaldehyde  often  contain  significant  amounts  of 
methanol  as  a  stabilizing  agent,  and  this  can  induce  the  same 
shrinkage  artifact. 

Commonly  used  crosslinking  fixatives  include  glutaralde- 
hyde,  formaldehyde,  and  ethylene  glycol-i>w-succinimidyl  succi¬ 
nate  (EGS).  These  fixation  chemicals  form  covalent  crosslinks  that 
are  determined  by  the  active  groups  in  each  compound. 

The  ideal  fixative  should  penetrate  tissues  quickly,  act  rapidly, 
and  preserve  the  cellular  structure  before  the  cell  can  react  to 
produce  structural  artifacts.  Unfortunately,  no  single  agent  has 
emerged  as  the  ideal  fixative  and  a  pragmatic  approach  to  the  use 
of  a  particular  fixative  is  determined  by  the  experimental  require¬ 
ments.  Although  tissue  fixation  is  incompletely  understood,  an 
understanding  of  the  characteristics  of  chemical  fixatives  can 
provide  a  rational  approach  to  their  use. 

Glutaraldehyde 

Since  its  introduction  in  1962,  glutaraldehyde  has  been  used  exten¬ 
sively  for  EM  specimen  preparation  (Sabatini  et  al.,  1962).  This 
diadehyde  is  composed  of  five  carbons  with  a  molecular  weight 
of  100.12.  Extensive  evaluation  of  the  preservation  properties  of 
glutaraldehyde  by  EM  has  shown  that  it  preserves  subcellular 
structures  well  (Sabatini  et  al.,  1963,  1964;  Barmett  et  al.,  1964). 
Glutaraldehyde  has  been  less  popular  as  a  fixative  for  fluorescence 
microscopy  because  it  renders  tissue  autofluorescent  and  destroys 
antibody-binding  sites.  Fortunately,  the  autofluorescence  can 
usually  be  adequately  attenuated  by  post-treating  samples  with 
NaBH4  (Tagliaferro  et  al.,  1997).  Recent  work  utilizes  ethanol 
extraction  to  remove  autofluorescent  chemicals  or  Sudan  black  to 
quench  autofluorescent  signals  (Baschong  et  al,  2001). 

The  chemistry  of  glutaraldehyde  fixation  is  complex  and  not 
fully  understood.  Glutaraldehyde  forms  a  Schiff’s  base  with  amino 
groups  on  proteins  and  polymerizes  via  Schiff’s  base  catalyzed 
reactions  (  Johnson,  1985;  Tashima  et  al,  1987).  The  ability  to  poly¬ 
merize  allows  glutaraldehyde  to  form  extended  crosslinks  (Meek 
and  Chapman,  1985).  Glutaraldehyde  reacts  with  the  e-amino 


group  of  lysine  and  will  react  with  the  a-amino  group  amino  acids. 
It  will  also  react  with  tyrosine,  tryptophan,  histidine,  phenylalanine, 
and  cysteine  (Hayat,  1989).  While  glutaraldehyde  fixes  proteins 
rapidly,  its  main  disadvantage  is  its  relatively  slow  rate  of  penetra¬ 
tion  into  the  cell  compared  to  formaldehyde.  Other  problems  asso¬ 
ciated  with  glutaraldehyde  fixation  are  the  propensity  of  many  cells 
to  form  membrane  blebs  as  they  die  (as  is  common  to  all  the  alde¬ 
hyde  fixatives)  and  its  inability  to  crosslink  neutral  lipids,  DNA,  or 
RNA  (Hopwood,  1975).  Not  all  proteins  are  inactivated  by  glu¬ 
taraldehyde,  and  some  proteins  can  still  be  extracted  from  fixed 
tissue  (Blanchette-Mackie  and  Scow,  1981;  Hayat,  1989). 

Glutaraldehyde  is  supplied  commercially  as  a  25%  or  8%  solu¬ 
tion.  Commercial  preparations  of  glutaraldehyde  may  contain  sig¬ 
nificant  amounts  of  impurities,  which  can  affect  the  reproducibility 
of  the  fixation  method  or  induce  fixation  artifacts.  In  our  experi¬ 
ence,  it  is  important  to  use  only  those  commercial  preparations  that 
list  the  impurities.  We  have  tended  to  use  the  glutaraldehyde  solu¬ 
tions  supplied  by  Polysciences  (Warrington,  PA)  and  have  noted 
satisfactory  reproducibility  between  commercial  lots. 

As  the  glutaraldehyde  solution  ages,  it  polymerizes  and  turns 
yellow.  Because  this  changes  the  reproducibility  of  the  fixation  and 
can  produce  artifacts,  it  is  important  to  identify  old  solutions  and 
to  store  the  fixative  so  as  to  inhibit  polymerization  (Hayat.  1989). 
As  the  polymerization  rate  increases  with  temperature  and  at  high 
pH,  it  can  be  minimized  by  storing  both  unbuffered  glutaraldehyde 
and  working  solutions  at  -20°C.  These  solutions  should  be  thawed 
only  once  and  used  the  same  day. 

One  useful  way  to  monitor  the  quality  of  glutaraldehyde  is  to 
measure  the  absorbance  of  a  0.5%  solution  at  235  nm  and  280  nm. 
Pure  glutaraldehyde  has  an  A235/A280  ratio  of  less  than  0.2.  Arti¬ 
facts  tend  to  occur  with  ratios  of  >2.0  (Bowers  and  Maser,  1988). 

A  variety  of  buffers  and  co-fixatives  can  be  used  with  glu¬ 
taraldehyde.  The  advantages  and  changes  in  fixation  chemistry  that 
occur  when  it  is  used  in  combination  with  other  fixatives  have  been 
reviewed  elsewhere  (Hayat,  1981;  Hayat,  1986). 

Formaldehyde 

As  mentioned  above,  commercial  solutions  of  formaldehyde 
contain  significant  amounts  of  methanol.  Formaldehyde  fixation 
for  confocal  microscopy  should  be  prepared  fresh  from 
paraformaldehyde  dissolved  in  water  (see  protocol  described 
below).  Formaldehyde  (MW  30.0)  crosslinks  proteins  by  forming 
methylene  bridges  between  reactive  groups.  The  rate-limiting  step 
is  a  deprotonation  of  amino  groups,  hence  the  pH  dependence  of 
the  crosslinking  reaction.  Other  functional  groups  that  are  reactive 
with  formaldehyde  include  amido,  guanidine,  thiol,  phenol,  imi- 
dazolyl,  and  indolyl  groups  (Hayat,  1989).  When  lysine  is  added 
to  the  fixation  buffer,  it  can  participate  in  the  crosslinking  reaction, 
and  this  is  the  basis  for  its  inclusion  in  the  paraformaldehyde- 
lysine-periodate  fixation  first  described  by  McLean  and  Nakane 
(1974). 

One  advantage  of  formaldehyde  as  a  fixative  is  its  ability  to 
crosslink  nucleic  acids.  This  makes  it  the  preferred  fixative  for  in 
situ  hybridization.  This  aldehyde  does  not  crosslink  lipids  and,  in 
some  cell  types,  it  produces  extensive  vesiculation  of  the  plasma 
membrane.  In  some  cases,  small  amounts  of  CaCl2  in  the  fixation 
buffer  have  been  shown  to  stabilize  the  plasma  membrane  during 
fixation.  However,  labile  structures  such  as  microtubules  are  not 
well  preserved  by  formaldehyde  at  physiologic  pH  (Sato  et  al., 
1976;  Wild  et  al.,  1987).  Observation  of  the  nucleus  by  phase- 
contract  microscopy  during  formaldehyde  fixation  reveals  marked 
changes  in  the  size  and  shape  during  fixation  (J.  DeMey,  personal 
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communication).  During  the  fixation  process,  the  nucleus  oscil¬ 
lates,  alternating  between  swelling  and  shrinking.  It  is  not  clear 
what  happens  to  the  nuclear  contents  during  this  time. 

The  rate  of  protein  crosslinking  is  slower  with  formaldehyde 
than  with  glutaraldehyde,  however,  formaldehyde  penetrates  cells 
10  times  faster  (Hopwood,  1967).  Attempts  have  been  made  to  try 
to  optimize  fixation  by  mixing  glutaraldehyde  with  formaldehyde. 
The  idea  is  to  take  advantage  of  the  rapid  penetration  of  formalde¬ 
hyde  into  the  tissue  and  the  high  speed  of  glutaraldehyde  crosslink¬ 
ing  (Karnovsky,  1965).  Although  what  happens  to  the  fixation 
chemistry  when  these  mixtures  are  employed  is  not  clearly  under¬ 
stood,  in  our  experience  Karnovsky’s  fixative  has  been  useful  for 
some  preparations  and  appears  to  preserve  the  3D  structure  well. 

EGS  is  another  bifunctional  crosslinking  agent  that  reacts  with 
primary  amino  groups  and  with  the  e-amino  groups  on  lysine.  A 
major  advantage  of  this  fixative  is  its  reversibility.  The  crosslinks 
are  cleavable  at  pH  8.5,  an  attractive  feature  because  reversible 
crosslinking  may  be  used  to  restore  antigen  binding  sites  (Abdella 
et  al,  1979).  This  fixative  is  useful  for  membrane-bound  proteins, 
but  its  limited  solubility  in  water  makes  its  performance  highly 
variable.  Sulfated  forms  of  this  crosslinking  agent  [ethylene  gly- 
cobis  (sulfo-succimidylsuccinate)]  (Pierce  Co.,  Pierce,  IL)  are 
available  and  are  water  soluble  (Abdella  et  al.,  1979).  We  have 
not  tried  this  particular  crosslinking  agent  as  yet  but  it  is  also 
cleavable. 


FIXATION  STAINING  AND 
MOUNTING  METHODS 

The  fixation  methods  described  below  have  been  optimized  for  the 
confocal  fluorescence  microscope  in  studies  in  which  MDCK  cells 
have  been  grown  on  opaque  filter  supports  (Bacallao  and  Stelzer, 
1989).  The  cells  grow  to  a  uniform  height  of  18  pm  and  form  a 
dense  monolayer  under  the  growth  conditions  we  used.  These  fix¬ 
ation  and  staining  protocols  have  been  tested  using  the  criteria 
described  in  the  next  section. 

Glutaraldehyde  Fixation 
Stock  Solutions 

•  8%  glutaraldehyde  EM  grade  (Polysciences,  Warrington,  PA). 

•  80  m M  Kpipes,  pH  6.8,  5  m/V7  EGTA,  2  m/V7  MgCl2,  both  with 
and  without  0.1%  Triton  X-100  depending  on  the  target 
antigen;  use  Triton  if  you  intend  to  stain  cytoskeletal  proteins. 

•  Phosphate-buffered  saline  without  Ca2+MG+  (PBS-). 

•  Phosphate-buffered  saline  without  Ca2+MG+,  pH  8.0. 

Preparation  of  the  Stock  Solutions 

EM  grade  glutaraldehyde  was  obtained  from  Polysciences.  It  is 
supplied  as  an  8%  aqueous  solution.  When  a  new  vial  is  opened, 
the  glutaraldehyde  is  diluted  to  0.3%  in  a  solution  of  80mM 
Kpipes,  pH  6.8,  5mM  EGTA,  2mM  MgCl2>  0.1%  Triton  X-100. 
The  aliquots  are  stored  at  -20°C.  Prior  to  each  experiment,  a  fresh 
aliquot  is  used  soon  after  thawing.  These  aliquots  are  never  frozen 
again  or  reused  because  this  causes  a  loss  of  efficacy.  Bring  up  the 
pH  of  the  PBS  by  adding  a  few  drops  of  6 N  NaOH  to  PBS. 

Fixation  Protocol 

1.  Warm  lOOmL  of  80mM  Kpipes,  pH  6.8,  5mM  EGTA, 
2  m M  MgCl2  without  Triton  X-100  to  37°C  in  a  beaker. 


2.  Pour  off  the  media  in  the  apical  well  of  the  Costar  filter.  Dip 
the  entire  filter  plus  filter  holder  into  this  modified  80  mM 
Kpipes  buffer  for  5  s. 

3.  Transfer  the  filter  to  the  6- well  plate  supplied  with  the  poly¬ 
carbonate  filters  to  permit  convenient  fixation  and  washing 
steps. 

4.  Fix  the  cells  for  lOmin  with  0.3%  glutaraldehyde  +0.1%  Triton 
X-100  at  room  temperature.  The  glutaraldehyde  fixative  is 
added  to  both  the  apical  (2mL)  and  basal  (3mL)  portions  of 
the  filter.  During  all  the  incubation  steps  and  washes,  the  6- 
well  plate  is  agitated  on  a  rotary  shaker. 

5.  During  the  fixation  period,  weigh  out  3-  to  10-mg  aliquots  of 
fresh  NaBH4.  The  NaBH4  powder  is  then  stored  in  50  mL  ster¬ 
ilized  conical  tubes  with  screw  caps.  The  NaBH4  should  be 
kept  in  an  anhydrous  state,  preferably  under  dry  nitrogen  gas 
because  it  is  a  very  strong  reducing  agent  and,  when  combined 
with  water,  hydrogen  gas  is  released.  Explosions  in  the  labo¬ 
ratory  setting  have  been  reported  so  this  agent  should  be 
used  with  care. 

6.  Aspirate  the  fixative,  and  dip  the  entire  filter  successively  in 
three  separate  lOOmL  beakers  containing  PBS-. 

7.  Add  PBS-,  pH  8.0  to  the  NaBH4  to  make  a  final  concentration 
of  1  mg/mL.  Add  3  mL  of  this  solution  to  the  apical  portion  of 
the  cell  and  4  mL  to  the  basal  chamber.  Incubate  1 5  min  at  room 
temperature.  You  should  see  gas  bubbles  in  the  solution  during 
this  step.  Repeat  this  step  two  more  times  using  freshly  dis¬ 
solved  NaBH4.  The  adjustment  of  the  pH  to  8.0  increases  the 
half-life  of  NaBH4  in  solution.  This  step  is  essential  to  decrease 
the  autofluorescence  of  the  glutaraldehyde-fixed  cells. 

8.  Wash  the  cells  with  PBS-  by  dipping  the  preparation  in  three 
beakers  containing  PBS-.  Return  the  filters  to  the  6-well  plate 
with  PBS-  bathing  the  apical  and  basal  side.  The  filter  is  now 
ready  for  immunofluorescence  staining. 

pH  Shift/Formaldehyde  Fixation 

The  pH  shift/formaldehyde  method  was  first  used  for  fixing  rat 
brain,  in  which  it  showed  excellent  preservation  of  neuronal  cells 
and  intracellular  compartments  (Berod  et  al,  1981).  This  tech¬ 
nique  applies  the  formaldehyde  to  the  tissue  twice:  once  at  near 
physiological  pH  to  halt  metabolism,  and  then  again  at  high  pH, 
where  the  crosslinking  action  of  the  fixative  is  more  effective. 

Stock  Solutions 

•  40%  formaldehyde  (Merck)  in  H20. 

•  80m M  Kpipes,  pH  6.8,  5mM  EGTA,  2mM  MgCl2. 

•  lOOmM  NaB407  pH  11.0. 

•  Phosphate-buffered  saline  without  Ca2+/Mg2+  (PBS-)  pH  8.0. 

•  Phosphate-buffered  saline  without  Ca2+/Mg2+  (PBS-)  and  both 
with  and  without  0.1  %  Triton  X-100. 

•  Premeasured  10-mg  aliquots  of  dry  NaBH4. 

Preparation  of  the  Stock  Solutions 

Preparation  of  the  formaldehyde  stock  solution  is  based  on  the 
description  by  Robertson  and  colleagues  (1963).  Forty  grams  of 
paraformaldehyde  (CH20)n;  MW  30.3  is  added  to  100mL  of  H20. 
While  continuously  stirring,  the  mixture  is  heated  above  to  70°C. 
A  few  drops  of  6N  NaOH  are  added  to  dissolve  the  formaldehyde, 
but  do  not  allow  the  solution  to  boil.  The  stock  solution  is  divided 
into  aliquots  and  stored  at  -20°C.  Prior  to  use,  aliquots  are  thawed 
by  warming  in  a  water  bath.  The  formaldehyde  is  diluted  to  2%  to 
4%  in  both  the  Kpipes  and  sodium  borate  buffers.  For  our  pur¬ 
poses,  a  3%  solution  of  formaldehyde  was  adequate  for  preserv- 
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ing  both  the  structure  and  antigenic  determinants  on  a  wide  variety 
of  cell  organelles.  The  pH  of  the  Kpipes  buffer  is  brought  to  6.5 
with  1 N  HC1  after  the  formaldehyde  has  been  added. 

PBS-,  pH  8.0  is  made  by  adding  a  few  drops  of  6 N  NaOH 
to  normal  PBS,  made  without  calcium  or  magnesium;  lOOmM 
NaB407  is  titrated  to  pH  11.0  by  adding  6 N  NaOH  to  the  buffer. 

Fixation  Protocol 

1 .  Pour  off  the  media  in  the  apical  well  of  the  filter. 

2.  Dip  the  filters  in  80 mM  Kpipes  solution  pre-warmed  to  37°C. 

3.  Add  3mL  of  3.0%  formaldehyde  in  the  80mM  Kpipes  solu¬ 
tion  to  the  basal  chamber  of  the  6- well  dish,  and  add  2mL  of 
this  solution  to  the  apical  surface  of  the  cells.  Incubate  the  cells 
with  agitation  on  a  rotary  table  for  5  min  at  room  temperature. 

4.  Aspirate  the  formaldehyde/Kpipes  solution;  then  add  3  mL  of 
3%  formaldehyde  in  lOOmM  NaB407,  pH  11.0,  to  the  basal 
side  and  2mL  to  the  apical  side  of  each  filter.  Incubate  with 
agitation  on  a  rotary  table  for  10 min  at  room  temperature. 

5.  Weigh  out  two  10-mg  aliquots  of  NaBH4  for  each  filter  and 
store  in  a  conical  tube  with  a  screw  cap. 

6.  Aspirate  the  fixation  solution.  Wash  the  filters  by  successively 
dipping  the  filters  in  three  beakers  containing  lOOmLof  PBS-. 

7.  Dissolve  each  aliquot  of  NaBH4  in  10 mL  PBS-,  pH  8.0  (final 
concentration  of  NaBH4  should  be  1  mg/mL).  Vortex  the  solu¬ 
tion  briefly,  and  add  it  to  the  apical  (2mL)  and  basal  (3mL) 
portions  of  the  filters.  Incubate  15  min  while  shaking  the  filters 
on  a  rotary  table.  Repeat  this  step  one  more  time  using  a  fresh 
solution  of  NaBH4  in  PBS-,  pH  8.0. 

8.  Wash  the  filters  by  successively  dipping  the  filters  in  three 
beakers  containing  100 mL  of  PBS-.  The  filters  can  be  stored 
overnight  at  4°C  with  PBS-/0.1%  NaN3. 

9.  The  sample  is  permeabilized  by  washing  with  PBS  (-)  +  0.1% 
Triton  X-100. 

Immunofluorescence  Staining 

1.  Cut  the  filter  from  its  plastic  holder.  Be  sure  to  note  which  side 
of  the  filter  has  the  cells  layered  on  it!  Cut  the  filter  into  four 
squares  using  a  sharp  scalpel  while  keeping  it  wet  with  PBS. 
To  ensure  that  the  cell  side  of  the  filter  can  be  readily  identi¬ 
fied,  we  routinely  cut  a  slit  in  the  upper  right  corner  of  the  filter. 
The  filter  is  cut  into  squares  because  this  tends  to  give  a  flat 
field  of  cells  after  the  filter  has  been  mounted.  Dividing  a  filter 
into  quadrants  with  one  rounded  edge  causes  the  filter  to  ripple 
during  mounting. 

2.  Wash  the  filter  squares  in  PBS-  containing  0.2%  fish  skin 
gelatin  (FSG),  which  is  used  as  a  carrier  protein  instead  of 
albumin  in  this  protocol  (Sigma,  St.  Louis,  MO)  and  the  appro¬ 
priate  percentage  of  detergent.  The  amount  of  detergent  must 
be  determined  empirically.  Usually  0. 1  %  to  0.3%  Triton  X- 1 00 
will  suffice.  Wash  the  filter  squares  in  a  6-well  plate.  All 
washes  are  done  in  4mL  of  solution,  at  room  temperature, 
with  agitation.  Unless  otherwise  stated,  the  filters  are  washed 
for  15  min  after  every  change  of  washing  buffer. 

3.  Place  a  50(1  L  drop  of  the  first  antibody  diluted  in  PBS- 
containing  0.2%  FSG  on  a  piece  of  parafilm  on  the  bottom  of 
a  petri  dish. 

4.  Place  a  filter  square,  cell-side  down,  on  the  antibody  solution. 
Place  a  piece  of  wet  Whatman  filter  paper  in  the  petri  dish, 
well  away  from  the  antibody  solution  and  then  cover  to  form 
a  small,  humidified  chamber. 

5.  Incubate  at  37°C  for  1  h  (for  specimens  that  are  <10(tm  thick, 
35  min  is  an  adequate  incubation  time). 


6.  Wash  the  filter  twice  with  PBS-  containing  0.2%  FSG. 
Follow  this  with  three  successive  washes  with  PBS-. 

7.  Wash  the  filter  once  more  with  PBS-  containing  0.2%  FSG. 

8.  Add  the  second  antibody  as  described  for  the  first  antibody. 

9.  Incubate  at  37°C  for  1  h. 

10.  Wash  the  filters  in  PBS-  containing  0.2%  FSG  twice. 

11.  Wash  three  times  in  PBS-. 

12.  Incubate  once  in  PBS  containing  0.2%  Triton  X-100  for  5  min. 

13.  Wash  twice  in  PBS-  for  5  min  each  time. 

The  filter  squares  are  ready  for  mounting. 

Mounting  the  Specimen 

Place  four  drops  of  clear  acrylic  nail  polish  on  a  microscope  slide 
to  make  corner  supports  for  a  coverslip.  Each  drop  should  be  at 
a  point  corresponding  to  the  corner  of  a  coverslip.  We  use  22  X 
22  mm  1.5  coverslips.  Place  the  filter  square  in  the  center  of  the 
area  demarcated  by  the  nail  polish.  Take  care  to  ensure  that  the 
cells  are  facing  up.  Put  a  drop  of  50%  glycerol  /  50%PBS  / 
0.1%NaN,  /  100 mg/mL  DABCO  ( 1 ,4-diazabicyclo  [2.2.2.] 
octane;  Sigma)  on  the  filter.  Carefully  place  the  coverslip  over  the 
filter.  Avoid  trapping  air  bubbles  in  the  specimen  mount  and  make 
sure  that  the  comers  lie  on  the  drops  of  polish.  Aspirate  the  excess 
glycerol  medium.  Put  four  drops  of  nail  polish  on  the  four  corners 
of  the  coverslip  to  stabilize  the  mount.  Alternatively,  shards  of 
broken  coverslips  can  be  used  as  coverslip  supports.  Once  the  nail 
polish  on  the  corners  has  dried,  the  entire  mount  can  be  sealed  with 
nail  polish.  The  specimen  should  be  viewed  within  24  h  because 
these  are  not  permanent  mounts. 

Semi-permanent  samples  can  be  made  by  post-fixing  the  filter 
in  4%  formaldehyde  dissolved  in  lOOmM  Na  cacodylate,  pH  7.5, 
for  30  min  at  room  temperature  followed  by  quenching  with  50mM 
NH4C1  in  PBS  for  15  min.  Post-fixed  samples  can  maintain  excel¬ 
lent  labeling  characteristics  for  over  6  months  when  stored  at  4°C 
or  at  -20°C.  Figure  18.4  shows  the  unfortunate  result  of  an  im¬ 
proper  mounting  method.  In  this  case,  the  coverslip  was  not 
adequately  supported  because  the  acrylic  nail  polish  posts  were  set 
too  far  apart:  clearly,  attention  to  the  details  can  determine  the 
success  or  failure  of  specimen  preparation. 

CRITICAL  EVALUATION  OF  LIGHT 
MICROSCOPY  FIXATION  AND 
MOUNTING  METHODS 

Immunofluorescence  labeling  methods  have  been  widely  em¬ 
ployed  in  cell  biology  (Osborn  et  al.,  1980;  Osborn  and  Weber, 
1982).  The  major  concerns  when  using  these  methods  to  study  cell 
morphology  have  been  to  preserve  the  antigenic  and  structural 
integrity  of  the  specimen.  In  general,  the  goal  is  to  immobilize  the 
antigen  quickly,  while  preserving  the  cell’s  organization.  All  too 
frequently,  the  best  preservation  methods  destroy  the  epitopes 
required  for  antibody  binding.  Because  this  is  a  particular  problem 
with  glutaraldehyde  (Nakane,  1975;  Cande  et  al. ,  1977;  Weber 
et  al.,  1978),  formaldehyde  is  often  used  as  a  fixative  despite  its 
poorer  preservation. 

Our  initial  attempts  to  use  the  “classical”  fixation  and  mount¬ 
ing  methods  for  normal  epi-fluorescence  microscopy  yielded 
images  that  were  markedly  deformed  in  the  third  dimension.  It 
became  apparent  that  these  methods  had  been  specifically  designed 
to  shrink  the  cells  flat  to  the  coverslip  in  order  to  reduce  the  out- 
of-focus  light  in  standard  epi-fluorescence  images.  To  avoid  this, 
it  became  necessary  to  meld  some  of  the  fixation  methods  devel- 
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oped  for  EM  with  the  techniques  of  classic  immunofluorescence. 
This  meant  that,  in  addition  to  the  requirements  for  structural  and 
antigenic  preservation,  the  3D  spatial  preservation  had  to  be  con¬ 
sidered  as  well.  This  matter  has  also  been  a  concern  for  workers 
using  scanning  EM  and  those  doing  stereographic  analysis  of  TEM 
images.  Because  samples  being  prepared  for  immunofluorescence 
microscopy  do  not  have  to  be  dried,  the  major  concern  we  had  was 
shrinkage  induced  by  fixation  or  dehydration.  This  problem  has 
been  examined  by  several  workers  (Tooze,  1964;  Boyde  and 
Maconnachie,  1979.  1981;  Lee  et  al.,  1979,  1982;  Wangensteen  et 
al.,  1981;  Lee,  1984).  Glutaraldehyde  fixation  has  been  found  to 
induce  cell  shrinkage  when  hypertonic  buffers  are  used.  Addition¬ 
ally,  Lee  has  shown  that  the  total  osmolarity  of  the  fixative  and  the 
type  of  buffer  used  determine  the  extent  of  gross  specimen  shrink¬ 
age  (Lee  et  al.,  1982;  Lee,  1984).  These  findings  have  important 
implications  for  the  preparation  of  tissue  samples  for  two-photon 
microscopy  where  thicker  tissue  sections  can  be  imaged. 

Other  fixation  procedures  can  also  affect  the  degree  of  cell 
shrinkage.  For  example,  osmium  tetroxide  fixation  causes  a  vari¬ 
able  degree  of  volume  shrinkage  in  erythrocytes.  It  has  been  sug¬ 
gested  that  the  amount  of  shrinkage  induced  by  this  fixative 
depends  on  the  interplay  between  the  electrostatic  interactions 
between  charged  protein  particles  and  osmotic  forces  (Tooze, 
1964).  A  recent  review  and  study  on  the  effect  of  formaldehyde 
fixation  on  cell  volume  found  that  rat  liver  strips  shrank  in  length 
by  only  3%  when  fixed  at  room  temperature.  The  liver  strips  were 
immersed  in  fixative  and  observed  by  video  microscopy,  but  the 
assessment  of  shrinkage  was  only  measured  in  the  length  dimen¬ 
sion.  Raising  the  temperature  to  39°C  diminished  the  amount  of 
shrinkage  observed  (Fox  et  al.,  1985).  It  was  also  noted  that,  in 
general,  increasing  the  concentration  of  formaldehyde  paradoxi¬ 
cally  seemed  to  cause  swelling  of  the  sample.  This  effect  has  not 
been  satisfactorily  explained  but  has  been  noted  by  other  investi¬ 
gators  (Bradbury  and  Meek,  1960).  The  shrinkage  is  due  to  the 
extremely  high  total  osmolality  of  fixative  solutions  con¬ 
taining  formaldehyde.  It  should  be  noted  that  a  3%  solution  of 
paraformaldehyde  has  a  far  higher  osmolality  (approximately 
lOOOmOsm)  than  that  of  a  3%  solution  of  glutaraldehyde 
(300mOsm)  because  of  the  large  difference  in  their  molecular 
weights  (Fox  et  al.,  1985).  The  buffer  should  be  optimized  to  be 
isotonic  with  the  specimen  when  paraformaldehyde  is  used. 

Our  overall  approach  to  resolve  these  conflicting  requirements 
reflected  a  bias  toward  immunofluorescence  studies.  We  started 
with  fixation  methods  that  still  preserved  the  epitope  binding  sites 
for  our  antibody  label  and  then  tried  modifications  designed  to  be 
freely  transferred  from  one  system  to  another.  An  example  of  this 
occurred  when  the  paraformaldehyde/pH  shift  fixation  protocol 
was  adopted  to  study  microtubule/kinetochore  interactions  during 
mitosis  in  newt  lung  epithelial  cells.  While  the  microtubules  were 
well  preserved,  it  was  found  by  differential  interference  contrast 
(DlC)-video  microscopy  that  the  chromosomes  continued  to  move 
during  the  low-pH  incubation  step  (A.  Merdes,  unpublished  obser¬ 
vations).  This  indicated  that  the  cells  were  not  immediately  immo¬ 
bilized  by  the  first  step  of  the  fixation  and,  as  a  result,  this  method 
was  not  useful  for  these  cells  in  this  particular  study.  This  example 
points  out  the  advantage  of  observing  the  specimen  under  phase 
contrast  or  DIC  during  fixation  to  detect  potential  artifacts. 

Use  of  the  Cell  Height  to  Evaluate 
the  Fixation  Method 

In  order  to  generate  3D  images  that  accurately  reflect  in  vivo  cel¬ 
lular  architecture,  it  was  necessary  to  employ  fixation  methods  that 


minimize  cell  shrinkage  or  distortion.  To  study  this  issue,  we  took 
advantage  of  the  ability  of  the  fluorescent  lipid  analog  C6-NBD- 
ceramide  to  label  the  plasma  membrane  in  vivo  (Lipsky  and 
Pagano,  1985).  The  plasma  membrane  of  MDCK  cells,  grown  to 
confluence  on  filter  supports,  was  labeled  as  described  by  van  Meer 
and  colleagues  (1987).  The  height  of  the  living  cells  was  deter¬ 
mined  from  randomly  selected  vertical  sections  using  confocal  xz- 
images  (Fig.  18.1).  The  samples  were  then  fixed  and  stained  with 
probes  that  recognized  either  actin  filaments  or  microtubules,  using 
procedures  described  previously.  Actin  staining  and  microtubule 
staining  were  used  as  markers  for  the  cell  height  in  the  fixed  cells 
because  these  networks  lie  close  to  the  plasma  membrane.  Fur¬ 
thermore,  the  preservation  of  the  cytoskeletal  network  is  particu¬ 
larly  sensitive  to  fixation  methodology.  The  mean  height  of  the 
fixed  cells  was  then  determined  again  at  randomly  selected  sites 
and  the  results  from  the  two  groups  (fixed  vs.  in  vivo  cells)  were 
analyzed  by  Student’s  t  test.  Figure  18.1(A)  shows  the  plasma 
membrane  labeling  observed  in  cells  labeled  in  vivo  with  C6-NDB- 
ceramide  and  Figure  18.1(B)  shows  immunofluorescence  labeling 
of  another  part  of  the  same  sample  after  glutaraldehyde  fixation. 
Both  samples  were  mounted  in  PBS.  The  height  and  shape  of  both 
samples  are  quite  similar.  We  observed  that  the  shape  of  the  apical 
dome  was  very  sensitive  to  fixation  artifacts  and  that  glutaralde¬ 
hyde  fixation,  with  or  without  Triton  X-100  in  the  buffer,  preserved 


FIGURE  18.1.  Vertical  optical  sections  of  MDCK  cells.  MDCK  cells  were 
plated  on  polycarbonate  filters  and  grown  for  5  days  in  culture.  The  cells  have 
formed  a  columnar  epithelium  of  fairly  uniform  height.  (A)  C6-NBD-ceramide 
labeling  of  the  plasma  membrane.  Note  the  curved  appearance  of  the  apical 
plasma  membrane.  (B)  Microtubule  staining  in  glutaraldehyde-fixed  cells.  Note 
that  the  cell  height  and  apical  membrane  curvature  are  roughly  comparable. 
Bar  =  10  pm. 
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the  cell  height  most  accurately  with  very  little  shrinkage  detected 
as  long  as  the  buffer  was  isotonic. 

Less  than  5%  increase  in  the  cell  height  was  caused  by  pH 
shift/formaldehyde  fixation.  We  consider  this  degree  of  distortion 
acceptable  in  some  well-defined  situations.  However,  fixation  with 
paraformaldehyde  alone  had  a  paradoxical  effect  on  cell  size  noted 
above  when  the  pH  shift  method  was  not  used.  Higher  concentra¬ 
tions  of  paraformaldehyde  (4%)  dissolved  in  low-ionic-strength 
buffers  were  less  likely  to  cause  cell  swelling.  Lower  concentra¬ 
tions  of  paraformaldehyde  in  hyperosmotic  buffers  either  showed 
no  change  in  cell  size  or  caused  shrinkage.  It  became  apparent  that 
no  single  buffer  yielded  optimal  fixation  when  paraformaldehyde 
was  used  as  a  fixative. 

Figure  18.2  shows  stereo  images  of  isolated  cells  fixed 
with  methanol  and  then  the  formaldehyde/pH  shift  method. 
The  methanol  has  caused  a  flattening  of  the  area  overlying  the 
nucleus  and  a  10%  to  20%  decrease  in  cell  height.  Whenever 
methanol  was  used  on  confluent  cells  as  either  a  fixative  or  as  a 
permeabilization  agent,  it  caused  a  20%  to  50%  decrease  in  cell 
height.  We  consider  it  inappropriate  to  use  this  agent  on  samples 
prepared  for  3D  examination  in  a  confocal  fluorescence 
microscope. 

Use  of  Cell  Height  to  Evaluate  Mounting  Media 

Shrinkage  is  often  associated  with  removing  biological  structures 
from  the  aqueous  environment  and  placing  them  in  less  polar  sol¬ 
vents  having  a  lower  dielectric  constant  (Boyde  and  Maconnachie, 
1979,  1981).  Confocal  .rz-images  were  also  employed  to  evaluate 


the  effects  of  different  mounting  methods  and  mounting  media. 
However,  as  all  such  media  have  different  indices  of  refraction  (r|) 
and  as  the  apparent  specimen  height  measured  in  this  way  is  only 
accurate  if  the  specimen  is  immersed  in  a  medium  having  the  T| 
for  which  the  objective  is  corrected  (see  Chapter  20,  this  volume ; 
Hell  et  al.,  1993),  these  measurements  should  be  considered  more 
comparative  than  quantitative. 

Gross  distortion  is  produced  if  the  coverslip  touches  the  spec¬ 
imen  (Fig.  18.4).  To  avoid  this,  we  suspended  the  coverslip  using 
four  posts  of  nail  polish  as  mentioned  above. 

The  best  mounting  media  was  found  to  be  50%  glycerol  in 
PBS,  which  showed  no  detectable  shrinkage  in  fixed  specimens 
compared  to  in  vivo  labeled  cells.  Mowoil  and  Gelvatol  caused  a 
10%  decrease  in  cell  height  in  glutaraldehyde-hxed  cells.  This 
amount  of  shrinkage  was  considered  to  be  significant  because  the 
apical  domes,  seen  in  vivo,  in  polarized  MDCK  cells  were  com¬ 
pletely  flattened  by  these  mounting  agents.  The  mounting  media 
appeared  to  have  an  even  greater  effect  on  the  shape  of  formalde¬ 
hyde-fixed  cells,  probably  due  to  the  lower  degree  of  crosslinking 
in  formaldehyde-fixed  specimens. 

Well-Defined  Structures  Can  Be  Used  to 
Evaluate  Fixation  Methods 

A  second  method  used  to  evaluate  fixation  was  to  examine  struc¬ 
tures  that  have  a  well-defined  morphology.  Because  we  had  an 
interest  in  microtubule  organization  in  epithelial  cells,  we  exam¬ 
ined  the  preservation  of  mitotic  spindles  in  our  sample  prepara¬ 
tions.  Changes  in  the  natural  symmetry  of  the  mitotic  spindle,  due 


FIGURE  18.2.  Stereo  image  of  isolated  MDCK 
cells.  MDCK  cells  were  plated  at  a  low  density, 
one  day  prior  to  fixation  and  staining.  This  image 
was  reconstructed  from  a  series  of  optical 
sections  taken  in  the  x-y  plane  from  consecutive 
“^’’-positions,  0.4  pm  apart.  No  other  image  pro¬ 
cessing  was  performed.  Each  field  is  made  up  of 
512  x  512  pixels.  The  image  is  the  average  of 
four  scans  per  line.  (A)  Microtubule  staining  in 
methanol-fixed  cells.  Note  the  flattening  over  the 
nuclear  region.  The  distance  from  the  top  of  the 
cell  to  bottom  is  4.1pm.  (B)  Microtubule  stain¬ 
ing  in  cells  fixed  with  the  formaldehyde/pH-shift 
method.  No  flattening  is  noted  and  the  micro¬ 
tubules  are  well  preserved.  The  distance  from 
the  top  of  the  cell  to  the  bottom  is  4.5  pm.  Bar  = 
10  pm. 
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FIGURE  18.3.  (A)  Stereo  images  of  the  Golgi 
apparatus  in  confluent  MDCK  cells  labeled  in  vivo 
with  C6-NBD-ceramide  made  under  the  same  con¬ 
ditions  as  Figure  18.2.  Note  the  convoluted  mor¬ 
phology.  The  distance  from  the  top  of  the  Golgi  to 
the  bottom  is  6.4pm.  (B)  Stereo  images  of  the 
Golgi  apparatus  in  confluent  MDCK  cells  labeled 
by  a  monoclonal  antibody.  Cells  were  hxed  by  the 
formaldehyde/pH-shift  method  described  in  this 
chapter.  Note  that  the  overall  morphology  is 
similar  to  the  ceramide  labeled  morphology.  The 
image  was  generated  as  described  above.  Bar  = 
10  pm. 


to  fixation  or  mounting  artifacts,  were  easily  detected.  Breaks  in 
the  microtubules  or  shortening  of  those  within  the  spindle  could 
be  seen  readily,  and  such  changes  were  used  to  disqualify  some 
fixation  methods. 

In  general,  formaldehyde  fixation  yielded  poorly  preserved 
microtubules,  however,  the  formaldehyde/pH  shift  fixation  method 
gave  acceptable  microtubule  preservation  of  the  mitotic  spindles. 
When  compared  directly  with  glutaraldehyde  fixed  cells,  the 
formaldehyde/pH  shift  method  was  not  so  good,  but  it  was  an 
acceptable  compromise.  In  direct  comparisons,  the  staining  of 
formaldehyde/pH  shift-fixed  specimens  was  slightly  lower  in 
intensity  than  that  in  specimens  fixed  with  formaldehyde  alone. 
However,  we  found  it  was  useful  for  double-immunofluorescence 
labeling  with  antigenic  epitopes  that  were  destroyed  by  glu¬ 
taraldehyde  fixation.  The  periodate-lysine-formaldehyde  fixation 
described  by  McLean  and  Nakane  (1974)  did  not  preserve  mitotic 
spindles  well. 

Comparison  of  In  Vivo  Labeled  Cell  Organelles 
with  Immunolabeled  Cell  Organelles 

Sometimes  membrane  structures  can  be  distorted  even  though  the 
fixation  method  does  not  affect  cell  height.  As  noted  above,  light 
microscopic  observation  of  cell  nuclei  during  fixation  with 
formaldehyde  revealed  marked  changes  in  nuclear  size  and  shape 
during  the  pH  6.5  step  of  the  fixation  that  showed  it  was  unsuit¬ 
able  for  the  examination  of  nuclear  membrane  antigens,  at  least  at 
this  tonicity. 


The  use  of  vital  fluorescent  dyes  such  as  rhodamine  123,3,3'- 
dihexyloxacarbo-cyanine  (DiOC6)  and  C6-NBD-ceramide  permits 
comparison  of  the  effects  of  subsequent  fixation  on  the  morphol¬ 
ogy  of  the  mitochondria,  rough  endoplasmic  reticulum  (RER),  and 
Golgi  apparatus,  respectively  (Walsh  et  al.,  1979;  Terasaki  et  al., 
1984;  Lipsky  and  Pagano,  1985;  see  Chapter  19,  this  volume ). 

MDCK  cells  undergo  striking  changes  in  the  morphology  of 
their  Golgi  apparatus  during  the  formation  of  a  polarized  epithe¬ 
lium  (Bacallao  et  al,,  1989).  The  morphology  in  the  final  polarized 
state  was  examined  in  vivo  using  the  fluorescent  lipid  analog,  C6- 
NBD-ceramide.  The  morphology  seen  in  vivo  was  very  similar  to 
that  in  samples  fixed  and  stained  using  a  Golgi-specific  monoclonal 
antibody  generously  supplied  by  Dr.  M.  Bornens  (Fig.  18.3)  and  to 
that  seen  in  non-ciliated  epithelial  cells,  as  determined  by  an  analy¬ 
sis  of  thick  sections  viewed  at  low  magnification  in  the  high-voltage 
electron  microscope  (HVEM)  (Rambourg  et  al.,  1989).  Both  glu¬ 
taraldehyde  fixation,  and  the  pH  shift/formaldehyde  fixation  pro¬ 
tocol,  preserved  the  morphology  of  the  Golgi  apparatus  well. 

GENERAL  NOTES 

The  repeated  use  of  borohydride  in  these  fixation  protocols  was 
found  to  decrease  endogenous  cellular  fluorescence  significantly. 
Other  quenching  agents,  such  as  ammonium  chloride  and  lysine, 
were  also  tried,  but  borohydride  worked  best.  This  was  a  crucial 
obstacle  to  overcome  in  our  work  because  our  specimens  had  high 
cell  densities  when  the  MDCK  cells  grew  to  confluence  on  mem- 
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brane  filter  supports.  Initially,  high  endogenous  background  fluo¬ 
rescence  caused  our  images  to  have  poor  contrast,  obscuring 
important  details. 

These  fixation  methods  also  work  well  with  thinner  cells 
grown  on  coverslips,  but  some  shortening  of  the  fixation  time  is 
necessary.  Typically,  a  4-min  fixation  in  the  pH  6.5  buffer  and  an 
8-min  fixation  in  the  pH  1 1 .0  buffer  worked  well  with  glass-grown 
cells.  Although  in  some  tissues  there  is  a  tendency  for  formalde¬ 
hyde  to  induce  vesiculation  of  cell  membranes,  this  artifact  was 
not  observed  in  all  tissues. 

Both  saponin  and  Triton  X-100  have  been  used  as  permeabi- 
lization  agents  in  the  formaldehyde  fixation  method.  Triton  X-100 
has  been  most  effective  when  used  after  the  fixation  was  complete, 
while  saponin  worked  best  when  included  with  the  borate  buffer. 
NP-40  worked  very  well  for  specimens  in  which  the  microtubules 
were  stained  with  monoclonal  antibodies  to  a  and  [3  tubulin. 

We  have  used  DABCO  at  a  concentration  of  lOOmg/mL  as 
an  antibleaching  agent  (Langanger  et  al.,  1983).  Yellowing  of 
the  specimen  occurs  1  month  after  mounting  when  DABCO  is 
included  with  the  mounting  media,  but  this  does  not  seem  to 
produce  a  significant  change  in  the  images  produced  from  such 
samples.  A  recent  paper  suggested  that  5  M  NaN3  is  a  better 
antibleaching  agent  than  DABCO  (Bock  et  al.,  1985),  however,  in 
our  experience,  we  found  the  reverse  (Merdes  et  al.,  1991).  n- 
Propylgallate  has  been  tried  as  an  antibleaching  agent  but  in  our 
hands  it  caused  a  dimming  of  the  fluorescent  signal.  We  did  not 
use  p-phenylenediamine  as  an  antibleaching  agent  because  this 
agent  destroys  the  sample  over  time  (Langanger  et  al..  1983). 
Some  investigators  have  tried  to  make  mixtures  of  antibleaching 
agents  such  as  5%  n-propylgallate-0.0025%  /7-phenylenediamine 
dissolved  in  glycerol.  Another  mixture  suggested  by  Peter  Hahn 
(Thomas  Jefferson  University,  Philadelphia,  PA)  is  a  0.25%  p- 
phenylenediamine,  0.0025%  DABCO,  5%  n-propylgallate  dis¬ 
solved  in  glycerol  with  a  pH  value  of  approximately  7.6.  These 
investigators  did  note  some  increase  in  background  signal, 
however  (see  also  Chapter  39,  this  volume). 

The  pH  of  the  mounting  media  can  be  an  important  parameter 
to  consider,  especially  when  fluorescein  is  used  as  a  fluorophore. 
The  fluorescent  emission  of  fluorescein  conjugates  increases  up  to 
pH  10.  For  additional  insights  on  the  effect  of  pH  on  fluorescent 
compounds,  the  readers  are  referred  elsewhere  (Hiramoto  et  al., 
1964;  Klugerman,  1965). 

Labeling  Samples  with  Two  or  More  Probes 

One  of  the  unique  aspects  of  3D  imaging  is  that  it  gives  one  the 
power  to  determine  spatial  relationships.  However,  this  can  also 
pose  problems  if  one  can  see  only  the  stained  structure.  For 
example,  in  our  early  work  using  C6-NBD-ceramide  to  study 
the  Golgi  morphology,  we  found  ourselves  dissatisfied  with  the 
images.  One  frequently  wanted  to  know  where  the  Golgi  appara¬ 
tus  was  located  with  reference  to  other  cellular  structures  (see  Fig. 
18.3).  Sometimes  a  reference  image  can  be  provided  by  detecting 
and  displaying  the  light  scattered  back  by  optical  inhomogeneities 
in  the  specimen.  This  signal  can  be  obtained  without  interfering 
with  the  collection  of  the  fluorescence  signal  (Pawley  et  al,  1993; 
Chapter  2,  this  volume)  but,  as  doing  so  requires  equipment  that 
is  only  now  becoming  generally  available,  it  has  been  more 
common  to  image  other  cellular  structures  using  a  second  fluores¬ 
cent  dye. 

Our  early  attempts  to  label  these  structures  simultaneously  led 
to  unforeseen  difficulties.  The  first  major  pitfall  was  the  finding 
that  fluoroscein  isothiocyanate  (FITC)  and  rhodamine  could  not  be 


used  simultaneously  in  combination  with  the  filter  sets  then  avail¬ 
able  to  us.  Although  the  cutoff  filters  were  supposed  to  eliminate 
signals  above  530nm,  we  were  unable  to  separate  cleanly  the  FITC 
image  from  the  rhodamine  image.  This  was  due  in  part  to  the  fact 
that  the  confocal  microscope  we  used  had  an  argon-ion  laser  with 
lines  at  528.7  and  476.5  nm,  so  even  when  the  528.7  line  is  used, 
FITC  is  still  slightly  excited  and  the  cutoff  filters  cannot  com¬ 
pletely  eliminate  the  FITC  signal.  At  the  shorter  wavelength,  we 
experienced  problems  with  fluorescence  energy  transfer  because 
the  light  emitted  from  the  FITC  was  exciting  the  rhodamine.  This 
problem  was  partially  overcome  by  combining  FITC  with  Texas 
Red  for  double-labeling  experiments.  Each  of  the  two  laser 
wavelengths  was  used  to  excite  a  single  fluorophore,  but  using 
528.7  nm  excitation,  a  higher  cutoff  filter  (580 nm)  could  then  be 
used  to  block  out  the  FITC  signal.  There  were  no  detectable  prob¬ 
lems  with  fluorescence  energy  transfer  with  this  combination; 
however,  it  did  present  another  difficulty.  Although  the  laser  we 
used  has  lines  available  at  528.7  and  476.5  nm,  the  power  at  these 
wavelengths  varies  markedly  with  time  and  temperature,  and,  in 
addition,  if  a  double  dichroic  is  not  used  (Chapter  9,  this  volume), 
changing  the  filters  to  select  a  different  wavelength  can  produce 
mis-registration  between  the  two  images.  Both  of  these  situations 
can  produce  sets  of  images  that  are  not  matched  in  intensity,  and 
the  detector  gain  must  be  adjusted  to  “normalize"  them. 

Another  useful  combination  was  Lucifer  Yellow  and  Texas 
Red,  which  were  used  to  study  the  endocytic  compartments  in 
filter-grown  MDCK  cells  in  vivo. 

Using  a  krypton-argon  laser,  which  has  lines  at  488  nm  and 
567  nm,  has  reduced  difficulties  with  bleed-through.  The  567  nm 
line  is  closer  to  the  excitation  maximum  of  Texas  Red  and,  with 
the  proper  configuration  of  cutoff  filters,  allows  the  user  to  label 
specimens  with  FITC  and  Texas  Red  (see  Chapters  16  and  36,  this 
volume). 

We  have  successfully  double-labeled  specimens  with  FITC 
and  rhodamine/Texas  Red  using  a  helium-argon  (He-Ar)  laser. 
One  of  the  cellular  components  is  labeled  with  two  secondary  anti¬ 
bodies  having  the  same  antigenic  specificity.  The  secondary  anti¬ 
body  mixture  contains  a  1 : 4  molar  ratio  of  rhodamine/Texas  Red 
conjugated  antibodies.  We  found  that  there  was  no  bleed-through 
when  the  FITC  signal  was  imaged.  However,  using  the  514nm  line 
gave  an  excellent  signal  from  the  rhodamine/Texas  Red  mixture 
(Bacallao  and  Garfinkel,  1994). 

Some  consideration  must  be  given  prior  to  the  experiment  as 
to  what  structures  will  be  labeled  with  which  fluorophore.  In  our 
experience,  perhaps  because  of  the  higher  quantum  efficiency  of 
the  detector  or  the  lower  diffraction  limit,  FITC  appears  to  give 
images  with  better  contrast  than  does  Texas  Red,  so  structures  with 
fine  detail  (e.g..  microtubules)  tend  to  yield  a  better  image  when 
stained  with  FITC  (Fig.  18.4).  In  samples  that  have  been  labeled 
with  both  FITC  and  Texas  Red,  the  FITC  image  must  be  obtained 
first,  followed  by  the  Texas  Red  image,  because  FITC  bleaches 
rapidly  even  with  antibleaching  agents  present.  Most  microscopes 
now  permit  the  simultaneous  acquisition  of  images  produced  by 
these  fluorophores  (Chapters  2  and  9,  this  volume). 

Triple  Labeling 

One  promising  method  for  triple  labeling  involves  the  use  of 
immunogold,  silver-enhancement  labeling  techniques  (Lackie  et 
al.,  1985;  Scopsi  and  Larsson,  1985;  Bastholm  et  al,  1986;  Birrell 
and  Hedbert,  1987;  Danscher  et  al.,  1987).  This  labeling  method 
has  been  used  successfully  to  stain  cell  adhesion  plaques  (Paddock, 
1989,  2002). 
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FIGURE  18.4.  Deformation  of  a  sample  due  to  improper  mounting.  A  con¬ 
fluent  monolayer  of  MDCK  cells  was  labeled  in  vivo  with  C6-NBD-ceramide. 
The  coverslip  was  placed  on  the  acrylic  spacing  mounts  incorrectly.  The  apical 
surface  has  been  completely  flattened.  The  basolateral  membranes  are  no  longer 
vertical  probably  due  to  shearing.  Bar  =  10  pm. 

A  third  label  can  be  added  to  a  double-labeled  fluorescent 
sample  using  gold-conjugated  antibodies,  which  can  be  imaged  in 
the  backscattered  light  mode  (Pawley  et  al,,  1993;  Linares-Cruz 
et  al.,  1994).  Image-processing  techniques  should  allow  one  to 
overlay  all  three  images.  Once  again,  the  cellular  structures  to  be 
studied  should  be  matched  with  the  particular  labeling  method 
employed.  In  our  initial  attempts  to  study  mitotic  spindles  using 
immunogold  labels,  the  gold  was  so  dense  that  it  acted  as  a  mirror, 
preventing  us  from  obtaining  an  image  below  the  upper  half  of  the 
spindle. 

We  have  routinely  used  three  or  more  fluorescent  probes  on 
single  samples  to  image  various  intracellular  compartments  using 
the  Zeiss  510  confocal  microscope  equipped  with  an  Enterprise 
laser  (Spectra  Physics,  Mountain  View,  CA),  two  helium-neon 
(He-Ne)  lasers  and  a  Kr-Ar  laser.  Laser  lines  at  353  nm,  383  nm, 
488  nm,  512nm.  563  nm,  and  650 nm  are  available  for  imaging. 
Appropriate  emission  filter  sets  are  available  on  this  system  that 
make  four-color  fluorescence  imaging  routine.  We  have  found  it 
to  be  particularly  useful  to  routinely  label  nuclear  compartments 
with  DAPI  or  Hoescht  3314.  Computer  programs  designed  to 
measure  the  extent  of  co-localization  between  two  or  more  labeled 
marker  proteins  give  correlations  greater  than  0.5  for  any  two 
markers,  even  if  there  is  no  visual  evidence  for  co-localization. 
This  is  because  the  nucleus  comprises  approximately  50%  of  total 
cell  volume,  so  any  cytoplasmic  protein  excluded  from  the  nucleus 
is  automatically  constrained  to  the  other  50%  of  the  total  volume. 
By  routinely  labeling  the  nuclear  compartment,  we  can  make 
masks  that  remove  the  nucleus  from  the  analyzed  volumes  to 
obtain  realistic  correlation  coefficients.  A  computer  program  that 
give  percentage  of  co-localization  and  correlation  coefficients  is 
under  development  and  is  called  CORR3D  (Christopher  Constan¬ 
tine,  manuscript  in  preparation).  It  will  be  available  as  shareware 
at  http://www.nephrology.iupui.edu. 

Preparation  of  Tissue  Specimens 

Preparation  of  tissues  for  examination  by  confocal  microscopy  is 
complicated  by  problems  of  fixative  penetration,  the  heteroge¬ 
neous  cellular  composition  of  tissues,  and  the  presence  of  the 
extracellular  matrix.  Tissue  slices  are  fixed  by  two  general  ap¬ 
proaches:  immersion  and  perfusion.  With  immersion  fixation,  the 
tissue  is  dissected,  cut  into  small  pieces,  and  immersed  in  the  fix¬ 


ation  solution.  The  dissection  must  be  performed  carefully  in  order 
to  avoid  damaging  the  specimen.  Penetration  of  the  fixative  is 
dependent  on  the  thickness  of  the  specimen  and  the  type  of  fixa¬ 
tive  (formaldehyde  penetrates  faster).  During  immersion  fixation, 
cells  are  exposed  to  anaerobic  conditions;  a  gradient  of  fixative 
concentration  means  that  deeper  regions  are  often  fixed  less  well 
than  superficial  regions. 

Fixation  of  tissues  by  perfusion  is  preferable  in  most  exam¬ 
ples  because  the  fixative  reaches  all  of  the  tissue  more  rapidly 
(Ericsson  and  Biberfeld,  1967;  Fahimi,  1967;  Petersen,  1977; 
Nowell  and  Pawley,  1980),  and  the  cells  are  less  likely  to  develop 
anoxic  damage  (Rostgaard  et  al,,  1993).  The  perfusion  pressure 
must  be  carefully  controlled  because  otherwise  this  can  be  a  source 
of  tissue  damage.  A  recommended  list  of  organ-specific  perfusion 
pressures  has  been  reported  by  Hayat  (1989).  We  have  used  either 
diluted  Karnovsky’s  fixative  or  paraformaldehyde  fixative  with 
a  perfusion  pump  to  fix  kidneys  with  satisfactory  morphological 
preservation.  The  osmolality  of  the  paraformaldehyde  fixation 
buffer  must  be  optimized  depending  on  the  region  of  the  renal 
tubule  one  wishes  to  study  (Hayat,  1989). 

Preparing  tissue  for  confocal  microscopy  brings  separate  con¬ 
cerns.  Because  excitation  wavelengths  are  shifted  toward  infrared, 
it  is  possible  to  image  deeper  into  tissue  (Brakenhoff  et  al,,  1996; 
Soeller  and  Cannell,  1996).  We  routinely  image  100  pm  thick 
vibratome  sections  labeled  with  a  variety  of  antibodies  and  fluo¬ 
rescent-tagged  lectins.  The  preparation  method  for  labeling  thick 
sections  is  provided  here. 

Labeling  Thick  Sections 

1.  Perfuse-fix  anesthesized  mice  with  4%  paraformaldehyde 
(PFA)  (made  in  lx  PBS).  Flush  ice  cold  lx  PBS  through  vas¬ 
culature  via  left  ventricle  before  PFA  perfusion. 

2.  Cut  vibratome  sections  of  kidneys  (50-200 pm  thickness). 

3.  Wash  sections  at  room  temperature,  2  to  4h,  in  lx  PBS  (vig¬ 
orously  on  orbital  shaker/rocker).  We  use  14 mL  in  a  15  mL 
Falcon  tube  (air  bubble  helps  with  agitation).  Carefully  trans¬ 
fer  wet  sections  with  soft  bristle  brush  or  one  tip  of  forceps  — 
don’t  squeeze  tissue! 

4.  Block:  incubate  tissue  sections  in  ~200pL  blocking  buffer  — 
sections  should  be  in  small  PCR  tubes  (blocking  buffer  is  0.1% 
to  1%  Triton  X-100*,  1%  to  2%  BSA,  lx  PBS,  make  fresh). 
Put  back  on  rotator/shaker  for  1  to  2h. 

5.  Add  primary  antibody  diluted  in  blocking  buffer  (we  use  200 
pL  total  volume  in  small  PCR  tubes)  —  place  labeled  tubes  in 
dry  50  mL  Falcon  tube  and  rotate  overnight  on  rotator/rocker 
4°C  (or  4+h  at  room  temperature,  depending  on  thickness  of 
sections). 

6.  Next  morning  repeat  wash  (see  step  3). 

7.  Incubate  in  fluorescent-labeled**  secondary  antibody  diluted 
in  blocking  buffer  (same  conditions  as  step  5  above).  Also 
good  time  to  add  fluorescent-labeled  lectins  (e.g.,  from  Vector 
labs,  usually  around  1 :200  dilution),  DAPI***  or  phalloidin. 
Put  PCR  tubes  in  dry,  foil-wrapped  50  mL  Falcon  tube  (protect 
from  light),  place  on  rocker  in  4°C  refrigerator,  overnight. 
Lectins  that  work  well  in  mouse  tissues: 

A.  peanut  agglutinin-rhodamine  (PNA)  proximal  tubules  and 
collecting  ducts 

B.  lotus  tetragonolobus-fluorescein  (LTG)  proximal  tubules 

C.  dolichos  biflorus-rhodamine  (DBA)  collecting  ducts 

D.  lens  culinaris  agglutin  (LCA)  GBM,  mesangial  matrix 

8.  Next  morning  wash  sections  in  lx  PBS  (in  15  mL  blue  cap 
tubes),  2  to  4  hours  or  overnight. 
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9.  Transfer  to  round  base  of  clean  coverslip  dishes  in  prepara¬ 
tion  for  imaging  (pre-measure  coverslip  thickness  with 
micrometer).  Keep  wet  in  PBS.  To  prevent  tissue  from  float¬ 
ing  around,  you  may  need  to  stabilize  (not  squish!  with  a  cap 
of  cooled  agarose  or  a  second  coverslip  (square  or  round). 
Avoid  air  bubbles! 

10.  Adjust  microscope  objective  collar  to  match  coverslip 
thickness. 

11.  Capture  z-stacks  at  0.4  pm  intervals  with  60x  water- 
immersion  objective. 

12.  Render  3D  image  with  Voxx  or  other  3D  reconstruction  soft¬ 
ware  packages. 

Important  Points 

•  *1%  is  better  than  0.1%  Triton  X-100. 

•  **Better  depth  of  imaging  is  achieved  with  Rhodamine  as  the 
label  for  the  fluorescent  probe  than  fluorescein. 

•  ***DAPI  can  be  added  5  min  before  imaging  (must  be 
washed). 

Refractive  Index  Mismatch 

Unfortunately,  the  imaging  of  thick  sections  is  often  hampered  by 
T|  mismatch,  which  can  lead  to  a  significant  loss  of  both  resolution 
and  signal  intensity.  The  problem  becomes  serious  when  one  sec¬ 
tions  more  than  10  pm  into  an  aqueous  specimen  while  using  a 
high  numerical  aperture  (NA)  oil  objective  (Hell  et  al.,  1993).  The 
problem  can  be  solved  by  using  objective  lenses  designed  for  water 
immersion  together  with  coverslips  with  lower  T|  such  as  CYTOP, 
which  has  T|  =  1.34  (compared  to  T|  =  1.33  for  water)  and  95% 
transmittance  in  the  range  of  visible  light  (developed  by  Asahi 
Glass,  Yokohama,  Japan).  Water-immersion  objectives  that  are 
designed  to  work  without  glass  coverslips  can  take  advantage  of 
this  new  material. 

Another  solution  to  the  RI  problem  is  to  mount  the  specimen 
in  a  media  that  matches  both  the  refractive  index  of  immersion  oil 
and  that  of  a  standard  coverslip.  Mounting  media  with  T|  =  1.518 
can  be  made  from  mixtures  of  glycerol,  borate,  and  potassium 
iodide  (W.  McCrone,  personal  communication).  Mounting  media 
with  a  range  of  refractive  indexes  can  be  bought  from  R.P.  Cargille 
Labs  (NJ)  or  McCrone  Accessories  and  Components  (Westmont, 
IL). 

Table  18.1  lists  values  of  T)  measured  with  a  Bausch  and  Lomb, 
Abbe  3-L  refractometer  by  Dr.  M.  Wessendorf  (University  of 


TABLE  18,1.  Refractive  Index  of  Common  Mounting  Media 


Mounting  Media 

Refractive  Index" 

Gel/mount  (Biomeda) 

1.3641 

Methyl  salicylate  (Sigma) 

1.5409 

Dimethyl  sulfoxide  (Sigma) 

1.4836 

VectaShield  (Vector  Labs) 

1.4577 

DPX  (Fluka) 

1.5251 

50%  glycerol/PBS/DABCO 

1.4159 

Water 

1.3381 

Cargille  index  of  refraction  liquids 

1.460-1. 700* 

5%  n-propyl  gallate/0.0025%  p-phenylene  gallate 
(PPD)  dissolved  in  glycerol 

1.4739 

0.25%  PPD,  0.0025%  DABCO,  5%  n-propyl  gallate 
dissolved  in  glycerol 

1.4732 

“Corrected  to  20°C. 

ftCan  be  ordered  as  a  set  of  liquids  with  refractive  index  intervals  between  each 
samples  as  low  as  0.002. 


TABLE  18.2.  Refractive  Index  of  Different  Tissue 
and  Organs 


Organ/Tissue 

Refractive  Index 

Spleen 

1.443  +  0.002 

Liver 

1.448  +  0.002 

Kidney 

Cortex 

1.444  +  0.002 

Medulla 

1.438  +  0.002 

Pancreas 

1.435  +  0.002 

Intestinal  wall 

1.436  +  0.002 

Fat 

1.472  +  0.002 

Bone 

1.556  +  0.002 

Cartilage 

1.492  +  0.002 

Muscle 

1.431  +0.002 

Lung 

1.342  +  0.002 

Gall  bladder  wall 

1.350  +  0.002 

Blood  (uncoagulated) 

Serum 

1.330 

Formed  elements 

1.432  +  0.003 

Coagulated  blood 

1.465  +  0.003 

Gray  matter 

1.395  +  0.002 

White  matter 

1.467  +  0.002 

Cerebellum 

1.470  +  0.002 

From  Biswas  and  Gaupta  (2002). 


Minnesota,  St.  Paul,  MN)  and  Dr.  Y.  Prakesh  (Mayo  Clinic, 
Rochester,  MN)  for  a  number  of  commercially  available  mounting 
media.  Table  18.2  lists  the  refractive  index  of  commonly  imaged 
tissues  (Biswas  and  Gupta,  2002). 

Screening  Antibodies  on 
Glutaraldehyde-Fixed  Specimens 

In  our  experience,  glutaraldehyde  fixation  best  preserves  the  struc¬ 
tural  and  spatial  integrity  of  the  cell.  The  main  difficulty  with  glu¬ 
taraldehyde  fixation  is  the  frequent  loss  of  epitope  antigenicity. 
However,  there  may  be  an  alternative  to  formaldehyde  fixation  to 
avoid  this  problem.  It  is  possible  to  screen  monoclonal  antibodies 
against  antigens  that  had  been  fixed  in  glutaraldehyde.  Frequently, 
clones  are  screened  against  samples  fixed  with  methanol  or 
formaldehyde,  but  then  they  are  actually  used  on  glutaraldehyde  - 
fixed  specimens.  Clearly,  this  process  may  produce  disappointing 
results.  Better  results  should  be  obtained  if  only  those  monoclonal 
antibodies  that  bind  to  glutaraldehyde-fixed  proteins  are  used  for 
detailed  structural  studies. 

Microwave  Fixation 

Microwave  heating  with  and  without  chemical  fixatives  has  been 
used  for  specimen  preparation  (Thoolen,  1990;  Benhamou  et  al., 
1991;  Jackson,  1991).  The  intense  heat  generated  by  microwave 
radiation  directly  coagulates  the  proteins  and  also  accelerates 
chemical  fixation.  It  is  not  clear  what  artifacts  are  created  by  this 
fixation  method,  but  some  denaturation  of  protein  is  to  be 
expected,  and  there  may  be  problems  with  reproducibility  re¬ 
lated  to  the  precise  location  of  the  specimen  in  the  chamber,  etc. 
(Giberson  and  Demaree,  1995).  Some  investigators  have  combined 
chemical  fixation  with  microwave  treatment  to  accelerate  the  rate 
of  chemical  fixation  with  satisfactory  tissue  preservation  (Jamur 
et  al.,  1995;  Sawitzky  et  al.,  1996).  Clearly,  further  development 
of  this  fixation  method  might  be  productive. 
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FIGURE  1 8.5.  Stereo  image  of  the  actin  cytoskele- 
ton  in  MDCK  cells.  The  cells  were  labeled  with 
rhodamine  phalloidin  after  fixation  with  glutaralde- 
hyde.  The  image  was  reconstructed  using  Advanced 
Visualization  Software  (Stardent,  CA)  on  a  Kubota 
Pacific  workstation.  (Photo  courtesy  of  A.  Garfinkel 
and  S.  Monke.) 


A  review  of  the  literature  shows  that  the  confocal  microscope 
is  often  used  only  to  produce  2D  images  and  that  its  ability  to  gen¬ 
erate  data  sets  suitable  for  3D  reconstruction  has  been  underused. 
Advances  in  image-processing  and  image-analysis  techniques  now 
provide  the  biologist  with  an  array  of  quantitative  3D  measurement 
tools  (Chapters  14  and  15,  this  volume).  Figure  18.5  shows  such 
a  reconstruction  of  the  actin  cytoskeleton  of  some  MDCK  cells 
grown  on  filter  supports  in  which  the  images  are  cut-away  views 
generated  by  the  Advanced  Visualization  Software  program  (Star- 
dent,  CA)  supported  by  a  Kubota  Pacific  workstation  (generously 
provided  by  A.  Garfinkel  and  S.  Monke,  UCLA,  Los  Angeles,  CA). 
By  applying  this  computer  analysis  system  to  this  type  of  data,  one 
can  determine  the  relative  intensity  values  for  every  voxel  in  the 
image,  calculate  the  volume  of  the  actin  network  in  the  cell,  and 
determine  the  volume  that  demonstrates  more  than  some  thresh¬ 
old  level  of  staining  as  well  as  displaying  a  stereo  image  showing 
its  3D  spatial  distribution. 

Two-photon  microscopy  is  being  used  to  study  tissue  mor¬ 
phology  due  to  its  ability  to  penetrate  deep  into  tissue  at  infrared 
wavelengths.  In  our  experience  we  routinely  image  up  to  100 
microns  into  tissue.  For  our  preparations  this  has  been  the  limit 
that  an  image  can  be  acquired.  Flowever,  we  have  not  performed 
a  rigorous  assessment  of  fixation  conditions  versus  imaging  depth. 
It  is  possible  that  optimizing  fixation  conditions  and  matching  the 
refractive  index  of  the  mounting  media  may  increase  the  maximal 
depth  of  image  acquisition.  The  optimal  fluorophores  have  yet  to 
be  determined  for  imaging  deep  into  tissues  and  it  may  be  possi¬ 
ble  to  obtain  clearer  images  by  measuring  the  point  spread  func¬ 
tions  at  various  tissue  depths  to  deconvolved  confocal  images 
obtained  at  greater  depths. 

It  is  important  to  remember  that  the  accuracy  of  any  three- 
dimensional  reconstruction  depends  on  the  extent  to  which  the 
specimen  examined  in  the  microscope  retained  the  structural  fea¬ 
tures  it  had  in  vivo. 


CONCLUSION 

In  molecular  cell  biology,  the  microarchitecture  of  organisms  is 
rapidly  becoming  a  major  subject  for  analysis.  For  example,  the 
temporal  sequence  of  morphogenetic  changes  readily  observable 
in  embryos  makes  an  excellent  system  to  test  the  effect  of  gene 
deletions,  protein  overproduction,  and  changes  in  transcriptional 
control  on  the  development  of  an  organism.  The  ability  of  the  con¬ 
focal  microscope  to  render  accurate,  3D  images  using  immuno¬ 
fluorescence  techniques  means  that  it  will  be  a  major  tool  in  the 
analysis  of  morphogenesis.  To  ensure  that  these  3D  images  carry 
real  3D  information,  it  is  important  that  whenever  possible,  struc¬ 
tural  morphology  obtained  by  various  preservation  methods  should 
be  compared  to  that  observed  in  vivo.  In  some  cases,  a  compro¬ 
mise  will  have  to  be  made  with  respect  to  fixation  methods  in  order 
to  preserve  the  antigenicity  of  a  particular  protein.  These  compro¬ 
mises  should  only  be  made  if  the  in  vivo  data  suggest  that  the 
compromise  does  not  affect  the  relevance  of  the  data.  Only  vigor¬ 
ous  attention  to  the  details  of  specimen  preparation,  and  in  partic¬ 
ular,  constant  comparison  between  living  and  prepared  specimens, 
can  ensure  that  an  accurate  understanding  of  the  3D  structure  of 
the  living  cell  is  achieved. 
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INTRODUCTION 

If  a  picture  is  worth  a  thousand  words,  then  a  movie  may  be  worth 
a  million  words.  Microcinematography  and,  later,  video 
microscopy  have  provided  great  insight  into  biological  phenom¬ 
ena.  One  limitation,  however,  has  been  the  difficulty  of  imaging  in 
three  dimensions.  In  many  cases,  observations  have  been  made  on 
cultured  cells  that  are  thin  to  start  with  or  tissue  preparations  that 
have  been  sectioned. 

The  development  of  the  first  beam-scanning  confocal  micro¬ 
scope  was  motivated  by  the  goal  of  making  observations  in  the 
tissues  of  living  organisms  (Petran  et  al.,  1986).  The  optical  sec¬ 
tioning  capability  of  the  confocal  or  multi-photon  (MP)  micro¬ 
scope  allows  one  to  make  thin-slice  views  in  intact  cells  or  even 
intact  animals.  Confocal  microscopes  are  now  fairly  common,  and 
because  they  employ  non-ionizing  radiation,  they  are  increasingly 
being  used  to  study  living  cells  and  tissue  preparations. 

What  are  the  specific  challenges  of  applying  confocal  imaging 
to  studies  of  living  cells?  First,  the  experimenter  must  do  no  harm. 
Arguably,  the  main  obstacle  in  living  cells  microscopy  is  not 
“getting  an  image”  but  doing  so  without  upsetting  the  cell.  To  be 
useful,  the  study  must  be  carried  out  on  a  biological  system  that 
retains  normal  function  and  can  be  subjected  to  controlled  condi¬ 
tions  while  on  the  stage  of  the  microscope.  Often,  environmental 
variables  such  as  temperature,  C02,  or  pH  must  be  regulated  and/or 
an  efficient  directional  perfusion  system  must  be  used.  Unfortu¬ 
nately,  the  difficulty  of  keeping  cells  alive  and  functioning  on 
the  microscope  discourages  many  researchers.  This  chapter  is 
designed  to  help  them  succeed  well  enough  to  become  convinced 
of  the  importance  and  utility  of  this  approach. 

Other  difficulties  are  more  specific  to  confocal  fluorescence 
microscopy.  All  studies  with  fluorescence  benefit  from  collecting 
as  much  of  the  emitted  fluorescent  light  as  is  possible,  but  this  is 
particularly  important  for  studies  of  living  cells  because  photo¬ 
dynamic  damage  and  consequent  alteration  in  normal  cell  behav¬ 
ior  is  a  very  real  possibility  (see  Chapters  16  and  38,  this  volume). 
Therefore,  optimizing  microscope  photon  collection  efficiency  is 
crucial  for  successful  confocal  microscopy  of  living  cells. 

Another  difference  between  living  and  fixed  cell  studies  is  the 
element  of  time.  All  living  processes  have  an  inherent  time  course, 
and  the  imaging  system  must  produce  images  at  the  appropriate 
rate  to  show  the  changes  involved.  The  amount  of  light  necessary 
to  obtain  the  data  must  be  apportioned  over  time  so  that  enough 
images  can  be  obtained  to  describe  the  process  under  investigation 
without  damaging  the  cells.  Although  early  confocal  microscopes 
had  a  relatively  slow  scan  speed,  newer  technology  now  permits 


very  rapid  image  collection  to  explore  spatially  and  temporally 
dynamic  biological  processes  (see  Chapter  10,  this  volume).  In 
single-beam  scanning  systems,  the  field  of  view  often  is  reduced 
to  achieve  higher  imaging  speeds. 

The  fluorescent  probes  used  in  studies  of  living  cells  must  not 
impair  normal  cell  function.  Immunofluorescence,  which  has  been 
used  so  successfully  to  localize  molecules  in  fixed  cells,  has  not 
been  practical  in  living  cells.  However,  there  are  now  many  com¬ 
mercially  available  fluorescent  probes  for  structural  and  physio¬ 
logical  studies  of  cells  and  tissues  (see  Chapters  16  and  17,  this 
volume).  Of  even  more  importance,  the  “green  revolution”  based 
on  the  green  fluorescent  protein  (GFP)  has  changed  the  landscape 
and  is  ushering  in  an  exciting  period  of  biological  imaging  of  pro¬ 
teins  in  living  cells,  and  of  various  cell  types  in  living,  intact 
tissue  preparations  (Chalfie  et  al.,  1994;  Bastieans  and  Pepperkok, 
2000). 

Although  confocal  microscopy  of  living  cells  is  difficult,  its 
usefulness  was  demonstrated  over  15  years  ago  in  two  pioneer¬ 
ing  studies.  Cornell-Bell  and  colleagues  (1990)  used  confocal 
microscopy  to  make  a  major  discovery:  the  existence  of  glutamate- 
stimulated,  transcellular  Ca2+  waves  in  astroglia.  In  the  same  year, 
confocal  microscopy  was  used  to  characterize  developmental 
changes  in  an  intact  animal  by  imaging  neuronal  axons  and  their 
growth  cones  in  the  developing  brain  of  a  tadpole  (O’Rourke  and 
Fraser,  1990).  Ever  since  these  pioneering  studies,  there  has  been 
an  increasing  use  of  confocal  microscopy  to  study  dynamic 
processes  in  an  array  of  diverse  biological  preparations.  When  the 
second  edition  of  this  volume  appeared  in  1995,  about  80  papers 
using  confocal  microscopy  on  living  cells  were  found.  Today 
(early  2005),  there  are  over  500  published  studies  using  live-cell 
confocal  imaging. 

While  live-cell  applications  of  confocal  imaging  have 
expanded  significantly  over  the  past  decade,  multi-photon  imaging 
(see  Chapters  21  and  28,  this  volume)  is  poised  to  make  a  similar 
impact  on  live-cell  and  tissue  studies  in  the  next  decade.  There 
are  trade-offs,  however.  MP  imaging  can  be  useful  especially  for 
very  deep  penetration  in  tissues  (>100  pm)  where  non-descanned 
detection  increases  signal  substantially,  but  on  thinner  specimens, 
the  actual  damage/excitation  may  be  greater  for  MP  than  for 
single-photon  confocal  imaging  (Tauer,  2002;  see  also  Chapter  38, 
this  volume).  Moreover,  the  cost  differential  is  such  that  one 
could  have  two  to  three  graduate  students  working  away  on 
disk  scanners  or  simpler  beam-scanning  confocal  units  for  every 
one  on  a  MP  unit.  At  any  rate,  most  of  the  topics  covered  in 
this  chapter  are  relevant  for  both  single-photon  confocal  and  MP 
excitation. 
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OVERVIEW  OF  LIVING-CELL  CONFOCAL 
IMAGING  TECHNIQUES 

Although  live-cell  imaging  often  involves  time-lapse  microscopy 
to  monitor  cell  movements,  modern  approaches  are  extending 
these  observations  well  beyond  simply  making  movies  of  cell 
structure.  Increasingly,  time-lapse  imaging  is  being  integrated  with 
specialized  techniques  for  monitoring,  measuring,  and  perturbing 
dynamic  activities  of  cells  and  subcellular  structures.  Below  we 
summarize  some  major  techniques  available  for  studying  the 
dynamic  organization  of  molecules  and  cells  in  live  biological 
specimen.  These  techniques  are  summarized  in  Table  19.1. 

Time-Lapse  Fluorescence  Imaging 

Time-lapse  fluorescence  imaging  involves  repeated  imaging  of  a 
labeled  specimen  at  defined  time  points,  thereby  permitting  studies 
on  the  dynamic  distribution  of  fluorescently  labeled  components 
in  living  systems.  Imaging  can  be  performed  in  one,  two,  or  three 
spatial  dimensions:  one-dimensional  (ID)  imaging  involves  rapid 
and  repeated  imaging  of  single  scan  lines;  two-dimensional  (2D) 
imaging  involves  repeated  imaging  of  single  focal  planes;  and 
three-dimensional  (3D)  imaging  involves  repeated  imaging  of 
multiple  focal  planes  in  a  thick  specimen.  The  time  intervals  for 
sequential  image  collection  can  range  from  sub-second  to  days  or 
even  months  (e.g.,  Gan  et  al.,  2003). 

Many  small  molecule,  vital  fluorescent  probes  that  give  highly 
specific  cellular  or  subcellular  patterns  of  labeling  are  now  avail¬ 
able  (see  below  and  Chapters  16  and  17,  this  volume).  In  addition, 
GFP  or  GFP-related  proteins  are  now  routinely  fused  to  other  pro¬ 
teins  of  interest,  and  the  inherent  brightness  and  photostability  of 
many  of  these  fluorescent  proteins  make  them  well  suited  for  the 
repeated  imaging  needed  for  time-lapse  studies.  Together,  these 
fluorescent  probes  are  affording  a  seemingly  limitless  array  of  pos¬ 
sibilities  for  imaging  molecular  components  in  live  cells. 

Multi-Channel  Time-Lapse 
Fluorescence  Imaging 

The  plethora  of  excellent  vital  fluorescent  labels  with  varying 
spectral  characteristics  (including  spectral  variants  of  GFP) 
allows  multi-label  experiments  to  visualize  the  relative  distribu¬ 
tion  of  several  different  cell  or  tissue  components  simultaneously. 
Advances  in  imaging  technology  have  facilitated  automated  col¬ 
lection  of  more  than  one  fluorescent  channel  (either  sequentially 
or  simultaneously)  with  improved  ability  to  maximize  signal  col¬ 
lection  and  to  separate  partially  overlapping  signals. 

In  addition  to  studies  using  multiple  fluorescent  tags,  multi¬ 
channel  data  collection  permits  ratiometric  imaging  of  single 
probes  whose  spectral  properties  (absorption  or  emission)  change 
depending  on  ionic  conditions,  such  as  the  Ca2+  sensitive  physio¬ 
logical  indicator,  indo-1  (see  Chapter  42,  this  volume). 

Spectral  Imaging  and  Linear  Unmixing 

Increasingly,  experiments  are  incorporating  multiple  fluorescent 
probes  within  single  cells  or  tissues  to  define  the  differential  dis¬ 
tribution  of  more  than  one  labeled  structure  or  molecular  species. 
Such  multi-color  or  multi-spectral  imaging  experiments  require 
adequate  separation  of  the  fluorescent  emissions,  and  this  is  espe¬ 
cially  problematic  when  the  spectra  are  substantially  overlapping. 
Spectral  imaging  utilizes  hardware  to  separate  the  emitted  light 
into  its  spectral  components.  Linear  unmixing  is  a  computational 


process  related  to  deconvolution  that  uses  the  spectra  of  each  dye 
as  though  it  were  a  point-spread  function  at  a  fixed  location  to 
“unmix”  the  component  signals  (Tsurui  et  al.,  2000;  Lansford 
et  al.,  2001;  Hiraoka  et  al,  2002).  Although  together  these  analy¬ 
tical  tools  can  be  used  to  discriminate  distinct  fluorophores  with 
highly  overlapping  spectra  (Zimmermann  et  al.,  2003),  they  do  so 
at  the  cost  of  requiring  that  significantly  more  photons  be  detected 
from  each  pixel. 

Fluorescence  Recovery  After  Photobleaching 

Fluorescence  recovery  after  photobleaching  (FRAP),  also  known 
as  fluorescence  photobleaching  recovery  (FPR),  is  a  technique  for 
defining  the  diffusion  properties  of  a  population  of  fluorescently 
labeled  molecules  (Axelrod  et  al.,  1976;  Koppel  et  al,,  1976;  for 
review,  see  Lippincott-Schwarz  et  al.,  2003).  Typically,  a  spot  or 
line  of  intense  illumination  is  used  to  bleach  a  portion  of  a  fluo¬ 
rescent  cell,  and  the  recovery  of  fluorescent  signal  back  into  the 
bleached  area  from  adjacent  areas  is  monitored  over  time  (usually 
seconds  to  minutes).  Although  this  technique  can  yield  quantita¬ 
tive  information  on  the  diffusion  coefficient,  mobile  fraction,  and 
binding/dissociation  of  a  protein,  care  has  to  be  taken  not  to  use 
so  much  power  in  the  bleach  beam  that  the  cellular  structure  is  dis¬ 
rupted  (Bloom  and  Webb,  1984;  Flock  etal.,  1998;  see  also  Figures 
49.10  through  49.14,  this  volume).  Quantitative  assessments  of 
FRAP  data,  which  can  be  confounded  by  uncertainties  in  the 
experimental  and  biological  parameters  in  living  cells,  may  benefit 
from  computer  simulations  (Weiss,  2004). 

Fluorescence  Loss  in  Photobleaching 

This  technique  utilizes  repeated  photobleaching  in  an  attempt 
to  bleach  all  fluorophores  within  a  given  cellular  compartment 
(Lippincott-Schwarz  et  al,  2001).  Thus,  fluorescence  loss  in  pho¬ 
tobleaching  (FLIP)  can  be  used  to  assess  the  continuity  of  mem¬ 
brane  bounded  compartments  (e.g.,  endoplasmic  reticulum  [ER]  or 
Golgi  apparatus)  and  to  define  the  diffusional  properties  of  com¬ 
ponents  within,  or  on  the  surface  of,  these  compartments. 

Fluorescence  Resonance  Energy  Transfer 

Fluorescence  resonance  energy  transfer  (FRET)  is  a  technique  for 
defining  interactions  between  two  molecular  species  tagged  with 
different  fluorophores  (Stryer,  1978;  Sekar  and  Periasamy,  2003). 
It  takes  advantage  of  the  fact  that  the  emission  energy  of  a  fluo¬ 
rescent  “donor”  can  be  absorbed  by  (i.e.,  transferred  to)  an  “accep¬ 
tor”  fluorophore  when  these  fluorophores  are  in  nanometer 
proximity  and  have  overlapping  spectra  (see  Chapters  16,  27,  and 
45,  this  volume). 

Fluorescence  Lifetime  Imaging 

This  technique  measures  the  lifetime  of  the  excited  state  of  a  flu¬ 
orophore  (Lakowicz  et  al.,  1992  and  Chapter  27,  this  volume). 
Each  fluorescent  dye  has  a  characteristic  “lifetime”  in  the  excited 
state  (usually  1-20  ns),  and  detection  of  this  lifetime  can  be  used 
to  distinguish  different  dyes  in  samples  labeled  with  multiple  dyes. 
Fluorescence  lifetime  imaging  (FLIM)  can  be  utilized  in  conjunc¬ 
tion  with  FRET  analysis  because  the  lifetime  of  the  donor  fluo¬ 
rophore  is  shortened  by  FRET.  In  fact,  FLIM  can  improve  the 
measurement  during  FRET  analysis  because  the  fluorescence  life¬ 
time  is  independent  of  the  fluorophore  concentration  and  excita¬ 
tion  energy  (Bastiaens  and  Squire,  1999;  Elangovan  et  al,  2002; 
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Chen  et  al.,  2003;  Chapter  27,  this  volume).  However,  the  lifetime 
can  be  modulated  by  environmental  considerations  (e.g.,  pH,  ion 
concentration),  and  this  change  can  be  used  to  measure  changes  in 
the  concentration  of  certain  ions  (Lin  et  al,  2003). 

Fluorescence  Correlation  Spectroscopy 

Fluorescence  correlation  spectroscopy  (FCS)  measures  sponta¬ 
neous  fluorescence  intensity  fluctuations  in  a  stationary  micro¬ 
scopic  detection  volume  (about  lfL)  (Magde  et  al,  1974).  Such 
intensity  fluctuations  represent  changes  in  the  number  or  quantum 
yield  of  fluorescent  molecules  in  the  detection  volume.  By  ana¬ 
lyzing  these  fluctuations  statistically,  FCS  can  provide  information 
on  equilibrium  concentrations,  reaction  kinetics,  and  diffusion 
rates  of  fluorescently  tagged  molecules  (Elson,  2001).  An  advan¬ 
tage  of  this  approach  is  the  ability  to  measure  the  mobility  of 
molecules  down  to  the  single  molecule  level  and  to  do  so  using  a 
light  dose  orders  of  magnitude  lower  than  used  for  FRAP. 

Fluorescence  Speckle  Microscopy 

The  dynamic  growth  and  movement  of  fluorescently  labeled  struc¬ 
tures  can  be  difficult  to  analyze  when  these  structures  are  densely 
packed  and  overlapping  within  living  cells.  Fluorescent  speckle 
microscopy  (FSM)  is  a  technique  compatible  with  widefield  or 
confocal  microscopy  (Adams  et  al,  2003)  that  uses  a  very  low 
concentration  of  fluorescently  labeled  subunits  to  reduce  out-of¬ 
focus  fluorescence  and  improve  visibility  of  labeled  structures  and 
their  dynamics  in  thick  regions  of  living  cells  (Waterman-Storer 
et  al.,  1998).  This  is  accomplished  by  labeling  only  a  fraction  of 
the  entire  structure  of  interest.  In  that  sense,  it  is  akin  to  perform¬ 
ing  FCS  over  an  entire  held  of  view,  albeit  with  more  focus  on 
spatial  patterns  than  on  quantitative  temporal  analysis.  FSM  has 
been  especially  useful  for  defining  the  movement  and  polymer¬ 
ization/depolymerization  of  polymeric  cytoskeletal  elements,  such 
as  actin  and  microtubules,  in  motile  cells  (Salmon  et  al.,  2002). 

Photo- Uncaging/Photoactivation 

Photo-uncaging  is  a  light-induced  process  of  releasing  a  “caged” 
molecule  from  a  caging  group  to  produce  an  active  molecule 
(Politz,  1999;  Dorman  and  Prestwich,  2000).  A  variety  of  caged 
molecules  have  been  synthesized  and  used  experimentally,  but  in 
some  examples  cages  have  been  used  to  mask  a  fluorophore,  induc¬ 
ing  a  non- fluorescent  state.  Excitation  light  of  -350  nm  is  used  to 
break  photolabile  bonds  between  the  caging  group  and  fluo¬ 
rophore,  thereby  uncaging  the  fluorophore  and  yielding  a  fluores¬ 
cent  molecule. 

A  related  technique  utilizes  genetically  encoded,  photoactivat- 
able  fluorescent  proteins,  of  which  there  are  currently  about  a 
dozen  (for  review,  see  Patterson  and  Lippincott-Schwartz,  2004). 
Two  examples  include  a  photoactivatable  (PA)  form  of  GFP,  called 
PA-GFP  (ex/em:  504/517),  which  shows  a  100-fold  increase  in 
fluorescence  following  irradiation  at  413  nm  (Patterson  and 
Lippincott-Schwartz,  2002),  and  Kaede  (ex/em:  572/582),  which 
shows  a  2000-fold  increase  following  irradiation  at  405  nm  (Ando 
et  al.,  2002;  see  also  Figure  8.37,  this  volume). 

An  extension  of  the  photoactivation  approach,  termed 
reversible  protein  highlighting,  has  been  developed  (Ando  et  al., 
2004).  This  involves  reversible,  light-induced  conversion  of  a  coral 
protein,  Dronpa,  between  fluorescent  and  non-fluorescent  states. 
One  study  used  this  approach  to  monitor  fast  protein  dynamics  in 
and  out  of  cell  nuclei  (Ando  et  al.,  2004).  Thus,  photo-uncaging 


and  photoactivation  are  complementary  to  FRAP  and  can  be  used 
in  conjunction  with  time-lapse  imaging  to  mark  and  follow  a  pop¬ 
ulation  of  molecules  in  order  to  study  their  kinetic  properties 
within  living  cells. 

Optical  Tweezers/Laser  Trapping 

Optical  tweezers,  or  single  beam  laser  traps,  use  the  “radiation 
pressure”  of  a  stream  of  photons  emitted  from  an  infrared  laser  to 
“trap”  small  objects  (often  a  protein-coated  bead)  and  to  move 
them  around  (Sheetz,  1998;  Kuo,  2001;  Chapters  5  and  9,  this 
volume).  This  technique  has  been  especially  useful  for  quantifying 
forces  generated  by  motor  protein  movement  (Ashkin  et  al.,  1990; 
Block  et  al.,  1990;  Kuo  and  Sheetz,  1993)  or  the  strength  of  adhe¬ 
sions  mediated  by  cell  adhesion  molecules  (e.g.,  Schmidt  et  al., 
1993;  Baumgartner  et  al,  2003).  Although  “laser  tweezers”  often 
are  used  in  widefield  imaging  systems,  they  also  have  been  incor¬ 
porated  into  confocal  (Visscher  and  Brakenhoff,  1991)  and  MP 
(Goksor  et  al,  2004)  imaging  systems. 

Physiological  Fluorescence  Imaging 

The  availability  of  fluorescent  physiological  indicators  extends 
live-cell  confocal  and  MP  imaging  studies  beyond  structural 
aspects  to  study  cell  and  tissue  physiology  (Niggli  and  Egger, 
2004;  Rubart,  2004;  Wang  et  al,  2004).  Calcium  indicators  have 
been  the  most  commonly  used  physiological  probes  because 
calcium  is  a  central  signal  transduction  molecule  and  in  many  cell 
preparations  the  calcium-sensitive  probes  give  robust  signals. 
These  signals  often  are  temporally  resolvable  in  full  field  scans  as 
calcium  transients  that  persist  for  several  seconds.  Fast  scanning 
systems,  or  line-scanning  mode  in  laser  scanning  systems,  have 
been  used  to  resolve  more  rapid  calcium  events  (e.g.,  Fan  et  al., 
1999;  Wang  et  al.,  2004).  Although  non-ratiometric,  visible  wave¬ 
length  calcium  indicators  (e.g.,  fluo-3,  calcium  green)  have  been 
more  widely  used  in  confocal  applications,  some  studies  have 
employed  ultraviolet  (UV)  excited  ratiometric  calcium  indicators, 
such  as  indo-1  (e.g.,  Pasti  et  al,  2001). 

In  addition  to  calcium  indicators,  other  fluorescent  physiolog¬ 
ical  probes  are  useful  for  reporting  various  ions  including  sodium, 
magnesium,  potassium,  and  chloride,  pH,  heavy  metals  such  as 
zinc,  and  membrane  potential,  to  name  a  few  (see  Chapter  42,  this 
volume).  Although  many  of  these  probes  are  small  molecules, 
genetic  (GFP-based)  probes  have  been  developed  (see  Miyawaki, 
2003)  and  are  being  incorporated  into  transgenic  animals  (e.g., 
Hasan  et  al.,  2004).  In  combination  with  state-of-the-art  confocal 
and  MP  imaging  systems,  these  probes  will  increasingly  permit 
detailed  spatio-temporal  analyses  of  physiological  processes  in 
intact  tissues  and  organisms  (Ashworth,  2004). 

Combining  Fluorescence  and  Other 
Imaging  Modalities 

Although  advancements  in  fluorescence  imaging  technology 
coupled  with  the  availability  of  a  multitude  of  vital  fluorescent 
probes  have  combined  to  make  fluorescence  the  method  of  choice 
for  most  high  resolution  studies  of  living  cells,  it  is  sometimes 
advantageous  to  combine  fluorescence  imaging  with  other  imaging 
modalities.  For  example,  differential  interference  contrast  (DIC) 
microscopy  can  be  used  in  conjunction  with  laser-scanning  confo¬ 
cal  microscopy  to  simultaneously  monitor  the  whole  cell  in  DIC 
mode  while  imaging  the  phagocytic  uptake  of  fluorescent  micro¬ 
spheres  (Hook  and  Odeyvale,  1989)  or  the  distribution  of  fluores- 


TABLE  19.1.  Overview  of  Live  Cell  Fluorescence  Confocal  Imagine  Techniques 
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cently  tagged  proteins  and  molecules  (e.g.,  Adams  et  al.,  1998) 
within  these  cells.  Although  it  is  difficult  to  perform  DIC  and 
epi-fluorescence  imaging  both  simultaneously  and  optimally  in 
widefield  microscopy,  it  is  somewhat  easier  to  ensure  that  the 
fluorescence  signal  is  not  subjected  to  the  light  loss  that  occurs  in 
the  analyzer  used  as  part  of  the  DIC  system  if  one  uses  a  single¬ 
beam  confocal.  Thus,  the  DIC  image  can  be  collected  from  a 
fluorescently  labeled  specimen  using  transmitted  light  that  would 
otherwise  be  wasted.  Recently,  differential  phase  contrast  (DPC) 
has  been  implemented  in  a  scanning  laser  microscope  system 
(Amos  et  al. ,  2003),  and  this  may  offer  additional  capabilities  where 
DIC  optics  are  unsuitable.  Notably  for  live-cell  imaging,  DPC 
reportedly  needs  20  times  less  laser  power  at  the  specimen  than  DIC. 


GENERAL  CONSIDERATIONS  FOR  CONFOCAL 
MICROSCOPY  OF  LIVING  CELLS 

What  factors  must  be  considered  when  performing  a  live-cell  con¬ 
focal  imaging  experiment  or  observation?  The  major  factors  are  to 
(1)  label  the  preparation  in  order  to  clearly  visualize  the  biologi¬ 
cal  component  of  interest,  (2)  maintain  the  preparation  in  a  condi¬ 
tion  that  will  support  normal  cell  or  tissue  health,  and  (3)  image 
the  specimen  with  sufficient  spatial  and  temporal  resolution  in  a 
way  that  does  not  perturb  or  compromise  it.  Table  19.2  outlines 
several  of  the  most  important  experimental  considerations  for  live¬ 
cell  imaging,  including  the  most  common  problems  and  some 
potential  solutions. 


TABLE  19.2.  Experimental  Considerations  for  Live  Cell  Imaging 


Consideration 

Problem 

Potential  solution(s) 

1  Temperature 

Many  biological  phenomena  are  temperature  sensitive. 

•  Use  stage  heaters;  inline  perfusion  heaters;  objective  lens  heaters; 
environmental  boxes. 

•  Take  precautions  against  stage  drift:  —  increase  thermal  mass, 

—  use  open-loop  controls 

2  Oxygenation 

Most  live  biological  specimens  require  02  (and  removal 
of  C02)  to  remain  healthy.  Oxygen  may  become 
depleted  in  closed  chambers. 

•  Use  a  perfusion  chamber. 

•  Exchange  used  chamber  media  with  oxygenated  media 
intermittently  or  continuously. 

•  Increase  volume  of  chamber  to  promote  health. 

3  pH 

Metabolism  of  live  biological  tissues  can  induce  severe 
pH  changes  in  chamber  media  over  time. 

•  Monitor  chamber  pH. 

•  Use  HEPES  (10-25mM)-buffered  media. 

•  Exchange  chamber  media  intermittently  or  continuously  (perfusion). 

•  Use  media  without  phenol  red  pH  indicator. 

4  Humidity 

Stage  heating  (especially  with  forced  air)  may  cause 
evaporation  from  an  open  chamber,  leading  to 
dramatic  changes  in  salinity  and  pH. 

•  Use  closed  chamber  configuration  (perfusion  chamber). 

•  Use  humidified  environmental  box. 

•  Use  auto-fill  system  for  open  chambers. 

5  Fluorescence 

Weakly  fluorescent  probes  or  low  concentration  of 

•  Increase  pixel  dwell  time. 

signal  strength 

probes  can  yield  weak  signals  that  produce  images 
with  low  signal-to-noise  ratio. 

•  Open  confocal  pinhole  aperture  (e.g.,  to  >2  Airy  disks). 

•  Maximize  throughput  of  emission  pathway  (e.g.,  in  spectral  imaging 
systems  with  variable  spectral  filters). 

•  Use  line  or  frame  averaging  to  improve  signal-to-noise  ratio. 

•  Adjust  illumination  (filling)  of  back  aperture  of  objective  lens. 

6  Channel  bleed- 

In  biological  specimens  labeled  with  multiple 

•  Image  separate  fluorescence  channels  sequentially  (either  line-by- 

through  or 

fluorescent  probes,  signals  from  one  channel  may  be 

line  or  frame-by-frame  in  scanning  systems). 

cross-talk 

detected  in  other  channels. 

•  Use  spectral  imaging  and  linear  unmixing  algorithms. 

•  Use  modern,  hard-coated  interference  filters  and  dichroics. 

7  Photobleaching 

Fluorescent  probes  bleach  with  repeated  illumination. 

Some  fluorescent  probes  bleach  quickly. 

•  Reduce  incident  illumination.  Then  reduce  it  again! 

•  Use  fade  resistant  dyes. 

•  Open  confocal  pinhole  aperture. 

•  Maximize  throughput  of  emission  pathway  (e.g.,  in  spectral  imaging 
systems  with  variable  spectral  filters). 

•  Reduce  pixel  dwell  time  (in  scanning  systems). 

•  Reduce  frequency  of  image  capture. 

•  Blank  laser  beam  during  flyback  (in  scanning  systems). 

•  Only  scan  specimen  when  actually  collecting  data. 

•  To  improve  S/N,  always  deconvolve  3D  data  before  viewing. 

8  Spatial 

Some  observations  require  very  high  spatial  resolution 

•  Use  high  NA  objectives. 

resolution 

in  x-y  or  z . 

•  Reduce  size  of  confocal  pinhole  aperture  (to  ~1  Airy  disk). 

•  Increase  spatial  sampling  frequency  (guided  by  Nyquist  theorem). 

•  Increase  electronic  zoom  (but  avoid  empty  magnification). 

•  Decrease  step  size  in  z-stacks. 

•  Use  water-immersion  objective  lenses  to  reduce  spherical  aberration. 

•  Deconvolve  the  images. 

9  Temporal  resolution 

Some  biological  phenomena  are  rapid  relative  to  the  rate 
of  image  collection  (especially  problematic  with  laser 
scanning  confocal  systems). 

•  Reduce  field  of  view  (e.g.,  collect  fewer  horizontal  lines). 

•  Reduce  pixel  dwell  time  (e.g.,  increase  scan  speed). 

•  Reduce  spatial  sampling  frequency  (e.g.,  reduce  pixel  array  from 
1024  to  512). 

10  Focus  drift 

Live  biological  specimens  on  heated  microscope  stages, 
or  features  within  live  specimens  (e.g.,  mitotic  cells), 
can  move  relative  to  a  fixed  focal  plane. 

•  Collect  a  z-stack  of  images,  and  reconstruct  these  images  following 
the  observation. 

•  Manual  focus  adjustments  may  be  required  periodically. 

•  Auto-focus  methodology  may  be  employed  in  some  cases. 
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Maintenance  of  Living  Cells  and 
Tissue  Preparations 

In  Vitro  Preparations 

Specimen  maintenance  is  a  very  important  part  of  any  live  imaging 
study  and  usually  requires  both  mechanical  ingenuity  and  insight 
into  the  biology  of  the  cell  or  tissue  under  study.  The  specimen 
chamber  must  keep  the  cells  or  tissues  healthy  and  functioning 
normally  for  the  duration  of  the  experiment  while  allowing  access 
to  the  microscope  objective.  This  can  be  particularly  difficult  when 
high-numerical-aperture  (NA)  oil-  or  water-immersion  lenses  are 
used.  In  many  cases,  there  must  also  be  a  controlled  and  efficient 
way  to  introduce  a  reagent  to  perturb  a  particular  cellular  process. 
Other  important  factors  are  simplicity,  reliability,  and  low  cost.  It  is 
advisable  to  monitor  the  conditions  within  the  imaging  chamber 
carefully.  It  may  be  helpful  to  use  microprobes  that  can  detect  pH, 
02,  and  C02  (e.g.,  Lazar  Research  Laboratories,  Inc.,  Los  Angeles, 
CA). 

The  early  closed  perfusion  chambers  designed  by  Dvorak  and 
Stotler  (1971 )  and  later  by  Vesely  and  colleagues  (1982)  were  inex¬ 
pensive  and  permitted  high-resolution  transmitted  light  observa¬ 
tion.  They  relied  on  an  external  heater  that  warmed  the  entire  stage 
area  for  temperature  control. 

Setups  for  different  cells  vary  widely.  Mammalian  cells  proba¬ 
bly  pose  the  greatest  problems.  McKenna  and  Wang’s  article  (1986) 
is  a  general  introduction  to  the  problems  associated  with  keeping 
such  cells  alive  and  functioning  on  the  microscope  stage.  This  article 
discusses  culture  chamber  design  as  well  as  strategies  for  control¬ 
ling  pH,  osmolarity,  and  temperature.  The  authors  describe  their  own 
chamber,  in  which  temperature  is  controlled  by  heating  the  air  in  a 
box  surrounding  the  stage  area,  and  mention  earlier  designs  such  as 
the  resistively  heated  Lieden  Culture  System  first  described  by  Ince 
and  colleagues  (1983)  and  later  improved  by  Forsythe  (1991). 

Strange  and  Spring  (1986)  describe  their  setup  for  imaging 
renal  tubule  cells  where  temperature,  pH,  and  C02  are  controlled. 
They  provide  a  detailed  account  of  the  problems  of  establishing 
laminar  flow  perfusion  systems,  temperature  regulation,  and  main¬ 
tenance  of  pH  by  C02  buffering.  Somewhat  later,  Delbridge  and 
colleagues  (1990)  describe  a  sophisticated,  open-chamber  super¬ 
fusion  system  permitting  programmed  changes  of  media,  precision 
control  of  media  surface  height,  and  temperature  regulation 
between  4°C  and  70°C  using  a  Peltier  device  to  control  the  per¬ 
fusate  temperature.  Myrdal  and  Foster  (1994)  used  a  temperature- 
stabilized  liquid  passing  through  a  small  coil  suspended  in  media 
filling  a  plastic  Nunc  chamber  to  provide  temperature  control  for 
confocal  observations  of  the  penetration  of  fluorescent  antibodies 
into  solid  tumor  spheroids.  An  automatic  system  maintained  fluid 
level  and  bathed  the  area  in  C02  but  special  precautions  were 
required  to  prevent  drift  of  the  confocal  focus  plane  during  long 
time-lapse  sequences.  Methods  for  observing  microglial  cell 
movements  in  mammalian  brain  slices  are  described  in  detail  in  a 
later  section  of  this  chapter. 

Chambers  have  even  been  built  for  the  microscopic  observa¬ 
tion  of  cells  as  they  are  being  either  frozen  or  thawed  in  the  pres¬ 
ence  of  media  that  could  be  changed  during  the  process  (e.g., 
Walcerz  and  Diller,  1991).  In  this  case,  computer-controlled  pumps 
deliver  temperature-controlled  nitrogen  gas  at  between  -120°C 
and  100°C  to  special  ports  connected  to  a  temperature  cell  (-55°C 
to  60°C)  that  forms  the  upper  boundary  of  the  perfusion  chamber. 
Other  ports  carry  either  the  perfusate  or  a  separate  nucleating  agent 
to  the  cell  chamber  itself. 

More  recently,  a  specialized  in  vitro  cell  culture  system  has 
been  developed  to  maintain  mammalian  neuronal  cells  for  over  a 
year  (Potter  and  DeMarse,  2001)! 


There  are  several  companies  that  provide  ready-made  micro¬ 
scope  stage  chambers,  temperature-control  units,  automated  per¬ 
fusion  systems,  and  a  variety  of  related  accessories.  These  are 
summarized  in  Table  19.3. 

In  Vivo  Preparations 

The  ultimate  goal  of  many  research  programs  is  to  understand  the 
normal  (or  abnormal)  structure  and  function  of  molecules,  cells, 
and  tissues  in  vivo,  that  is,  in  the  living  organism  functioning 
within  its  native  environment  (Frosting,  2002;  Megason  and 
Fraser,  2003).  There  has  been  some  remarkable  progress  recently 
on  extending  high  resolution  confocal  and  MP  imaging  in  this 
direction,  especially  in  preparations  that  are  essentially  translucent. 
Several  model  organisms,  including  zebrafish  (Cooper  et  at, 
1999),  frog  (Fraser  and  O’Rourke,  1990;  Robb  and  Wylie,  1999), 
fruit  fly  (Paddock,  2002),  leech  (Baker  et  al.,  2003),  and  worm 
(Crittenden  and  Kimble,  1999),  have  emerged  as  excellent  prepa¬ 
rations  for  cellular  and  molecular  imaging  studies  spanning  a 
variety  of  biological  questions.  As  an  example,  studies  in  the 
zebrafish  have  been  carried  out  on  the  structural  development  of 
vasculature  (Lawson  and  Weinstein,  2002;  Isogai  et  al.,  2003),  cell 
division  (Gong  et  al.,  2004;  Das  et  al.,  2003),  neuronal  migration 
(Koster  and  Fraser,  2001),  axonal  pathfinding  (Dynes  and  Ngai, 
1998),  synapse  formation  (Jontes  et  al.,  2000;  Niell  et  al.,  2004), 
and  synaptic  plasticity  (Gleason  et  al.,  2003),  to  name  a  few.  Phys¬ 
iological  studies  in  zebrafish  have  included,  for  example,  imaging 
intracellular  calcium  during  gastrulation  (Gilland  et  al.,  1999),  in 
the  intact  spinal  cord  (O’Malley  et  al.,  1996;  Gahtan  et  al.,  2002), 
and  in  brain  (Brustein  et  ah,  2003).  Other  examples  related  to 
studies  of  embryos  are  covered  in  Chapter  43.  Each  of  these  bio¬ 
logical  preparations  embodies  its  own  unique  set  of  specimen 
mounting  and  maintenance  challenges.  Indeed,  it  is  sometimes 
necessary  to  anesthetize  the  preparation  to  prevent  it  from  crawl¬ 
ing  or  swimming  away  during  the  imaging  session! 

Perhaps  the  most  difficult  conditions  involve  imaging  in  a  living 
mammal,  an  undertaking  for  which  the  confocal  or  MP  microscope 
enjoys  the  twin  advantages  of  epi-illumination  and  optical  section¬ 
ing  that  make  it  possible  to  view  solid  tissues  without  mechanical 
disruption.  Confocal  microscopy  has  long  been  an  important  tool 
for  in  vivo  imaging  of  eye  tissues  non-invasively  (Petran  et  al., 
1986;  Jester  et  al.,  1991,  1992;  Masters,  1992;  Petroll  et  al.,  1992, 
1993;  Poole  et  al.,  1993).  In  terms  of  imaging  interior  tissues,  early 
studies  described  methods  for  examining  microcirculation  of  the 
brain  cortex  in  anesthetized  rats  (Dirnagl  et  al.,  1992)  or  changes  in 
kidney  tubules  during  ischemia  (Andrews  et  al.,  1991).  Confocal 
microscopy  also  has  been  used  to  image  leukocyte-endothelium 
interactions  during  infections  through  closed  cranial  windows 
(Lorenzl  et  ah,  1993).  More  recently,  MP  has  been  used  to  image 
live  mammalian  brain  tissues  in  vivo,  either  through  a  cranial 
window  (Svoboda  et  ah,  1997;  Trachtenberg  et  ah,  2002),  fiber¬ 
optic  coupled  devices  (Mehta  et  ah,  2004),  or  directly  through  the 
intact  but  thinned  skull  (Yoder  and  Kleinfeld,  2002;  Zhang  et  al., 
2005).  Dual-channel  MP  imaging  also  has  been  used  to  image  other 
tissues  in  vivo,  including  lymphoid  organs  (e.g..  Miller  et  al.,  2002). 
It  is  generally  accepted  that  MP  imaging  is  superior  to  single-photon 
confocal  for  these  in  vivo  imaging  studies  (Cahalan  et  al.,  2002). 

Fluorescent  Probes 

Except  in  those  cases  where  an  adequate  image  can  be  derived 
from  either  the  backscattered-light  signal  or  from  autofluores¬ 
cence,  confocal  microscopy  of  living  cells  is  dependent  on  the 
properties  and  availability  of  suitable  fluorescent  probes.  In  addi¬ 
tion  to  binding  specifically  to  what  one  is  interested  in  studying. 


Source 


TABLE  19.3.  Commercially  Available  Chambers  for  Live  Cell  Imaging 

Description/Features 


Contact  Info 


20/20  Technology,  Inc. 

Bldg.  2,  Unit  A 
311  Judges  Road 
Wilmington,  NC  28405  USA 
ALA  Scientific  Instruments  Inc. 

1100  Shames  Dr. 

Westbury,  NY  11590  USA 
ASI  /  Applied  Scientific 
Instrumentation  Inc. 

29391  W.  Enid  Rd. 

Eugene,  OR  97402  USA 
AutoMate  Scientific,  Inc. 

336  Baden  Street 

San  Francisco,  California  94131  USA 

Bellco  Glass,  Inc. 

340  Edrudo  Road, 

Vineland,  NJ  08360  USA 
BioCrystal  Ltd 
OptiCell 

575  McCorkle  Blvd. 

Westerville,  OH  43082  USA 
Bioptechs,  Inc. 

3560  Beck  Road 
Butler,  PA  16002  USA 
Bioscience  Tools  —  CB  Consulting 
Inc.,  4527  52nd  Street,  San  Diego,  CA 
92115  USA 

C&L  Instruments,  Inc. 

314  Scout  Lane 
Hummelstown,  PA  17036  USA 
CellBiology  Trading 
Hinsbeker  Berg  28a 
Hamburg,  22399  Germany 
Dagan  Corporation 
2855  Park  Avenue, 

Minneapolis,  Minnesota  55407  USA 
Digitimer  Ltd 
37  Hydeway 
Welwyn  Garden  City 
Hertfordshire,  AL7  3BE,  England 
Grace  Bio-Labs,  Inc. 

P.O.  Box  228 
Bend,  OR  97709  USA 


Harvard  Apparatus  Inc. 

84  October  Hill  Rd. 

Holliston,  MA  01746  USA 
Integrated  BioDiagnostics 
Schellingstrasse  4 
80799  Miinchen,  Germany 
Intracel,  Ltd. 

Unit  4  Station  Road 

Shepreth,  Royston 

Herts,  SG8  6PZ  England 

In  Vitro  Systems  &  Services  GmbH 

Rudolf- Wissell-Str.  28 

37079  Gottingen,  Germany 

Life  Imaging  Services 

Kaegenstrasse  17 

CH-4153  Reinach,  Switzerland 

MatTek  Corporation 

200  Homer  Avenue 

Ashland,  MA  01721  USA 

Molecular  Probes,  Inc. 

2985 1  Willow  Creek  Road 
Eugene,  OR  97402  USA 


Heating,  cooling,  atmosphere  control  instrumentation 
for  microscopy. 


Microincubators  and  temperature  control;  Peltier  heating  & 
cooling  pre-stage;  recording  chambers;  inline  perfusion 
heating  tube. 

Supplier  for  Solent  and  Bioptechs  incubation  chambers. 


Programmable  controlled  perfusion  systems,  temperature 
control,  valves  and  fittings,  oocyte  perfusion  chamber,  Petri 
dish  perfusion  chamber,  sub-millisecond  switching, 
submerged  and  interface  tissue  and  brain  slice  chambers. 

Sykes-Moore  culture  chambers;  used  with  stationary  culture 
when  medium  is  changed  intermittently. 

OptiCell  is  a  sterile,  sealed  cell  culture  environment 

between  two  optically  clear  gas-permeable  growth  surfaces  in 
a  standard  microtiter  plate- sized  plastic  frame  with  ports  for 
access  to  the  contents. 

Live  cell  microscopy  environmental  control  systems. 

Thermal  regulation  of  specimen  and  objective,  electronic 
control  and  integration  of  temperature  and  perfusion. 

Glass  bottom  Petri  dishes;  ultra-thin  imaging  chambers; 
temperature  control;  perfusion  systems;  small  volume 
delivery  systems;  ultra-fast  temperature/solution  switch. 

Fluorometers  and  fluorometer  components  for  steady-state 
fluorescence  measurements;  complete  fluorescence  systems 
for  photometry  and  fluorescence  imaging. 

Microinjection  and  incubation;  EMBL  live  cell  observation 
chamber. 

Microscope  stage  temperature  controller;  perfusion  controller. 


AutoMate  Scientific,  Medical  Systems  and  Scientific  Systems 
Design  incubators,  chambers,  and  perfusion  systems. 


Manufactures  3-D  microporous  coatings  on  microscope 
slides,  and  a  variety  of  “press  to  seal”  enclosures  for 
microarrays,  cell  culture,  and  high  throughput  cytochemistry, 
hybridization,  cytogenetics,  and  fluorescent  imaging 
applications. 

Variety  of  valve  controlled  perfusion  systems. 


The  p-slide  family  of  live  cell  imaging  flow  chambers;  suited 
for  optical  studies  of  hydrodynamic  shear  stress  on  biofilms 
or  adhesion  studies  on  cell  layers. 

WillCo  glass  bottomed  dishes;  Bioptechs  micro-environmental 
control  systems. 


Gas-permeable  plastic  foil  (bioFOLIE  25);  sterile  tissue 
culture  dish  (petriPERM);  Petri  dish  with  gas-permeable 
base;  two-compartment  system. 

Ludin  imaging  chamber;  microscope  temperature  control 
system 

Glass  bottom  culture  dishes 


Attofluor  cell  chamber  designed  for  viewing  live-cell 
specimens  on  upright  or  inverted  microscopes.  Chamber 
gaskets  for  imaging,  perfusion,  and  incubation. 


TEL:  1-910-791-9226 
WEB:  http://20-20tech.com/ 


TEL:  516-997-5780 
WEB:  www.alascience.com 
EMAIL:  sales@alascience.com 
TEL:  541-461-8181 
WEB:  http://www.asiimaging.com/ 
EMAIL:  info@ASIimaging.com 

TEL:  415-239-6080 

WEB:  http://www.autom8.com/ 

EMAIL:  info@autom8.com 

TEL:  1-800-257-7043 

WEB:  http://www.bellcoglass.com/ 

EMAIL:  cservice@bellcoglass.com 

TEL:  614-818-0019 

WEB:  http://www.opticell.com 

EMAIL:  sales@opticell.com 

TEL:  724-282-7145 

WEB:  http://www.bioptechs.com/ 

EMAIL:  info@bioptechs.com 

TEL:  1-877-853-9755 

WEB:  http://biosciencetools.com/ 

EMAIL:  info@biosciencetools.com 

TEL:  1-717-564-9491 

WEB:  http://www.fluorescence.com/ 

EMAIL:  support@fluorescence.com 

TEL:  49-0-40-53889432 

WEB :  http://cellbiology-trading.com/ 

EMAIL:  info@cellbiology-trading.com 

TEL:  612-827-5959 

WEB:  http://www.dagan.com/ 

EMAIL:  support@dagan.com 
TEL:  +44  (0)  1707  328347 
WEB:  http://www.digitimer.com/ 
EMAIL:  sales@digitimer.com 

TEL:  1-800-813-7339 

WEB:  http://www.gracebio.com/ 

EMAIL:  custservice@gracebio.com 


TEL:  508-893-8999 

WEB :  http://www.harvardapparatus.com 
EMAIL:  bioscience@harvardapparatus.com 
TEL:  +49  (0)89  /  2180  64  19 
WEB:  http://www.ibidi.de/ 

EMAIL:  info@ibidi.de 

TEL:  01763  262680 

WEB:  http://www.intracel.co.uk/ 

EMAIL:  intracel@intracel.co.uk 

TEL:  ++49  551  500  97-0 
WEB:  http://www.ivss.de/ 

EMAIL:  info@ivss.de 
TEL:  ++41  (0)61  7116461 
WEB:  http://www.lis.ch/ 

EMAIL:  info@lis.ch 
TEL:  1-800-634-9018 

WEB :  http://www.glass-bottom-dishes.com/ 
EMAIL:  DishInfo@mattek.com 
TEL:  1-541-465-8300 
WEB:  http://www.probes.com/ 

EMAIL:  tech@probes.com 


Confocal  Microscopy  of  Living  Cells  •  Chapter  19  389 


Source 

PeCon  GmbH 

ZiegeleistraBe  50 
89155  Erbach 
Germany 

Physitemp  Instruments,  Inc. 

154  Huron  Avenue 
Clifton,  New  Jersey  07013  USA 
SDR  Clinical  Technology 
213  Eastern  Valley  Way 
Middle  Cove,  NSW  2068  Australia 
Solent  Scientific  Limited 
14  Matrix  Park,  Talbot  Road, 
Segensworth  P015  5AP,  UK 
Stratech  Scientific,  Ltd. 

Unit  4  Northfield  Business  Park, 
Northfield  Road, 

Soham,  Cambridgshire  CB7  5UE  UK 
Warner  Instruments,  Inc. 

1125  Dixwell  Avenue, 

Hamden,  CT  06514,  USA 
WillCo  Wells  BV 
WG  Plein  287 

1054  SE  Amsterdam,  The  Netherlands 
World  Precision  Instruments,  Inc. 

175  Sarasota  Center  Boulevard 
Sarasota,  Florida  34240  USA 


TABLE  19.3.  ( Continued) 

Description/Features 

Live  cell  imaging  solutions  including  stage  heating  and 
cooling,  C02  and  02  regulation,  and  evaporation  reduction. 


Heating  &  cooling  stages  (Peltier)  for  microscopes  (—20°  to 
+100°C);  custom  thermal  stages. 

Physiological  recording  chambers  &  accessories  for  use  on 
the  microscope  stage;  bath  perfusion;  temperature  control. 

Manufacturers  of  full  enclosure  incubation  chambers  for 
research  inverted  microscopes,  confocal  microscopes  and 
multi-photon  microscopes. 

CoverWell  imaging  chambers  are  designed  to  stabilize  and 
support  thick  and  free-floating  specimens  for  confocal 
microscopy  and  imaging  applications. 

Full  range  of  recording,  imaging,  and  perfusion  chambers; 
perfusion  and  valve  control. 

WillCo-dish  glass  bottom  dishes. 


FluoroDish  glass-bottom  culture  dish;  Air-Therm  ATX  Air 
Heater  Controller;  programmable  automated  multi-channel 
perfusion  system. 


Contact  Info 

TEL:  0049  (0)  7305  95666-0 

WEB :  http://www.pe-con.de/pecon/index.htm 

EMAIL:  info@pecon.biz 

TEL:  1-973-779-5577 

WEB:  http://www.physitemp.com/ 

EMAIL:  physitemp@aol.com 
TEL:  +61-2-9958-2688 
WEB:  http://www.sdr.com.au/ 

EMAIL:  sdr@sdr.com.au 
TEL:  +44  (0)870  774  7140 
WEB:  http://www.solentsci.com/ 

EMAIL:  sales@solentsci.com 
TEL:  +44  (0)1353  722500 
WEB:  http://www.stratech.co.uk/ 


TEL:  1-800-599-4203 

WEB:  http://www.wameronline.com/ 

EMAIL:  support@warneronline.com 

TEL:  ++31(0)20  685  0171 

WEB:  http://www.willcowells.com 

EMAIL:  info@willcowells.com 

TEL:  941-371-1003 

WEB:  http://www.wpiinc.com/ 


the  fluorescent  probe  should  produce  a  strong  signal  and  be  both 
slow  to  bleach  and  non- toxic.  Chapters  16  and  17  discuss  fluores¬ 
cent  dyes  that  have  been  used  in  published  work  with  confocal 
microscopy  in  detail. 

Many  dyes  are  useful  when  introduced  to  the  medium  sur¬ 
rounding  cells  to  be  labeled.  Some  of  the  classic  and  most  com¬ 
monly  used  cell  stains  include  Dil  for  labeling  the  plasma 
membrane  (Honig  and  Hume,  1986;  Baker  and  Reese,  1993), 
DiOC6(3)  for  labeling  internal  membranes  (Terasaki  et  al.,  1984), 
NBD-ceramide  and  bodipy-ceramide  which  label  the  Golgi  appa¬ 
ratus  (Pagano  et  al.,  1991),  rhodamine  123  which  labels  mito¬ 
chondria  (Johnson  et  al.,  1980),  potential  sensitive  dyes  such  as 
DISBAC2(3)  (see  Fig.  8.65,  this  volume)  (Loew,  1993),  and  FM 
1-43  (Betz  et  al.,  1992)  which  is  used  to  follow  plasma  membrane 
turnover  and  vesicular  release.  Fluorescent  ion  indicators  such  as 
Fluo-3  (Minta  et  al.,  1989)  can  either  be  microinjected  or  added 
to  the  media  in  a  cell-permeant  acetoxymethylester  form  that 
becomes  trapped  inside  the  cell  after  being  cleaved  by  intracellu¬ 
lar  esterases  (see  Chapter  42,  this  volume). 

Minimizing  Photodynamic  Damage 

Once  the  cells  are  labeled  and  on  the  microscope,  one  is  faced  with 
the  challenge  of  collecting  data  without  compromising  the  cell  or 
bleaching  the  label.  In  practice,  the  major  problem  is  light-induced 
damage.  Fluorescent  molecules  in  their  excited  state  react  with 
molecular  oxygen  to  produce  free  radicals  that  can  then  damage 
cellular  components  and  compromise  cell  health  (Dixit  and  Cyr, 
2003;  see  also  Chapters  38  and  39,  this  volume). 

In  addition,  several  studies  suggest  that  components  of  stan¬ 
dard  culture  media  might  also  contribute  to  adverse  light-induced 
effects  on  cultured  cells  (see  Siegel  and  Pritchett,  2000).  Some 
early  studies  (Spierenburg  et  al.,  1984;  Zigler  et  al.,  1985,  1991; 
Lepe-Zuniga  et  al.,  1987)  indicated  a  phototoxic  effect  of  N- 2- 


hydroxyethylpiperazine-A^-2-ethanesulfonic  acid  (HEPES)  con¬ 
taining  media  on  cells  under  some  circumstances.  It  seems  possi¬ 
ble  that  this  effect  might  be  more  directly  related  to  inadequate 
levels  of  bicarbonate  (Cechin  et  al.,  2002).  Other  studies  suggest 
that  riboflavin/vitamin  B2  (Zigler  et  al. ,  1985;  Lucius  et  al.,  1998) 
and  the  essential  amino  acid  tryptophan  (Griffin  et  al.,  1981;  Silva 
et  al.,  1991;  Silva  and  Godoy,  1994;  Edwards  et  al.,  1994)  may 
mediate  phototoxic  effects.  Whether  these  effects  occur  under 
typical  confocal  imaging  conditions  is  unknown,  but  many  of  the 
photoeffects  are  reduced  by  antioxidants,  so  it  seems  advisable  to 
maintain  antioxidants  (and  some  bicarbonate,  as  well)  in  the  spec¬ 
imen  chamber  (see  below)  and  to  use  photons  with  great  efficiency. 

Improving  Photon  Efficiency 

There  are  several  strategies  to  minimize  the  amount  of  excitation 
light  required  to  collect  data  (see  Chapters  2  and  9,  this  volume, 
for  more  details).  Briefly,  higher-NA  objective  lenses  collect  more 
of  the  fluorescent  emission.  For  a  given  lens,  there  is  also  a  theo¬ 
retical  optimal  setting  of  the  zoom  magnification  that  best  matches 
the  resolution  required  to  the  allowable  dose  (see  Chapter  4,  this 
volume).  When  the  focus  plane  is  more  than  5  pm  from  the 
coverslip,  water-immersion  lenses  should  be  used  to  avoid  the 
signal  loss  caused  by  spherical  aberration  when  using  an  oil  lens 
(see  Chapters  7  and  20,  this  volume ). 

Another  way  to  reduce  light  damage  is  to  minimize  the  dura¬ 
tion  of  the  light  exposure  during  the  experimental  setup.  For 
example,  one  should  try  to  focus  as  rapidly  as  possible  and  turn 
off  the  light  source  as  soon  as  the  focus  range  has  been  chosen.  In 
addition,  in  single-beam  scanning  systems,  make  sure  that  your 
scanner  is  set  up  to  blank  the  laser  beam  during  scan  retrace.  Oth¬ 
erwise,  areas  on  both  sides  of  the  imaged  area  will  receive  a  very 
high  light  exposure  as  the  beam  slows  down  to  change  direction. 

Finally,  photon  efficiency  can  be  maximized  by  using  the  best 
mirrors,  the  correct  pinhole  size  for  the  resolution  required  (in  x. 
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y,  and  -;),  and  photodetectors  that  yield  the  highest  quantum  effi¬ 
ciency  at  the  wavelength  of  the  signal. 

Antioxidants 

As  noted  above,  one  can  also  reduce  photodynamic  damage  by 
adding  antioxidants  to  the  medium.  Oxyrase  (Oxyrase  Inc., 
http://www.oxyrase.com)  is  an  enzyme  additive  used  to  deplete 
oxygen  in  order  to  grow  anaerobic  bacteria.  It  has  been  used  at  0.3 
unit/mL  to  reduce  photodynamic  damage  during  observations  of 
mitosis  (Waterman- Storer  et  al.,  1993).  Another  approach  is  to 
include  ascorbic  acid  in  the  medium.  This  reducing  agent  is  typi¬ 
cally  used  at  0. 1  to  1.0  mg/mL  but  has  been  used  at  up  to  3  mg/mL. 
A  recent  confocal  study  of  calcium  transients  in  isolated  chon¬ 
drocytes  reported  a  relationship  between  laser  intensity  and  the  fre¬ 
quency  of  Ca2+  oscillations  and  cell  viability:  Ca2+  events  were 
more  frequent  and  cell  viability  was  decreased  with  higher  laser 
intensity  (Knight  et  al.,  2003).  Treatment  with  ascorbic  acid 
reduced  the  Ca2+  events  and  improved  cell  viability  (see  also 
Chapters  16  and  17,  this  volume ). 

THE  ONLINE  CONFOCAL  COMMUNITY 

Confocal  microscopy  of  living  cells  is  an  area  of  active  research 
where  individuals  are  constantly  developing  new  techniques  and 
approaches.  One  way  to  keep  up  with  current  practice  is  to  join 
about  1600  others  who  subscribe  to  the  Confocal  e-mail  listserver. 
This  can  be  done  by  registering  at  the  listserver  Web  site,  located 
at  http://listserv.buffalo.edu/archives/confocal.html.  You  will  then 
begin  to  receive  messages  from  other  microscopists.  Recent  topics 
have  included  discussions  on  such  diverse  issues  as  autofluores¬ 
cence  problems,  glass-bottomed  culture  chambers,  damage  to  live 
cells  during  FRAP  experiments,  and  announcements  of  confocal 
workshops.  The  listserver  also  has  an  extensive,  searchable  archive 
dating  back  to  1991,  and  this  is  freely  accessible. 

A  CONVENIENT  TEST  SPECIMEN 

Rnebel  et  al.  (1990)  showed  that  onion  epithelium  ( Allium  cepa) 
is  a  simple  preparation  that  can  be  used  as  a  convenient  test  spec¬ 
imen  for  confocal  microscopy  of  living  cells.  Figure  19.1  shows 


FIGURE  19.1.  A  convenient  test  specimen  for  confocal  microscopy  of  cells 
in  a  living  tissue:  onion  epithelium.  This  drawing  shows  how  to  obtain  onion 
epithelium.  As  described  in  the  text,  the  epithelium  is  stained  with  DiOC6(3), 
and  the  ER  and  mitochondria  within  the  cells  provide  a  bright  and  motile 
specimen. 


how  to  prepare  onion  epithelium.  First,  a  small  square  of  a  layer 
is  cut  out  using  a  razor  blade.  A  forceps  is  used  to  peel  off  the  thin 
epithelium  on  the  inner  surface  of  the  onion  layer.  The  epithelium 
is  then  put  onto  a  microscope  slide,  covered  with  a  drop  or  two  of 
staining  solution  containing  DiOC6(3),  a  marker  of  mitochondria 
and  endoplasmic  reticulum,  and  coverslipped.  The  stock  solution 
of  DiOC6(3)  (0.5  mg/mL  in  ethanol)  can  be  kept  indefinitely  if  pro¬ 
tected  from  light  in  a  scintillation  vial.  The  staining  solution  is  a 
1 : 1000  dilution  in  water  on  the  day  of  the  experiment.  The  center 
of  these  cells  is  usually  occupied  by  a  large  vacuole,  and  the  ER 
and  mitochondria  are  located  in  a  thin  cytoplasmic  region  near  the 
plasma  membrane.  Motion  of  the  ER  is  relatively  quick  and  easily 
detected  in  consecutive  1-s  scans. 


SPECIFIC  EXAMPLE  I:  VISUALIZING 
CHROMATIN  DYNAMICS  USING  VERY  LOW 
LIGHT  LEVELS 

It  is  clear  from  the  discussion  above  that  microscopy  of  living  cells 
has  become  a  technique  of  major  importance  in  cell  biology:  it 
can  be  used  to  tell  us  where  molecules  are  located,  when  they 
become  localized,  how  fast  they  are  moving,  with  which  molecules 
they  are  interacting,  and  how  long  they  stay  attached  to  these  mol¬ 
ecules.  All  these  properties  can  be  observed  in  the  natural  envi¬ 
ronment  of  the  living  cell.  The  major  limiting  factor  in  live-cell 
imaging  is  phototoxic  effect  of  light  used  for  the  observation  of 
the  cell.  Here  we  will  address  some  practical  issues  of  phototoxic¬ 
ity  based  on  our  experience  in  imaging  chromatin  dynamics  in 
living  cells  (Manders  et  al.,  1996;  1999;  2003;  Verschure  et  al., 
1999;  Mone  et  al.,  2004). 

Phototoxicity 

A  large  number  of  photochemical  reactions  are  responsible  for  the 
phototoxic  effect  of  light.  Light  can  be  absorbed  by  cellular  com¬ 
ponents  and  induce  chemical  alterations  in  their  molecular  struc¬ 
ture.  For  example,  UV  light  is  absorbed  by  DNA  (absorption  peak 
at  280  nm),  directly  inducing  DNA  damage.  Here  we  assume  that, 
working  with  visible  light,  the  direct  photodamage  is  negligible. 
In  fluorescently  labeled  cells,  the  main  source  of  photodamage  is 
the  production  of  reactive  oxygen  species  (ROS)  including  singlet 
oxygen  ('CL),  superoxide  (-CL-),  hydroxyl  radical  (HO),  and 
various  peroxides.  These  activated  oxygen  species  react  with  a 
large  variety  of  easily  oxidizable  cellular  components,  such  as 
proteins,  nucleic  acids,  and  membrane  lipids.  Singlet  oxygen  is 
responsible  for  much  of  the  physiological  damage  caused  by 
reactive  oxygen  species.  For  the  production  of  singlet  oxygen,  the 
fluorescent  label  acts  as  a  photosynthesizer  in  a  photochemical 
reaction  where  dioxygen  (302)  converts  into  singlet  oxygen  ('02). 
Singlet  oxygen  mainly  modifies  nucleic  acid  through  the  selec¬ 
tive  oxidation  of  deoxyguanosine  into  8-oxo-7,8-dihydro-2'- 
deoxyguanonine.  Proteins  and  lipids  also  will  be  damaged  by  ROS. 
Phototoxicity  likely  depends  on  several  variables: 

•  The  photochemical  properties  of  the  fluorescent  molecule. 

Some  molecules  induce  more  phototoxicity  than  others, 
depending  on  the  lifetime  of  their  triplet  state.  For  photo¬ 
dynamic  therapy  (PDT),  dedicated  molecules  called  photosen¬ 
sitizers  have  been  designed  in  order  to  induce  a  maximum 
damage  in  tissue  for  the  treatment  of  cancer  (e.g.,  halogenated 
fluorescein  is  much  more  toxic  than  fluorescein).  Another 
property  that  influences  the  phototoxicity  of  a  molecule  is  the 
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local  environment  of  the  molecule.  The  active  fluorophore  of 
a  GFP  molecule  is  positioned  on  the  inside  of  the  protein, 
within  the  barrel  structure  (the  “(3-can”).  Probably  this 
hydrophobic  protein  environment  contributes  to  the  relatively 
low  phototoxicity  of  GFP  compared  with  naked  fluorophores 
such  as  fluorescein  or  rhodamine. 

•  The  subcellular  location  of  the  fluorescent  molecule.  When 
fluorescent  molecules  are  situated  close  to  DNA,  the  damag¬ 
ing  effect  of  singlet  oxygen  is  more  pronounced.  Despite 
several  DNA-repair  mechanisms,  the  cell  will  not  continue  its 
cell  cycle  (arrest)  and  may  even  die  if  there  is  too  much  DNA 
damage.  Therefore,  fluorophores  in  the  cytoplasm  seem  to 
induce  less  phototoxicity  than  fluorophores  in  the  nucleus. 

•  The  concentration  of  fluorophore.  It  is  clear  that  there  is  a 
relationship  between  the  local  concentration  of  fluorophore 
and  the  level  of  phototoxicity.  We  assume  a  linear  relationship 
between  fluorophore  concentration  and  toxicity,  although  this 
has  not  been  assessed  directly  and  is  complicated  by  the  fact 
that  if  there  is  more  dye,  one  need  use  less  excitation. 

•  The  excitation  intensity.  Fluorescent  cells  in  a  dark  incuba¬ 
tor  are  quite  happy  for  weeks  as  long  they  are  maintained  in 
the  dark.  As  the  word  phototoxicity  implies,  photons  are 
needed  to  induce  toxicity  in  a  fluorescently  labeled  specimen. 
We  usually  assume  a  linear  relationship  between  excitation 
light  dose  and  toxicity,  although  the  temporal  regimen  of  the 
excitation  may  be  important  to  how  cells  handle  the  accumu¬ 
lation  of  phototoxic  biproducts.  Phototoxicity  is  dependent  on 
the  wavelength  of  light  in  the  sense  that  the  wavelength  of  the 
toxic  excitation  light  matches  the  excitation  curve  of  the  fluo¬ 
rophore.  In  other  words,  it  is  the  excited  fluorophore  that  is 
toxic.  Koenig  also  found  that,  with  two-photon  excitation,  the 
damage  is  proportional  to  the  number  of  molecular  excitations 
(see  Chapter  38,  this  volume). 

There  is  no  clear  evidence  for  differences  in  phototoxicity 
between  green,  red,  or  far-red  fluorophores.  In  principle,  excited 
Cy5  can  be  as  toxic  as  excited  FITC.  However,  the  wavelength  of 
excitation  light  can  be  a  factor  when  imaging  in  thick  specimen 
because  stronger  incident  illumination  is  needed  for  comparable 
excitation  of  shorter  wavelength  fluorophores  due  to  increased 
tissue  scatter  at  shorter  wavelengths. 

Reduction  of  Phototoxicity 

For  many  researchers,  phototoxicity  is  a  serious  (and  annoying!) 
limitation  of  their  observations  of  living  cells.  When  you  do  not 
look  at  a  cell  it  is  alive,  but  the  moment  you  start  to  observe  how 
it  lives,  it  is  killed  by  the  light  used  to  observe  it.  In  experiments 
so  far,  we  have  succeeded  in  obtaining  acceptable  time  series  of 
living  cells  by  carefully  optimizing  all  steps  in  the  imaging  process 
in  an  effort  to  achieve  (i)  maximal  signal-to-noise  ratio  (S/N),  (ii) 
maximal  spatial  and  temporal  resolution,  and  (iii)  minimal  photo¬ 
toxic  effects.  Specifically,  phototoxicity  has  been  minimized  by  (i) 
using  radical  scavengers  (e.g.,  trolox)  in  the  culture  medium  and 
(ii)  using  culture  medium  without  phenol-red.  Most  important  of 
all.  however,  is  minimizing  the  total  excitation  light  dose.  The 
excitation  light  dose  is  the  product  of  the  light  intensity  and 
the  exposure  time.  Decreasing  either  the  excitation  intensity  or  the 
excitation  dose  implies  a  loss  of  fluorescent  signal.  It  is  inevitable 
that  a  reduction  of  light  dose  puts  a  limitation  on  the  S/N  and  the 
spatial  and  temporal  resolution.  The  art  of  successful  live-cell 
microscopy  is  finding  the  balance  between  image  quality  and  cell 
vitality. 


Improving  Image  Quality  in  Low-Dose 
Microscopy 

Figure  19.2(A)  shows  a  single  time  frame  projection  from  a  3D 
time-series  of  a  HeLa  cell  expressing  the  fluorescent  histone  fusion 
protein,  H2B-GFP.  In  the  time  series  shown  in  Figure  19.2(B),  the 
cell  is  in  late  telophase  at  the  start  of  the  imaging  and  proceeds 
into  interphase  during  the  movie.  This  movie  shows  data  from  a 
study  on  the  dynamics  of  chromatin  during  decondensation 
(Manders  et  ah,  2003 ).'  In  these  experiments,  the  excitation  light 
intensity  was  kept  below  150nW  and  the  total  exposure  time2  of 
a  cell  that  was  3D  imaged  for  3h  was  not  more  than  70s.  Under 
these  conditions  the  total  light  dose  was  approximately  10 Jem-2. 
In  experiments  where  we  used  a  higher  dose  of  light  we  observed 
phototoxic  effects,  such  as  cell  cycle  arrest  and  cell  death. 

Reducing  the  total  light  dose  during  an  experiment  requires 
that  the  number  of  3D  images  in  the  sequence  (temporal  sampling 
rate)  be  low.  Because  of  this  limited  sampling  rate,  live-cell  movies 
are  usually  under-sampled  in  time  according  to  the  Nyquist  crite¬ 
rion.  As  a  result,  such  movies  often  show  cells  that  nervously  move 
from  one  place  to  another  and  sometimes  suddenly  rotate.  We  have 
applied  an  image  processing  procedure  to  correct  for  all  the  move¬ 
ments  (translation  and  rotation)  of  the  cell.  For  each  3D  image  of 
the  time  sequence,  a  translation  and  rotation  transform  vector  was 
calculated  in  order  to  obtain  a  best  fit  with  the  previous  image  in 
the  sequence.  After  a  series  of  such  transformations,  a  new  movie 
was  produced  showing  a  stable  cell  that  does  not  move  or  rotate. 
Only  internal  movements  are  visible.  After  this  correction  proce¬ 
dure,  we  applied  a  simple  Gaussian  spatial  filter  to  reduce  noise  in 
the  image  [Fig.  19.2(C,D)].  We  also  applied  a  temporal  filter  by 
adding  to  each  voxel  of  the  3D  image  at  each  time-point  the  value 
for  that  voxel  in  the  previous  and  subsequent  image  multiplied  by 
an  intensity  factor  of  0.5.  Our  experience  is  that  temporal  filtering 
makes  the  movie  easier  to  interpret. 

Low-Dose  Imaging  Conclusion 

The  success  of  live-cell  microscopy  is  very  much  dependent  on 
minimizing  or  avoiding  any  toxic  effect  of  light  on  the  biological 
system  under  observation.  A  certain  dose  of  light  may  induce 
serious  DNA  damage  that  may  arrest  the  cell  cycle,  whereas  the 
diffusion  coefficient  of  a  certain  protein  is  not  influenced  at  all  at 
the  same  dose.  In  the  experimental  example  shown  here  [Fig. 
19.2(B)],  we  used  only  150nW  of  incident  beam  power.  This  dose 
was  found  to  be  phototoxic  in  other  experiments  using  fluorescein 
instead  of  GFP,  and  it  was  found  necessary  to  drop  the  laser  power 
to  50nW  [Fig.  19.2(E)].  These  power  levels  are  far  lower  than  (i.e., 
<1%  of)  those  commonly  used  in  confocal  microscopy,  a  circum¬ 
stance  facilitated  at  least  in  part  by  the  fact  that  the  chromosomes 
are  quite  heavily  stained. 

Our  collective  experiences  indicate  that  the  effect  of  photo¬ 
toxicity  depends  on  the  cell  type,  the  stage  of  the  cell  cycle,  the 
fluorophore,  the  observed  biological  process,  and  many  other 
experimental  conditions.  We  conclude  that  there  is  no  general 
guideline  for  the  maximum  allowable  laser  power:  it  must  be 
assessed  empirically  for  each  experimental  condition.  As  a  general 


1  Both  movies  are  on  the  Springer  Web  site  associated  with  this  book. 

2  Power  levels  were  measured  using  a  photometer  sensor  that  was  oil-coupled 
to  the  specimen  side  of  an  oil-coupled  coverslip. 
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FIGURE  19.2.  (A)  A  single  time  frame 
from  a  time-series  of  a  HeLa  cell  expressing 
the  fluorescent  histone  fusion  protein,  H2B- 
GFP.  The  cell  is  in  late  telophase.  (B)  A 
time-series  of  another  cell  shows  how  it  pro¬ 
ceeds  into  interphase  during  chromatin 
decondensation.  In  this  experiment,  the 
excitation  light  intensity  was  kept  below 
150nW.  Higher  laser  power  induced  cell 
cycle  arrest.  (C,  D)  Application  of  a  simple 
Gaussian  spatial  filter  reduces  noise  in  the 
raw  image  (C)  and  improves  image  quality 
(D).  (E)  Time-series  showing  chromatin 
structure  in  a  cell  proceeding  from  G2  (left) 
to  prophase  (right).  Note  that  the  images 
were  captured  with  a  very  low  power  dose 
(50  nW).  See  the  supplemental  video  movie 
at  http://www.springer.com/0-387-25921-X. 
For  details,  see  Manders  and  colleagues 
(2003). 


rule,  however,  images  of  living  cells  are  almost  always  more  noisy 
than  images  of  fixed  preparations  because  the  incident  illumina¬ 
tion  intensity  needs  to  be  kept  to  a  minimum  to  maintain  cell  via¬ 
bility.  A  noisy  image  in  which  one  can  see  what  is  absolutely 
essential  is  of  more  use  than  a  “better"’  image  of  a  damaged  cell. 
Keep  in  mind  that  not  all  types  of  damage  are  equally  easy  to 
detect.  Damage  may  disturb  (or  create!)  a  monitored  process,  it 
may  interfere  with  cell  division,  or  it  may  cause  the  cell  to  bleb 
and  pop!  We  recommend  that  the  experimentalist  BEGIN  by 
assuming  that  “to  observe  is  to  disturb.”  Measure  the  power  level 
coming  out  of  the  objective.  Try  your  experiment  again  using  twice 
the  power,  and  again  using  half  the  power.  Make  sure  that  you  can 
explain  any  “behavioral”  differences  between  these  runs. 

We  show  here  that  some  simple  image  processing  (deconvo¬ 
lution)  can  help  facilitate  the  analysis  of  live-cell  imaging  data  by 
reducing  noise  and  improving  the  point-to-point  coherence  when 
viewing  a  time-series  of  images.  Note  the  reduction  in  noise  level 
between  Figure  19.2(C)  and  19.2(D). 


SPECIFIC  EXAMPLE  II:  MULTI-DIMENSIONAL 
IMAGING  OF  MICROGLIAL  CELL  BEHAVIORS 
IN  LIVE  RODENT  BRAIN  SLICES 

The  example  above  serves  to  illustrate  that  confocal  microscopy 
is  an  important  tool  for  studying  dynamic  subcellular  processes  in 
live,  isolated  cells.  Many  biologists  are  also  interested  in  under¬ 
standing  dynamic  structural  and  functional  aspects  of  cells  within 


the  context  of  a  natural  tissue  environment.  As  noted  above,  con- 
focal  and  multi-photon  imaging  have  been  applied  to  intact,  nor¬ 
mally  functioning  systems  such  as  the  eye,  skin,  or  kidney.  Some 
recent  studies  have  even  extended  these  observations  beyond 
superficial  tissues  to  deep  tissues  of  the  brain  (e.g.,  Mizrahi  et  al., 

2004) . 

However,  some  tissues  are  much  less  accessible,  or  it  may  be 
of  interest  to  be  able  to  experimentally  perturb  or  control  the 
system  under  study.  For  these  purposes,  the  in  vitro  tissue  slice  has 
been  an  important  experimental  preparation  (e.g.,  Gahwiler  et  al., 
1997).  Smith  and  colleagues  (1990)  were  among  the  first  to  show 
the  feasibility  of  imaging  the  structure  and  physiology  of  living 
mammalian  brain  tissue  slices  at  high  resolution  using  fluores¬ 
cence  confocal  microscopy.  Since  then  many  confocal  studies  of 
both  the  structural  and  the  physiological  dynamics  of  cells  in  tissue 
slices  have  appeared,  and  it  seems  that  interest  in  imaging  in  vitro 
tissues  is  continuing  to  grow.  We  will  address  here  some  of  the 
most  common  problems,  challenges,  and  limitations  inherent  in 
confocal  studies  of  live  tissue  slices.  These  points  will  be  illus¬ 
trated  by  drawing  from  our  own  time-lapse  studies  in  live  tissue 
slices  of  developing  mammalian  central  nervous  system  (CNS) 
(Dailey  and  Smith,  1993,  1994,  1996;  Dailey  et  al.,  1994;  Dailey 
and  Waite,  1999;  Marrs  et  al,  2001;  Stence  et  al,  2001;  Gross- 
mann  et  al.,  2002;  Petersen  and  Dailey,  2004;  Benediktsson  et  al., 

2005) . 

Some  of  the  major  problems  encountered  when  imaging 
fluorescently  labeled  cells  in  live  tissue  slices  are: 

•  Attaining  a  suitable  level  and  specificity  of  staining. 

•  Maintaining  cell/tissue  health:  pH,  temperature,  oxygen,  etc. 


Confocal  Microscopy  of  Living  Cells  •  Chapter  19  393 


•  Keeping  cells  in  focus:  Can  be  an  immense  problem  when  fol¬ 
lowing  cells  over  long  periods  of  time: 

•  Movement  of  the  microscope  stage,  especially  when  stage 
heaters  are  used. 

•  Movement  of  the  tissue:  apparent  movement  that  is  really 
caused  by  movement  of  the  focal  plane  within  the  speci¬ 
men;  natural  movement  of  whole  organisms  or  those 
caused  by  heartbeat,  etc. 

•  Movement  of  cells  within  the  tissue,  for  example,  cell 
migration,  extension/retraction  of  cell  processes. 

•  Movement  related  to  experimental  procedures,  for 
example,  stimulus-induced  osmotic  changes. 

•  Attaining  a  useful  image  with  a  high  S/N  of  cells  deep 
within  tissue: 

•  Imaging  away  from  damaged  tissue  surfaces. 

•  Light  scatter  by  the  tissue. 

•  The  problem  of  spherical  aberration. 

•  Handling  data:  viewing,  storing,  retrieving,  and  analyzing 
four-dimensional  (4D)  data  sets: 

•  Short  term:  monitoring  experiments  on  the  fly;  adjusting 
focus. 

•  Long  term:  accessibility  and  security  of  archived  data. 

We  have  been  exploring  the  dynamic  behavior  of  a  type  of 
brain  cell,  termed  microglia,  following  brain  tissue  injury.  These 
cells  undergo  a  dramatic  transformation  (“activation”)  from  a 
resting,  ramified  form  to  an  amoeboid-like  form  within  a  few  hours 
after  traumatic  tissue  injury.  Activation  of  microglia  is  triggered 
by  signals  from  injured  cells  (including  neurons),  and  this  mobi¬ 
lizes  microglia  to  engage  neighboring  dead  and  dying  cells.  Natu¬ 
rally,  these  events  are  best  studied  in  the  context  of  a  complex 
tissue  environment  containing  the  native  arrangement  of  tissue 
components;  thus  time-lapse  confocal  microscopy  is  well  suited  to 
examine  these  events.  The  general  approach  we  have  taken  is  to 
label  the  cell  surface  of  microglia  with  fluorescent  probes  and,  sub¬ 
sequently,  to  follow  the  dynamic  movements  of  these  cells,  as  well 
as  their  interactions  with  other  labeled  cells,  within  live  tissue 
slices  continuously  over  periods  of  time  up  to  28  h  (Dailey  and 
Waite,  1999;  Stence  et  al.,  2001;  Grossmann  et  al.,  2002;  Petersen 
and  Dailey,  2004). 

Preparation  of  Central  Nervous  System 
Tissue  Slices 

A  useful  method  of  preparing  and  maintaining  live  brain  tissue 
slices  for  microscopy  is  based  on  the  organotypic  (roller-tube) 
culture  technique  of  Gahwiler  (1984)  or  the  static  filter  culture 
technique  of  Stoppini  and  colleagues  (1991).  Briefly,  these  tech¬ 
niques  involve  rapidly  removing  the  tissue  of  interest  (in  this  case, 
neonatal  rat  or  mouse  hippocampus),  then  slicing  the  tissue  with  a 
manual  tissue  chopper  (Stoelting,  Chicago,  IL)  at  a  thickness  of 
300  to  400  |im.  Others  have  used  a  vibratome  or  custom-built 
instruments  akin  to  an  egg  slicer.  In  the  case  of  the  roller  tube  tech¬ 
nique,  the  tissue  slices  are  secured  to  an  alcohol-cleaned  coverslip 
(11  X  22mm)  with  a  mixture  of  chicken  plasma  (lOpL;  Cocalico) 
and  bovine  thrombin  (10pL;  Sigma).  Collagen  gels  (Vitrogen; 
CellTrix;  O’Rourke  et  al.,  1992)  and  Cell-Tak  (BioPolymers  Inc.; 
Barber  et  al.,  1993)  have  also  been  used  successfully  to  attach 
slices.  In  the  case  of  the  plasma  clot,  the  slices  are  adherent  within 
about  10  min,  at  which  point  the  coverslips  are  placed  in  a  test  tube 
with  1  niL  of  HEPES-buffered  culture  media  containing  25% 
serum.  The  tubes  are  kept  in  a  warm  box  (37°C)  and  rotated  at  12 
rph  in  a  roller  drum  tilted  at  5°  to  the  horizontal.  In  the  case  of  the 


static  filter  cultures,  brain  slices  are  placed  on  porous  cell  culture 
inserts  in  6-well  plates  containing  ~lmL  of  bicarbonate-buffered 
culture  media  per  well.  The  filter  cultures  are  maintained  at  36°C 
in  a  5%  CO2  incubator.  In  either  case,  these  “organotypic”  culture 
methods  provide  a  means  for  maintaining  tissue  slices  in  vitro  for 
up  to  several  weeks  (Gahwiler  et  al.,  1997). 

Fluorescent  Staining 
Microglia 

Often  it  is  most  useful  to  label  only  a  small  percentage  of  the  total 
number  of  cells  within  a  tissue  volume,  and  in  certain  cases  it  is 
desirable  to  label  only  an  identified  subset  of  cells.  Vital  fluores¬ 
cent  probes  must  be  non-toxic  and  resistant  to  photobleaching.  In 
the  case  of  microglia,  there  are  several  commercially  available  flu¬ 
orescent  conjugates  (FITC,  Alexa  Fluor-488,  -568,  or  -647)  of  a 
highly  selective,  non-toxic  lectin  (IB4)  derived  from  Griffonia  sim- 
plicifolia  seeds  (Sigma;  Molecular  Probes).  This  lectin  (IB4)  has 
an  exclusive  affinity  for  a-D-galactosyl  sugar  residues  on  glyco¬ 
proteins  and  glycolipids  (Wood  et  al.,  1979).  In  mammalian  brain 
tissues,  IB4  labels  only  microglia  and  endothelial  cells  lining  blood 
vessels  and  capillaries  (Streit  and  Kreutzberg,  1987).  The  simulta¬ 
neous  labeling  of  microglial  cell  populations  and  blood  vessels  has 
revealed  novel,  dynamic  interactions  between  these  structures 
(Grossmann  et  al. ,  2002).  Incubation  of  brain  slices  or  slice  cul¬ 
tures  for  1  h  in  IB4-containing  medium  (5  |ig/mL)  is  sufficient  for 
robust  labeling  of  microglia  up  to  ~50pm  deep  within  tissues 
(Kurpius  and  Dailey,  2005). 

Nuclei  of  Live  or  Dead  Cells 

One  of  a  variety  of  fluorescent  DNA-binding  dyes  is  used  to  label 
live  or  dead  cell  nuclei  in  brain  tissue  slices  (Dailey  and  Waite, 
1999;  Petersen  and  Dailey,  2004).  To  visualize  the  nuclei  of  live 
cells,  we  use  one  of  the  membrane-permeant  dyes  that  have  spectra 
in  the  far-red,  SYT059  (abs/em:  622/645  nm)  or  SYT061  (620/647 
nm).  For  labeling  the  nuclei  of  dead  cells,  we  use  one  of  the  mem¬ 
brane  impermeant  dyes,  Sytox  Green  (504/523  nm),  Sytox  Orange 
(547/570nm),  or  To-Pro-3  (642/661  nm)  (all  from  Molecular 
Probes).  All  DNA-binding  dyes  are  applied  for  10  to  20min 
(1 : 10,000).  These  dyes  are  used  in  combination  with  fluorescently 
tagged  IB4  to  image  microglial  behaviors  in  relation  to  other  cells. 

Maintaining  Tissue  Health  on  the 
Microscope  Stage 

Image  data  obtained  from  compromised  tissue  is  useless  at  best, 
and  deceiving  at  worst.  For  example,  CNS  slice  physiologists  have 
long  known  that  oxygen  deprivation  can  have  severe  effects  on 
synaptic  activity,  although  CNS  tissues  from  developing  animals 
seem  to  have  a  fairly  high  resistance  to  hypoxia  (Dunwiddie, 
1981). 

It  is  not  always  easy  to  assess  the  health  of  living  tissue  on  the 
microscope  stage,  but  in  the  case  of  dynamic  processes  such  as  cell 
division  or  cell  migration,  one  would  at  least  expect  that  the  cells 
perform  these  activities  at  rates  near  that  expected  based  on  other 
methods  of  determination.  Also,  one  should  become  suspicious  if 
the  rate  of  activity  consistently  increases  or  decreases  over  the 
imaging  session.  For  example,  exposure  of  fluorescently  labeled 
axons  to  high  light  levels  can  reduce  the  rate  of  extension  or  cause 
retraction.  In  contrast,  high  light  levels  can  produce  a  long-lasting 
increase  in  the  frequency  of  Ca2+  spikes  in  Fluo-3-labeled  astro¬ 
cytes  in  cultured  brain  slices.  In  many  cases,  there  will  not  be  a 
useful  benchmark  for  determining  phototoxic  effects,  but  consis- 
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tent  changes  during  imaging  will  serve  to  warn  the  concerned 
microscopist.  It  may  be  worth  sacrificing  a  few  well-labeled  preps 
to  determine  if  different  imaging  protocols,  such  as  lower  light 
levels  or  longer  time  intervals  between  images,  significantly  alter 
the  biological  activity  under  study. 

Requirements  for  maintaining  healthy  tissue  during  imaging 
dictate  specimen  chamber  design.  A  closed  specimen  chamber  has 
the  advantage  of  preventing  evaporation  during  long  experiments 
and  stabilizing  temperature  fluctuations  caused  by  this  evapora¬ 
tion.  We  found  that  microglia  in  tissue  slices  maintained  in  a  closed 
chamber  (volume  ~lmL)  with  HEPES-buffered  culture  medium 
remain  viable  and  vigorous  for  about  6h,  after  which  point  the 
chamber  medium  acidifies  and  cell  motility  declines.  However, 
when  the  old  chamber  medium  is  exchanged  with  fresh  medium, 
the  cells  jump  back  to  life  again.  This  crude  method  of  periodic 
medium  exchange  has  supported  continuous  observation  of  Dil- 
labeled  migrating  cells  in  tissue  slices  on  the  microscope  stage  for 
as  long  as  45h  (O’Rourke  et  ah,  1992).  However,  when  using  this 
approach,  one  runs  the  risk  of  mechanically  disturbing  the  chamber 
or  inducing  a  temperature  change  and  thereby  causing  a  jump  in 
focus. 

A  more  elegant  method  for  medium  exchange  and  introduction 
of  reagents  involves  continuous  superfusion.  A  variety  of  perfu¬ 
sion  chambers  with  either  open  or  closed  configurations  are  avail¬ 
able  (see  Table  19.3).  Sometimes  it  is  necessary  to  design  and 
construct  very  sophisticated  temperature  and  fluid-level  control 
systems  (e.g.,  Delbridge  et  al..  1990;  or  Walcerz  and  Diller,  1991). 
Such  chambers  permit  very  rapid  exchange  of  medium  (seconds 
rather  than  minutes),  which  is  necessary  for  physiological  experi¬ 
ments  requiring  high  time  resolution.  There  are  also  now  com¬ 
mercially  available,  programmable,  automated  perfusion  systems 
that  permit  rapid  switching  between  one  of  several  perfusion  chan¬ 
nels  (see  Table  19.3).  Some  experimental  conditions  require  only 
relatively  simple,  low-cost  chambers  and  perfusion  systems,  such 
as  the  one  depicted  in  Figure  19.3  (see  also  Dailey  et  al,  2005). 
We  have  used  such  a  design  to  continuously  superfuse  tissue  slices 


on  the  microscope  stage  for  many  hours.  The  tissues  seem  to 
remain  healthy  for  at  least  20  h  when  perfused  ( 10-20  mL/h)  with 
either  the  culture  medium  (see  Dailey  et  ah,  1994)  or  normal 
saline,  both  of  which  are  buffered  with  25mM  HEPES. 

Specimen  heating  is  essential  for  many  experiments,  but  this 
can  induce  an  agonizing  battle  with  focus  stability  (see  below)  as 
the  chamber  and  stage  components  heat  up.  Because  there  is 
always  a  time  lag  between  when  the  sensor  of  the  temperature 
control  detects  that  the  temperature  is  too  high  (or  low)  and  the 
time  that  the  heater  is  able  to  warm  the  whole  stage,  the  actual 
temperature  of  most  stage  heaters  is  always  slowly  oscillating,  a 
fact  that  causes  the  focus  plane  to  shift  in  a  periodic  manner.  A 
sufficient  period  of  preheating  can  sometimes  reduce  this  problem. 
Another  approach  is  to  use  a  modified  hair  dryer  to  blow  warm  air 
onto  both  the  chamber  and  the  stage  (see  Dailey  et  al.,  2005)  or 
an  egg-incubator  heater  to  heat  all  the  air  in  an  insulated  box  sur¬ 
rounding  the  entire  microscope  (Potter,  2004). 

It  is  also  important  to  monitor  the  temperature  of  the  per¬ 
fusing  medium  very  near  to  the  specimen.  A  low-cost  micro¬ 
processor  temperature  controller  that  reduces  fluctuations  in  the 
heating/cooling  cycle  can  be  obtained  from  Omega  Engineering 
(Stamford,  CT). 

Imaging  Methods 

It  should  by  now  be  evident  that  a  primary  concern  when  imaging 
living  cells  is  photon  collection  efficiency.  This  is  especially  true 
when  imaging  dynamic  processes,  such  as  cell  migration,  over 
long  periods  of  time.  Higher  collection  efficiency  will  afford  effec¬ 
tively  lower  excitation  light  levels,  thus  permitting  more  frequent 
sampling  or  observations  of  longer  duration. 

We  have  been  using  commercially  available  laser  scanning 
confocal  systems  (Leica  TCS  NT  and  Leica  SP2  AOBS).  For  illu¬ 
mination,  the  microscopes  are  equipped  with  multiple  lasers.  Both 
systems  offer  simultaneous  excitation  and  detection  in  more  than 
one  epi-fluorescent  channel.  In  addition,  the  SP2  acousto-optic 
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FIGURE  19.3.  Schematic  diagram  of  a  simple  perfusion  system  for  maintaining  tissue  slices  on  the  microscope  stage.  The  gravity  perfusion  system  is  con¬ 
structed  using  readily  available  syringes,  three-way  stopcocks,  and  tubing.  The  perfusion  rate  is  adjusted  using  a  simple  thumb-wheel  regulator.  Perfusate  is 
removed  through  a  vacuum  line  and  flask. 


Confocal  Microscopy  of  Living  Cells  •  Chapter  19  395 


beam  splitter  (AOBS)  system  offers  increased  flexibility  by  pro¬ 
viding  continuously  variable  wavelength  selection  in  the  excita¬ 
tion  pathway.  This  permits  customizing  the  spectral  detection  to 
maximize  throughput  in  separate  channels.  The  Leica  objectives 
that  we  find  most  useful  are  a  dry  20X/0.75  PlanApo  lens,  and  a 
63X/1.2  water-immersion  PlanApo  objective  lens. 

Imaging  Deep  Within  Tissue 

Often  the  goal  of  studies  in  tissue  slices  is  to  examine  biological 
processes  within  a  cellular  environment  that  approximates  that 
found  in  situ.  In  the  case  of  tissue  slices,  it  is  usually  desirable  to 
image  as  far  from  cut  tissue  surfaces  as  possible  to  avoid  artifacts 
associated  with  tissue  damage.  However,  the  cut  surfaces  of  devel¬ 
oping  CNS  tissue  slice  cultures  contain  a  plethora  of  astrocytes, 
activated  microglia,  and  a  mat  of  growing  neuronal  processes. 
Time-lapse  imaging  of  these  regions  provides  striking  footage  of 
glial  cell  movements,  proliferation,  and  phagocytosis  (Stence 
et  al.,  2001;  Petersen  and  Dailey,  2004). 

If  high-NA  (1.3,  1.4)  oil-immersion  lenses  are  used  to  focus 
more  than  5-10  pm  into  most  living  specimens,  spherical  aberra¬ 
tion  will  severely  reduce  resolution  and  consequently  diminish 
signal  levels  (see  also  Chapters  1,  7  and  20,  this  volume).  Instead 
one  should  use  water-immersion  lenses,  or  on  tissue  having  a 
refractive  index  (RI)  around  1 .48,  an  objective  designed  for  glyc¬ 
erin  immersion.  Table  18.2  in  the  preceding  Chapter  lists  RI  values 
for  a  number  of  tissues.  Efforts  to  image  more  than  50pm  into 
tissue  is  also  complicated  by  a  number  of  other  factors: 

•  Weak  staining  of  cells  due  to  poor  dye  penetration. 

•  Light  scatter  by  the  overlying  tissue  components. 

•  Spherical  aberration. 

The  first  problem  can  be  overcome  if  the  dye  can  be  injected 
into  the  tissue  with  a  minimum  of  disruption,  or  if  tissues  can  be 
harvested  from  transgenic  animals  expressing  fluorescent  proteins 
in  subsets  of  cells  (e.g.,  Zuo  et  ah,  2004).  Light  scatter  by  the 
tissue  can  be  minimized  by  using  longer-wavelength  dyes  (see 
Chapters  28  and  37,  this  volume ).  Indeed,  we  find  that  a  long- 
wavelength  (Alexa  Lluor-647;  Molecular  Probes)  conjugate  of 
IB4  noticeably  improves  visibility  of  labeled  cells  in  deeper 
portions  of  the  tissue  slice.  Imaging  at  longer  wavelengths  may 
also  reduce  phototoxic  effects  because  the  light  is  of  lower  energy. 
Linally.  the  problem  of  spherical  aberration  can  be  improved  by 
using  water-immersion  objective  lenses  or  the  recently  introduced 
automatic  spherical  aberration  corrector  (Intelligent  Imaging 
Innovations,  Denver,  CO;  and  also  Chapters  1,  7,  and  20,  this 
volume). 

Keeping  Cells  in  Focus 

The  optical  sectioning  capability  of  the  confocal  microscope  can 
be  simultaneously  a  blessing  and  a  curse.  On  the  one  hand,  thin 
optical  sections  reduce  out-of-focus  flare  and  improve  resolution. 
However,  with  such  a  shallow  depth  of  focus,  even  very  small 
changes  in  the  position  of  the  objective  lens  relative  to  the  object 
of  interest  within  the  specimen  can  ruin  an  otherwise  perfect  exper¬ 
iment.  This  is  a  particular  problem  when  imaging  thin,  tortuous 
structures  such  as  axons  or  dendritic  spines  within  neural  tissue. 
A  moving  focal  plane  can,  for  example,  give  one  the  erroneous 
impression  of  dendritic  spine  extension  or  retraction.  This  problem 
is  compounded  when  imaging  cells  and  cell  processes  that  are  in 
fact  actively  moving  within  the  tissue. 


One  obvious  approach  is  to  image  the  cells  in  four  dimensions 
(3D  x  time).  This  can  absorb  some  changes  in  tissue  and  stage 
movement  as  well  as  help  track  cells  that  move  from  one  focal 
plane  to  another.  In  addition,  our  strategy  has  been  to  image  with 
the  detector  pinhole  aperture  substantially  open,  corresponding  to 
a  pinhole  size  roughly  4  Airy  units  in  size.  Although  this  reduces 
the  axial  resolution  slightly,  it  has  the  dual  advantage  of  achieving 
a  higher  S/N  at  a  given  illumination  intensity,  as  well  as  thicken¬ 
ing  the  optical  section.  On  our  microscope  systems,  the  open 
pinhole  configuration  gives  an  apparent  optical  section  thickness 
of  about  3  pm  when  using  a  20x  NA  0.7  objective.  Thus,  for  each 
time  point,  we  collect  about  15  images  at  axial  step  intervals  of 
~2pm.  The  guiding  principle  here  is  to  space  the  image  planes  in 
the  axial  dimension  so  as  to  maximize  the  volume  of  tissue 
imaged  but  not  lose  continuity  between  individual  optical-section 
images. 

When  these  image  stacks  are  collected  at  ~5-min  intervals  at 
power  levels  of  ~50  to  75  pW  (back  aperture  of  objective),  IB4- 
labeled  cells  do  not  appear  to  suffer  phototoxic  effects  and  can  be 
imaged  continuously  for  over  20  h.  Image  stacks  can  be  recom¬ 
bined  later  using  a  maximum  brightness  operation.  Unfortunately, 
even  when  z-axis  stacks  of  images  are  collected,  tissue  movements 
can  be  so  severe  as  to  necessitate  a  continuous  “tweaking"  of  the 
focus.  Thus,  it  is  helpful  to  monitor  image  features  on  the  screen 
(by  making  fiduciary  marks  on  an  acetate  sheet  taped  to  the 
monitor  screen),  or  to  store  the  data  in  such  a  way  that  they  are 
quickly  accessible  and  can  be  reviewed  on  the  fly  to  make  correc¬ 
tive  focus  adjustments. 

Ideally,  one  would  like  an  automated  means  of  maintaining 
the  desired  plane  of  focus,  especially  for  long  imaging  sessions. 
Although  there  are  several  autofocus  methods  that  work  for  simple 
specimens  (e.g.,  Lirestone  et  ah,  1991),  imaging  structures  in  3D 
tissue  presents  a  significant  challenge  because  there  is  no  single 
image  plane  on  which  to  calculate  focus. 

Handling  the  Data 

Imaging  tissue  in  3D  over  time  solves  some  problems  but  gener¬ 
ates  others.  In  particular,  how  does  one  deal  with  the  gigabytes  of 
image  data  that  are  often  obtained  in  a  single  experiment?  Lortu- 
nately,  improvements  in  desktop  computer  performance  and 
storage  capacity,  coupled  with  lower  cost,  make  this  much  less  of 
a  problem  than  it  used  to  be.  Desktop  workstations  now  contain 
hard  drives  with  very  large  storage  capacities  (hundreds  of  giga¬ 
bytes),  and  archiving  methods  and  media  are  readily  accessible  and 
inexpensive.  We  typically  store  newly  acquired  data  on  a  network 
server  for  image  processing  (such  as  spatial  filtering  or  deconvo¬ 
lution)  and  analysis,  then  archive  the  data  onto  compact  disks 
(CDs)  or  digital  video  disks  (DVDs).  This  is  a  technology  that  is 
sure  to  continue  rapid  advancement,  making  it  easier  to  store  and 
access  image  data.  At  the  end  of  the  time-lapse  experiment,  the  z- 
axis  image  stacks  are  combined  in  a  variety  of  ways  for  viewing. 
Lor  time-lapse  studies,  it  is  generally  most  useful  to  produce  a  set 
of  “extended  focus”  images  for  viewing  time-points  in  rapid  suc¬ 
cession.  Depending  on  the  file  format,  these  image  series  can  be 
viewed  in  one  of  a  number  of  image  viewers,  including  Scion 
Image  or  ImageJ  (freely  available  from  Scion  Corp.).  To  create  the 
projection  images,  we  use  a  maximum  brightness  operation 
running  in  a  custom-written  Pascal  macro  to  construct  a  2D  rep¬ 
resentation  of  each  3D  data  set  (Stence  et  al.,  2001).  When  the 
axial  step  interval  is  appropriate  (see  above),  portions  of  single 
cells  that  pass  through  the  various  focal  planes  appear  contiguous. 
Alternatively,  the  image  stacks  can  be  reassembled  into  a  set  of 
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red-green  3D  stereo  images.  Such  images,  when  played  in  rapid 
succession,  provide  3D  depth  information  as  well  as  time  infor¬ 
mation  in  thick  tissue  samples.  If  more  than  one  fluorescent 
channel  is  used,  they  can  be  combined  to  create  multicolor  images. 

Results 

The  ability  to  collect  3D  image  data  sets  over  long  periods  of  time 
and  at  relatively  short  time  intervals  has  revealed  new  information 
on  the  dynamics  of  microglial  cell  activation,  migration,  and 
cell-cell  interactions  in  live  mammalian  brain  tissues.  Here  we 
offer  some  examples  of  the  kinds  of  data  that  can  be  obtained, 
along  with  some  possible  modes  of  data  analysis. 

In  Figure  19.4,  we  show  an  example  of  a  one-channel,  single 
focal  plane  confocal  time-lapse  observation.  Several  IB4-labeled 
microglial  cells  in  a  live,  “roller  tube”  slice  culture  were  repeat¬ 
edly  imaged  at  intervals  of  a  few  minutes  to  reveal  movements  of 
microglial  cells  over  time.  These  movements  can  be  quantified 
using  automated  edge  detection  [Fig.  19.4(A)].  Computer  analysis 
using  2D  Dynamic  Image  Analysis  Software  (DIAS;  Soli,  1995) 
shows  differences  in  motile  behavior  of  three  cells,  as  determined 
by  tracing  the  cell  perimeter  [Fig.  19.4(B)].  Cell  motility  move¬ 
ments  can  be  displayed  as  regions  of  protrusion  (green)  or  retrac¬ 
tion  (red)  by  comparing  cell  shape  over  successive  time-points 
[Fig.  19.4(C)].  Many  parameters  of  cell  shape  and  movement  can 
be  quantified  easily  using  DIAS  (Soli,  1995,  1999;  Heid  et  al., 
2002). 

Dual-channel,  time-lapse  confocal  imaging  of  a  specimen 
labeled  with  more  than  one  fluorophore  can  provide  information 
on  the  dynamic  relationship  between  cell  components  or  diverse 
cell  types.  In  Figure  19.5,  we  show  a  two-channel  time-lapse  series 
of  IB4-labeled  microglia  and  cell  nuclei.  Images  from  each  channel 
(green  and  red)  were  collected  simultaneously  using  a  two- 
detector  system.  The  nuclear  marker  (SYTO  61)  labels  all  cells, 
including  microglia.  Time-lapse  imaging  shows  the  location  of 


nuclei  within  migrating  cells  [Fig.  19.5(A)],  and  DIAS  analysis 
can  be  used  to  plot  and  analyze  the  movements  of  nuclei 
[Fig.  19.5(B)], 

Dual-channel,  3D  time-lapse  confocal  imaging  can  yield  infor¬ 
mation  on  the  relative  movement  and  location  of  structures  labeled 
with  two  or  more  fluorophores  within  a  thick  tissue  specimen.  In 
the  example  shown  in  Figure  19.6,  the  movements  of  IB4-labeled 
microglia  are  observed  in  the  context  of  dead  cell  nuclei  labeled 
with  To-Pro-3,  a  membrane-impermeant  DNA  binding  dye.  The 
stereo  images  reveal  the  movements  of  microglia  on  the  surface  of 
the  tissue  slice  as  well  as  deep  within  the  tissue.  Such  information 
is  being  used  to  study  differences  in  the  behaviors  and  cell-cell 
interactions  of  microglia  in  a  brain  tissue  environment.  Note,  for 
example,  the  rapid  microglial  movement  and  phagocytosis  of  a 
dead  cell  nucleus  in  the  time-lapse  series  [Fig.  19.6(D)]. 

In  summary,  the  4D  and  five-dimensional  (5D)  imaging 
methods  outlined  here  capture  more  of  the  events  occurring  within 
the  tissue  and  also  provide  the  researcher  with  assurance  that 
apparent  changes  in  the  length  of  even  the  smallest  branches  and 
fine  filopodia  are  not  due  to  the  structures  moving  in  and  out  of 
the  focal  plane.  They  also  provide  a  means  for  exploring  the 
dynamic  interactions  between  different  molecules,  cells,  and  cell 
types  within  thick  biological  specimen. 

Conclusion 

Imaging  of  living  specimens  at  a  high  S/N  can  be  achieved  with 
very  low  levels  of  incident  illumination. 

Imaging  with  a  fairly  large  pinhole  strikes  a  balance  between 
image  quality  and  focus  problems  during  long-term  observation  of 
live  tissue. 

Four-dimensional  time-lapse  imaging  is  useful  for  following 
moving  cells  and  fine  cellular  processes  within  complex  tissues. 
Under  optimal  conditions,  3D  data  can  be  collected  over  long 
periods  of  time  (>20h)  at  relatively  short  time  intervals  (~5min). 
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FIGURE  19.4.  Use  of  a  Dynamic  Image  Analysis  System  (DIAS;  Soli,  1995.  1999)  to  characterize  microglial  motility  behaviors  in  time-lapse  imaging  exper¬ 
iments.  (A)  Boundaries  (red  lines)  of  three  FITC-IB4-labeled  microglial  cells  were  defined  by  automated,  computer-assisted  edge  detection.  (B)  Tracings  of  the 
three  cells  show  motility  behaviors  over  a  2  h  period.  The  cell  centroid  ( black  dots)  were  computed  and  plotted  for  each  time-point.  Note  that  all  three  cells  are 
motile,  but  cell  2  does  not  locomote.  (C)  Areas  of  new  protrusion  (green)  and  resorption  (red)  are  shown  for  cell  1  at  4  min  intervals.  The  new  centroid  (open 
dot)  is  shown  in  relation  to  the  former  centroid  (filled  dot). 
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FIGURE  19.5.  Two-channel  time-lapse  imaging  of  microglia  {green)  and  cell  nuclei  {red).  (A)  Time  lapse  sequence  shows  capability  of  simultaneously  imaging 
IB4-labeled  microglia  and  nuclei  of  cells  using  a  cell  membrane  permeant  DNA  binding  dye  (Syto61).  Note  the  nuclei  {arrows)  in  two  different  migrating 
microglial  cells.  The  sequence  spans  156  min.  Only  a  small  portion  of  the  original  field  of  view  is  shown.  (B)  Automated  tracing  of  the  movement  of  cell  nuclei 
shows  paths  taken  by  nuclei  of  cells  in  the  experiment  show  in  (A).  Nuclei  were  detected  by  DIAS  software  (Soli,  1995).  Only  a  select  subset  of  nuclei  are 
shown.  Note  differences  in  the  movement  among  different  cells. 
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FIGURE  19.6.  Two-  and  three-dimensional,  two-channel  time-lapse  imaging  of  microglial  movements  and  phagocytosis  of  dead  cell  nuclei  in  live  rat  brain 
tissue  slice  cultures  (P6+2DIV).  (A)  Microglia  are  stained  with  a  fluorescent  lectin,  FITC-IB4.  (B)  Dead  cell  nuclei  are  stained  with  To-Pro-3.  (C)  Merged  image 
of  (A)  and  (B).  The  image  represents  a  projection  of  nine  optical  sections  spanning  40 |tm  in  --depth.  Images  were  captured  using  a  20x/0.7  dry  objective.  (D) 
Time-lapse  sequence  of  a  portion  of  the  field  of  view  above  showing  phagocytic  clearance  of  a  dead  cell  nucleus  (arrowhead)  by  a  locomotory  microglial  cell 
(arrow).  Note  that  the  microglial  cell  maintains  a  rapid  rate  of  locomotion  as  it  sweeps  over  and  picks  up  the  dead  cell  nucleus.  (E-G)  3D  stereo  projections  of 
images  in  A  through  C  above.  Use  red-green  glasses  (red  over  left  eye)  to  view  depth.  (H)  3D  stereo  time-lapse  sequence  showing  mitosis  of  a  microglial  cell 
(arrow)  near  the  surface  of  the  brain  slice  culture.  The  small,  round  objects  represent  the  condensed  nuclei  of  dead  cells.  Time  is  shown  in  minutes.  See  the  sup¬ 
plemental  video  movies  at  http://www.springer.com/0-387-25921-X  (Adapted  from  Petersen  and  Dailey,  2004.) 


Multi-channel  (5D)  imaging  can  provide  information  on  the 
dynamic  relationship  of  different  cell  or  tissue  components.  Future 
developments  should  address  constraints  on  high-resolution  imag¬ 
ing  deep  (>50|im)  within  tissue.  Improvements  will  likely  be 
achieved  by  using  water-immersion  lenses  and  external  automatic 
spherical  aberration  correctors  (see  Chapter  20,  this  volume )  and 
by  employing  longer  wavelength  dyes  to  reduce  light  scatter  by 
the  tissue  and  to  minimize  phototoxic  effects. 


FUTURE  DIRECTIONS 

For  confocal  microscopy  of  living  cells,  the  most  important  char¬ 
acteristic  of  the  instrument  is  its  efficiency  in  collecting  and  detect¬ 
ing  the  fluorescence  emission  light  from  the  specimen  (Chapter  2, 
this  volume).  Any  improvement  in  this  efficiency  reduces  the 
amount  of  light  damage  and  allows  the  gathering  of  more  data.  The 
increased  data  can  either  be  in  the  form  of  more  images,  images 


Confocal  Microscopy  of  Living  Cells  •  Chapter  19  399 


with  less  statistical  noise,  or  images  obtained  with  greater  spatial 
or  temporal  resolution.  Newer  models  of  existing  commercial  con- 
focal  microscopes  have  substantially  improved  photon  efficiency. 
In  addition,  there  have  been  technological  improvements  in  the 
ability  to  separate  the  excitation  and  fluorescence  emission  of  flu- 
orophores,  providing  greater  flexibility  for  multi-channel  imaging 
and  quantitative  image  analysis  in  live  cells  and  tissues.  Finally, 
the  advantages  of  either  Gaussian-filtering  2D  data  or  deconvolv¬ 
ing  3D  data  to  reduce  the  effects  of  Poisson  noise  are  now  widely 
appreciated.  Routine  application  of  this  approach  can  reduce  the 
light  load  to  the  specimen  by  a  factor  of  from  10  to  100  while  still 
producing  images  with  the  same  apparent  resolution  and  signal-to- 
noise  ratio. 

Technological  and  conceptual  advancements  are  also  likely  to 
push  the  spatial  and  temporal  resolution  and  other  modes  of  fluo¬ 
rescence  microscopy  (e.g.,  Gustafsson,  1999;  Hell  et  al.,  2004; 
Chapters  13,  30,  and  31,  this  volume).  Some  of  these  approaches 
(e.g.,  4Pi-microscopy)  look  promising  for  live  cells  (Gugel  et  al ., 
2004),  but  their  potential  for  widespread  use  in  biological  appli¬ 
cations  has  yet  to  be  established,  and  there  are  limitations  on  the 
sample  thickness  (Gustafsson,  1999  and  Chapter  21,  this  volume). 
In  addition,  higher  resolution  implies  smaller  pixels  and  therefore 
more  photons/square  micrometer  and  more  bleaching  and  toxicity. 
Undoubtedly,  there  will  be  more  improvements  and  wider  appli¬ 
cations  along  these  lines  in  the  future. 

Although  it  is  difficult  to  predict  the  future  of  confocal 
microscopy  of  living  cells,  as  confocal  microscopy  (and  its  richer 
cousin,  multi-photon  microscopy)  are  in  all  probability  the  optimal 
methods  for  studying  the  3D  structure  of  living  cells,  the  future 
seems  sure  to  be  bright! 
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Microscopy  Induced  by  Refractive  Index  Mismatch 
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INTRODUCTION 

Modern  optical  microscopes  are  so  good  that  many  scientists 
forget  that  these  instruments  only  provide  their  optimal  perfor¬ 
mance  if  they  are  used  under  certain  operating  conditions.  Typical 
users  may  be  unaware  of  the  very  existence  of  such  limitations 
either  because  they  may  unwittingly  work  within  the  limits  or 
because  they  fail  to  recognize  their  effects.  It  is  probably  also 
correct  to  assume  that  the  manufacturer  does  not  intend  to  dis¬ 
courage  purchase  by  emphasizing  the  pitfalls  that  unavoidably 
arise  from  the  physics  of  imaging.  However,  advanced  micro- 
scopists  tend  to  use  their  instruments  at  the  limits  of  their  perfor¬ 
mance.  They  wish  to  use  lasers  from  the  ultraviolet  (UV)  to  the 
infrared  (IR),  special  emission  lines  from  very  large  arc-lamps, 
charge-coupled  device  (CCD)  cameras  with  a  dynamic  range  of 
up  to  16  bits,  photon-counting  avalanche  photodiodes  with 
unmatched  sensitivity,  and  several  fast  photomultiplier  tubes.  They 
want  to  observe  two  or  more  dyes  simultaneously.  They  expect  the 
stage  to  remain  in  a  stable  position  for  several  hours,  possibly 
while  going  through  heating  and  cooling  cycles,  and  sometimes 
they  want  to  record  low-level  fluorescence  emissions  from  rather 
thick  specimens  mounted  in  an  aqueous  medium  with  high  numeri¬ 
cal  aperture  (NA)  oil-immersion  lenses.  Is  this  possible?  Can  one 
expect  an  off-the-shelf  product  to  perform  well  under  all  these  cir¬ 
cumstances?  The  answer  is:  Within  certain  limits,  yes,  you  can. 
The  issue  is  to  specify  and  recognize  these  limits. 

This  chapter  describes  the  problems  that  occur  when  observ¬ 
ing  specimens  that  are  mounted  in  a  medium  whose  refractive 
index  is  different  from  that  of  the  immersion  liquid.  Classic  exam¬ 
ples  are  live  cells  kept  in  a  physiological  buffer  solution  or  even 
fixed  cells  kept  in  a  glycerol-based  mountant  that  are  imaged  by 
an  oil-immersion  lens  of  large  numerical  aperture.  This  chapter 
first  outlines  the  physics  of  the  situation,  both  for  confocal  and 
multi-photon  microscopy,  then  presents  the  results  of  a  theoretical 
investigation,  compares  them  with  a  series  of  experiments,  and 
finally  draws  conclusions  that  are  particularly  relevant  to  the  quan¬ 
titative  observation  of  (living)  biological  specimens. 

THE  SITUATION 

Figure  20.1  describes  a  common  situation  encountered  in 
microscopy.  The  sample  is  mounted  between  a  coverslip  and  a 
glass  slide,  which  in  fact  can  be  another  coverslip,  and  is  immersed 
in  a  special  mounting  medium,  such  as  an  aqueous  buffer  or  a  more 
viscous  solution  based  on  glycerol.  Coverslip  glass  has  a  refrac¬ 
tive  index  (RI)  n  =  1.518.  The  immersion  oil  between  the  cover- 


slip  and  the  objective  lens  is  assumed  to  have  the  same  n.  The  n 
of  the  mounting  medium  around  the  sample  will  usually  be  dif¬ 
ferent  from  that  of  the  glass  and  of  the  immersion  oil.  Water  has 
an  index  of  n  =  1.33  and  glycerol  has  n  =  1.47.  The  sample  itself 
will  have  an  n  that  is  not  much  different  from  that  of  the  mount¬ 
ing  medium  and  slightly  higher  than  that  of  water  (see  Chapter  18, 
this  volume ,  for  the  RIs  of  common  mounting  media). 

A  light  ray  emerging  from  an  oil-immersion  objective  lens  that 
is  coupled  to  the  coverslip  with  the  appropriate  oil  will  not  be 
refracted  until  it  passes  the  interface  from  the  coverslip  into  the 
mounting  medium.  The  light  ray  is  usually  only  slightly  affected  by 
the  sample  itself  and  is  assumed  to  carry  on  straight  towards  the 
focal  region  once  it  has  passed  the  interface  between  the  mounting 
medium  and  the  coverslip.  The  discussion  can  therefore  be 
restricted  to  the  effects  caused  by  the  change  in  n  at  the  glass- 
medium  boundary  and  to  the  distance  from  this  interface  to  the  focus 
point  somewhere  inside  the  sample.  What  effects  can  be  expected? 

•  A  light  ray  is  refracted  at  the  glass-medium  interface.  The 
angle  of  the  ray  is  changed;  therefore,  the  different  rays  focus 
at  different  positions  along  the  z-axis  than  they  would  in  a  per¬ 
fectly  matched  optical  system.  In  microscopy,  ni  is  usually 
larger  than  n2,  and  the  focus  is,  therefore,  closer  to  the  cover- 
slip  than  under  ideal  conditions.  The  position  of  an  object  will 
then  appear  to  be  further  away  from  the  coverslip.  If  were 
smaller  than  n2,  the  focus  would  be  further  from  the  coverslip 
than  it  should  be  and  the  object  would  then  appear  to  be  closer 
to  the  coverslip. 

•  Whenever  light  is  refracted,  some  light  is  also  reflected  (Born 
and  Wolf,  2002).  As  refraction  occurs  only  when  the  angle  of 
incidence  is  lower  than  the  angle  of  total  internal  reflection, 
the  NA  of  the  immersion  system  is  effectively  reduced. 

•  Perfect  imaging  is  only  possible  if  the  wavefront  remains 
spherical.  Any  deviation  from  sphericity  results  in  a  larger 
spread  of  the  focus  and  hence  in  a  reduction  in  both  spatial  res¬ 
olution  and  peak  intensity. 

•  This  spreading  of  the  focus  means  that  the  image  of  the  focal 
spot  focused  back  towards  the  confocal  pinhole  is  also  spread. 
This  second  defocus  effect  means  that  less  light  penetrates  the 
pinhole,  and  the  observed  intensity  decreases  still  more. 


THEORY 

The  calculations  are  performed  in  a  vectorial  theory  following  Hell 
and  colleagues  (1992).  The  sample  object  is  a  layer  of  fluorophore 
immersed  in  the  mounting  medium.  The  immersion  oil  and  the 
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FIGURE  20.1 .  Terminology  used  for  the  calculation  of  point  spread  functions 
in  optically  mismatched  systems.  A  spherical  wavefront  emerges  from  the 
objective  lens.  The  wavefront  is  described  by  its  electric  field  strength  E  and 
its  direction  k ,  while  the  objective  lens  is  described  by  its  focal  length  and  its 
numerical  aperture  (NA)  =  n i  sin(a).  The  sample  consists  of  two  coverslips, 
each  with  a  refractive  index  of  //  and  the  mounting  medium  with  a  homoge¬ 
neous  refractive  index  of  n2.  The  immersion  medium  between  the  upper  cover- 
slip  and  the  objective  lens  has  the  same  n  as  the  coverslip.  In  a  perfectly 
matched  system,  n2  is  equal  to  n2  and  the  geometrical  focus  is  a  distance  NFP 
(nominal  focal  position)  away  from  the  glass/mounting  medium  interface.  In  a 
mismatched  system.  «,  is  not  equal  to  n2.  The  focus  suffers  from  a  focal  shift 
and  is  found  at  AFP  (actual  focal  position).  The  theory  describes  the  calcula¬ 
tion  of  the  electric  field  strength  E  in  a  point  P(x,y,z)  close  to  both  AFP  and 
NFP. 


coverslip  have  n  =  and  the  mounting  medium  n  =  n2.  Focusing 
into  the  sample  is  achieved  by  mechanically  varying  the  distance 
between  the  objective  lens  and  the  bottom  of  the  coverslip.  The 
distance  between  this  surface  and  the  geometrical  focus  in  a  per¬ 
fectly  matched  system  is  referred  to  as  the  nominal  focal  position 
(NFP).1  The  difference  between  the  NFP  and  the  actual  focal  posi¬ 
tion  (AFP)  is  referred  to  as  the  focal  shift  in  the  optically  mis¬ 
matched  system.  We  wish  to  calculate  the  AFP  and  the  intensity  at 
a  point  P(x,y,z)  in  the  vicinity  of  the  AFP.  The  optical  system  is 
described  by  the  wavelength  of  the  incident  light  ray  (X),  the  NA 
of  the  objective  lens,  and  the  diameter  of  the  aperture  in  the  lens. 
In  a  modern  optically  perfectly  matched  microscope,  the  so-called 
infinity-corrected  lens  is  assumed  to  accept  a  perfectly  planar 
incoming-wavefront  and  produce  a  perfectly  spherical  outgoing 
wavefront  that  produces  an  aberration  free  point  spread  function 
(PSF)  at  the  focal  point.  We  note  that  our  considerations  apply  to 
any  point  within  the  field  of  view  specified  for  the  lens. 

In  a  confocal  microscope,  a  point  source  is  used  to  define  the 
extent  and  the  position  of  illumination,  whereas  a  point  detector 
discriminates  against  any  light  emitted  outside  a  certain  region.  In 
physical  terms,  the  effective  PSF  (Hell  et  al.,  1992)  of  the  confocal 
fluorescence  microscope  is  given  by  the  product  of  the  illumi¬ 
nation  intensity  and  detection  PSF: 


1  It  should  be  noticed  that  the  NFP  is  the  actual  distance  of  a  feature  in  the 
object  from  the  surface  of  the  coverslip. 


hmf  X 


(1) 


where  hm  denotes  the  amplitude  of  the  illumination  light  in  the 
focal  region  and  hia  is  the  amplitude  distribution  for  the  detection, 
which  is  similar  to  hm,  but  is  calculated  for  the  wavelength  of  flu¬ 
orescence  emission.  So,  while  lim  is  proportional  to  the  light  field 
used  for  illumination,  and  l/;,„l2  to  its  intensity,  the  effective  con¬ 
focal  PSF  hrf  is  proportional  to  the  probability  that  a  given  focal 
coordinate  contributes  to  the  signal  at  the  confocal  detector.  If  the 
excitation  and  emission  wavelengths  are  rather  similar,  it  follows 
that  \hiU I2  =  l/?detl2.  In  this  case,  hcf  is  proportional  to  the  fourth  power 
of  the  illumination  amplitude  and  hence  to  the  square  of  the  illu¬ 
mination  intensity  (Wilson  and  Sheppard,  1984),  as  indicated  on 
the  right-hand  side  of  Eq.  1 .  This  quasi-quadratic  signal  depen¬ 
dence  of  the  recorded  intensity  on  the  illumination  intensity  causes 
a  drop  of  the  detected  fluorescence  from  points  away  from  the  geo¬ 
metrical  focus  and  is  the  actual  physical  reason  why  a  confocal 
setup  defines  a  confined  recording  volume  in  three-dimensional 
(3D)  space. 

The  z-response  7(z)  to  infinitely  thin  fluorescent  planes  and  the 
response  /edge(:t)  to  half-volumes  in  the  z-direction  are  of  practical 
importance  as  well  because  they  quantify  the  ability  of  a  micro¬ 
scope  to  distinguish  planes  that  are  stacked  in  the  z-direction: 

oo  oo  z 

I(z)  =  J  J  hcf(r)dx  dy  and  Iedge{z)  =  J  I(z')dz'  (2) 


A  volume  confinement  can  also  be  achieved  by  multi-photon 
excitation  of  the  fluorophores  (Denk  et  al.,  1990;  Sheppard  and 
Gu,  1992a,  1992b)  (Chapters  28  and  37,  this  volume).  The  proba¬ 
bility  of  two  photons  being  simultaneously  absorbed  by  the  same 
molecule  is  proportional  to  the  square  of  the  local  intensity  (Kaiser 
and  Garret,  1961).  Therefore,  the  effective  PSF  of  a  microscope 
based  on  two-photon  absorption  is  described  by  the  square  of  the 
illumination  PSF,  that  is,  the  fourth  power  of  the  field  amplitude 
for  illumination: 


hi  *v  = 


(3) 


The  similarity  to  Eq.  1  indicates  that  the  illumination  process 
defines  a  volume  in  a  manner  similar  to  the  combined  illumination 
and  detection  processes  in  a  confocal  single-photon  excitation  flu¬ 
orescence  microscope  (Hell  and  Stelzer,  1992;  Sheppard  and  Gu, 
1992b;  Stelzer  et  al.,  1994).  This  means  that  all  the  problems  dis¬ 
cussed  for  single-photon  confocal  microscopy  are  also  encoun¬ 
tered  in  two-photon  fluorescence  microscopes.  The  only  real 
difference  is  that  the  latter  requires  wavelength  doubling.  Addi¬ 
tional  confocalization  of  the  system  means  that  the  effective  PSF 
is  given  by  hCf2ln.  =  \hm\A  X  l/?detl2  and  for  an  «-photon  confocalized 
system  we  obviously  have 


cflhv 


\hj  x  \hA. 


(4) 


In  an  aberration-free  system,  the  calculation  of  the  fields  hm 
and  ha#  is  rather  straightforward  because  these  functions  solely 
depend  on  the  wavelength  and  the  NA.  By  contrast,  when  focus¬ 
ing  through  RI  interfaces,  the  evaluation  of  hm  and  hia  is  compli¬ 
cated  by  the  fact  that,  loosely  speaking,  it  has  to  be  calculated  once 
for  medium  /q  and  then  for  medium  n2  (Hell  et  al.,  1993).  The 
latter  publication  treated  the  problem  with  specific  regard  to 
confocal  microscopy  and  quantitatively  predicted  all  the  effects 
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encountered  with  refractive  index  mismatched  samples.  Therefore, 
in  this  chapter  we  will  follow  the  argument  presented  in  that  pub¬ 
lication.  In  the  meantime,  significant  advancements  in  the  formu¬ 
lation  of  this  theory  have  been  made.  These  are  considered  later  in 
this  chapter  (Torok  et  al.,  1995;  Egner  and  Hell.  1999). 

Starting  from  simple  terms,  the  calculation  basically  requires 
the  solution  of  a  variational  problem  in  optics,  that  is,  the  appli¬ 
cation  of  Fermat’s  principle  from  a  point  of  the  converging  spheri¬ 
cal  wavefront  to  the  point  of  the  focal  region  in  question  [Fig. 
20.2(A)],  According  to  the  Huygens-Fresnel  construction,  each 
point  on  the  spherical  wavefront  is  a  source  of  secondary  spheri¬ 
cal  wavelets  (Hopkins,  1943;  Li  and  Wolf,  1981).  In  a  matched 
medium,  we  have 

| 

h(r)  =  cj  j  A(F)  —  K(x)exp(iks)dF  (5) 

F  S 

where  X(F)  denotes  the  wavefront  amplitude  over  the  surface  F  of 
the  spherical  wavefront  and  dF  is  the  surface  element  (see  Fig. 
20.2);  s  is  the  distance  between  the  origin  of  the  wavelet  q  and  the 
point  P,  and  %  is  the  angle  of  inclination  between  the  normal  at  q 
and  the  direction  from  q  to  P. 

=  +C°S(X))  (6) 

In  an  aplanatic  objective  lens,  the  wavefront  X(F)  is  given  for 
in  the  .x-direction  polarized  light  of  Amplitude  A,  by  Richards  and 
Wolf  (1959): 

<cos(0)  +  (l-cos(0))sin2(cp)  % 

A(F)  =  AVcos(0)  (l-cos(0))cos(cp)sin(tp)  (7) 
^  sin(0)sin(cp)  , 

For  the  numerical  calculation  of  h{r),  the  term  K(j)  can  be 
neglected  because  it  varies  by  only  a  small  amount.  Because  the 
calculation  is  restricted  to  the  volume  close  to  the  geometrical 
focus,  1/5  remains  a  constant  and  can  therefore  also  be  neglected. 
The  function  h(r)  can  be  simplified  to 


am  ( cos(0)  +  (l-cos(0))sin2(cp)  2 

h(r)  =  cj  |  Vcos(6)  (1-  cos(0))cos(cp)sin(cp) 

00  sin(0)sin(cp)  , 

exp(iks)sm(Q)d(p  dQ  (8) 

The  final  problem  is  the  determination  of  s,  which,  as  is 
pointed  out  above,  is  a  variational  problem  and  a  careful  analysis 
of  the  light  transition  at  the  interface  between  n,  and  n2,  to  which 
one  solution  has  been  provided  (Hell  et  al, ,  1993). 

If  the  NFP  is  small  compared  to  the  focal  length  of  the  objec¬ 
tive  lens,  an  assumption  which  is  well  met  in  confocal  microscopy, 
the  variational  problem  can  be  solved  analytically  for  the  region 
near  the  NFP  (Egner  and  Hell,  1999).  Torok  and  colleagues 
derived  an  identical  solution  (T5rok  et  al.,  1995)  using  a  plane 
wave  expansion  of  the  light  field,  that  is,  the  Debye  approxima¬ 
tion  instead  of  the  Huygens-Fresnel  construction.  In  both  cases  the 
solution  for  h(r)  is  given  by: 

A’(/o(F)  +  /2(F)c°s(2cp))" 

h(r)  =  c  /2(r)sin(2tp)  (9) 

v  i  2/1(r)cos(cp), 

Where  the  diffraction  integrals  /„  are  defined  by: 

a 

/„:=  J  v'cos(01)  sin(01)(xs  +  xp  cos(02))/o(/:«1  V*2  +y 2  sin(01)) 

0 

exp(ik(<I)(NFP)  +  n2zcos(02)))6/01 

a 

/j :  =  J  VcoI(Uj  sin(01  )(x;)  sin(0 ,  ))/2  ( /c/tj  2Jx1+y2  sin(0 1 )) 

0 

exp(ik((b(NFP)  +  /7,zcos(0,)))^01 

a 

I2:=  J  Vcos(0,)  sin(01)(xs  -  xp  cos(02))/2(/«i,  V*2  +y2  sin(0,)) 

0 

exp(ik(A>(NFP)  +  n2zcos(Q2)))dQ1  (10) 


FIGURE  20.2.  (A)  The  variational  approach  to  the  calculation  of  the  electric  field  strength  E  at  a  point  P(x,y,z )  close  to  both  the  AFP  and  the  NFP.  The  geom¬ 
etry  is  basically  the  same  as  described  in  Figure  20.1.  The  problem  is  to  minimize  the  distance  +  s2(x).  It  can  be  shown  that  this  sum  depends  on  the  angle 
and  on  the  origin  of  the  beam  in  the  primary  spherical  wavefront.  The  calculation,  therefore,  has  to  search  for  those  beams  that  contribute  to  the  field  at  posi¬ 
tion  P  according  to  Fermat’s  principle.  For  each  point  P(*,;y,z)  in  the  object,  the  contribution  is  found  by  integrating  the  complex  electric  field  across  the  whole 
exit  aperture.  This  makes  the  calculations  somewhat  tedious.  (B)  If  the  NFP  is  small  compared  to  the  focal  length  of  the  objective  lens,  which  is  always  the  case 
in  microscopy,  the  calculation  becomes  much  easier  as  only  the  transmission  and  refraction  for  a  beam  incident  upon  the  boundary  with  an  angle  0!  has  to  be 
known. 
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J„  are  Bessel  functions  of  the  first  kind  and  ;;th  order  and  xp>s  are 
Fresnel  transmission  coefficients  for  s-  and  p-polarized  light  (Born 
and  Wolf,  2002).  The  aberration  function 

®(AFP)  =  -NF P(n ,  cos^)  -  n2  cos(02))  (11) 

depends  on  the  nominal  focusing  position,  the  azimuth  angle  0  and 
therefore  on  the  aperture  angle  a  and  the  difference  of  the  refrac¬ 
tive  indices  n,  and  n2.  The  differences  between  the  calculations  of 
the  illumination  and  the  detection  PSFs  are  the  wavelength  and  the 
term  V cos(0)  which  is  omitted. 

RESULTS  OF  THEORETICAL  CALCULATIONS 

The  theory  described  above  does  not  result  in  an  analytical  descrip¬ 
tion  of  the  PSF.  The  PSFs  have  to  be  calculated  numerically  as  a 
function  of  the  NA,  excitation  and  emission  wavelengths,  NFP,  nu 
and  n2.  In  order  to  illustrate  how  focusing  into  a  mismatched 
medium  affects  the  PSF,  Figure  20.3  shows  jcz-images  of  calcu¬ 
lated  PSFs  (logarithmic  scale)  and  the  corresponding  experimen¬ 
tal  through-focus  series  (linear  scale)  for  focusing  with  a 
water-immersion  lens  either  10  pm  deep  into  water  [Fig. 
20.3(A,B)]  or  into  immersion  oil  [Fig.  20.3(C,D)].  In  the  mis¬ 
matched  case,  the  main  maximum  is  shifted,  becomes  relatively 
broader,  and  drops  in  peak  intensity  (Fig.  20.4).  In  addition,  the 
PSF  loses  its  axial  symmetry  with  respect  to  the  main  maximum 
whereby  focusing  above  the  AFP  leads  to  a  different  image  from 
focusing  below  the  AFP  [Fig.  20.3(D)],  an  effect  that  is  not  present 
in  the  matched  case  [Fig.  20.3(B)]. 


The  results  of  several  calculations  are  summarized  in  Figures 
20.4,  20.6,  20.7,  20.8,  20.9,  and  20.10  and  in  Tables  20.1  and  20.2 
for  water  and  for  glycerol  mounting  media.  Figure  20.4  shows  the 
integrated  intensity  for  various  NFPs  using  water  as  the  mounting 
medium.  The  first  image  indicates  the  ideal  situation  encountered 
with  a  fluorophore  mounted  in  immersion  oil.  The  following 
images  show  again  that  the  integrated  intensity  is  smeared  along 
the  optical  axis,  and  an  additional  peak  appears  below  the  main 
maximum.  The  main  maximum  itself  is  shifted,  drops  in  peak 
intensity,  and  becomes  relatively  broader.  These  values  can  be 
evaluated  to  obtain  the  focal  shift  [(NFP)  -  (AFP)],  the  drop  in 
peak  intensity,  and  the  full-width  half-maximum  (FWHM)  of  the 
main  peak.  While,  because  of  its  convoluted  shape  (Fig.  20.5),  it 
is  difficult  to  specify  a  simple  metric  that  quantifies  the  sharpness 
of  a  spherically  aberrated  focal  spot,  the  FWF1M  of  the  main  peak 
is  relatively  simple  to  measure  and  hence  has  become  the  most 
common  measure  of  the  xy  or  axial  resolution. 

Calculations  for  some  numbers  encountered  in  real  situations 
have  been  combined  with  the  experimental  results  in  Figures  20.6, 
20.7,  and  20.8.  Figure  20.9  shows  the  focal  shift  of  the  excitation 
PSF  of  a  0.6  oil-immersion  objective  lens  directly  as  a  function  of 
NFP  for  various  refractive  indices.  Figure  20. 10  also  plots  the  focal 
shift  but  for  an  NA  1 .4  oil-immersion  objective  lens.  The  AFP  is 
regarded  as  the  position  of  the  global  maximum  of  the  PSF  along 
the  optical  axis  rather  than  the  center  of  gravity  or  some  other 
measure  of  this  complex  shape. 

An  important  result  of  these  calculations  is  that  the  effects 
of  spherical  aberration  increase  rapidly  with  («!  -  n2),  NA,  and 
distance  of  the  object  from  the  coverslip  (NFP).  As  long  as  the 
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FIGURE  20.3.  Influence  of  a  mismatched  medium  on  the  PSF.  (A)  and  (C)  show  JK-images  of  calculated  PSFs  in  logarithmic  scale  for  focusing  with  a  water- 
immersion  lens  (n  =  1.334)  10|rm  deep  into  water  and  immersion  oil  («  =  1.518),  respectively.  The  inlets  in  the  lower  left  corners  show  the  central  part  of  the 
PSFs  in  linear  scale.  (B)  and  (D)  show  corresponding  experimental  through-focus  series  in  a  linear  scale.  These  series  represent  exactly  what  one  would  see 
when  focusing  through  a  point-like  object.  Focusing  into  a  mismatched  medium  causes  a  shift  and  a  broadening  of  the  main  maximum.  The  aberrated  PSF  can 
be  clearly  identified  in  the  through-focus  series  as  focusing  above  the  AFP  leads  to  a  different  image  from  focusing  below  the  AFP.  The  image  series  was  made 
of  a  mirror  specimen  by  J.  Pawley  using  a  Zeiss  Axioskpp  50.  with  a  40X/NA1.2  C-Apo  objective,  and  recorded  with  a  Sony  TRV-900  camcorder  using  the 
zoom  lens  to  provide  the  high  magnification. 
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FIGURE  20.4.  Integrated  z-responses  for  various 
penetration  depths  (NFP)  in  water  for  an  NA  =1.3 
oil-immersion  objective  lens.  The  excitation  and 
emission  wavelengths  were  514nm  and  590  nm, 
respectively.  (A)  Ideal  situation  in  oil,  penetration 
depths  of  (B)  5  pm,  (C)  1 0  pm,  (D)  1 5  pm,  (E)  20  pm, 
and  (F)  25  pm.  All  curves  are  normalized  to  the  ideal 
situation  encountered  with  immersion  oil  as  the 
mounting  medium.  The  point  spread  function  is 
obviously  not  confined  to  the  minimal  volume  but 
instead  continues  to  spread  the  larger  NFP  becomes. 
This  causes  a  decrease  of  the  maximal  intensity,  an 
increase  of  the  full-width  half-maximum,  and  a  focal 
shift.  The  intensity  is  distributed  among  several  axial 
peaks  of  which  at  least  two  are  clearly  visible.  Please 
note  that  the  NFP  axis  has  been  offset. 


FIGURE  20.5.  Sample  used  for  measuring  the  edge  response  in  a  confocal  laser-scanning  microscope.  (A)  The  sample  consists  of  two  coverslips  or  a  cover- 
slip  and  a  glass  slide  with  a  fluorophore  dissolved  in  the  mounting  medium  (water,  glycerol,  immersion  oil)  in  between.  The  sample  thickness  (upper/lower  inter¬ 
face  distance)  is  at  most  100  pm.  In  this  sample,  the  concentration  of  the  fluorophore  is  zero  inside  the  glass  and  abruptly  reaches  a  high  concentration  when 
moving  the  probe  along  the  optical  axis.  (B)  During  the  experiment,  the  point  spread  function  penetrates  the  mounting  medium  through  a  coverslip.  The  mea¬ 
sured  fluorescence  intensity  signal  depends  on  the  penetration  depth.  The  fluorescence  emission  is  maximal  when  the  point  spread  function  is  completely  inside 
the  sample.  Because  the  point  spread  function  has  a  finite  size,  it  has  a  response  curve  with  a  finite  steepness.  The  slopes  close  to  the  interfaces  can,  therefore, 
be  used  to  determine  the  extent  of  the  point  spread  function  along  the  optical  axis,  and  this  is  the  axial  resolution  of  the  instrument. 


Aberrations  in  Confocal  and  Multi-Photon  Fluorescence  Microscopy  Induced  by  Refractive  Index  Mismatch  •  Chapter  20  409 


TABLE  20.1.  Result  of  Calculations  for  Glycerol3'4  TABLE  20.2.  Result  of  Calculations  for  Water3 


Axial 

Axial 

Lateral 

Edge 

PSF 

PSF 

NFP 

Focal  Shift 

Normalized 

FWHM 

FWHM 

FWHM 

(|4m) 

(pm) 

Intensity 

(pm) 

(pm) 

(pm) 

0 

0 

100 

0.53 

0.47 

0.16 

5 

-0.28 

95 

0.53 

0.47 

0.16 

10 

-0.55 

91 

0.53 

0.47 

0.16 

15 

-0.83 

78 

0.555 

0.47 

0.16 

20 

-1.10 

62 

0.65 

0.50 

0.18 

25 

-1.33 

50 

0.81 

0.57 

0.18 

30 

-1.54 

40 

0.97 

0.77 

0.20 

50 

-2.30 

31 

1.00 

0.72 

0.20 

"For  various  nominal  focal  positions  (NFP),  the  focal  shift,  the  normalized  inten¬ 
sity,  the  axial  edge  response,  the  axial  width  of  the  PSF,  and  the  lateral  width  of  the 
PSF  have  been  calculated  for  an  NA  =1.3  oil-immersion  objective  lens  and  excita¬ 
tion  and  emission  wavelengths  of  514nm  and  590  nm,  respectively. 

^The  values  for  an  NFP  =  0  are  the  ideal  values  if  immersion  oil  is  used  as  the  mount- 

ing  medium. 


Axial 

Axial 

Lateral 

Edge 

PSF 

PSF 

NFP 

Focal  Shift 

Normalized 

FWHM 

FWHM 

FWHM 

(pm) 

(pm) 

Intensity 

(pm) 

(pm) 

(pm) 

0 

0 

100 

0.53 

0.47 

0.16 

5 

-1.0 

60 

0.68 

0.58 

0.20 

10 

-1.83 

39 

0.90 

0.70 

0.20 

15 

-2.57 

28.50 

1.08 

0.88 

0.23 

20 

-3.30 

23 

1.22 

0.93 

0.24 

25 

-4.02 

19 

1.37 

1.00 

0.24 

30 

-4.72 

16.60 

1.45 

1.11 

0.24 

50 

-7.57 

11 

1.79 

1.37 

0.24 

"For  various  nominal  focal  positions  (NFP),  the  focal  shift,  normalized  intensity, 
axial  edge  response,  axial  width  of  the  PSF,  and  lateral  width  of  the  PSF  have  been 
calculated  for  an  NA  =1.3  oil-immersion  objective  lens  and  excitation  and  emis¬ 
sion  wavelengths  of  514nm  and  590  nm,  respectively.  The  values  for  an  NFP  =  0 
are  the  ideal  values  if  immersion  oil  is  used  as  the  mounting  medium. 

spherical  aberrations  are  below  a  certain  threshold,  the  focal  shift 
depends  linearly  on  the  NFP  which  can  be  used  to  correct  the 
apparent  thickness  of  a  sample.  If  the  spherical  aberrations  exceed 
the  threshold,  the  relation  between  the  NFP  and  the  focal  shift 
becomes  nonlinear  which  leads  to  image  distortions  along  the  z- 
axis  and  a  rapid  drop  in  intensity  as  the  excitation  and  emission 
PSF  will  be  substantially  displaced.  This  effect  is  most  prominent 
with  high  NAs  and  very  low  for  oil-immersion  lenses  having  an 
NA  of  <0.85. 

EXPERIMENTS 

To  verify  these  calculations,  Rhodamine  6G  was  dissolved  in 
water,  glycerol,  and  immersion  oil  to  form  ~1(T5M  solutions.  The 
solutions  were  mounted  between  a  glass  slide  and  a  coverslip  using 


droplets  of  dried  nail  polish  as  spacers.  The  samples  were  placed 
either  onto  a  home-built  confocal  microscope  (Stelzer  et  al.,  1991) 
[Figs.  20.6,  20.7(A),  20.8]  or  a  commercial  confocal  microscope 
(TCS  SP2,  Leica  Microsystems  Heidelberg,  Mannheim,  Germany) 
(Martini  et  al.,  2002)  [Fig.  20.7(B)].  The  dye  was  excited  at  a 
wavelength  of  488  nm  and  observed  either  above  530 nm  [Figs. 
20.6,  20.7(A),  20.8]  or  between  495  nm  and  530nm  [Fig.  20.7(B)]. 
For  the  data  presented  in  the  Figures  20.6,  20.7(A)  and  20.8,  a 
Zeiss  Apochromat  lOOx  oil  objective  with  an  adjustable  aperture 
(NA  0.8-1. 4)  was  used,  whereas  the  data  of  Figure  20.7(B)  was 
recorded  with  a  lOOx  adjustable  aperture  Leica  oil-immersion  lens 
(HCX  PL  APO  1.4-0. 7)  and  a  lOOx,  1.35  NA  Leica  glycerol- 
immersion  lens  (HCX  PL  APO,  GLYC  CORR).  The  instruments 
were  used  to  record  .rz-images,  that  is,  images  in  a  plane  parallel 
to  the  optical  axis  (Fig.  20.5).  These  images  started  with  the  focus 
in  the  coverslip  and  ended  with  the  focus  in  the  glass  slide.  The 
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FIGURE  20.6.  The  experimental  and  the  theoretical  edge-response  curves  for 
immersion  oil.  The  theoretical  curve  was  fitted  to  the  experimental  data  set  to 
comply  with  the  apparent  sample  thickness  and  the  maximum  intensity  in  the 
sample.  The  theoretical  calculations  assume  a  perfect  match  of  the  refractive 
indices  of  the  immersion  oil,  the  glass,  and  the  mounting  medium.  As  shown, 
this  condition  is  almost,  but  not  perfectly,  fulfilled  under  real  conditions.  The 
theory  predicts  a  slightly  steeper  slope. 


FIGURE  20.7.  The  experimental  and  the  theoretical  edge-response  curves  for 
glycerol.  Focusing  with  an  oil-immersion  lens  into  glycerol  leads  to  a  signifi¬ 
cant  drop  in  image  brightness  (A).  The  theoretical  curve  was  fitted  to  the  exper¬ 
imental  data  set  to  comply  with  the  apparent  sample  thickness  and  the 
maximum  intensity  in  the  sample.  The  intensity  decrease  is  perfectly  repro¬ 
duced.  (B)  By  using  a  glycerol-immersion  lens  the  drop  in  image  brightness 
can  be  prevented. 
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NFP  [pm] 

FIGURE  20.8.  Two  edge-response  curves  in  a  water  sample.  The  graph  shows 
the  intensities  as  a  function  of  NFP  for  NA  1.4  and  0.8.  Both  curves  were 
recorded  with  the  same  sample  in  the  same  position.  This  is  possible  because 
the  objective  lens  (Zeiss  Plan-Apochromat  lOOx,  NA  1.4)  has  an  adjustable 
NA.  The  figure  demonstrates  that  the  loss  in  resolution  and  the  loss  in  inten¬ 
sity  are  due  to  the  effects  of  high  NA  and  not  to  quenching  or  bleaching. 
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FIGURE  20.1 0.  Dependence  of  the  focal  shift  for  an  NA  =1.4  oil-immersion 
objective  lens  on  the  nominal  focal  position  for  various  refractive  indices 
between  1.33  and  1.49.  For  such  a  high  NA  objective  lens,  the  variation  is  only 
linear  up  to  a  certain  maximum  NFP,  which  strongly  depends  on  the  difference 
in  the  refractive  index  and  the  aperture  angle.  After  that,  the  focal  shift  starts 
to  show  some  kind  of  oscillation.  In  the  linear  regime  the  curves  can  be  used 
to  correct  the  apparent  thickness  of  a  sample  in  a  mounting  medium  with  a 
known  refractive  index  by  multiplying  it  by  an  appropriate  factor. 


64  central  columns  were  then  averaged  to  generate  the  edge- 
response  curves  (see  Eq.  6)  shown  in  Figures  20.6  through  20.8. 
These  graphs  resemble  intensity  signals  as  a  function  of  the  NFP. 
Of  interest  is  the  edge  steepness  on  both  sides  and  the  variation  in 
peak  intensity  with  depth.  The  results  are  summarized  in  Table 
20.3.  Because  it  was  not  possible  to  systematically  vary  the  thick¬ 
ness  of  the  dye  layer  (usually  between  50  pm  and  100  pm),  the 
focal  shift  was  not  measured  directly. 
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FIGURE  20.9.  Dependence  of  the  focal  shift  for  an  NA  =  0.6  oil-immersion 
objective  lens  on  the  nominal  focal  position  for  various  refractive  indices 
between  1.33  and  1.49.  For  such  a  low  NA  objective  lens,  the  variation  is  linear 
for  the  whole  range.  The  curves  can  be  used  to  correct  the  apparent  thickness 
of  a  sample  in  a  mounting  medium  with  a  known  refractive  index  by  multi¬ 
plying  it  by  an  appropriate  factor. 


OTHER  CONSIDERATIONS 
Dry  Objectives 

The  use  of  dry,  high-NA  lenses  for  the  observation  of  wet  speci¬ 
mens,  or  even  those  sealed  behind  glass  coverslips,  causes  aberra¬ 
tions  that  are  much  worse  than  for  any  of  the  situations  described 
above. 

First,  the  difference  between  the  refractive  indices  of  the 
immersion  and  embedding  media  is  larger  than  for  any  other  type 
of  objective.  Therefore,  focusing  even  a  few  micrometers  into  the 
sample  will  result  in  a  severe  drop  of  intensity  and  resolution.  Of 
course,  this  can  be  avoided  by  adjusting  the  correction  collar.  Most 
high-NA  dry  objectives  are  equipped  with  such  a  collar  but  the 
correct  adjustment  only  works  for  a  single  plane.  Every  change  of 
the  NFP  will  result  in  serious  degradation  of  the  PSF.  For  the  same 
reason,  dry  objectives  are  very  sensitive  to  the  coverslip  thickness. 

Second,  due  to  total  internal  reflection,  the  maximum  angle 
transmitted  as  the  emerging  light  moves  from  glass  to  air  is  41° 
and  49°  as  it  moves  from  water  to  air.  The  PSF  is  therefore  dom¬ 
inated  by  the  illumination  PSF.  The  situation  is  much  relaxed  when 
using  lenses  having  an  NA  of  0.6  or  less  because  the  angle  inside 
the  water  layer  is  then  26°  and  relatively  uncritical.  On  the  other 


TABLE  20.3.  Result  of  the  Experiments  with 
Water  and  Glycerol 


Water 
(NA  =  0.8) 

Water 
(NA  =  1.3) 

Glycerol 
(NA  =  1.3) 

Upper  edge  (pm) 

1.5 

1.0 

1.0 

Lower  edge  (pm) 

2.0 

5.7 

2.3 

Ratio  upper/lower 

1.3 

5.7 

2.3 

Aberrations  in  Confocal  and  Multi-Photon  Fluorescence  Microscopy  Induced  by  Refractive  Index  Mismatch  •  Chapter  20  411 


FIGURE  20.11.  Optical  spherical  aberration  correction.  Images  of  aqueous  PTK2  cells  stained  with  Oregon  green  anti-tubulin  and  made  using  an  NA  1.4  oil 
objective.  Replacing  microns  of  glass  by  water  introduces  distinct  spherical  aberrations  [(A)  xy  image;  (B)  xz  PSF;  (C)  raw  xz  image;  (D)  deconvolved  xz  image] 
that  are  not  present  in  the  corrected  images  [(E)  xy  image;  (F)  xz  PSF;  (G)  raw  xz  image;  (H)  deconvolved  xz  image].  The  loss  in  peak  intensity  and  contrast  is 
clearly  visible  in  the  images  of  the  same  xy  plane  (A,E).  The  zx  images  of  the  fluorescent  microsphere  show  the  asymmetry  in  the  z  direction  and  the  loss  in 
axial  resolution  (B,F).  As  a  result,  after  deconvolution,  tubulin  fibers  lying  on  top  of  each  other  can  be  clearly  distinguished  only  in  the  unaberrated  case. 


hand,  the  axial  resolution  improvement  produced  by  confocal 
microscopy  under  these  conditions  is  so  low  that  it  is  fair  to  ask  if 
it  makes  sense  to  use  such  a  microscope  under  these  circumstances. 

Refractive  Index,  Wavelength,  and  Temperature 

The  refractive  index  is  not  a  constant  but  depends  on  the  wave¬ 
length  (a  phenomenon  referred  to  as  dispersion)  and  on  the 
temperature  of  the  medium  (especially  with  liquids).  Standard 
immersion  oil  has  a  refractive  index  of  1.518  at  the  nc  line  of  the 
Hg  spectrum  (486 nm)  and  a  temperature  of  23°C,  but  performance 
away  from  these  standard  conditions  may  vary  depending  on  the 
manufacturer  of  the  oil.  Perfect  imaging  is,  therefore,  not  straight¬ 
forward  to  achieve.  Special  immersion  oils  and  gels  with  many  dif¬ 
ferent  RI  and  varying  temperature  behavior  are  available  through 
Cargille  Laboratories  (Cedar  Grove,  NJ). 

Spherical  Aberration  Correction 

The  loss  in  intensity  and  resolution  caused  by  RI  mismatch 
induced  spherical  aberrations  can  highly  degrade  the  imaging 
process.  Therefore,  it  is  advisable  to  reduce  the  RI  mismatch  as 
much  as  possible  (see  section  Consequences).  Unfortunately,  this 


is  not  always  possible  in  a  specific  experimental  setting.  For 
example,  when  using  dipping  water-immersion  objective  lenses 
together  with  a  coverslip,  the  spherical  aberrations  caused  by  the 
water-glass-water  transitions  induce  spherical  aberrations  that  are 
much  stronger  than  those  shown  in  the  preceding  examples.  For¬ 
tunately,  it  is  possible  to  cancel  out  aberrations  emerging  at  one 
point  in  the  imaging  process  by  introducing  negative  aberrations 
at  another  point.  For  example,  in  water-immersion  objective 
lenses,  this  is  done  inside  the  lens  and  the  spherical  aberration  can 
be  adjusted  using  a  correction  collar.  Of  course  this  balancing  can 
also  be  performed  anywhere  else  in  the  optical  path  by  either  using 
a  deformable  mirror  (Booth  et  al.,  2002),  or  a  spatial  light  modu¬ 
lator  or  altering  the  effective  tube  length  (Sheppard  and  Gu, 
1992b).  While  the  first  two  approaches  can  also  correct  for  non- 
spherical  aberrations,  the  latter  can  only  be  used  for  balancing 
spherical  aberrations.  Figure  20.11  shows  an  example  for  aberra¬ 
tion  correction  by  altering  the  effective  tube  length  through  a  com¬ 
mercially  available  system  (SAC,  Intelligent  Imaging  Innovations 
Inc.,  Denver,  CO). 

Replacing  micrometers  of  glass  by  water  results  in  pronounced 
spherical  aberration.  One  can  recognize  the  severe  loss  in  inten¬ 
sity  and  contrast  in  Figure  20. 11  (A, E),  an  xy  image  of  tubulin  in 
PTK2  cells  stained  with  Oregon  green.  The  axial  resolution  is  also 
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strongly  reduced  [Fig.  20. 11(B,F)],  an  effect  that  be  compensated 
for  by  deconvolution  only  approximately  (see  Chapter  24,  this 
volume).  While  in  the  aberrated  case,  it  is  difficult  to  distinguish 
individual  fibers  lying  near  each  other  [Fig.  20.11(C,D)],  they  are 
well  separated  in  the  aberration-free  images  [Fig.  20. 1 1(G,H)]. 

It  is  important  to  point  out  that  much  of  the  charm  of  correct¬ 
ing  spherical  aberration  by  moving  optical  components  that  are  not 
mounted  in  the  objective  lens  comes  from  the  fact  that  the  motion 
of  such  a  correcting  element  can  now  be  driven  by  a  motor.2  As 
all  optical  devices  that  change  the  spherical  correction  of  an  optical 
system  by  tube  length  alteration  also  change  its  focal  length, 
changing  the  correction  implies  creating  a  shift  in  the  focal  plane. 
However,  if  the  motion  of  the  corrector  is  driven  by  a  computer, 
this  same  computer  can  also  adjust  the  focus  control  in  a 
compensatory  direction  to  maintain  the  focus  plane.  The  SAC  is 
the  first  device  to  do  this  automatically.  One  hopes  that  such 
devices  may  soon  be  incorporated  into  microscope  stands  as  a 
standard  attachment,  similar  to  a  magnification  shifter  or  a 
Bertrand  lens. 

CONCLUSION 

The  effects  mentioned  in  this  chapter  are  so  important  that  most 
practicing  microscopists  will  have  already  encountered  them.  The 
effects  are  mainly  visible  when  using  lenses  with  a  high  NA  or, 
more  correctly,  when  using  lenses  whose  cone  angle  is  larger  than 
40°.  Some  of  the  older  confocal  microscopes  did  not  fill  the  back- 
focal  plane  completely  (see  Chapter  2,  this  volume)  and  the  higher 
angles  were  basically  not  present,  especially  when  using  high-NA 
objectives  of  relatively  low  magnification  (-40x),  as  such  lenses 
have  very  large  entrance  pupils.  Therefore,  it  can  be  assumed  that 
these  problems  will  become  more  obvious  as  the  quality  of  the 
instruments  improves  and  more  scientists  work  with  aqueous 
specimens. 

Consequences 

The  qualitative  relevance  of  these  experiments  for  the  observation 
of  thick  specimens  mounted  in  an  aqueous  medium  with  a  high- 
NA,  oil-immersion  objective  in  a  confocal  microscope  is  as 
follows: 

•  There  is  a  serious  loss  of  intensity  when  recording  optical  sec¬ 
tions  in  focal  planes  away  from  the  coverslip  and  this  loss 
becomes  greater  as  the  distance  from  the  coverslip  increases. 

•  The  PSF  suffers  from  a  serious  axial  smearing;  consequently, 
the  axial  resolution  is  lower. 

•  The  xy-resolution  is  also  lower  because  the  high-NA  rays  no 
longer  reach  the  focal  spot. 

•  Less  fluorescence  intensity  is  detected  behind  the  confocal 
pinhole. 

•  The  distances  measured  along  the  optical  axis  are  smaller  than 
the  geometrical  distances  in  the  object.  The  benchmark  to 
remember  is  that  in  water,  the  focal  shift  is  13A.  at  a  penetra¬ 
tion  depth  of  KXVi. 


2  Other  methods  proposed  for  the  automatic  correction  of  spherical  aberrations 
include  deformable  mirrors  and  spatial  light  modulators.  Like  the  SAC,  these 
devices  would  be  introduced  after  the  objective,  where  their  operation  could 
be  computer  controlled  and  any  adjustments  could  be  coupled  with  any 
changes  in  stage  position  that  might  be  required. 


•  In  a  mismatched  system,  the  only  part  of  the  specimen  that  can 
be  observed  without  the  severe  effects  described  above  is  that 
immediately  next  to  the  coverslip. 

•  The  problem  is  encountered  in  every  single-photon  confocal 
and  two-photon  scanning  microscope. 

The  physical  relevance  of  these  experiments  is  that,  in  the  case 
of  a  noticeably  mismatched  sample,  a  single  PSF,  or  a  single 
optical  transfer  function  (OTF)  is  not  valid  throughout  the  sample. 
As  the  PSF  or  OTF  can  be  defined  only  locally,  a  global  decon¬ 
volution  to  remove  the  effect  of  spherical  aberration  using  the 
system  response  is  unlikely  to  be  successful. 

This  chapter  also  shows  that  an  aberrated  PSF  can  be  accu¬ 
rately  calculated  from  an  initial  unaberrated  PSF.  The  deviations 
follow  an  approximately  linear  relationship,  and  a  simple  linear 
expansion  is  likely  to  be  able  to  describe  the  PSF  in  different  loca¬ 
tions  of  the  object  fairly  well.  However,  this  does  not  mean  that 
the  effect  can  be  easily  corrected.  If  not  counteracted  optically,  the 
loss  in  signal  level  and  resolution  cannot  be  recalled. 

Practical  Strategies  to  Reduce  Refractive 
Index  Mismatch 

The  biologist  can  avoid  these  problems  by  adhering  to  the  fol¬ 
lowing  guidelines: 

•  Always  use  the  appropriate  objective  lens.  High  resolving 
objective  lenses  like  NA  1.4  oil-immersion  lenses  are  usually 
designed  to  be  used  for  thin  specimens  mounted  behind  a 
coverslip.  For  other  experimental  settings,  like  imaging 
through  the  bottom  of  a  petri  dish,  there  are  other  more  appro¬ 
priate  lenses  available,  particularly  those  that  incorporate  a 
correction  collar  for  spherical  aberration  minimization.  These 
should  always  be  properly  adjusted. 

•  Thick  specimens  should  only  be  observed  in  matched  systems. 
This  means  the  RI  of  the  mounting  medium  and  the  immer¬ 
sion  medium  should  be  identical  at  all  wavelengths  of  interest. 
Therefore,  water-immersion  and  glycerol-immersion  lenses 
should  be  used  for  water-  and  glycerol-embedded  specimens, 
respectively. 

•  Avoid  situations  in  which  the  sample  is  thin  but  far  away  from 
the  coverslip.  This  often  happens  when  the  object  is  either 
attached  permanently  to  the  glass  microscope  slide  or  when 
viewing  cells  grown  on  filters  but  separated  from  the  protec¬ 
tive  coverslip  by  an  unknown  amount  of  aqueous  medium. 

•  If  you  cannot  match  the  RI  of  the  mounting  medium  to  that  of 
the  immersion,  try  to  get  its  refractive  index  as  similar  as  pos¬ 
sible.  This  can  be  accomplished  by  preparing  mixtures  of  glyc¬ 
erol  and  water  or  by  adding  large  amounts  of  dextrose  to  the 
mounting  medium.  This  will  at  least  reduce  the  amount  of 
spherical  aberrations  incident  at  a  certain  focusing  depth. 

•  For  unmatched  systems,  increase  the  size  of  the  detection 
pinhole  as  a  function  of  the  penetration  depth.  This  will  not 
improve  the  PSF,  but  it  will  increase  the  detectable  signal 
without  any  further  losses  in  resolution,  and  at  least  the  bright¬ 
ness  will  not  decrease  so  much  as  the  image  series  extends 
farther  from  the  coverslip  surface. 

•  If  you  have  to  image  deeper  into  unmatched  samples  and  no 
appropriate  lens  is  available,  use  a  lower  NA  immersion  objec¬ 
tive.  As  demonstrated  in  Figure  20.8,  for  small  mismatches,  the 
loss  in  intensity  is  much  less  below  NA  0.85.  However,  the 
lower  NA  has  its  own  disadvantages:  lower  light  collection  and 
lower  resolution.  As  shown  in  Table  20.3,  looking  through 
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50|dm  of  aqueous  specimen  with  an  NA  1.4  oil  lens  decreases 
the  axial  resolution  by  a  factor  of  6  and  the  intensity  becomes 
as  low  as  11%.  If  you  use  instead  a  NA  0.8  lens,  the  axial 
resolution  at  the  surface  will  be  (0.8/1.4)2  =  3  times  worse  and 
only  36%  of  the  fluorescence  photons  of  the  oil  lens  will  be 
collected  due  to  the  smaller  aperture  angle.  At  an  imaging  depth 
of  50  pm  things  will  have  changed:  The  peak  light  intensity  will 
drop  to  only  about  33%,  which  is  about  the  same  signal  as  we 
would  get  with  the  oil  lens  (36%  *  33%  =  12%)  and  the  reso¬ 
lution  will  be  6  /  (3  *  1 .3)  =  1 .5  times  that  of  the  oil  lens.  Clearly, 
this  is  an  improvement,  and  the  advantage  becomes  even  more 
prominent  as  the  sample  becomes  thicker.  On  the  other  hand, 
imaging  through  a  20pm  layer  of  water  gives  about  the  same 
image  with  either  lens,  and  any  features  closer  to  the  coverslip 
will  be  imaged  better  by  the  larger  NA  lens.  Of  course,  one 
must  keep  in  mind  that  imaging  through  ~10pm  of  water  with 
an  oil  immersion  lens  leads  to  a  seriously  aberrated  PSF. 

•  Spherical  aberration  (and  not  absorption)  is  the  major  source 
of  the  reduction  in  fluorescence  signal  with  focus  depth  com¬ 
monly  noted  by  most  practicing  confocal  microscopists. 

•  Optimal  observation  of  thick  living  specimens  occurs  when  the 
sample  is  observed  using  water-immersion  objectives  and  no 
coverslip.  Upright  microscopes  are  best  suited  for  this  purpose 
because  the  water  is  then  less  likely  to  leak  into  the  objective. 
However,  the  absence  of  a  coverslip  may  cause  the  specimen 
to  move  as  the  focus  plane  changes  to  record  3D  data. 
Under  these  conditions,  one  may  be  able  to  use  either  water- 
immersion  lenses  corrected  for  use  with  a  coverslip  (see 
Chapter  7,  this  volume )  or  CYTOP  plastic  coverslips,  which 
have  n  =  1.34,  and  as  a  result  can  be  used  with  normal  water- 
immersion  lenses. 

•  If  the  experimental  setting  does  not  allow  for  spherical  aber¬ 
ration  minimization,  think  about  aberration  balancing  inside 
the  optical  light  path  by  introducing  a  spherical  aberration  cor¬ 
rection  device. 
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Interaction  of  Light  with  Botanical  Specimens 

Ping-Chin  Cheng 


INTRODUCTION 

In  recent  years,  plant  biotechnology  has  become  an  important 
sector,  with  the  agricultural  and  pharmaceutical  industries  pro¬ 
moting  research  activities  in  plant  sciences.  This  has  placed 
increased  demands  on  sophisticated  optical  microscopy  to  assist 
plant  research.  While  most  of  the  discussion  of  confocal  and  multi¬ 
photon  fluorescence  microscopy  concentrates  on  the  imaging  of 
animal  tissues  and  cells,  very  little  attention  has  been  paid  to  the 
imaging  of  botanical  specimens.  As  a  result,  plant  researchers  fre¬ 
quently  have  to  rely  on  the  imaging  technology  developed  pri¬ 
marily  from  animal  work.  While  there  is  nothing  wrong  with  using 
suitable  technology  developed  for  other  systems,  in  this  case 
allowance  must  be  made  for  the  optical  characteristics  of  botani¬ 
cal  specimens.  Unlike  animal  cells,  plant  cells  are  designed  to 
interact  with  light,  either  to  harvest  energy  or  to  protect  sensitive 
organelles.  As  a  result,  plant  cells  are  generally  more  optically 
active  and  heterogeneous  and  contain  significantly  more  absorb¬ 
ing,  scattering,  and  fluorescent  structures.  In  addition,  many  botan¬ 
ical  specimens  exhibit  birefringence  and  are  also  capable  of 
generating  strong  second  and  third  harmonic  signals. 

The  specimen  is  an  integral  part  of  any  microscope  system  and  its 
optical  characteristics  directly  affect  the  performance  of  the  micro¬ 
scope.  To  image  botanical  specimens  effectively,  one  must  first 
understand  their  optical  properties.  Plant  tissues  are  generally 
bulkier  than  their  animal  counterparts  and  their  cells  are  usually 
larger.  As  most  plant  cells  contain  a  significant  number  of  fluores¬ 
cent  molecules  tightly  coupled  to  the  photosynthetic  system, 
they  interact  with  the  illumination  very  differently  from  animal 
cells. 

Optical  microscopy  is  likely  to  cause  significant  disturbances 
to  the  physiology  of  plant  cells.  For  example,  the  autofluorescence 
intensity  and  spectra  varies  with  the  duration  and  intensity  of  the 
illumination  (Cheng  et  al.,  2000a;  2001).  This  chapter  discusses 
the  various  optical  and  fluorescence  properties  that  affect  the 
optical  microscopy  of  plant  specimens.  A  number  of  plant  tissues, 
both  photosynthetic  and  non-photosynthetic,  will  be  used  to  illus¬ 
trate  the  imaging  characteristics  of  botanical  specimens,  including 
their  ability  to  produce  useful  and  informative  contrast  using 
second  and  third  harmonic  generation  signals  (SHG  and  THG).  In 
addition,  the  methods  suitable  for  imaging  various  plant  organs  and 
tissues  are  discussed. 


LIGHT  ATTENUATION  IN  PLANT  TISSUE 

The  transmission  of  light  incident  on  plant  tissue  is  mainly  deter¬ 
mined  by  the  amount  of  absorption  and  scattering  in  the  specimen. 
Because  it  is  technically  difficult  to  separate  these  two  factors,  we 
will  refer  to  their  joint  effects  as  attenuation.  In  this  article,  we  esti¬ 
mate  attenuation  using  a  detector  equal  to  the  size  of  a  parallel 
illumination  beam  and  placed  behind  a  specimen  with  a  detecting 
solid  angle  no  larger  than  5°.  The  measured  transmission  intensity 
(4ran)  is  approximated  as 

Aran  A)  Alba  A 

where  I0  is  the  incident  intensity,  7abs  is  the  intensity  lost  due  to 
absorption,  and  Is  is  the  decrease  of  transmitted  intensity  due  to 
light  scattering  out  of  the  beam. 

Linear  Absorption 

As  water  is  the  most  abundant  material  in  living  systems,  light 
absorption  of  water  determines  the  usable  spectral  range  of  optical 
microscopy  of  living  material.  Figure  21.1  shows  the  absorption 
spectrum  of  water  from  200  to  1400  nm.  Note  that  water  is  trans¬ 
parent  at  the  shorter  wavelengths,  with  two  minor  absorption  peaks 
at  approximately  1000  and  1200nm,  but  becomes  opaque  at 
wavelengths  longer  than  1300nm.  Besides  water,  botanical  tissue 
consists  of  organic  molecules  similar  to  those  found  in  animal 
tissue,  plus  cellulose,  chlorophylls,  anthocyanins,  flavonols,  lignin, 
sporopollenin,  polyesters  (cuticle),  starch,  inorganic  deposits,  and 
other  light  absorbing  pigments.  The  pigments  can  play  havoc  with 
labeling  studies  while  cellulose  structures  scatter  and  refract  the 
light.  Fortunately,  the  latter  problem  is  really  a  cloud  with  a  “silver 
lining”  because  such  structures  are  also  very  efficient  at  produc¬ 
ing  SHG  and  THG  signals. 

Figure  21.1  shows  the  attenuation  spectra  of  plant  tissue  (rice 
leaf,  Oryza  sativa)  and  animal  tissue  (chicken  dermis).  Compared 
to  animal  tissue,  plant  tissue,  particularly  photosynthetic  tissue,  has 
significantly  higher  absorbance  in  the  visible  spectrum  but  similar 
attenuation  in  the  near  infrared  (NIR)  range.  It  is  evident  that  the 
most  transparent  region  of  both  animal  and  plant  tissue  is  around 
1300nm.  Figure  21.2  shows  the  attenuation  spectra  of  a  number 
of  other  plant  tissues  including  maize  leaf  (green),  etiolated  maize 
leaf  (yellow),  carrot  (orange),  and  rose  petal  (red). 
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FIGURE  21 .1 .  Absorption  spectrum  of  water  (A)  and  attenuation  spectra  of  living  rice  leaf  ( Oryza  sativa  L.)  and  chicken  dermis  (B).  The  low  absorption  (ji  < 
0.5)  window  of  water  between  200  to  1300nm  determines  the  useful  range  of  optical  microscopy  for  living  biological  specimens.  Water  becomes  opaque  for 
wavelengths  longer  than  1300nm.  Note  that,  while  the  photosynthetic  pigments  in  plant  tissue  produce  additional  absorption  peaks  in  the  blue  and  red  region 
not  found  in  animal  tissue,  both  types  of  specimen  show  little  attenuation  in  the  NIR.  The  blue  and  green  lines  show  the  typical  operating  wavelengths  of 
Ti:  sapphire  and  Cr:forsterite  lasers,  respectively.  The  colored  arrows  indicate  the  upper  limit  of  two-  and  three-photon  excitation.  The  gray  bars  indicate  the 
tuning  range  of  the  respective  lasers. 
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FIGURE  21.2.  Attenuation  spectra  of  various  plant  petals  and  leaves. 
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Using  Arabidopsis  thaliana  leaf  as  an  example,  Figure  21.3(A) 
is  an  attenuation  spectrum  showing  a  general  increase  in  optical 
density  towards  the  shorter  wavelengths.  Two  prominent  absorp¬ 
tion  peaks  are  contributed  by  leaf  pigmentation,  mainly  the  chlo¬ 
rophylls.  The  attenuation  spectrum  of  an  acetone-extracted, 
pigment-free  leaf  shows  the  removal  of  both  peaks  and  lower 
overall  attenuation  [Fig.  21.3(B)].  Figure  21.3(C)  shows  the  atten¬ 
uation  of  protoplasts  in  culture  medium  (ca.  2  X  104  cells/mL). 
Because  the  light-scattering  cell  wall  is  now  absent,  the  attenua¬ 
tion  at  400  nm  is  very  similar  to  that  at  1000  nm.  However,  in  the 
more  fibrous  Marsilea  quadrifolia  leaf,  where  scattering  is  signif¬ 
icant,  removal  of  the  pigments  does  little  to  lower  the  light  atten¬ 
uation  (Fig.  21.4).  It  is  worth  noting  that,  over  the  spectral  range 
of  520  to  850  nm,  the  light  attenuation  is  very  similar  in  the  leaves 
of  both  species.  These  spectra  show  that  using  NIR  (700-900  nm) 
illumination  for  multi-photon  microscopy  offers  only  a  limited 
penetration  advantage  over  the  shorter  wavelength  light  used  in 
single-photon  excitation.  Although  a  significant  increase  in  pene¬ 
tration  depth  can  be  gained  by  using  wavelengths  in  the  1200nm 
range  (Fricker  et  al .,  1992;  White  et  al.,  1996;  Lin  et  al.,  2000c, 
2001;  Chu  et  al.,  2003a),  resolution  will  be  lower  because  of  the 
longer  wavelength  and  because  most  objectives  are  corrected  only 
for  visible  light.  On  the  other  hand,  if  the  refractive  index  (RI)  of 
a  sample  does  not  match  that  of  the  medium  in  which  it  is 
immersed,  the  use  of  a  longer  wavelength  often  reduces  the  aber¬ 
rations  caused  by  this  mismatch  (because  all  RIs  drop  with  wave¬ 
length  and  the  differences  between  the  RIs  of  the  different 
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FIGURE  21 .3.  Attenuation  spectra  of  (A )  Arabidopsis  thaliana  protoplast,  (B) 
water-logged  whole  leaf  and  EtOH-extracted  leaf,  and  (C)  acetone-extracted 
leaf. 


FIGURE  21 .4.  Attenuation  spectra  of  a  living  leaf  of  Marsilea  quadrifolia  and 
after  ethyl  alcohol  extraction. 


components  become  smaller),  and  this  factor  compensates  some¬ 
what  for  the  loss  of  resolution. 

In  addition  to  the  700  to  1300nm  region,  green  plant  tissue 
shows  a  secondary  low  attenuation  window  around  550  nm.  Select¬ 
ing  a  flurophore  with  an  emission  peak  within  this  region  improves 
the  detectable  fluorescence  signal  as  long  as  there  is  no  overlap¬ 
ping  autofluorescence.  In  addition,  the  550 nm  region  is  suitable 
for  backscattered  light  (BSL)  imaging  within  the  visible  spectrum. 
Fluorochromes  with  their  fluorescence  emission  longer  than 
650nm,  such  as  Alexa  Fluor  680,  Alexa  Fluor  750,  Cy5.5,  and  Cy7 
(Molecular  Probes  Inc.),  can  be  very  useful  for  botanical  speci¬ 
mens.  In  this  spectral  region,  the  absorption  of  plant  material  is 
low,  relatively  little  autofluorescence  is  produced,  and  the  signal 
is  still  suitable  for  high  quantum  efficiency  (QE),  Si-based 
photodetectors. 


Nonlinear  Absorption 


In  nonlinear  optics,  the  nonlinear  optical  response  can  be  described 
by  the  power  series 


Pit)  =  e0%(1)£(0  +  ^oXa)E(t)2  +  e„)c,3)£(03  +  •  •  • 

=  p(1)(0  +  P(2)(0  +  p(3)(0  + . . . 


where  /<2)  is  the  the  second-order  nonlinear  susceptibility  and  %<3) 
is  the  third-order  nonlinear  susceptibility. 

Although  the  nonlinear  optical  response  of  man-made  materi¬ 
als  has  been  extensively  studied,  the  nonlinear  absorption  proper¬ 
ties  of  plant  specimens  are  poorly  understood.  Figure  21.5  shows 
a  transmission  versus  incident  intensity  curve  of  a  flurophore 
(APSS;  Cheng  et  al.,  1998)  at  780nm.  Although  the  absorption 
increases  as  the  incident  power  increases,  one  should  note  that  a 
specimen  showing  little  absorbance  in  the  NIR  may  exhibit  sig¬ 
nificant  nonlinear  (two-photon)  absorption  at  high  peak  illumina¬ 
tion  intensity,  and  significant  energy  can  thus  be  absorbed  by  the 
specimen.  This  is  the  case  for  the  chloroplast.  Even  though  the 
APSS  fluorophore  shows  low  absorbance  in  the  NIR  region,  sig¬ 
nificant  two-photon  absorption  is  evident.  The  shaded  bar  in  Figure 
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FIGURE  21.5.  Nonlinear  absorption  of  APSS  fluorophore  solution.  The  light 
blue  bar  represents  the  range  of  power  densities  used  in  a  typical  multi-photon 
fluorescence  microscopy  setup.  In  general,  higher  power  density  occurs  when 
a  high-NA  objective  is  used  because  the  focal  spot  is  smaller. 


21.5  represents  the  power  density  used  in  a  typical  multi-photon 
fluorescence  microscope.  Under  high  power  density  IR  illumina¬ 
tion,  significant  cellular  damage  can  result  (Konig  et  al.,  1995, 
1996a, b,  1997;  Oehring  et  al.,  2000;  Chapter  38,  this  volume ;  and 
see  paragraph  below  on  light-specimen  interactions). 


Scattering 

Both  Rayleigh  and  Mie  scattering  occur  in  botanical  specimens. 
Rayleigh  scattering  is  produced  by  small  particles  (less  than  a  tenth 
of  the  wavelength)  and  is  more  effective  at  shorter  wavelengths; 
the  intensity  of  the  Rayleigh-scattered  light  I  is 


1  = 


8jt4Mx2 

X 4R2 


(1  +  COS20) 


where  N  is  the  number  of  scatters,  a  is  the  polarizability,  and  R  is 
the  distance  from  the  scatterer. 

Clearly,  Rayleigh  scattering  is  highly  wavelength  dependent: 


Therefore,  the  wavelength-dependent  attenuation  observed  in  the 
spectrum  (Fig.  21.6)  is  mainly  due  to  Rayleigh  scattering. 

In  contrast,  Mie  scattering  predominates  for  particles  that  are 
larger  than  a  wavelength.  This  scattering  produces  a  pattern  similar 
to  an  antenna  lobe,  with  a  sharper  and  more  intense  forward  lobe 
for  larger  particles.  Mie  scattering  is  generated  mainly  by  the 
organelles  in  the  cell  and,  while  not  strongly  wavelength  depen¬ 
dent,  the  scattering  intensity  is  proportional  to  the  square  of  the 
ratio  of  the  RI  of  the  feature  to  the  RI  of  the  media  (see  Fig.  8.2, 
this  volume).  Figure  21.6  shows  the  contribution  of  scattering  to 
the  light  attenuation  in  hydrated  and  methyl  salicylate-cleared 
parenchyma  cells  in  a  maize  stem  (Cheng  et  al.,  2001a).  By  index¬ 
matching  many  of  these  small  particles,  the  clearing  process 
greatly  reduces  scattering  within  a  specimen.  This  allows  one  to 
image  significantly  deeper  into  the  tissue. 

In  a  simulation  of  biological  specimens,  Gu  and  colleagues 
(2000)  have  demonstrated  that  Mie  scattering  is  the  dominant  scat¬ 


tering  event  in  milk  and  that,  as  long  as  spherical  aberration  is  not 
a  problem,  scattering  is  the  determining  factor  for  the  depth  limit 
in  multi-photon  microscopy.  Figure  21.7  shows  a  high-intensity 
NIR  beam  incident  on  a  1  mm  cuvet  containing  diluted  (20% 
dilution)  2%  homogenized  milk;  note  the  two-photon  excited  flu¬ 
orescence  in  the  second  cuvet  and  the  scattered  NIR  in  the 
milk-containing  cuvet. 

Between  the  outer  layers  of  epidermal  cells,  leaves  consist 
mainly  of  mesophyll  tissue,  each  cell  being  rich  in  photosynthetic 
pigments  confined  within  light-scattering  cell  walls.  The  outer 
surface  of  epidermal  cells  is  generally  covered  with  a  layer  of 
cuticle  that  has  a  refractive  index  significantly  different  from  that 
of  water.  As  a  result,  the  cuticle  surface  reflects  incident  light  par¬ 
ticularly  that  impinging  at  low  angles,  preventing  some  of  it  from 
entering  the  leaf  and  effectively  reducing  the  NA  of  the  objective. 
The  cuticle  surface  may  be  covered  with  a  complex  surface  pattern 
(Cheng  et  al.,  1979a,  1981,  1986;  Cutler  et  al.,  1982),  which 
further  increases  scattering  and  lowers  the  intensity  of  the  illumi¬ 
nation  penetrating  the  tissue. 

Specialized  cells  on  the  leaf  surface,  such  as  heavily  walled 
guard  cells  and  silica  cells  (frequently  found  in  grasses;  Jones  and 
Handreck,  1967),  provide  additional  scattering  centers  that  can 
also  significantly  attenuate  the  illumination.  Finally,  a  significant 
volume  of  the  leaf  is  occupied  by  an  air  space  that  becomes  the 
major  reflection  and  scattering  center  within  the  tissue. 

One  method  commonly  used  to  combat  these  problems  in 
freshly  excised  tissue  is  to  soak  the  leaf  in  air-deficient  culture 
medium/water  (Cheng  et  al.,  2001a)  or  to  place  the  leaf  in  water 
but  under  a  partial  vacuum  to  remove  the  air  trapped  in  the  meso¬ 
phyll.  Air-deficient  water  is  prepared  by  filling  a  bottle  with  boiling 
water,  recapping  the  bottle  when  the  water  is  hot  and  without  any 
trapped  air  bubbles,  and  then  letting  the  water  cool  down.  The  bub¬ 
bling  and  high  temperature  removes  dissolved  air.  Soaking  leaves 
in  this  air-deficient  water  can  quickly  and  efficiently  dissolve  away 
air  bubbles  trapped  in  tissue  without  the  damage  caused  when  the 
air  trapped  in  the  leaf  expands  under  vacuum.  Figure  21.8  shows 


FIGURE  21.6.  Attenuation  spectra  of  maize  stem  (pith)  in  water,  xylene,  and 
methyl  salicylate  (after  dehydration  with  EtOH).  All  spectra  were  obtained 
from  the  same  spot  on  the  sample. 
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FIGURE  21.7.  Demonstration  of  high  penetration  of  NIR  beam  in  turbid  medium.  High  intensity  IR  beam  incident  on  a  cuvet  filled  with  diluted  milk.  The 
ballistic  photons  leaving  the  milk  are  intense  enough  to  excite  2P  fluorescence  in  the  dye  solution  in  the  second  cuvet. 


significantly  less  scattering  from  waterlogged  leaves  and  this 
allows  an  increase  in  imaging  depth. 

If  living  tissue  observation  is  not  required,  the  high  scattering 
of  botanical  specimens  can  be  minimized  by  fixation,  dehydration, 
and  clearing  procedures  commonly  used  in  traditional  microtech¬ 
niques.  Figure  21.6  demonstrates  the  effect  of  such  a  clearing  pro¬ 
cedure  on  the  scattering  properties  of  the  parenchyma  cells  of 
maize  stem  (pith).  Untreated,  these  cells  have  no  pigmentation  and 
appear  white  to  the  naked  eye  (mainly  due  to  Mie  scattering). 
However,  after  clearing,  the  tissue  becomes  nearly  transparent.  The 
absence  of  the  absorption  peak  at  1450  nm  is  due  to  the  absence 
of  water. 


Optical  Density  of  Leaf 


FIGURE  21 .8.  Attenuation  spectra  of  a  live  maize  leaf  before  and  after  being 
waterlogged.  Note  the  significant  decrease  in  attenuation  [nearly  1  optical 
density  (OD)  unit  in  the  red]  due  to  the  elimination  of  the  air  spaces  in  meso- 
phyll  tissue.  This  significantly  improves  the  penetration  of  the  excitation  light 
in  single-photon  excitation  and  also  greatly  increases  the  returning  fluorescence 
signal. 


REFRACTIVE  INDEX  HETEROGENEITY 

Any  optical  heterogeneity  within  the  specimen  or  deviation  in  the 
refractive  index  from  the  design  specification  of  the  objective  can 
produce  spherical  aberration  (Chapter  20,  this  volume).  Structures 
having  different  refractive  indexes  can  also  affect  the  flatness  of 
an  optical  section.  Recently,  spherical  aberration  introduced  by  a 
living  specimen  has  focused  attention  on  the  need  to  process  spec¬ 
imens  properly  to  minimize  refractive  index  mismatches  within 
them  (Cheng  and  Cheng,  2001;  Cheng  et  al.,  2002;  Pawley,  2002). 

Examples  of  optical  heterogeneity  in  plant  specimens  can  be 
demonstrated  by  laying  maize  tissue  sections  on  the  surface  of  a 
slide  made  of  fluorescent  plastic  (Cheng  et  al.,  2002).  Figure  21.9 
shows  an  xz  image  of  such  a  specimen.  Refractive  index  hetero¬ 
geneity  within  the  tissue  slice  causes  the  image  of  the  surface  of 
the  fluorescent  slide  not  to  be  flat  as  it  should  be  (marked  by 
horizontal  line).  Figure  21.10  shows  xy,  xz,  and  yz  views  of  a 
submerged  leaf  of  Marsilea  quadrifolia  using  two-photon  fluores¬ 
cence  microscopy.  Note  the  smearing  of  the  image  in  the  deeper 
part  of  the  tissue  (xz  and  yz)  as  the  result  of  specimen-introduced 
spherical  aberration.  Apparently,  the  depth  limit  is  around  50  to  60 
|im  in  this  case.  Inspection  of  the  xz  and  yz  sections  reveals  that 
the  lower  left  portions  of  the  xy  image  were  obtained  near  the 
surface  of  the  specimen  while  the  upper  right  portion  of  the  image 
was  obtained  through  30  pm  of  tissue.  Significant  differences  in 
image  resolution  and  signal  level  are  evident. 

Historically,  this  heterogeneity  is  overcome  by  dehydration 
and  by  clearing  and  mounting  the  tissue  sections  between  a  slide 
and  coverslip  in  proper  mounting  medium.  This  procedure  matches 
the  optical  properties  of  the  specimen  to  the  design  conditions  of 
the  objective  lens,  and  should  not  be  overlooked  in  modem  wide- 
field,  confocal,  and  multi-photon  microscopy.  When  imaging  fixed 
tissues  and  cells,  always  use  a  clearing  agent  to  reduce  scattering 
and  aberrations. 

A  number  of  clearing  methods  are  used  in  optical  microscopy, 
most  of  them  involving  the  use  of  organic  solvents  such  as  methyl 
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FIGURE  21 .9.  Optical  xz  section  of  a  piece  of  maize  stem  (pith)  mounted  on  a  flu¬ 
orescent  plastic  slide  (Chroma  Technology,  Inc.,  and  AMIL  Technology)  showing  how 
the  optical  heterogeneity  of  the  tissue  section  dramatically  affects  the  image  of  the 
homogeneous  fluorescent  plastic  slide.  The  horizontal  line  demarcates  the  slide 
surface.  Note  that  the  image  of  the  surface  of  the  plastic  slide  is  not  flat  because  RI 
variations  in  the  overlying  section  cause  the  focal  length  of  the  relatively  low-NA 
objective  to  vary. 
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FIGURE  21.10.  Two-photon  xy,  xz, 
and  yz  sections  of  young  leaf  (the 
submerged  form)  of  M.  quadrifolia. 
The  reduction  of  signal  strength  and 
image  quality  as  a  function  of  depth 
is  evident.  The  white  lines  on  the  xy 
section  indicate  the  positions  where 
the  xz  and  yz  sections  were  obtained; 
similarly,  the  white  lines  on  the  xz 
and  yz  images  indicate  the  corre¬ 
sponding  position  of  the  xy  image. 
The  image  was  obtained  by  two- 
photon  fluorescence  microscopy 
of  a  water-mounted  spe-cimen  using 
800  nm  excitation  with  a  water- 
immersion  objective  on  Olympus 
Fluroview  FV300  confocal  micro¬ 
scope. 
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salicylate  (r)  =  1.52),  xylene  (r|  =  1.5055),  or  toludine  (r)  =  1.4961) 
(Stelly  et  al .,  1984;  Cheng  et  al,  1993).  Examples  using  methyl 
salicylate  include  the  study  of  ovules  of  Zepliyranthaceae  (Crane 
and  Carman,  1987),  grass  ovules  (Young  et  al.,  1979),  and  maize 
apical  meristem  (Bommineni  et  al.,  1990,  1993,  1995).  For 
hydrated  tissue,  aqueous  based  glycerol-Ppda,  glycerol  (r)  = 
1.4746)  and  recently  FocusClear  (r)  =  1.43)  and  MountClear  (r)  = 
1 .43;  Pacgen,  BC,  Canada)  can  be  used  in  conjunction  with  Immer¬ 
sion  Solution-M  (r|  =  1.43).  All  these  methods  work  well  with 
botanical  specimens. 

In  addition  to  the  agents  mentioned  above,  several  clearing 
techniques  have  been  developed  for  the  study  of  whole-mount 
plant  tissue  in  the  past  three  decades.  These  include  the  BB-4  1/2 
clearing  fluid  (Herr,  1971),  the  improved  BB-4  1/2  (Herr,  1974), 
and  the  mixture  of  benzyl  benzoate  and  dibutyl  phthalate  (2:1,  v/v. 
Crane  and  Carman,  1987).  Examples  of  the  applications  of  these 
clearing  agents  have  appeared  in  many  botanical  publications,  such 
as  the  use  of  Herr's  BB-4  1/2  for  the  study  of  ovules,  pollen,  and 
pollen  tubes  (Fredrikson,  1992),  and  the  use  of  benzyl  benzoate 
and  dibutyl  phthalate  mixture  in  the  study  of  apomixis  in  Elymus 
(Crane  and  Carman,  1987).  Both  Herr’s  BB-4  1/2  and  BB-4  1/2 
fluids  turn  dark  brown  after  2  to  4  weeks  in  the  light  at  room 
temperature  because  the  clove  oil  photo-oxidizes  (Herr,  1992). 
Because  the  browning  results  in  a  significant  increase  in  light 
absorbance,  it  is  important  to  keep  the  clearing  agents  in  dark 
bottles  and  in  the  refrigerator.  Substituting  the  clove  oil  with 
dibutyl  phthalate  produces  a  superb  photostable  clearing  agent 
(Herr’s  BB-DP-4  1/2;  Herr,  1992). 

BIREFRINGENT  STRUCTURES  IN 
PLANT  CELLS 

In  crystalline  materials,  it  is  well  known  that  different  refraction 
indices  may  be  associated  with  different  crystallographic  orienta¬ 
tions.  Mineral  crystals  frequently  show  two  distinct  indices  of 


refraction,  and  are  referred  to  as  birefringent  materials.  If  the  y- 
and  z-directions  are  equivalent  in  terms  of  the  crystalline  forces, 
then  the  .r-axis  is  unique  and  is  called  the  optic  axis  of  the  mater¬ 
ial.  The  propagation  of  light  along  the  optic  axis  is  independent  of 
its  polarization:  its  electric  field  is  everywhere  perpendicular  to  the 
optic  axis  and  is  called  the  ordinary-  or  o-wave.  The  light  wave 
with  its  E-field  parallel  to  the  optic  axis  is  called  the  extraordinary- 
or  e-wave. 

Birefringence  has  to  do  with  anisotropy  in  the  binding  forces 
between  the  atoms  forming  the  crystal.  It  can  be  visualized  as  the 
atoms  having  stronger  “springs”  holding  them  together  in  some 
crystalline  directions  than  in  others.  A  number  of  structures  in  plant 
tissue  exhibit  birefringence.  These  include  microtubules,  spindles, 
secondary  cell  wall,  cuticle,  surface  wax,  starch  granules,  and  Si02 
deposits  (bio-opals).  For  example,  paracrystalline  cellulose  in  the 
cell  walls  of  trichomes  of  Arabidopis  thaliana  shows  birefringence 
(Brininstool,  2003).  Figure  21.11  shows  the  birefringent  properties 
of  maize  starch  granules  as  observed  under  a  conventional  polar¬ 
ization  microscope. 

Placing  a  birefringent  material  between  a  pair  of  crossed  polar¬ 
izers  gives  rise  to  interference  colors.  When  light  passes  through 
a  polarizer  to  produce  linearly  polarized  light  and  that  light  then 
passes  through  a  piece  of  birefringent  material,  the  light  is  broken 
up  into  two  components.  Because  the  index  of  refraction  for  one 
of  them  is  larger  than  for  the  other,  that  component  will  lag  in 
phase  (retardance)  and  these  components  will  emerge  from  the 
specimen  out  of  phase  with  each  other.  Then  if  the  light  is  passed 
through  a  crossed  polarizer  (the  analyzer),  only  the  part  of  each 
component  that  is  in  the  transmission  plane  of  the  analyzer  will 
emerge.  This  emerging  ray  consists  of  two  co-planar  components 
that  differ  in  phase.  If  the  refraction  indices  of  the  material  also 
change  with  wavelength,  then  for  a  given  thickness  of  birefringent 
material,  some  wavelengths  will  undergo  destructive  interference 
and  some  constructive,  giving  an  interference  pattern  of  changing 
colors  similar  in  appearance  to  the  interference  colors  of  a  thin  film 
of  oil  floating  on  water.  If  monochromatic  light  is  used,  interfer- 


FIGURE  21.11.  Birefringent  properties  in  the  secondary  wall  thickenings  in  the  endothelium  cells  of  maize  anther. 


Interaction  of  Light  with  Botanical  Specimens  •  Chapter  21  421 


FIGURE  21.12.  Dynamic  polarization  microscopy  showing  the  cell  wall  ori¬ 
entation  (color-coded  wheel  shows  s-axis)  in  the  root  of  A.  thaliana.  The 
images  were  obtained  by  using  Polscope.  (Images  courtesy  of  Cambridge 
Research  Instruments,  Woburn,  MA.) 


ence  fringes  are  produced.  These  interference  color/fringes  can  be 
used  to  study  the  thickness  and  internal  stress  of  plant  cuticles  (i.e., 
photoelasticity). 

The  use  of  either  a  conventional  or  a  dynamic,  tunable  polar¬ 
ization  microscope  (Polscope,  Cambridge  Instrument  Inc.)  can 
provide  image  contrast  specific  to  birefringent  structures  such 
as  the  microtubules  in  the  mitotic  spindle  during  cell  division. 
Because  of  the  nature  of  transmission  polarization  microscopy, 
only  low  intensity  light  is  required,  which  minimizes  phototoxic¬ 
ity.  However,  as  the  retardance  of  many  botanical  structures,  such 
as  cell  walls  and  starch  granules,  are  relatively  large,  contrast  pro¬ 
duced  by  low  retardance  structures  such  as  microtubules  (MT) 
(typical  retardance  for  a  single  MT  is  0.07  nm)  often  becomes  lost 
in  the  high  background  retardance  contributed  from  the  cell  wall. 
Figure  21.12  shows  the  orientation  of  cellulose  microfibrils  in  the 
root  cell  wall  of  A.  thaliana.  The  cellulose  microfibril  shows  excel¬ 
lent  birefringence  contrast.  The  color  indicates  the  orientation  of 
the  microfibril  molecules. 


FLUORESCENCE  PROPERTIES  OF  PLANTS 

Plant  tissue  generally  contains  a  large  number  of  light-absorbing 
pigments,  including  those  responsible  for  photosynthesis.  Many  of 
these  pigments  produce  autofluorescence  (Figs.  21.13,  21.14,  and 
21.15).  Figure  21.13  shows  a  collection  of  fluorescence  spectra 
excited  at  different  wavelengths  from  various  plant  specimens. 
Note  the  spectral  variation  as  the  excitation  wavelength  changes. 
Figure  21.14  shows  the  fluorescence  spectrum  of  maize  leaf  grown 
in  either  normal  lighting  or  dark  (etiolated)  conditions,  while 
Figure  21.15  demonstrates  the  fluorescence  spectra  of  a  methanol 
extract  of  etiolated  maize  leaf  at  various  excitation  wavelengths. 
Because  plant  tissue  consists  of  a  mixture  of  many  fluorescent 
compounds,  each  with  different  absorption  spectrum  and  quantum 
yield  (QY),  the  resulting  fluorescence  spectra  can  vary  greatly 
when  subject  to  different  excitation  wavelengths.  As  these  endoge¬ 
nous  fluorophores  produce  high  intensity  background  autofluores¬ 


cence  throughout  the  entire  visible  spectrum,  it  is  very  difficult  to 
separate  their  output  from  that  emitted  by  introduced  fluorescent 
probes. 

Even  worse,  the  emitted  autofluorescence  is  often  so  intense 
that  it  can,  in  turn,  excite  nearby  fluorescent  tags  at  a  wavelength 
different  from  the  applied  illumination.  Therefore,  the  apparent 
spectral  fingerprint  of  a  fluorescent  dye  may  vary  depending  on  the 
presence  or  absence  and  the  location  of  nearby  autofluorescing 
structures.  An  example  might  be  the  vacuole  found  in  many  flower 
petals.  Vacuoles  can  contain  large  amounts  of  anthocyanins  and 
flavonols  that  produce  intense  autofluorescence,  the  emission 
spectra  of  which  varies  according  to  local  pH  and  ionic  conditions. 
Any  fluorescently  labeled  organelle  near  the  vacuole  may  be 
excited  by  this  autofluorescence.  Figure  21.16  shows  an  xz  section 
of  the  petal  of  tree  peony  ( Paeonia  sujfruticosa),  the  green  channel 
is  the  BSL  image  and  the  red  channel  is  the  autofluorescence 
image. 

Changes  in  Emission  Spectra  Depending  on 
One-  Versus  Two-Photon  Excitation 

When  subjected  to  excitation  illumination  at  380nm,  the  living, 
freshly  excised  leaves  of  A.  thaliana  emit  both  strong  red  fluores¬ 
cence  peaks  at  684  nm  and  714nm  and  blue-green  fluorescence  in 
the  range  of  <550nm  [Fig.  21.17(A)].  In  contrast,  high  intensity 
excitation  by  760  nm  NIR  illumination  produces  a  significantly  dif¬ 
ferent  upconverting  fluorescence  spectrum,  as  shown  in  Figure 
21. 17(B).  The  green  and  red  fluorescence  now  peaks  at  500 nm  and 
688  nm,  respectively.  Under  380 nm  excitation,  the  mesophyll  pro¬ 
toplasts  of  A.  thaliana,  exhibit  strong  red  fluorescence,  peaking  at 
680  nm,  and  very  low  broadband  emission  in  the  450  to  550  nm 
range  [Fig.  21.17(C)).  The  two-photon-excited  fluorescence  spec¬ 
trum  of  these  protoplasts  is  similar  to  that  obtained  from  the  whole 
leaf  but  with  the  688  nm  peak  blue-shifted  to  approximately 
660  nm  [Fig.  21.17(D)],  It  is  important  to  note  that  the  fluorescence 
emission  spectra  generated  by  the  excitation  with  single-photon 
excitation  is  not  identical  to  that  produced  by  two-photon  excita¬ 
tion  at  double  the  wavelength.  Kennedy  and  Lytle  (1986)  have  also 
reported  a  difference  between  the  excitation  spectra  obtained  from 
single-photon  and  two-photon  excitation.  Therefore,  two-photon 
images  may  well  be  different  from  those  obtained  from  single¬ 
photon  excitation  with  half  of  the  wavelength. 

Microspectroscopy 

Because  of  the  complexity  of  plant  autofluorescence,  it  is  fre¬ 
quently  advisable  to  obtain  fluorescence  spectra  from  the 
organelles  of  interest  to  confirm  their  fluorescence  signature.  This 
can  be  achieved  using  either  single-photon  or  two-photon 
microspectroscopy  (Cheng  and  Cheng,  2000;  Kao  et  al.,  2000a, b; 
Chu  et  al.,  2001).  The  latter  has  the  advantage  of  providing  excel¬ 
lent  three-dimensional  spatial  resolution  of  the  location  from 
which  the  fluorescence  spectra  can  be  obtained. 

This  technique  can  also  be  used  to  separate  SHG  and  THG 
signals  from  background  fluorescence  (Sun  et  al.,  2001).  Figure 
21.18  shows  the  setup  of  a  home-built  microspectroscopy  system 
[Fig.  21.18(A)]  and  a  fluorescence  spectrum  of  a  parenchyma  cell 
in  maize  stem  [Fig.  21.18(B)];  the  spectrum  was  obtained  by  two- 
photon  excitation  (Xex  =  800  nm)  from  the  fluorescent  spot  indi¬ 
cated  by  the  arrow  in  Figure  21.18(C).  The  setup  consists  of  a 
linear  charge-coupled  device  (CCD)-equipped  (Roper  Scientific) 
Acton  2300i  spectrometer  connected  to  the  photo  port  of  an 
Olympus  1X70  inverted  microscope  via  a  fiber  bundle.  A 
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FIGURE  21.13.  (A-D)  Comparative  fluorescence  spectra  of  various  plant  materials  [the  petal  of  yellow,  orange,  and  red  varieties  of  Canna  L.,  spider  lily 
(. Hymenocallis  speciosa )  and  forget-me-not  {Plumbago  auriculata)].  The  interruption  in  the  spectra  {dashed  line)  is  the  estimated  spectrum  as  the  data  were 
contaminated  by  scattered  excitation  illumination  leaving  the  spectrometer  grating  as  a  2nd  order  diffraction  peak. 


FIGURE  21 .14.  Autofluorescence  spectra  of  normal  and  etiolated  maize  seedling  {Zea  mays  L.,  var.  Golden  Beauty)  excited  by  light  at  different  wavelengths. 
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Wavelength  nm 

FIGURE  21.15.  Fluorescence  spectra  of  the  leaf  methanol  extract  obtained 
from  etiolated  maize  seedlings  (Zea  mays  L.,  var.  Golden  Beauty)  shows  sig¬ 
nificant  change  in  fluorescence  spectra  when  excited  by  different  excitation 
wavelengths. 


mode-lock  ultra-fast  titanium-sapphire  laser  was  used  as  the  light 
source  for  two-photon  fluorescence  excitation. 

A  number  of  modern  confocal  microscopes  are  capable  of 
recording  fluorescence  spectra  and  these  should  be  used  to  under¬ 
stand  the  fluorescence  signature  and  photobleaching  dynamics  of 
the  specimen  under  study.  Figure  21.19  shows  an  example  of 
spectra  of  a  living  maize  chloroplast  as  a  function  of  light  expo¬ 
sure.  The  three  dimensional  (3D)  spectra  (intensity  vs.  wavelength 
vs.  time)  demonstrates  the  evolution  of  chloroplast  fluorescence 
when  it  is  subjected  to  light  at  high  enough  intensity  to  cause  sig¬ 
nificant  radiation  damage  (Konig  et  al.,  1995,  1996a,  1997;  Cheng 
et  al.,  2000a,  2001;  Kao  et  al.,  2000b;  Lin  et  al.,  2000a, b;  Chen 
et  al.,  2002). 

Fluorescence  spectra  obtained  from  botanical  specimens  are 
generally  contributed  by  the  emission  of  many  fluorophores,  fre¬ 
quently  with  overlapping  emission  peaks.  There  are  a  number  of 
methods  to  separate/minimize  the  overlapping  spectra,  such  as 
sequentially  exciting  the  specimen  with  various  excitation  wave¬ 
lengths  or  using  spectral  unmixing  techniques. 

The  term  spectral  unmixing  refers  to  a  series  of  computer  algo¬ 
rithms  that  are  used  to  separate  a  spectral  signal  recorded  from  a 
single  pixel  or  a  defined  area  containing  a  number  of  fluorophores 
into  separate  intensity  signals  for  each  one  of  them.  The  algorithms 


FIGURE  21 .1 6.  Two  channel  (backscattered  light  and  autofluorescence)  con- 
focal  xz  section  of  a  tree  peony  (Paeonia  suffruticosa)  petal. 


Arabidopsis  thaliana 


FIGURE  21.17.  Comparison  of  fluorescence  spectra  obtained  with  single¬ 
photon  and  two-photon  excitation  in  a  leaf  preparation  of  A.  thaliana.  (A) 
Single-photon  excited  fluorescence  spectrum  of  fresh,  untreated  leaf,  excited 
at  380nm,  (B)  fluorescence  spectrum  of  mesophyll  protoplasts  excited  with 
380nm,  (C)  two-photon  fluorescence  spectrum  of  leaf  excited  at  780nm,  (D) 
two-photon  fluorescence  spectrum  of  protoplast  excited  at  780nm.  (Ti:Sa 
mode-lock  laser  operated  at  82  MHz.  lOOfs  pulse  at  an  average  power  of 
600  mW,  beam  diameter  approximately  0.8  mm  in  diameter.) 


vary  depending  on  the  hardware  used  to  initially  record  the  spectral 
data  but  all  of  them  have  a  close  similarity  to  the  programs  used  to 
deconvolve  3D  widefield  data  or  data  from  energy-dispersive  X-ray 
spectrometers  (EDS)  in  electron  microscopy.  In  spectral  unmixing, 
the  spectrum  of  each  individual  dye  serves  as  if  it  were  one  of  a 
series  of  “point-spread  functions”  and  it  is  the  function  of  the 
unmixing  algorithm  to  make  the  best  fit  between  the  recorded  data 
and  a  weighted  sum  of  the  emission  spectra  of  the  dyes  (i.e.,  each 
one  acting  like  a  “PSF”  with  a  shape  and  a  position)  thought  to  be 
present.  This  is  based  on  the  assumption  that  the  spectra  of  the  indi¬ 
vidual  dyes  are  not  changed  by  their  being  combined  in  the  pixel  or 
area  of  interest  (i.e.,  that  micro-environmental  differences  are 
either  absent  or  inconsequential).  On  the  other  hand,  it  is  common 
to  use  signal  from  a  nearby  pixel(s)  that  is  felt  to  be  producing  only 
background  autofluorescence,  as  the  reference  to  unmix  the  pixel  of 
interest  and  obtain  a  background-free  spectrum.  This  assumes  that 
the  reference  background  also  represents  the  background  present  in 
the  measured  pixel(s). 


c 
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FIGURE  21.18.  (A)  A  multi-photon  microspectrometer  setup,  (B)  two-photon  excited  fluorescent  spot  ( arrow )  in  a  maize  stem.  (C)  two-photon  excited  fluo¬ 
rescent  spot  (arrow)  in  a  maize  stem  stained  with  Calcofluor  white,  and  (D)  the  recorded  fluorescence  spectra. 


For  example,  it  is  difficult  to  detect  green  fluorescent  protein 
(GFP)  expression  in  chloroplasts  because  the  chloroplast  contains 
not  only  strongly  fluorescing  chlorophylls,  but  also  the  many 
highly  fluorescent  associate  pigments  of  the  photosynthetic  appa¬ 
ratus,  some  of  which  emit  in  the  green  region.  Exciting  each  flu- 
orophore  sequentially  with  the  appropriate  excitation  wavelengths 
may  also  be  helpful  in  reducing  the  problem  of  crosstalk  between 


detection  channels.  Unfortunately,  multi-photon  excitation  gener¬ 
ally  causes  all  fluorophores  to  fluoresce,  creating  broadband  fluo¬ 
rescence  emission  for  which  the  only  really  effective  antidote  is 
unmixing  (Figs.  21.13  and  21.14). 

Figure  21.20  demonstrates  the  use  of  spectral  unmixing  in 
conventional  transmitted  and  epi-fluorescence  microscopy  of  the 
rhizome  of  lily  of  the  valley  ( Convalaria  majalis )  to  separate  the 


FIGURE  21.19.  (A)  Two-photon  excited,  time-lapse,  microspectroscopy  of  maize  (Zea  mays  L.,  Ohio  43)  chloroplast  in  mesophyll  protoplast,  (B)  conventional 
fluorescence  micrograph  of  a  maize  protoplast,  (C)  conventional  transmission  micrograph  of  a  maize  protoplast. 
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FIGURE  21 .20.  (A)  Conventional  fluorescence  microscopy  of  the  rhizome  of 
lily-of- the- valley  (Convalaria  majalis).  Classification  of  tissue  structures  based 
on  their  autofluorescence  spectra,  produced  four  images  (B-E).  (F)  False  color 
image  of  the  combination  of  (B-E).  Image  obtained  by  using  the  Nuance  Spec¬ 
tral  Imaging  System.  (Images  courtesy  of  Cambridge  Research  Instrument, 
Woburn,  MA.) 


overlapping  autofluorescence.  The  xy-K  image  was  obtained  by 
using  a  tunable,  liquid-crystal  (LC)-based,  bandpass  filter  at  20 nm 
bandwidth  and  30  nm  steps.  Different  autofluorescence  was  chosen 
in  various  regions  of  the  image,  and  spectral  unmixing  was  per¬ 
formed  on  the  image  to  remove  the  mixed  autofluorescence.  The 
technique  clearly  separates  the  fluorescence  from  parachyma  cells 
from  the  Casparian  band  (Nuance  Imaging  System,  Cambridge 
Research  Instruments,  Woburn,  MA). 

Light-Specimen  Interaction 
(Fluorescence  Emission) 

Photoinduced  damage  can  be  characterized  by  a  typical  fluores¬ 
cent  intensity  versus  dose  curve.  Although  usually  the  fluorescent 
intensity  decreases  as  a  function  of  exposure,  this  frequently  does 
not  apply  to  specimens  of  botanical  origin.  As  plant  cells  gener¬ 
ally,  and  particularly  those  capable  of  photosynthesis,  are  designed 
to  capture  and  utilize  light  energy,  light-specimen  interactions 
often  involve  a  series  of  pigment  systems,  and  are  much  more 
complex  than  those  observed  in  animal  cells.  On  the  other  hand, 
as  it  is  highly  possible  that  fluorescence  resonance  energy  transfer 
(FRET)  occurs  naturally  in  the  photosynthetic  system,  introducing 
an  exogenous  donor-acceptor  pair  for  a  FRET  study  may  be  com¬ 
plicated  by  FRET  from  endogenous  fluorophores  in  the  plant  cells. 
For  example,  one  often  notices  an  increase  in  chloroplast  fluores¬ 
cent  intensity  and  a  blue  shift  in  autofluorescent  spectrum  upon 
illumination.  Figure  21.21  shows  the  two-photon-excited  fluores¬ 
cence  spectra  of  a  protoplast  culture  versus  exposure  time.  Note 
the  initial  increase  in  fluorescent  intensity  in  the  630  nm  chloro¬ 
phyll  peak. 

Time-lapse,  two-photon  fluorescence  microscopy  ( xyt )  of  the 
mesophyll  protoplasts  of  A.  thaliana  yielded  interesting  results 
concerning  cell  viability  under  the  high  illumination  intensity  com¬ 
monly  used  in  multi-photon  fluorescence  microscopy.  Upon  NIR 
(760  nm)  irradiation,  the  protoplasts  emit  blue-shifted  fluorescence 
as  shown  in  the  two-channel,  false-color  image  [combination 
of  green  (-500 nm)  and  red  (-660 nm)  channels].  In  time-lapse 
studies,  it  was  observed  that  the  red  fluorescence  diminishes  much 
more  rapidly  than  the  green  fluorescence.  This  results  in  a  blue 
shift  of  the  fluorescence  as  a  function  of  irradiation  dose.  Because 


FIGURE  21.21.  Two-photon  fluorescence  spectra  of  Arabidopsis  thaliana 
protoplasts  (in  culture  medium)  versus  exposure  time.  Note  the  red  fluores¬ 
cence  intensity  first  increases  before  decreasing.  Excitation,  760  nm. 
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FIGURE  21.22.  Two-photon  time-lapse  microspectroscopy  of  a  single  chloroplast  (from  A.  thaliana  mesophyll  protoplast)  showing  complex  spectral  changes 
as  a  function  of  increasing  exposure  dose. 


this  rapid  decrease  in  red  fluorescence  is  accompanied  by  cellular 
uptake  of  neutral  red/propidium  iodide  or  the  rapid  loss  of  cyto¬ 
plasmic-loaded  Calcien  dye  (both  of  which  indicate  cellular 
damage),  it  might  be  possible  to  use  the  red-to-green  fluorescence 
intensity  ratio  as  an  indicator  of  cell  vitality  (Cheng  et  al.,  2000a, 
2001).  This  fluorescence  study  of  protoplasts  was  confirmed  by 
microspectroscopy  of  a  single  chloroplast,  as  shown  in  Figure 
21.22.  Figure  21.23  shows  an  area  that  received  a  single 
3.3  s/frame  scan  (8.4p.s  dwell  time/pixel)  at  768  x  512  pixels.  As 
the  total  irradiated  area  is  180  pm  X  120  pm  and  the  average  power 
was  measured  as  6.4  mW,  the  average  energy  striking  each  pixel 
is  approximately  54 nj  (i.e.,  8.4 ps  at  6.4 mW).  On  the  other  hand, 
considering  that  the  laser  was  operating  at  82  MHz  with  lOOfs 
pulse  with  a  numerical  aperture  (NA)  1.2  objective  lens,  the 
average  and  peak  power  densities  at  the  focal  point  are  approxi¬ 
mately  3  x  106W/cm2  and  3.9  x  10nW/cm2,  respectively.  A  sig¬ 
nificant  reduction  in  red  fluorescence  is  evident  after  such  scans. 
Detailed  analysis  revealed  that  the  red  fluorescence,  from  chloro- 
phyll-a  and  -b,  initially  increases  slightly  before  decreasing.  This 
initial  increase  is  in  agreement  with  the  study  of  protoplast  fluo¬ 
rescence  in  culture  (Fig.  21.21),  and  is  believed  to  be  a  physio¬ 
logical  response  to  adjust  the  photosynthetic  system  to  the  high 
light  intensity,  perhaps  involving  the  xenthophyll  cycle. 

As  pointed  out  above,  under  high  intensity  illumination,  cell 
survival  can  be  monitored  by  the  cytoplasmic  retention  of  Calcein 
dye  (loaded  with  Calcein- AM  form;  Molecular  Probe  Inc.)  or  by 
the  entrance  of  neutral  red/propidium  iodide  (cell  death).  It  has 
been  reported  that  multiple  scans  during  image  acquisition  causes 
neutral  red  to  enter  the  cells  and/or  Calcein  dye  to  escape  from 
them.  These  changes  are  then  followed  by  visible  structural 


damage  such  as  shriveling  or  bursting  of  the  cell.  Figure  21.24  is 
a  series  of  images  obtained  in  sequence  demonstrating  cell  damage 
under  high  intensity  NIR  illumination.  Note  the  cell  most  heavily 
loaded  with  Calcein  dye  was  damaged  first  (arrow).  The  high  con¬ 
centration  of  Calcein  dye  in  the  cell  significantly  increases  the  non¬ 
linear  absorption  of  NIR,  causing  more  rapid  damage. 

It  has  been  reported  that,  for  a  comparable  red  fluorescence 
signal  level,  single-photon  excitation  using  488  nm  allows  signifi¬ 
cantly  more  imaging  scans  before  the  cell  begins  to  show  photo¬ 
damage  (Cheng  et  al.,  2001a).  Typical  one-photon  illumination 
power  is  in  the  order  of  0. 1  -  0.5  mW  at  the  focal  point  (power 
density  at  5  x  104W/cm2  ~  2.5  x  105W/cm2),  more  than  6  orders 
of  magnitude  lower  in  power  density  than  the  peak  power  used  in 
the  two-photon  imaging  mode  (3.9  x  1011  W/cm2).  Although  two- 
photon  fluorescence  microscopy  limits  the  photon-induced  damage 
to  the  vicinity  of  the  focal  point,  while  in  single-photon  excitation 
damage  is  spread  over  the  entire  illuminated  volume,  many  studies 
now  show  that,  for  the  same  signal  level,  high  intensity,  pulsed 
NIR  illumination  produces  far  greater  damage  to  many  living  plant 
cells.  This  may  be  due  to  the  fact  that  in  a  multi-photon  excitation 
scheme,  broadband  absorption/excitation  involving  all  the 
absorbers  is  possible,  while  in  single-photon  excitation,  com¬ 
pounds  only  absorb  energy  if  they  have  an  absorption  peak(s)  at 
the  excitation  wavelength.  It  is  important  to  point  out  that  photo- 
induced  damage  occurs  not  only  via  the  absorption/excitation  of 
fluorescent  molecules,  but  also  because  of  absorption  by  any  of  the 
vast  majority  of  non-fluorescing  molecules. 

When  Schilders  and  Gu  (2000)  investigated  the  effect  of 
multiple  scattering  in  a  turbid  medium  in  single-photon  and 
two-photon  fluorescence  microscopy,  they  demonstrated  that 
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FIGURE  21.23.  Two-photon  fluorescence  image  of  A.  thaliana  protoplasts.  (A)  The  image  was  obtained  by  first  scanning  the  central  portion  of  the  field  (rec¬ 
tangular  area)  followed  by  a  second  scan  to  record  a  larger  view.  Note  the  single  scan  causes  significant  intensity  and  spectral  changes  in  the  autofluorescence 
(760  nm  NIR  excitation).  (B)  The  ratio  of  the  green  and  red  channel  intensity;  (C)  intensity  variation  plot  of  the  green  and  red  channels  showing  the  initial 
increase  in  intensity. 


two-photon  excitation  provides  significant  improvement  in  pene¬ 
tration  depth  in  terms  of  resolution  because  the  longer  wavelength 
is  scattered  less.  However,  the  two-photon  image  signal-to-noise 
(S/N)  was  appreciably  degraded  if  a  large  fraction  of  the  so-called 
ballistic  photons  in  the  excitation  beam  scattered.  In  addition,  it 
was  noted  that  the  effective  absorbance  of  a  biological  specimen 
in  NIR  (i.e.,  800  nm)  can  be  significantly  higher  at  high  illumi¬ 
nation  intensities  because  the  nonlinear  absorption  coefficient 
increases  with  power  level.  Thus,  the  nonlinear  absorption  cross- 


section  increases  as  the  beam  intensity  increases  towards  the  focal 
point  and  the  effective  opacity  of  the  specimen  can  be  much  higher 
than  the  linear  absorption  measured  at  lower  intensity  (Fig.  21.5). 

In  two-photon  fluorescence  imaging,  the  achievable  image 
quality  is  more  often  limited  by  low  signal  strength  than  by  the 
optical  resolution  per  se.  In  the  case  of  the  highly  scattering  M. 
quadrifolia  leaf  (Fig.  21.10),  the  optical  sectioning  depth  is  limited 
to  approximately  40  to  50  pm.  The  two-photon  image  was  obtained 
by  using  an  Olympus  Fluroview  300  confocal  scanner  and  the  flu- 


FIGURE  21 .24.  Series  of  two-photon  fluorescence  micrographs  showing  the  photodamage  of  Calcein-loaded  protoplast  of  A.  thaliana.  Note  the  cell  most  heavily 
loaded  with  dye  is  the  first  to  be  damaged  (excitation,  760 nm;  emission,  550nm  +  20nm  band;  peak  power  density  at  focus  is  approximately  3.9  X  1011  W/cm2 
at  8.4  gs  dwell  time/pixel). 
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orescence  signal  was  descanned  and  detected  with  the  built-in  pho¬ 
tomultiplier  tube  (PMT)  but  no  detector  aperture  was  used.  The  fact 
that  the  two-photon  excited  fluorescence  peaks  are  located  near  the 
attenuation  maximum  of  this  specimen  further  limits  the  detectable 
fluorescence.  For  comparison,  the  same  specimen  imaged  in  single¬ 
photon  mode  gave  a  usable  depth  of  approximately  30  pm. 

HARMONIC  GENERATION  PROPERTIES 

A  number  of  plant  structures  are  capable  of  efficiently  generating 
second  harmonic  signals.  These  include  stacked  membrane  struc¬ 
tures  such  as  grana,  starch  granules,  secondary  cell  wall,  cuticle 
and  cuticular  waxes,  and  silica  deposits  (bio-opals).  Starch  gran¬ 
ules  exhibit  high  conversion  efficiency  in  SHG,  in  fact,  a  piece  of 
potato  tuber  placed  in  an  unfocused,  ultra-fast  laser  beam  can  effi¬ 
ciently  generate  a  bright  SHG  beam  in  the  forward  direction  (Fig. 
40.10,  this  volume).  Due  to  the  radial  symmetry  of  the  molecular 
structure  of  a  starch  grain,  starch  granules  exhibit  birefringence, 
and  it  is  believed  that  this  molecular  arrangement  is  also  respon¬ 
sible  for  the  SGH  property.  The  efficiency  varies  with  potato 
variety,  presumably  because  of  differences  in  the  structure  and  size 
of  the  starch  granules. 

Cellulose  is  a  linear  molecule  without  branching.  Neighboring 
cellulose  chains  may  form  hydrogen  bonds  leading  to  the  forma¬ 
tion  of  microfibrils  (20  to  30  nm)  with  partially  crystalline  parts 
called  micelles.  This  highly  organized  structure  may  be  responsi¬ 
ble  for  the  strong  SHG  properties  of  cell  walls.  The  term  bio¬ 
photonic  crystal  is  used  to  describe  biological  structures  having 
the  physical  properties  required  to  generate  harmonic  signals  (Sun 
et  al.,  2001;  Cheng  et  al.,  2003). 

The  SHG  signal  depends  strongly  on  the  polarization  of  the 
illumination  beam.  Figure  40.9  shows  the  rotation  of  SHG  signal 
in  maize  silica  cell  as  the  polarization  of  the  illumination  changes. 
Because  removing  the  organic  matrix,  using  a  ZnCT-HCl  solution, 
destroys  the  SHG  signal,  it  seems  likely  that  it  is  the  arrangement 
of  the  amorphous  silica  in  the  organic  matrix  that  is  important  for 
SHG. 

As  the  third  harmonic  signal  is  generated  at  optical  interfaces, 
and  as  plant  tissues  are  frequently  highly  heterogeneous  in  terms 
of  refractive  index,  it  is  not  surprising  that  many  plant  tissues  gen¬ 
erate  strong  THG.  The  surfaces  of  organelles,  such  as  mitochon¬ 
dria  and  chloroplasts,  lipid  droplets,  storage  bodies  (e.g.,  starch 
granules  and  protein  bodies  in  seeds)  and  cell-wall  interfaces  are 


all  capable  of  generating  strong  THG  signals.  Figure  21.25  shows 
a  set  of  chloroplast  images  [THG  (a),  SHG  (b),  2P-fluorescence 
(c),  and  combined  image  (e)]  of  mesophyll  cells  of  Commelina 
communis  L.  obtained  using  150fs,  1230nm  illumination. 

THE  EFFECT  OF  FIXATION  ON  THE  OPTICAL 
PROPERTIES  OF  PLANTS 

When  live-cell  imaging  is  impractical  or  unnecessary,  fixation  by 
means  of  chemicals  provides  a  way  to  avoid/minimize  the  prob¬ 
lems  with  “uncleared”  botanical  specimens  that  are  cataloged 
above. 

While  the  primary  purpose  of  the  fixation  process  is  to  pre¬ 
serve  the  structure  of  interest,  it  may  also  be  desirable  to  remove 
any  constituents  that  might  interfere  with  the  observation.  For 
example,  alcohol-based  fixatives  (e.g.,  1 : 3  KtOH/acctic  acid; 
methanol,  etc.)  usually  extract  significant  amounts  of  pigments 
from  plant  tissue,  greatly  reducing  the  autofluorescence  back¬ 
ground  (Figs.  21.13,  21.14,  and  21.15).  These  alcohol-based 
fixatives  are  commonly  used  to  preserve  protein  structures.  The 
use  of  a  simple  aldehyde  fixative,  such  as  formaldehyde,  acroline, 
or  1:1:3  EtOH/formaldehyde/acetic  acid  mixtures,  is  also 
recommended. 

As  aldehyde  fixation  causes  condensation  reactions  with  tissue 
proteins  that  can  result  in  ring  formation  and  autofluorescence,  it 
is  normally  avoided  to  prevent  high  background  autofluorescence. 
However,  Fredrickson  used  autofluorescence  enhanced  in  this  way, 
combined  with  excitation  from  the  488  nm  Ar  line  to  study  the 
development  of  the  female  gametophyte  in  Epipactis  (Fredrikson, 
1992).  Because  glutaraldehyde,  a  di-aldehyde,  can  greatly  increase 
background  autofluorescence  by  forming  excess  crosslinkage  of 
proteins,  its  usage  should  generally  be  avoided  (for  exception,  see 
Chapter  18,  this  volume).  The  use  of  a  fixative  containing  1% 
freshly  prepared  para-formaldehyde  and  1.5%  acroline  in  0.05  M 
phosphate  buffer  (pH  7.0)  is  recommended  for  its  high  penetration 
and  excellent  ultrastructural  preservation  (Cheng  et  al.,  1979b;  use 
without  the  glutaraldehyde  component).  This  fixative  offers  rela¬ 
tively  low  fixative-induced,  autofluorescence  and  offers  the  possi¬ 
bility  of  both  SEM  and  TEM  studies  after  optical  observations  (see 
Chapter  49,  this  volume).  The  brownish  coloration  of  plant  tissue 
caused  by  natural  tannins  or  the  oxidized  products  of  fixation  can 
be  removed  by  the  treatment  in  Stockwell’s  solution  for  10  to 
20h  (Johansen,  1940;  Herr,  1985,  1992;  Palser  et  al.,  1989). 


FIGURE  21.25.  SHG,  THG,  and  two-photon  fluorescence  image  of  chloroplast.  (A)  The  THG  signal  is  generated  at  the  interfaces  of  refractive  index  while  the 
SHG  signal  (B)  is  generated  by  the  stacked  membranes  in  the  grana  (G)  and  by  the  starch  granules  (S).  The  2P-fluorescence  (2PF)  signal  is  the  result  of  chloro¬ 
plast  autofluorescence.  Panel  (D)  combines  images  of  (A).  (B),  and  (C). 
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LIVING  PLANT  CELLS 

As  it  is  not  possible  to  modify  the  optical  properties  of  a  living 
plant  cell,  one  can  only  optimize  the  imaging  method  to  achieve 
best  attainable  images.  These  methods  include  the  selection  of 
the  proper  excitation  wavelength(s),  a  suitable  detecting  wave¬ 
length  (if  possible),  a  suitable  water-immersion  or  dipping 
objective  and  the  choice  of  single-  or  multi-photon  excitation.  In 
addition,  imaging  modalities  such  as  BSL  imaging  or  SHG 
should  be  considered  as  these  methods  do  not  deposit  any 
energy  in  the  specimen.  Spectral  imaging  may  help  to  separate 
the  emission  from  the  fluorescent  tag(s)  from  that  caused  by 
autofluorescence. 

Unlike  many  common  cultured  animal  cells,  neither  suspen¬ 
sion-cultured  plant  cells  nor  free-living  algae  attach  to  culture 
dishes  or  to  any  other  substrates.  As  a  result,  immobilizing  sus¬ 
pending  cells  can  become  a  challenge.  Most  of  the  discussion  so 
far  in  this  chapter  has  been  focused  on  the  photosynthetic  tissues. 
The  following  sections  will  discuss  the  technical  issues  specific  to 
imaging  particular  organs  and  tissues. 


Callus,  Suspension  Culture  Cells  and  Protoplasts 

Parenchymal  tissues  can  be  cultured  in  suitable  artificial  media.  The 
cells  remain  in  an  undifferentiated  state  and  form  aggregates  called 
callus.  The  callus  remains  colorless  when  kept  in  the  dark,  but  gen¬ 
erally  turns  green  under  light.  Therefore,  depending  on  the  type  of 
callus,  one  may  encounter  different  levels  of  autofluorescence  back¬ 
ground.  Imaging  GFP,  yellow  fluorescent  protein  (YFP),  and  DsRed 
is  relatively  easy  in  a  colorless  callus,  but  spectral  overlap  will  occur 
in  green  callus.  In  order  to  improve  the  optical  properties  of  the 
specimen,  it  is  important  to  mount  the  callus  under  a  coverslip  and 
in  a  suitable  culture  medium,  so  that  there  is  no  air  interface  between 
the  water-immersion  objective  lens  and  the  callus. 

Suspension-Cultured  Cells 

Suspension-cultured  cells  are  relatively  easy  to  image  because 
most  of  them  develop  only  the  primary  cell  wall.  However,  immo¬ 
bilizing  suspension  cells  requires  some  effort.  Methyl-cellulose, 
agrose  (0.5%-l%)  and  polyvinyl  alcohol  gel  (such  as  K-Y  Jelly) 
can  be  used  to  immobilize  such  cells  in  a  chambered  slide  [Fig. 
21.26(A)].  Cells  can  also  be  held  in  a  specially  designed  chamber 


FIGURE  21 .26.  (A)  Chamber  slides  commonly  used  in  handling  cultured  plant  cells.  (B)  A  homemade,  larger  version  of  the  chambered  slide,  filled  with  agrose 
gel,  can  be  used  to  observe  the  roots  of  small  plants  —  note  the  opening  in  the  upper  lid  to  allow  plants  to  grow  and  the  bottom  of  the  Petri  dish  ( dotted  rec¬ 
tangular  area)  has  a  window  made  of  a  coverslip  having  the  proper  thickness.  (C)  A  specimen  chamber  designed  to  handle  loose  tissue  or  cultured  cells;  (D) 
diagrammatic  representation  of  the  chamber  shown  in  (C).  The  chamber  is  made  of  an  upper  and  a  lower  cover  glass  window  (green)  with  a  10  pm  Mylar  film 
(red)  located  just  below  the  upper  coverslip.  Cells  or  loose  tissues  are  held  between  the  upper  coverslip  and  a  Mylar  film.  The  lower  chamber  is  filled  with  water 
and  connected  to  a  water  reservoir  via  a  flexible  tube.  The  amount  of  “holding”  pressure  exerted  on  the  specimen  by  the  Mylar  film  can  be  adjusted  by  the  height 
of  the  water  head  (Cheng  et  al.,  1999a).  (Other  chambers  are  discussed  in  Chapter  19,  this  volume ) 
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such  as  that  shown  in  Figure  21.26(B)  (Cheng  et  al.,  1999a). 
Figure  21.27  shows  GFP-expressing  tobacco  cells  held  in  a 
chamber  slide  with  0.5%  agrose  gel.  Through-focus  series  of  flu¬ 
orescence  images  were  obtained  at  video  rate  using  the  widefield, 
epi-fluorescence  mode  and  subsequent  deconvolved  (AutoDeburr, 
AutoQuant  Inc.,  Troy,  NY)  to  obtain  optical  sections.  Success  in 
this  technique  requires  that  the  specimen  be  immobile  during  the 
entire  data  collection  process. 

Protoplasts  are  plant  cells  that  have  had  their  cell  walls 
removed  by  enzymatic  digestion.  When  suspended  in  liquid 
medium,  they  are  generally  spherical  in  shape  and  can  be  imaged 
using  the  same  methods  as  for  suspension  cells.  If  the  protoplast 
is  derived  from  a  mesophyll  cell,  it  may  contain  large  numbers  of 
chloroplasts  as  shown  in  Figure  21.28.  The  image  shows  the  red 
autofluorescing  chloroplasts  and  green  fluorescing  mitochondria  in 
a  mesophyll  protoplast  of  A.  thaliana.  The  cell  was  labelled  with 
Mitotracker  (Molecular  Probes  Inc.)  in  culture  medium  (Huang 
and  Chen,  1988). 

Calcein  AM  dye  (Molecular  Probes  Inc.)  provides  a  good 
measure  of  plant  cell  and  protoplast  viability.  The  AM-ester  salt 
can  be  loaded  into  the  protoplast.  After  incubation,  wash  with  fresh 
medium  and  mount  in  a  chamber  slide.  Lower  concentrations  of 
dye  may  be  necessary  as  excess  dye  loading  can  greatly  increase 
the  linear  and  nonlinear  opacity  of  the  cell.  This  results  in  a  sig¬ 
nificant  increase  in  energy  deposition  by  the  illumination  and  can 
pose  a  serious  problem  in  confocal  or  multi-photon  fluorescence 
microscopy.  Figure  21.24  shows  a  series  of  images  taken  from 
Calcein  AM-loaded  protoplasts  (leaf  of  Zea  mays  L.).  Note  the 
rapid  cellular  damage  in  the  cells  most  heavily  loaded  with 
dye. 


Meristem 

The  apical  meristem  is  the  site  of  the  tip  growth  that  is  one  of  the 
most  striking  features  of  plant  growth.  Although  meristems  poten¬ 
tially  have  an  unlimited  ability  to  divide,  division  is  controlled  and 
regulated  resulting  in  the  proper  initiation  of  leaves  and  suppres¬ 
sion  of  the  development  of  lateral  buds  (apical  dominance).  Con- 
focal  microscopy  and  multi-photon  fluorescence  microscopy  is 
well  suited  for  the  study  of  leaf  and  floral  initiation  in  the  meri¬ 
stem.  Backscattered  light  mode  can  be  used  to  perform  time-lapse 
imaging  on  the  transformation  of  vegetative  to  reproductive 
growth  in  the  apical  meristem.  On  the  other  hand,  multi-photon 
fluorescence  microscopy  can  be  used  to  image  cell  organization 
and  division  orientation  in  an  initiating  leaf  primordium 
(Bommineni  et  al.,  1990,  1993,  1995).  Figure  21.29  is  a  BSL 
image  of  a  maize  apical  meristem. 


Stem  and  Root 

In  the  stem  and  root,  vasculature  is  the  most  frequently  studied 
tissue.  The  vascular  bundle  consists  of  the  elements  of  water 
conduction,  the  xylem,  and  those  of  assimilate  conduction,  the 
phloem.  The  phloem  is  the  principal  food-conducting  tissue  in  vas¬ 
cular  plants.  Mature  phloem  elements  are  called  sieve  tubes.  At 
regular  intervals  callose  is  deposited  to  form  sieve  plates  between 
sieve-tube  members  and  these  can  be  detected  with  Resorcin  blue. 
Decolorized  aniline  blue  (pH  10)  can  be  used  for  staining  both  fresh 
sieve  tubes  and  those  fixed  in  EtOH-acetic  acid  (3:1).  The  amount 
of  callose  increases  as  the  cell  ages,  and  this  continuously  reduces 
the  diameter  of  the  pores.  Because  increased  callose  formation 
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FIGURE  21.27.  Widefield  fluorescence  microscopy  of  suspension-cultured 
GFP-tobacco  cells.  (A)  Projection  of  a  series  of  widefield,  epi-fluorescence, 
through-focus  images  (xy,  xz,  and  yz  views).  (B)  Projection  of  a  deconvolved 
image  stack  from  (A).  Fluorescence  images  were  obtained  with  blue  excitation 
and  recorded  as  550  nm  emission.  Image  was  recorded  with  a  low  cost  video¬ 
rate  CCD  camera  using  ImageAcquire  acquisition  software  (Ming-mei 
Technology  Inc.,  Taipei).  Deconvolution  was  performed  using  AutoDeblur 
v.9.3  (AutoQuant  Inc.).  (Image  obtained  at  the  2002  International  Course  on 
Live  Cell  Imaging  held  at  the  University  of  British  Columbia,  Vancouver, 
Canada.) 
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FIGURE  21.28.  A  two-channel  fluorescence  image  of  a  protoplast  of  A. 
thaliana.  The  image  shows  Mitotracker-labeled  mitochondria  (green)  (A)  and 
autofluorescent  chloroplasts  (red)  (B).  (C)  Combined  false-color  image  of 
both  fluorescence  channels. 


occurs  in  response  to  injury  or  infection,  this  staining  method  can 
also  be  used  in  pathological  studies  of  plant  tissue.  In  addition, 
callose  can  also  be  stained  by  the  use  of  Cellufluor.  The  Casparian 
band  in  root  can  be  stained  by  berberin  hemisulphate/aniline-blue 
(Brundrett  et  al.,  1990;  Hose  et  al.,  2001). 

One  of  the  common  features  of  vascular  plants  is  their  ability 
to  synthesize  lignin.  This  substance  is  deposited  in  the  cell  walls 
of  lignified  tissue,  particularly  the  xylem,  to  provide  rigidity  to  the 
otherwise  elastic  polysaccharide  cell  walls.  Lignin  is  a  high- 
molecular-weight  polymer  of  substituted  cinnamyl  alcohols  that 
contributes  about  20%  to  30%  of  the  dry  weight  in  woody  stems. 
It  gives  strong  autofluorescence  when  excited  by  ultraviolet 
(UV)/blue  light  or  by  multi-photon  NIR.  These  autofluorescence 
properties  can  be  used  to  study  vascular  bundles  or  secondary  wall 
thickenings.  Woody  root  tissues  that  contain  so  much  of  lignin  and 
alkaloids  produce  such  intense  autofluorescence  that  it  is  generally 
difficult  to  image  exogenous  dyes  in  them,  although  the  fluores¬ 
cence  emission  can  sometimes  be  separated  using  spectral  unmix¬ 
ing  (Fig.  21.20). 

Synchronous  mitosis  can  be  achieved  in  plant  tissue  by  the  use 
of  colchicine  or  hydroxyurea  (Carvalho  et  a /.,  2002).  This  tech¬ 
nique  is  commonly  used  in  the  treatment  of  root  tips  to  obtain  a 
high  number  of  mitotic  figures.  Polarization  microscopy  can  be 
used  in  the  study  of  the  dynamics  of  mitotic  spindles  consisting  of 
birefringent  microtubules. 

Mounting  Roots  for  Imaging 

The  glass-bottomed  Petri  dish  shown  in  Figure  21.26(B)  can  be 
used  in  the  study  of  root  tips  and  root  hairs.  A  rectangular  hole  was 
cut  at  the  bottom  of  the  Petri  dish  and  a  coverslip  was  glued  over 
it  with  SilGaurd.  Then,  0.5%  to  2%  agrose  gel  was  pipetted  into 
the  dish  and  allowed  to  set.  Seeds  can  be  set  on  the  surface  or 
embedded  in  agrose  in  one  side  of  the  dish  so  that  the  seed  can 
germinate  while  the  dish  is  placed  horizontally.  As  soon  as  the 
young  roots  reach  the  bottom  of  the  dish,  place  the  petri  dish  at  a 
45°  to  60°  angle  on  a  stand,  to  force  the  root  to  grow  downwards 


along  the  surface  of  the  cover  glass.  This  method  minimizes  the 
distance  of  root  tissue  from  the  coverslip,  creating  good  optical 
conditions  for  microscopy.  The  root  portion  of  the  dish  should  be 
kept  in  the  dark  and  an  opening  in  the  cover  may  be  needed  to 
allow  the  seedlings  to  emerge.  Smaller  plants,  such  as  Arabidop- 
sis,  can  be  seeded  in  a  small-chambered  glass  [Lab-Tek;  Fig. 
21.26(A)].  An  inoculation  of  a  fungal  plant  pathogen  can  be 
applied  to  the  agrose  after  germination  and  pathogen-root  inter¬ 
actions  can  be  studied  through  the  coverslip  window  at  the  bottom 
of  the  Petri  dish.  Figure  21.30  is  a  fluorescence  confocal  image  of 
the  root  tip  of  maize. 

Microspores  and  Pollen  Grains 

Microsporogenesis  can  be  imaged  either  by  isolating  the 
microspore  from  an  anther  or  by  studying  a  whole-mount  speci¬ 
men.  Isolated  pollen  mother  cells  or  microspores  can  be  handled 
the  same  way  as  suspension-cultured  cells.  However,  it  is  fre¬ 
quently  desirable  to  image  the  microspores  in  situ  (Pace  et  al., 
1987)  through  the  anther  wall.  Multi-photon  fluorescence 
microscopy  is  the  choice  for  this  application.  Because  high  reso¬ 
lution  is  generally  needed  to  study  this  type  of  specimen  (i.e.,  chro¬ 
mosomes,  microtubules,  etc.),  special  care  should  be  taken  to 
ensure  that  the  optical  properties  are  matched  as  closely  as  possi¬ 
ble  to  the  design  criteria  of  the  objective  lens.  If  the  tissue  can  be 
fixed,  stained,  and  cleared  in  methyl-salicylate,  it  can  produce 
exceptional  images  (Cheng  et  al.,  1999b;  Fig.  21.31).  In  addition, 
it  can  be  helpful  to  apply  adaptive  deconvolution  [i.e.,  “blind” 
deconvolution,  capable  of  finding  the  best-fit  PSF  for  3D  multi¬ 
photon  or  confocal  data  stacks  (see  Chapter  24,  this  volume)].  The 
three-dimensional  chromosomal  arrangement  can  also  be  obtained 
by  using  widefield  epi-fluorescence  microscopy  and  deconvolution 
(Agard  and  Sedat,  1983;  Agard  et  al.,  1989;  Dawe  et  al.,  1992); 
the  technique  is  particularly  attractive  when  DAPI  (Vergne  et  al. , 
1987)  is  used  for  staining  of  the  chromosomes  because  UV  exci¬ 
tation  is  readily  available  in  conventional  epi-fluorescence. 
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FIGURE  21 .29.  Confocal  image  of  the  apical  meristem  of  maize. 


FIGURE  21 .30.  Two-channel  confocal  image  of  root  apex  of  maize. 


Because  microtubules  are  birefringent  structures,  the  spindle  in 
the  dividing  cells  can  also  be  imaged  by  dynamic  polarization 
microscopy  (e.g.,  Spindleview  from  Cambridge  Research  Instru¬ 
ments,  Woburn,  MA).  However,  as  the  anther  wall  can  also  be 
highly  birefringent,  it  may  be  better  to  image  isolated  pollen 
mother  cells. 

Pollen  Grains 

Pollen  grains  are  difficult  specimens  to  image  as  they  are  fre¬ 
quently  both  highly  scattering  and  pigmented.  The  surface  of  the 
pollen  grain  is  covered  with  a  sporopollenin  exine  and  a  cellulose 
intine  (Heslop-Harrison  and  Dickinson,  1969).  The  exine  consists 
of  complex  fine  structures  that  make  the  pollen  grain  not  only  a 
highly  absorbing  but  also  a  highly  scattering  specimen.  Because 
pollen  grains  also  frequently  contain  a  large  number  of  starch  gran¬ 
ules  and  a  relatively  dense  cytoplasm,  significant  self-shadowing 
can  occur  when  imaging  them.  Figure  21.32(A)  shows  the  auto¬ 
fluorescence  spectra  of  maize  pollen.  As  it  is  clear  that  this  auto¬ 
fluorescence  background  is  high  for  almost  the  entire  visible 
spectrum,  when  studying  pollen  grains,  consider  fluorescent  dyes 
that  emit  in  the  NIR.  The  autofluorescent  signal  can  be  excited  by 
UV,  blue  light  or  multi-photon  of  NIR.  Figure  21.32(B,C)  shows 
a  pollen  tube  (PT)  penetrating  the  hair  (hr)  of  silk  (S).  Note  the 
high  autofluorescence  of  the  pollen  grain  (P).  Other  pollen  images 
can  be  found  in  Figures  8.49  and  8.60. 
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FIGURE  21.31.  (A)  Meiosis  of  maize  pollen  mother  cell.  (A,  B,  C)  Representative  images  from  an  optical  section  stack  of  maize  anther.  (D)  Approximate 
position  where  the  optical  sections  were  obtained  from  the  maize  anther.  (E)  Pachytene  chromosomes  in  the  pollen  mother  cell  (PMC);  this  image  was  obtained 
directly  through  the  four-layer  anther  wall  using  a  confocal  microscope  (excitation,  514nm;  emission,  600  nm).  (D)  Deconvolved  image  of  (E).  Specimen  was 
stained  with  Feulgen  reaction  and  cleared  in  methyl  salicylate.  IMS1  and  IMS2  are  intermicrosporangial  stripe  1  and  2,  respectively.  Co,  anther  connective;  E, 
epidermal  cell;  En,  endothecium;  M,  middle  layer;  T,  tapetum;  No,  nucleolus. 


Tapetum 

The  tapetum  film  and  the  orbicule  [Ubish  body.  Fig.  21.32(D)]  are 
also  made  of  sporopollenin  (a  polyester  of  several  monomers  of 
which  beta-carotene  and  zeaxanthin  are  the  most  common)  and 
have  the  same  autofluorescence  properties  as  pollen  exine  (Echlin 
and  Godwin,  1968;  Dickinson  and  Bell,  1972;  Cheng  et  al.,  2004) 
that  produces  intense  autofluorescence  [Fig.  21.32(E)].  Therefore, 
because  studying  tapetum  development  after  the  young  microspore 
stage  (onset  of  sporopollenin  deposition)  may  suffer  from  high 
background  autofluorescence,  it  may  help  to  use  fluorescent  probes 
excited  in  deep  red  or  even  NIR. 

Starch  Granules 

As  stated  above,  starch  granules  are  efficient  SHG  producers. 
Therefore,  when  imaging  pollen  with  a  multi-photon  fluorescence 


microscope,  it  is  highly  possible  that  the  scattered  SHG  signal 
can  significantly  contaminate  the  recorded  fluorescence  signal.  In 
this  case,  it  is  wise  to  check  the  fluorescence  spectrum  to  ensure 
that  SHG  signal  is  not  present.  Alternatively,  one  can  place  a 
long-pass  barrier  filter  to  remove  the  SHG  signal  from  the 
fluorescence-detecting  path.  This  SHG  light  can  also  excite  addi¬ 
tional  autofluorescence. 

Pollen  Germination 

Although  it  is  common  to  image  pollen  germination  on  the  surface 
of  an  agrose  gel,  imaging  the  penetration  of  a  pollen  tube  into  a 
style  can  be  more  challenging.  In  order  to  optically  match  the 
design  criteria  of  a  “non-dipping”  water-immersion  objective  lens, 
it  is  essential  that  a  coverslip  with  proper  thickness  be  added  on 
top  of  the  agrose  gel.  A  small  spacer  may  be  needed  in  between 
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FIGURE  21.32.  (A)  Single-photon-excited  fluorescence  spectra  of  maize  pollen  grains  (Zea  mays  L.,  var.  KYS).  The  spectra  were  obtained  by  exciting  pollen 
grains  with  350nm,  380nm,  400nm,  448 nm,  and  514nm.  Note  the  strong  autofluorescence  in  the  blue  and  green  regions  when  excited  with  UV  and  deep  blue 
light  (350,  380,  and  400nm),  and  significant  red  fluorescence  when  excited  with  blue  and  green  light  (448  and  514nm).  (B  and  C)  Single-photon,  two-channel 
fluorescence  image  of  germinating  pollen  grain  (P)  on  the  hair  (hr)  of  the  silk  (S).  Note  the  penetrating  pollen  tube  (PT).  (D)  SEM  image  of  isolated  tapetum 
film  with  orbicules  (O).  (E):  (Style)  autofluorescence  image  of  orbicules  in  maize  (excitation,  488 nm;  emission,  610nm). 


the  agrose  gel  and  the  coverslip  to  prevent  it  crashing  into  the 
germinating  pollen  grain.  Decolorized  aniline  blue  (pH  10)  can  be 
used  for  staining  both  EtOH-acetic  acid  (3:1)  fixed  and  fresh 
callosic  inner  wall  of  the  pollen  tube  (Pareddy  et  al.,  1989). 

Rapid  cytoplasmic  streaming  within  the  pollen  tube  usually 
requires  high-speed  imaging  devices  such  as  the  disk-scanning  con- 
focal  microscope  (for  more  about  fast  confocal  imaging  see 
Chapter  10,  this  volume).  High-speed,  ratiometric  imaging  also 
allows  the  study  of  Ca^  gradients  at  the  tip  of  the  pollen  tube  using 
Ca^  sensitive  dyes  (see  Chapter  42,  this  volume).  Figure  21.33 
shows  a  Ca++  ratiometric  image  of  the  tip  of  lily  pollen  tube,  imaged 
by  a  confocal  microscopy  using  the  Fluo-3/Fura  Red  scheme. 

Cuticles,  Hairs,  and  Waxes 

The  outer  surfaces  of  terrestrial  plants  are  generally  covered  with 
an  inert  polymer  of  cutin  and  waxes  (Cheng  et  ah,  1981,  1986; 


Cutler  et  al.,  1982;  Juniper  and  Jeffree,  1983;  Cheng  and  Walden, 
2005).  The  most  common  constituents  of  the  cuticle  are  9,10,18- 
trihydroxyoctadecanoic  acid  and  19,16-dihydroxyhexadecanoic 
acid  (Holloway,  1982).  The  reactive  groups  of  these  molecules  link 
and  crosslink  into  a  high-molecular-weight,  three-dimensional 
network.  There  is  usually  an  excess  of  free  hydroxyl  (-OH)  and 
carboxyl  (-COOH)  groups  that  can  be  stained  with  basic  dyes.  In 
some  cases,  such  as  that  of  tomato  fruit,  the  cuticle  is  pigmented 
with  carotinoids  and  shows  strong  autofluorescence.  Cuticle  can 
also  be  stained  by  the  fluorescent  dye  berberin  sulphate.  The 
reverse  (or  “negative”)  fluorescence  method  can  also  be  used  to 
study  the  surface  topography  of  a  cuticle  layer,  such  as  the  ridged 
cuticle  found  on  the  anther  surface  of  maize  (Cheng  et  al.,  1986). 

Terrestrial  plants  frequently  have  additional  wax  covering  the 
surface  of  the  cuticle.  These  waxes  can  be  laid  down  in  complex 
structures  (plates,  needles,  etc.)  and  can  be  highly  birefringent. 
Plant  surface  waxes  are  complex  chemical  mixtures  composed 
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FIGURE  21.33.  Ca1 1  ratiometric  image  (serial  optical  sections)  of  the  tip  of  lily  pollen  tube  (PT).  (Image  obtained  at  the  2002  International  Course  on  Live 
Cell  Imaging  held  at  the  University  of  British  Columbia,  Vancouver,  Canada.) 


mainly  of  long-chain  aliphatic  hydrocarbons  with  chain  lengths 
ranging  from  C2o  to  C35  (Bianchi  and  Salamini,  1975;  Bianchi 
etal.,  1977,  1978;  Tulloch,  1981;  Baker,  1982;Blaker  1989;  Cheng 
etal,  1986).  These  waxes  can  form  complex  surface  structures  and 
are  difficult  to  study  morphologically  because  of  their  sensitivity 
to  electron  beam  irradiation  in  a  scanning  electron  microscope.  The 
waxy  surface  structures  can  be  studied  by  using  a  confocal  micro¬ 
scope  with  either  BSL  imaging  or  reverse  contrast  fluorescence  in 
which  the  surface  is  wetted  by  the  fluorescent  solution.1 

The  air-cuticle-water  interfaces  are  optical  interfaces  between 
RI  =  1/1 .4/1.3.  This  difference  is  high  enough  to  effectively  lower 
the  solid-angle  of  high-NA  objectives.  The  air-cuticle-water 
interfaces  can  also  generate  THG  signal  when  illuminated  at  high 
intensity.  Trichomes  and  hairs  are  additional  strongly-scattering 
structures  on  the  surfaces  of  many  plants.  To  reduce  the  deleteri¬ 
ous  optical  effects  of  these  structures,  it  is  wise  to  mount  such  plant 
tissues  in  water  or,  if  possible,  in  Rl-matching  glycerol/water  solu¬ 
tion.  This  measure  reduces  the  RI  difference  between  the  medium 
and  the  epidermal  surface  (wax  and  cuticle)  from  1.0/1. 4 
(air/cuticle)  to  1.3/1. 4  (water/cuticle).  However,  if  one  is  trying  to 
image  the  organ  surface  (i.e.,  the  surface  of  the  leaf),  an  air/tissue 
interface  is  desirable  as  it  can  easily  be  imaged  using  BSL.  This 
imaging  modality  allows  researchers  to  monitor  the  development 
of  an  organ  or  tissue  over  a  long  period  of  time  using  low  light 
intensity  and  without  adding  fluorescent  dyes.  Cuticle  can  also  be 


1  To  ensure  low  staining  of  the  waxy  structures,  it  is  important  to  use  dyes  that 
have  a  low  partition  coefficient  in  waxes. 


separated  from  the  cell  wall  and  studied  in  this  way  (Holloway  and 
Baker,  1968).  Figure  21.34  shows  extended-focus  images  of  the 
surface  of  the  leaf  by  BSL  mode  (green)  and  underlying  cells  in 
fluorescence  mode  (red). 

Storage  Structures 

Starch  granules,  protein  bodies,  and  lipid  granules  are  commonly 
found  in  plant  cells.  In  most  cases,  if  the  density  of  these  granules 
is  relatively  low,  no  significant  influence  on  the  signal  strength  will 
be  detected.  However,  in  tissues  where  a  large  number  of  storage 
granules  is  present  (endosperm  cells,  pollen  grains,  and  other 
storage  cells),  significant  deterioration  both  in  image  quality  and 
signal  strength  should  be  expected.  This  is  particularly  serious 
when  the  granules  are  stained  with  dyes  that  have  strong  absorp¬ 
tion  characteristics  in  the  excitation  and/or  emission  wavelengths. 

Oils  found  in  plant  cells  commonly  have  refractive  indices 
higher  than  that  of  water.  For  example,  most  of  the  commercially 
used  plant  oils  have  refractive  indices  in  the  range  of  1.4  (Lide, 
1991).  Therefore,  these  substances  can  generally  be  imaged  in  the 
BSL  mode  as  well  as  by  using  fluorescent  dyes  such  as  Neutral 
Red  (Kirk,  1970)  or  Nile  Red.  The  fluorescence  excitation  and 
emission  maxima  of  these  dyes  may  vary,  depending  on  the 
hydrophobicity  of  the  environment.  Nile  Red  can  be  used  at  a  con¬ 
centration  of  lOOng/mL,  and  can  be  excited  with  the  488  nm/ 
514nm  Ar-ion  lines,  with  emission  detected  using  a  530nm/ 
590  nm  longpass.  Figure  21.35  shows  the  lipid  and  protein  storage 
granules  of  maize  endosperm  imaged  by  two-channel  fluorescence 
confocal  microscopy.  Note  the  dense  tissue  resulting  in  very 
shallow  penetration. 
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FIGURE  21 .34.  Extended-focus  image  of  the  leaf  surface  of  maize.  (A)  Ridged  cuticle  {arrow)  can  be  found  at  the  base  of  a  hair  (hr);  the  surface  of  the  inter¬ 
locking  epidermal  cells  (E)  is  clearly  visible.  (B)  Stomata  and  interlocking  epidermal  cells.  Note  that  the  secondary  wall  of  the  guard  cells  produces  intense 
autofluorescence.  Green,  backscattered  light  signal;  red,  autofluorescence  signal. 


Mineral  Deposits 

Mineral  deposits  are  frequently  found  in  and  around  cells  in  plants. 
For  example,  Si02  deposits  are  common  in  specialized  cells  and 
in  the  cell  wall  of  the  epidermal  cells  of  grasses  (Dayanandan  et 
al.,  1983;  Hodson  and  Sangster,  1988;  Cheng  et  al.,  1990;  Kim  et 
al.,  1990)  and  Equisetum.  The  Si02  deposition  is  amorphous  in 
nature  and  commonly  referred  to  as  biological  opal.  Bio-opals 
have  an  RI  of  around  1.42.  As  this  is  significantly  different  from 
that  of  the  surrounding  aqueous  medium,  it  produces  a  strong  BSL 
signal.  The  surface  of  these  deposits  can  be  rough  (on  the  hundreds 
of  nanometer  scale)  and  they  often  have  many  internal  lacunae 


from  a  remnant  of  cytoplasmic  strands.  Figure  21.36(A)  shows  a 
confocal  image  of  silica  cells  in  maize  ( Zea  mays  L.).  Silica  cells 
also  exhibit  strong  birefringence  [Fig.  21.36(B)]  (Sun  et  al.,  2001; 
Cheng  et  al.,  2003a;  Chu  et  al.,  2003a)  (Fig.  21.37). 

Diatom  frustules  are  also  made  of  silica  and  can  exhibit  strong 
birefringent,  scattering,  and  SHG  properties  (Fig.  21.38).  In  addi¬ 
tion,  calcium  oxalate  (Jauregui-Zuniga  et  al.,  2003),  and  calcium 
carbonate  (in  the  form  of  calcite  and  argonite)  and  other  mineral 
deposits  (Homer  and  Wagner,  1992)  are  present  in  plant  tissues  and 
generally  can  be  imaged  by  either  BSL  or  by  negative  staining 
(filling  the  surrounding  medium  with  fluorescent  dye  to  “high¬ 
light”  the  non-fluorescing  crystals)  (Cheng  and  Kriete,  1995). 


FIGURE  21 .35.  Two-channel  confocal  fluorescence  image  of  maize  scutellum  showing  the  lipid  and  protein  storage  granules:  (A)  near  surface.  (B)  5  pm  farther 
into  the  surface,  (C)  scanning  electron  microscope  image  of  the  same  type  of  structure.  Note  the  signal  degrades  rapidly  in  the  tissue  because  of  the  high  density 
of  fluorescent  features  in  the  specimen. 
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FIGURE  21.36.  Silica  cells  in  maize  leaf.  (A-D)  ZnCl2-HCl  isolated  cuticle  with  attached  silica  deposition  imaged  in  (A)  confocal  backscattered  light  mode, 
(B)  confocal  autofluorescence  mode  (FL),  and  (C)  conventional  transmission  image  (TR).  (D)  False  color  combination  of  (A)  and  (B).  The  arrow  in  (B)  indi¬ 
cates  the  center  of  the  silica  deposit  and  becomes  visible  in  the  isolated  specimen.  The  isolated  cuticle  was  mounted  in  water  on  a  microscope  slide.  The  sig¬ 
nificant  difference  in  RI  between  silica  and  water  provides  the  contrast  in  the  conventional  transmission  image.  (E)  Silica  cell  stained  in  Feulgen  showing  the 
cell  nuclei  (N)  in  fluorescence  mode  (blue)  and  silica  deposits  in  backscattered  light  mode  (red).  (F)  SEM  surface  view  (low-loss  electron  mode,  LLE)  of  leaf 
surface  showing  the  location  of  the  silica  cells  (Si)  (Wells  and  Cheng,  1992). 


FIGURE  21.37.  Two-photon,  extended  focus,  fluorescence  image  of  abnormal  vasculature  in  a  dwarf  mutant,  nal/nal,  of  maize  (Zea  mays  L.)  stained  with 
analine  blue.  The  secondary  wall  of  the  vascular  bundles  produce  intense  autofluorecence  when  excited  with  780  nm  NIR.  V,  vertically  oriented  vascular  bundle; 
C,  connecting  bundle. 
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Primary  and  Secondary  Cell  Walls 

Most  plant  cells  are  covered  with  an  extracellular  wall  composed 
mainly  of  cellulose  with  additional  lignins  and  sometimes,  mineral 
deposits.  Cellufluor  (Polysciences)  and  other  similar  brighteners 
can  be  used  in  the  study  of  cell  walls  in  both  UV  confocal  and 
multi-photon  fluorescence  microscopy.  These  dyes  are  frequently 
used  for  assaying  cell  wall  regeneration  of  plant  protoplasts  and 
studying  the  development  of  tracheary  elements  (Taylor  and 
Peterson,  2001).  Cellufluor  binds  non-specifically  to  beta-linked 
polysaccharides  such  as  chitin  and  cellulose.  In  addition,  cellulose, 
carboxylated  polysaccharides,  beta-1,  3-glucans,  pectin,  callose, 
and  certain  other  polysaccharides  in  the  extracellular  mucilages  of 
root  caps  can  be  stained  with  Calcofluor  White  M2R  [disodium 
salt  of  4,4-bis-(4-anilino-bis-diethylamino-S-triazinyl-2-aminol)- 
2,2-stilbene-disulfonic  acid;  Hughes  and  McCully,  1975].  This  flu- 
orochrome  stains  most  types  of  cell  walls  but  usually  not  cell 
contents.  Therefore,  it  is  a  useful  dye  for  demarcating  cellular 
boundaries  and  may  also  bind  to  lignin  and  some  regions  of  starch 
grains.  It  has  been  reported  that  Calcofluor  White  M2R  exhibits 
fluorescence  dichroism:  Two  distinct  colors  are  seen,  depending  on 
the  relationship  between  the  orientation  of  the  structural  elements 
in  the  wall  and  the  polarization  direction  of  the  illumination 
(Hughes  and  McCully,  1975).  If  the  plane  of  polarization  of  the 
illumination  beam  is  known,  one  can  study  the  orientation  of  the 
cell  wall.  Because  at  low  concentration  Calcofluor  has  no 
detectable  effect  on  the  root  growth,  it  may  also  be  used  as  a  vital 
dye.  A  typical  staining  protocol  follows. 

Calcofluor  Staining  Procedure  for  Fixed  and 
Fresh  Tissue  (Clark,  1981) 

1.  Fresh  or  3%  glutaraldehyde-fixed  tissue  can  be  used. 

2.  Stain  the  tissue  in  0.01%  Calcofluor  solution  for  20  to  60s. 

3.  Destain  in  water  for  1  min. 

4.  Mount  in  water  for  microscopy. 


In  contrast  to  the  primary  cell  wall,  the  secondary  wall 
contains  less  pectin  and  hemicellulose,  but  lignin  and  other  com¬ 
ponents  are  still  present.  Lignin  consists  of  polymerized  phenyl- 
propane  units  and  acts  as  a  structural  component.  The  other 
non-structural  components  found  in  the  secondary  wall  are  tannins, 
oligosaccharides,  and  glycoproteins.  Figure  21.37  shows  a  projec¬ 
tion  of  the  vasculature  of  a  dwarf  mutant  of  maize  (Zea  mays  L., 
nal/nal  homozygous,  Cheng  et  al.,  2001b).  Two-photon  fluores¬ 
cence  imaging  was  used  to  achieve  deep-tissue  imaging  at  low 
magnification.  In  certain  cells,  such  as  the  pollen  mother  cell  at  the 
Central  Callose  stage  (Cheng  et  a].,  1979a),  callosic  wall  is 
present.  Callose  is  a  helical  molecule  made  of  glucose  units  linked 
via  1,3  glycosidic  linkages  (callose  contains  also  1,4  and  1,6  link¬ 
ages).  Aniline  blue  binds  to  the  helix,  and  the  resulting  complex 
produces  yellow  fluorescence. 

Cell  walls  can  exhibit  strong  birefringence  and  have  been 
reported  to  have  strong  SHG  properties  (Cheng  et  al.,  2002b;  Chu 
et  al.,  2003a,b).  The  cell  wall  and  the  surrounding  water  are  suf¬ 
ficiently  different  in  RI  to  be  capable  of  generating  THG  signals. 

The  structural  components  of  the  cell  walls  of  algae  consist 
of  mannanes,  xylanes,  alginic  acid,  sulfonated  polysaccharides, 
silicon,  sporopollenin,  and  calcium  carbonate  (in  the  form  of 
calcite  and  argonite).  For  those  species  with  a  high  degree  of  min¬ 
eralization,  BSL  mode  is  ideal  for  imaging  the  surface  of  the  algae 
while  fluorescence  mode  is  used  for  cellular  organelles.  Figure 
21.38  shows  a  diatom  imaged  in  backscattered  (silica  surface)  and 
fluorescence  mode  (chloroplast).  Note  that  the  particle  (arrow) 
casts  a  shadow  on  the  diatom  below. 


Fungi 

Although  fungi  are  not  plants,  and  are  usually  classified  as  a  sep¬ 
arate  kingdom,  it  is  necessary  to  discuss  the  imaging  of  these 
important  organisms  here  as  many  fungi  are  plant  pathogens.  Yeast 
and  many  other  fungi  have  a  highly  reflective  chitin  surface  and 


FIGURE  21 .38.  Fluorescence  and  backscattered  light  images  of  diatoms.  (A)  The  silica  wall  produces  an  excellent  backscattered  light  signal  and  (B)  the  chloro- 
plasts  emit  strong  autofluorescence.  (C)  Transmission  image  showing  a  soil  particle  (P)  which  casts  a  shadow  in  the  fluorescence  image  (B)  and  produces  strong 
scattering  signal  in  the  BSL  image  (A). 
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are  frequently  difficult  to  image.  Imaging  the  penetration  of  fungal 
hyphae  into  plant  tissue  can  also  be  challenging  as  hyphae  often 
have  a  RI  higher  than  the  surrounding  medium. 

Chitin  is  a  polymer  that  consists  of  glucose  derivatives,  N- 
acetyl  glucosamine  units  connected  by  1,4  linkages.  Although 
occasionally  found  in  algae,  chitin  is  the  main  structural  compo¬ 
nent  in  the  cell  wall  of  most  fungi.  Because  Cellufluor  binds  non- 
specifically  to  beta-linked  polysaccharides  such  as  chitin  and 
cellulose,  high  purity  Cellufluor  in  deionized  water  (0.05%  solu¬ 
tion),  with  potassium  hydroxide  added  as  a  clearing  agent,  can  be 
used  for  the  identification  of  various  fungi.  This  dye  is  commer¬ 
cially  available  as  Fungi-Fluor  kit  from  Polysciences  (Warrington, 
PA).  Muller  and  Sengbusch  (1983)  have  shown  that  Calcufluor 
white  reveals  the  rhizoid  system  of  the  fungus  Blastocladiella 
anabaenae  on  Anabaenci  flosaquae  (a  filamentous  cyanobacteria). 


CONCLUSION 

Because  plant  cells  contain  light-scattering  cell  walls  and  a  high 
concentration  of  pigments,  they  pose  a  challenge  to  both  single¬ 
photon  confocal  and  multi-photon  fluorescence  microscopy.  In 
plant  tissues  free  of  pigmentation,  two-photon  fluorescence 
microscopy  outperforms  conventional  single-photon  confocal 
microscopy  because  of  its  increased  depth  penetration  (Bhawalkar 
et  ah,  1996;  Cheng  el  al.,  1998).  Flowever,  when  the  two-photon 
light  is  strongly  absorbed,  it  can  cause  even  more  damage  than 
single-photon  excitation. 

On  the  other  hand,  even  heavily  pigmented  plant  specimens  do 
show  significantly  lower  attenuation  at  wavelengths  longer  than 
lOOOnm,  and  therefore,  long-wavelength  two-  and  three-photon 
excitation  can  be  considered.  Results  indicate  that  significant  dif¬ 
ferences  exist  in  fluorescence  emission  spectra  between  single¬ 
photon  and  two-photon  excitation.  Therefore,  one  should  generally 
NOT  select  two-photon  excitation  wavelengths  by  simply  dou¬ 
bling  the  wavelength  of  the  linear  absorption  maximum  of  the 
fluorophore. 

Mesophyll  protoplast  viability  tests  indicate  that  heavily  pig¬ 
mented  (or  heavily  dye-loaded)  plant  cells  can  experience  exten¬ 
sive  photodamage  under  high-intensity  NIR  illumination.  Even 
though  photodamage  is  limited  to  the  immediate  vicinity  of  the 
focal  point  in  two-photon  fluorescence  microscopy,  overall  cell  via¬ 
bility  is  often  lower  than  with  single-photon  confocal  microscopy, 
assuming  that  the  criterion  is  to  obtain  the  same  fluorescence  signal 
level.  Therefore,  the  choice  of  using  single-photon  confocal 
microscopy  versus  multi-photon  fluorescence  microscopy  for  plant 
material  comes  down  to  a  careful  analysis  of  attenuation  and  exci¬ 
tation  spectra  to  determine  which  imaging  modality  is  most  likely 
to  allow  one  to  obtain  the  required  information. 
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INTRODUCTION 

Strictly,  to  obtain  true  confocal  imaging  the  detector  pinhole  must 
be  infinitesimally  small,  which  would  of  course  result  in  a  van¬ 
ishingly  weak  image  signal.  On  the  other  hand,  a  very  large  pinhole 
degrades  the  confocal  imaging  effect.  So  in  practice  it  is  necessary 
to  adopt  some  optimum  diameter  for  the  pinhole,  which  will 
depend  on  the  design  of  the  microscope,  how  it  is  operated,  and 
the  type  of  specimen.  The  resultant  imaging  performance  then  also 
depends  on  these  various  factors.  In  this  way  we  can  compare  the 
performance  of  different  designs  of  confocal  microscope,  and  also 
compare  them  with  widefield  (WF)  microscopes  that  have 
electronic  image  capture  coupled  with  digital  three-dimensional 
(3D)  image  restoration.  In  addition,  we  can  understand  how  best 
to  use  the  microscope  in  order  to  achieve  optimum  imaging 
performance. 

In  many  biological  applications  of  confocal  microscopy  the 
signal  level  is  weak.  In  fluorescence  microscopy,  the  fluorophore 
bleaches  after  exposure  to  light,  thus  limiting  the  total  exposure 
and  hence  the  number  of  photons  that  can  be  detected.  This  results 
in  shot  noise  in  the  final  image,  degrading  the  image  and  limiting 
the  number  of  gray  levels  that  can  be  perceived.  The  overall  image 
quality  can  be  determined  by  considering  the  signal-to-noise  ratio 
(S/N)  of  the  image.  If  the  only  source  of  noise  were  the  shot  noise 
on  the  detected  signal,  then  S/N  would  increase  as  the  pinhole 
diameter  was  increased.  However,  there  will  always  be  some 
unwanted  background  that  also  contributes  to  the  noise.  In  general, 
the  background  originating  from  non-specific  staining  or  autoflu¬ 
orescence,  from  either  the  specimen  or  optical  elements,  increases 
as  the  pinhole  diameter  is  increased  so  that  for  very  large  pinhole 
diameters  S/N  actually  decreases.  Thus,  there  is  an  optimum 
pinhole  size  that  is  large  enough  to  allow  the  signal  to  be  detected 
with  adequate  S/N,  but  small  enough  to  reject  the  noise  from  the 
background. 

For  a  given  imaging  frame  time,  it  is  tempting  to  increase  the 
signal  level  by  increasing  the  incident  laser  power.  However,  too 
high  a  laser  power  results  in  saturation  of  the  fluorophore.  As 
well  as  limiting  the  signal  that  can  be  detected,  this  also  has  the 
effect  of  giving  a  nonlinear  relationship  between  fluorophore 
concentration  and  detected  signal  so  that  quantitative  interpreta¬ 
tion  of  the  image  is  difficult.  These  effects  are  particularly  notice¬ 
able  in  the  fast-scanning  systems  that  are  often  needed  for  live-cell 
studies.  The  result  is  that  if  the  signal  is  limited  by  saturation  rather 
than  bleaching,  the  relative  performance  of  the  various  systems 
with  regard  to  S/N  is  very  different,  as  will  be  discussed  later. 


Similar  conclusions  hold  in  backscattered  light  (BSL)  imaging 
rather  than  fluorescence  imaging  (Cogswell  and  Sheppard,  1990; 
Sheppard  and  Cogswell,  1990;  Dunn  et  al.,  1996).  BSL  imaging 
is  growing  in  importance  for  clinical  diagnostic  and  biopsy  appli¬ 
cations  (Lucid  VivaScope).  The  confocal  principle  is  also  of 
importance  in  optical  coherence  tomography  (OCT)  (Huang  et  al., 
1991).  The  BSL  signal  from  biological  samples,  produced  by 
variations  in  specimen  refractive  index,  stains,  or  immuno-gold 
labeling  can  all  be  very  weak  ( 1 0-4 —  1 0-6  relative  to  the  incident 
power).  In  addition,  the  allowable  incident  laser  power  is  limited 
by  specimen  damage.  The  main  difference  (advantage)  in  BSL 
imaging  is  that  there  is  no  bleaching  of  the  signal,  so  that  if  the 
specimen  is  unchanging  in  time  there  is  in  principle  no  limit  to  the 
signal  that  can  be  obtained  over  a  long  period  of  time,  and  hence 
S/N  can  be  increased  to  any  desired  level. 


SOURCES  OF  NOISE 

Shot  Noise  and  Quantum  Efficiency 

As  described  in  the  previous  section,  the  most  important  source  of 
noise  is  shot  noise.  The  importance  of  shot  noise  is  that  it  is  the 
fundamental  limit  of  S/N.  [We  mention  here  that  an  active  area  of 
present  fundamental  research  is  in  reducing  noise  below  the 
classic  limit  as  a  result  of  quantum-mechanical  effects  (Slusher 
et  al.,  1985).]  Light  consists  of  particles  called  photons,  each  of 
which  has  an  energy  hv,  where  h  (=  6.626  x  10_347j)  is  Planck’s 
constant  and  v  is  the  frequency.  As  the  frequency  is  related  to  the 
wavelength  by 

c  =  vA,  (1) 

c  being  the  velocity  of  light  (=  2.998  X  108ms-1),  a  beam  of  light 
of  power  p  incident  for  a  time  t  contains  np  photons,  where 


Shot  noise  results  from  the  statistical  variation  in  the  number  of 
detected  photons,  which  obeys  a  Poisson  distribution.  The  Poisson 
distribution  is  a  skewed  distribution  for  small  numbers  of  photons, 
but  tends  to  the  symmetrical  normal  distribution  for  large  numbers. 
The  Poisson  distribution  has  the  property  that  its  mean  is  equal  to 
its  variance,  and  as  the  noise  is  given  by  the  square  root  of  the 
variance  the  shot  noise  on  a  signal  of  /?pis  *Jn~,  so  that 
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S/N  = 


(3) 


Suppose  now  the  light  is  detected  (Sheppard  et  al.,  1992)  by  a  photo¬ 
detector  of  quantum  efficiency  QP,  which  is  simply  the  fraction 
of  photons  which  are  detected  by  the  detector.  If  the  detector  also 
contributes  a  sensor  noise  of  n„  electrons  then  S/N  is  given  by 


S/N  = 


QeUp 

V QEnp  +  nl ' 


(4) 


The  overall  behavior  of  the  detector  for  a  given  number  of  inci¬ 
dent  photons  depends  on  both  the  quantum  efficiency  and  the 
sensor  noise:  for  a  large  number  of  photons,  S/N  is  proportional  to 
sIQe,  whereas,  for  a  very  small  number  of  photons,  it  is  propor¬ 
tional  to  the  ratio  QE/n„. 

We  can  consider  the  information  content,  b,  in  the  image,  in 
bits,  given  by  Shannon  (1949) 

b  =  log2(l  +  S//V).  (5) 


In  terms  of  the  number  of  discernible  gray  levels  g,  we  can  write 
b  =  log2(g),  (6) 

so  that  the  number  of  gray  levels  is  simply 

QrnP 


g  =  i+sy/v  =  i+ 


4 QEnp  +  nl 


(7) 


Thus,  if  SIN  is  zero,  we  can  only  perceive  a  single  gray  level,  that 
is.  the  image  is  featureless.  For  S/N  =  1,  we  see  two  gray  levels, 
corresponding  to  a  binary  (black  and  white)  image.  For  large 
values  of  S/N,  the  number  of  gray  levels  increases  linearly  with 
S/N.  Note  that  this  derivation  of  the  number  of  gray  levels  is  based 
on  the  information  content  in  the  image,  which  is  an  objective 
property.  Although  it  does  require  that  the  specimen  be  capable  of 
producing  the  measured  signal  levels  (i.e.,  having  a  certain  voxel 
staining  ratio),  it  does  not  rely  on  the  subjective  perception  of  an 
observer,  as  do  some  investigations  of  the  number  of  discernible 
gray  levels.  The  disadvantage  of  the  latter  approach  is  that  the 
number  of  perceived  gray  levels  is  altered  by  changing  the  prop¬ 
erties  of  the  display  (e.g.,  by  altering  the  lookup  table),  so  that  the 
results  are  no  longer  a  property  of  the  image  data  alone. 

In  Figure  22.1,  the  gray-level  behavior  of  a  photomultiplier 
tube  (PMT,  S20  photocathode,  QE  =  13%  and  n„  ~  0),  predicted  by 
Eq.  7,  is  compared  with  that  of  a  cooled  frame-transfer  charge- 
coupled  device  (CCD)  detector  ( QE  =  70%,  n„  =  9  RMS  elec¬ 
trons/pixel).  The  sensor  noise  of  the  PMT  has  been  neglected 
because  of  its  very  high  gain.1  The  behavior  of  an  ideal  detector 
( Qe  =  100%,  n„  =  0)  is  also  shown  for  comparison.  For  all  detec¬ 
tors  the  number  of  gray  levels  increases  from  unity  with  the 
number  of  photons  incident  on  each  pixel.  The  behavior  of  the 
PMT  is  limited  by  the  poor  quantum  efficiency.  It  is  clear  that  for 
greater  than  about  25  photons  incident  per  pixel  the  CCD  exhibits 
superior  performance,  whereas  for  a  smaller  number  of  photons 
the  photomultiplier  tube  is  better.  In  the  region  where  the 
photomultiplier  is  superior,  the  number  of  gray  levels  is  less  than 
1  +  13%  X  25,  or  about  three,  so  that  the  image  is  of  poor  quality. 


1  Actually,  the  PMT  has  considerable  multiplicative  noise  but,  in  this  discus¬ 
sion,  this  noise  has  been  accounted  for  by  assuming  a  fairly  low  QE. 


FIGURE  22.1.  The  number  of  gray  levels  discernible  in  an  image  formed 
using  various  detectors  under  different  illumination  conditions. 


Background  Noise 

Next,  we  can  incorporate  noise  emanating  from  the  background 
(stray  light)  in  the  measured  signal.  A  number  of  different  models 
for  stray  light  have  been  considered.  Here  we  consider  two  differ¬ 
ent  models.  In  the  first,  which  we  shall  term  Nl,  the  stray  light  is 
assumed  constant  in  intensity  over  the  detector  plane.  In  fluores¬ 
cence  this  would  be  an  appropriate  model  for  noise  that  originates 
from  autofluorescence  in  the  cements  and  coatings  of  the  optical 
elements  of  the  microscope  system,  particularly  the  objective  lens, 
as  in  single  molecule  fluorescence  studies.  It  is  also  a  good  model 
for  BSL  imaging  into  a  scattering  medium.  This  background  npBm 
can  be  assumed  to  be  distributed  uniformly  over  the  detector 
pinhole,  so  that  the  intensity  that  passes  through  the  pinhole  is  pro¬ 
portional  to  its  area  and  can  be  written 

Bm  =  avd/4’  (8) 

where  a  is  a  constant,  discussed  later,  and  vd  is  the  radius  of  the 
detector  in  normalized  coordinates 

vd={2Krd/X)smad,  (9) 

where  rc,  is  the  true  radius  of  the  pinhole  and  sinOQ  is  the  numer¬ 
ical  aperture  (NA)  of  the  system  in  detector  space  (=  nsina/M, 
where  n  sin  a  is  the  NA  of  the  objective  in  object  space,  and  M  is 
the  lateral  magnification  from  object  plane  to  detector).  A  value  of 
vd  =  3.83  corresponds  to  the  first  zero  of  the  Airy  disk,  that  is,  to 
1  Airy  unit.  For  simplicity,  the  X  of  the  fluorescent  radiation  is 
assumed  equal  to  that  of  the  incident  radiation  throughout  this 
chapter,  that  is,  there  is  no  Stokes  shift.  As  the  increase  in  X  for 
common  dyes,  such  as  fluoroscein  isothiocyanate  fluorescein,  rho- 
damine,  Texas  Red,  or  allophycocyanine,  is  usually  in  the  range 
3%  to  8%,  its  effect  of  the  behavior  of  the  system  is  unlikely  to  be 
very  significant. 

Model  N 1  has  been  found  useful  (Sheppard,  1991)  for  describ¬ 
ing  the  behavior  of  confocal  BSL  microscopy,  in  which  case  the 
background  can  come  from  specular  reflection  from  the  optical  ele¬ 
ments  and  light  scattered  from  within  the  microscope  body.  Of 
course,  in  fluorescence  microscopy,  both  these  contributions  are 
substantially  removed  by  the  barrier  filters.  Wells  and  colleagues 
(1990)  showed  in  their  experiments  on  fluorescence  microscopy 
that  N 1  was  not  a  good  model.  This  may  be  an  indication  that  auto¬ 
fluorescence  was  the  primary  cause  of  the  background.  Webb  and 
colleagues  (1990,  pp.  73-108)  have  considered  a  further  model  in 
which  the  signal  was  considered  to  originate  from  within  the  res¬ 
olution  element  while  detected  light  not  from  this  source  is  con¬ 
sidered  as  background.  This  model  has  the  disadvantages  that  the 


444  Chapter  22  •  C.J.R.  Sheppard  et  al. 


boundary  between  signal  and  background  is  arbitrary  and  that  the 
size  of  the  resolution  element  varies  with  the  parameters  of  the 
microscope.  It  also  assumes  an  infinite  field  together  with  a  finite 
object  thickness  (infinite  field  with  an  infinite  object  thickness 
gives  infinite  background  for  a  WF  microscope).  We  choose  to 
assume  a  finite  field  and  an  infinite  object  thickness  in  order  to 
stress  the  fact  that  reduction  of  the  field  size  is  an  important  factor 
in  improving  noise  performance.  Our  second  model  (N2)  assumes 
that  the  background  originates  uniformly  from  everywhere  within 
an  infinitely  large  fluorescent  object.  This  model  would  be  appro¬ 
priate  for  the  case  of  observing  fluorescence  labeling  in  the  pres¬ 
ence  of  background  object  autofluorescence.  In  this  case,  the 
intensity  that  passes  through  the  pinhole  npBN2 (v)  is  a  function  of 
the  pinhole  size,  as  will  be  described  in  the  next  section.  Both  these 
background  contributions  are  detected  by  the  photodetector  and 
the  statistical  variations  of  this  value  give  rise  to  noise  in  the 
image.  Thus,  the  overall  S/N  is  given  by 


S/N  -- 


QEnpF(Vd) 


^QEnPiF(Vd)  +aVd/4  +  BNliVd)\+nl  ’ 


(10) 


where  F(vd)  is  the  fraction  of  signal  light  incident  on  the  pinhole 
that  passes  through  to  the  detector. 


SIGNAL  LEVEL  IN  CONFOCAL  MICROSCOPES 


Consider  a  single  point  object  situated  at  the  focal  point  of  the 
microscope.  For  a  confocal  system  with  a  point  source,  the  inten¬ 
sity  in  the  detector  plane  is  an  Airy  disk 


'■W 


(11) 


where  Jn(v)  is  a  Bessel  function  of  order  n  and  the  factor  1/471  nor¬ 
malizes  the  total  intensity  in  the  detector  plane  to  unity.  The  nor¬ 
malized  signal  for  a  circular  detector  radius  vd  is  (Cox  and 
Sheppard,  1986) 


FM  = 


4k 


2nvdv 


=  l-Jo(vd)-Ji(vd\ 


(12) 


For  small  pinhole  radii,  we  have 


4. 


(13) 


For  a  slit  detector,  width  2vd,  placed  in  front  of  the  detector  (i.e., 
an  incoherent  slit  detector)  and  a  point  source,  the  normalized 
signal  we  denote  F2(vd).  Then  F2(vd )  is 


7 T  J 


2  f  Ht(2v) 

2 


dv 


(14) 


where  H,  is  a  Struve  function.  For  long  slits,  the  signal  from  a 
point  object  is  independent  of  the  length  of  the  slit.  For  small  vd 
the  normalized  signal  is  (Sheppard  and  Mao,  1988) 


F2{vd)  «  I6vd/3n2,  (15) 


so  that  the  signal  is  proportional  to  the  width  of  the  slit. 

For  a  square  detector,  side  2vd,  as  is  the  case  for  a  single  CCD 
element,  the  signal  can  be  evaluated  by  integrating  over  the  Airy 
disk.  As  the  signal  for  a  small  detector  is  proportional  to  the  area 
of  the  detector,  for  small  detectors  we  have 


F3  (vd)~vd/K- 


(16) 


FIGURE  22.2.  The  signal  level  from  a  point  object  as  a  function  of  detector 
aperture  shape  and  size. 


These  functions  are  illustrated  in  Figure  22.2.  It  is  seen  that  in  each 
case  the  normalized  signal  rises  from  zero  until  it  approaches  unity 
for  large  pinhole  diameters.  For  a  circular  pinhole  the  normalized 
signal  reaches  about  85%  at  vd  =  4,  and  thereafter  there  is  little 
increase  in  signal  level.  In  practice,  if  the  pinhole  diameter  is 
increased  further  the  signal  collected  from  a  thick  object  increases 
as  the  optical  sectioning  effect  is  weaker.  However,  this  increased 
signal  is  not  useful  as  it  represents  light  collected  from  out-of¬ 
focus  regions  of  the  specimen.  As  is  apparent  from  Figure  22.2, 
slit  apertures  can  give  increased  signal  strength,  but  this  is  associ¬ 
ated  with  poorer  optical  sectioning  performance  compared  with 
that  from  a  circular  aperture.  Figure  22.3  shows  the  effect  of 
pinhole  or  slit  width  on  various  imaging  properties  of  a  confocal 
fluorescence  microscope.  The  transverse  resolution  is  measured  as 
the  reciprocal  of  the  full-width  at  half-maximum  (FWHM)  of  the 
image  of  a  point  object,  normalized  by  that  for  a  point  detector. 
There  is  a  small  improvement  in  transverse  resolution  for  small 


-  Circular  aperture 

-  -  -  Slit  aperture 


Diameter  or  width  (Airy  units) 


FIGURE  22.3.  The  effect  of  pinhole  or  slit  size  on  various  imaging  proper¬ 
ties  of  a  confocal  fluorescence  microscope.  The  transverse  resolution  for  a 
pinhole  aperture  is  defined  as  the  reciprocal  of  the  full-width  at  half-maximum 
(FWHM)  of  the  image  of  a  point  object,  normalized  by  that  for  a  point  detec¬ 
tor.  The  axial  resolution,  for  pinhole  or  slit  apertures,  is  defined  as  the  recip¬ 
rocal  of  the  FWHM  of  the  image  of  a  thin  fluorescent  planar  object,  normalized 
by  the  FWHM  of  a  planar  object  in  a  confocal  BSL  microscope  with  point 
detector.  The  signal  level  from  a  planar  fluorescent  object  is  also  shown. 
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pinhole  sizes,  but  the  pinhole  radius,  vd  has  to  be  very  small  ( vd 
<  2)  for  this  to  be  appreciable.  The  behavior  is  identical  for  fluo¬ 
rescence  or  BSL  systems.  The  axial  resolution  is  measured  as  the 
reciprocal  of  the  FWHM  of  the  image  of  a  thin  fluorescent  planar 
object,  normalized  by  the  FWHM  of  a  planar  object  in  a  confocal 
BSL  microscope  with  point  detector.  The  axial  resolution  steadily 
degrades  as  the  pinhole  diameter  is  increased,  until  there  is  no 
7-resolution  for  a  large  detector,  equivalent  to  a  WF  microscope. 
Even  for  a  point  detector,  the  axial  resolution  is  poorer  than  for  a 
confocal  BSL  microscope.  The  signal  level  from  a  planar  fluores¬ 
cent  object  is  also  shown.  The  behavior  is  qualitatively  similar  to 
that  for  a  point  object,  as  shown  in  Figure  22.2.  As  the  pinhole 
radius  is  increased,  the  signal  level  increases,  but  the  resolution 
also  decreases,  so  that  there  is  an  optimum  behavior  for  some  inter¬ 
mediate  pinhole  radius.  Turning  now  to  the  slit  aperture,  although 
the  signal  level  is  greater  than  for  a  circular  pinhole  of  the  same 
width,  the  --resolution  is  worse.  Overall,  there  is  little  to  be  gained 
from  using  a  slit  aperture  from  the  point  of  view  of  image  quality. 

Experimental  results  (Sheppard  et  al.,  1991)  have  confirmed 
that  the  theoretically  predicted  behavior  is  observed  in  practice  for 
pinhole  radii  up  to  a  value  vd  of  about  100.  For  very  large  pinholes, 
optical  behavior  will  eventually  be  limited  by  other  apertures 
present  in  the  optical  system. 


SIGNAL-TO-NOISE  RATIO  FOR 
CONFOCAL  MICROSCOPES 


Qe,  N1,  and  Stain  Level 

Signal-to-noise  ratios  for  a  confocal  microscope  can  be  calculated 
from  Eq.  10.  In  the  following  we  shall  assume  for  simplicity  that 
the  sensor  noise  n„  is  zero  so  that 


s/n = 4q^tp 


V F{vd)  +  av2d/4  +  BN2{vd ) 


(17) 


In  the  rest  of  this  chapter  S/N  is  normalized  by  the  constant  factor 
v/Se^p- 

Let  us  consider  first  the  case  of  noise  model  N 1  so  that  BN2  is 
taken  as  zero,  and  after  normalization  (Sheppard  et  al.,  1991) 

F{vd) 


S/N  = 


^F(vd)  +  av2d/J4 


(18) 


d  '  dl 

For  a  circular  pinhole  and  small  values  of  vd,  using  Eq.  13  we  have 

v. 


S/N  = 


2Nl  +  a 


(19) 


so  that  a  represents  the  magnitude  of  the  background  from  noise 
model  N 1  relative  to  the  signal  for  a  small  circular  pinhole.  Thus, 
a  is  smaller  for  stronger  staining.  The  behavior  of  S/N  with  pinhole 
radius  for  different  strengths  of  background  is  shown  in  Figure 
22.4.  For  a  particular  value  of  a,  S/N  increases  as  the  pinhole 
radius  increases,  reaches  a  maximum  value  for  an  optimum 
pinhole  size,  and  then  decreases  as  1  tvd  as  the  pinhole  is  made 
larger.  For  a  wide  range  of  values  of  a,  the  optimum  pinhole  radius 
corresponds  to  values  of  vd  between  about  2  and  3.  Simply,  a  larger 
pinhole  lets  in  more  background  but  does  not  produce  a  signifi¬ 
cantly  stronger  signal.  Interestingly,  as  the  strength  of  the  signal 
relative  to  the  background  decreases  (i.e.,  a  increases),  the 
optimum  pinhole  radius  decreases  so  that,  contrary  to  expected 
behavior,  the  pinhole  should  actually  be  closed  down  for  very 
weak  signals.  For  a  value  of  a  of  unity,  for  example,  the  optimum 
pinhole  radius  has  decreased  to  vd=  2.3.  For  S/N  to  be  better  than 


FIGURE  22.4.  Variation  in  S/N  with  pinhole  size  for  different  strengths  of 
background  (noise  model  Nl:  background  from  autofluorescence  from  optical 
components). 


50%  of  that  achieved  at  the  optimum  pinhole  radius,  vd  must  be 
kept  in  the  range  between  0.8  and  5.6.  Using  a  larger  pinhole  radius 
further  decreases  S/N.  Thus,  using  a  pinhole  radius  that  is  too  large, 
as  well  as  not  giving  an  increase  in  useful  signal,  actually  results 
in  a  decrease  in  S/N  from  weakly  fluorescent  objects. 

On  the  other  hand,  it  should  be  noted  that  for  small  values  of 
a  the  maximum  in  S/N  is  broad,  so  that  choice  of  an  appropriate 
pinhole  diameter  is  not  critical  from  the  point  of  view  of  S/N. 

Thus,  if  the  background  is  weak,  the  pinhole  can  be  opened 
up  to  increase  the  number  of  detected  photons,  and  hence  S/N. 
Although  one  would  expect  this  to  reduce  the  z-resolution,  it  is 
very  common  for  the  z-resolution  to  be  limited  more  stringently 
by  low  signal  level  (and  the  resulting  inability  to  measure  pixel 
brightness  accurately)  than  by  purely  optical  considerations.  In  this 
case,  the  extra  signal  may  enable  one  to  make  a  better  estimate  of 
the  true  structure  of  the  dye  in  the  specimen  in  spite  of  the  large 
PSF. 

For  instance,  if  we  are  imaging  a  small,  bright,  spherical 
object  say  20  voxels  in  diameter,  surrounded  by  a  low  intensity 
background  (~1%  of  the  peak),  the  signal  obtained  from  a  voxel 
in  the  center  of  this  object  will  be  substantially  greater  and  the 
uncertainty  of  the  measurement  will  be  less  if  the  pinhole  is  opened 
from  1  to  perhaps  2  or  3  Airy  units.  Then  the  difference  between 
this  signal  and  that  from  a  nearby  background  voxel  located  above 
the  object  will  be  greater  but  the  absolute  value  of  this  background 
measurement  will  also  be  more  because  the  reduced  z-resolution 
means  that  the  sensed  volume  is  larger  and  contains  more  dye. 
Although  one  can  retrieve  some  of  this  lost  resolution  if  one  decon¬ 
volves  the  data  recorded,  using  a  PSF  measured  at  the  same 
pinhole  size,  this  will  be  decreasingly  effective  as  the  pinhole  gets 
bigger.  Specifically,  the  S/N  of  the  imaging  system  will  begin  to 
decrease  as  the  size  of  the  detection  PSF  exceeds  the  size  of  the 
object. 

From  this  discussion,  one  can  see  that  the  optimal  pinhole 
size  becomes  highly  dependent  on  the  details  of  the  3D  stain  dis¬ 
tribution  in  the  specimen  (both  signal  and  background),  and  that 
deconvolving  the  recorded  data  will  always  help.  However,  even 
though  one  can  calculate  the  S/N  for  some  limiting  conditions  (as 
is  done  below)  in  the  end,  one  must  often  determine  the  optimal 
pinhole  size  for  a  given  type  of  specimen  experimentally. 

For  large  values  of  a,  corresponding  to  weak  signals,  S/N  tends 
to  a  behavior  independent  of  the  value  of  a.  In  this  case,  again 
renormalizing  S/N,  we  obtain  for  a  circular  pinhole,  a  slit  detec¬ 
tor,  and  a  square  detector 
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FIGURE  22.5.  Variation  in  S/N  with  aperture  shape  and  size  for  weak  signal 
(noise  model  Nl):  (A)  circular  or  square  detector;  (B)  slit  detector  with  length 
2vs  =  2nvd. 


S/N  = 


Vttv,  ’ 


F2(Vd)  F3(Vd) 


2v . 


(20) 


respectively,  where  2vs  is  the  length  of  the  slit.  These  are  shown 
in  Figure  22.5.  The  peak  value  for  S/N  is  slightly  higher  for  a  cir¬ 
cular,  rather  than  a  square,  aperture.  For  the  circular  pinhole  the 
peak  S/N  occurs  at  about  0.52  Airy  units  ( vd  ~  2).  For  the  slit  aper¬ 
ture  the  optimum  S/N  occurs  at  0.48  Airy  units  (vd  =  1.84).  The 
S/N  is  decreased  relative  to  the  confocal  case  with  a  circular  detec¬ 
tor  by  a  factor  of  about  V77/2,  where  n  =  v. Jvd. 


N2  and  Detectability 

Turning  now  to  the  second  noise  model,  N2,  the  background  Bm 
resulting  from  a  featureless  volume  can  also  be  calculated  for  these 
various  systems  (Gu  and  Sheppard.  1991). 

The  intensity  in  the  detector  region  produced  by  a  volume 
object  is  given  by  the  convolution  of  the  intensity  point  spread 
function  of  the  illumination  lens  with  that  of  the  collection  lens. 
Assuming  these  are  identical,  this  is  conveniently  calculated  as  the 
3D  Fourier  transform  of  the  square  of  the  optical  transfer  function 
(OTF).  Then  the  normalized  intensity  in  the  plane  of  the  detector 
aperture  is 


/(v)  =  -kj(2-/)2(4  +  /)/0(v/)rf/.  (21) 

This  is  illustrated  in  Figure  22.6.  The  intensity  decays  monotoni- 
cally  with  distance  from  the  axis,  for  large  distances  decaying  as 
1/v3.  The  background  detected  can  then  be  calculated  by  integrat- 


FIGURE  22.6.  The  variation  in  the  intensity  of  the  fluorescence  arriving  at  the 
plane  of  the  detector  when  a  thick  featureless  volume  is  illuminated  by  an  axial 
focused  laser  beam.  The  horizontal  axis  is  calibrated  in  Airy  units,  with  zero 
being  the  optical  axis. 


ing  over  the  detector  aperture.  For  a  circular  pinhole  radius  vd  we 
obtain 


^2(vrf)  =  Tj(2-/)2(4  +  /)^Mrf/.  (22) 

Figure  22.7  shows  the  variation  in  background  with  pinhole  radius 
for  a  circular  pinhole.  For  small  pinhole  sizes  the  background  is 
vj/4  (as  in  Eq.  13),  that  is,  proportional  to  the  area  of  the  pinhole, 
but  for  large  pinholes  it  is  proportional  to  vd.  This  is  because  for 
large  vd  the  thickness  of  the  optical  section  is  proportional  to  vd. 
This  is  an  idealized  model:  the  situation  is  more  complex  when  the 
background  is  not  featureless  but  contains  structures  in  out-of- 
focus  planes. 

We  have  for  the  normalized  S/N  for  weak  signals 


S/N  = 


m  ' 


(23) 


We  have  termed  this  property  the  detectability  (Gan  and  Sheppard, 
1993)  of  the  system:  it  describes  the  ability  with  which  a  weakly 
fluorescent  point  object  can  be  detected  in  a  background  of  a  uni¬ 
formly  fluorescent  background.  Actually,  other  forms  of  detectabil¬ 
ity  have  also  been  defined  (Gan  and  Sheppard,  1993).  These 
describe  the  ease  with  which  a  point  object  can  be  detected  within 
a  background  uniform  over  a  plane  [the  two-dimensional  (2D) 
detectability],  and  the  ease  with  which  a  planar  object  can  be 
detected  within  a  background  volume  (the  axial  detectability). 
Flere  we  shall  only  discuss  the  3D  detectability,  appropriate  for 
detecting  a  point  in  a  background  volume. 


FIGURE  22.7.  The  background  from  a  thick  featureless  volume  as  a  function 
of  pinhole  radius. 
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FIGURE  22.8.  Variation  in  S/N  with  aperture  size  for  (A)  circular  and  (B)  slit  apertures,  for  weak  signal  (noise  model  N2:  background  from  autofluorescence 
from  specimen  bulk). 


The  S/N  for  N2  for  a  confocal  microscope  with  a  circular  or 
slit  detector  is  shown  in  Figure  22.8.  The  behavior  is  broadly  the 
same  as  for  the  case  of  N 1  in  that  S/N  reaches  a  maximum  value 
for  an  optimum  aperture  dimension,  which  is  vd  ~  2.4  for  the  cir¬ 
cular  pinhole,  and  vrf  ~  1  for  the  slit  aperture.  For  the  case  of  noise 
model  N2,  for  large  circular  pinholes  S/N  decays  as  1 

Multi-Photon  Fluorescence  Microscopy 

The  S/N  can  be  calculated  in  a  similar  way  for  multi-photon  fluo¬ 
rescence  microscopy  (Gauderon  and  Sheppard,  1999).  The  S/N  for 
model  N2  for  a  multi-photon  microscope  with  a  circular  detector 
is  shown  in  Figure  22.9.  In  this  case,  for  large  pinholes  the  S/N 
does  not  tend  to  zero  as  in  the  single-photon  fluorescence  micro¬ 
scope,  so  the  S/N  can  be  normalized  to  unity  for  large  pinhole 
diameters  (i.e.,  non-descanned  detection).  It  is  found  that  the  S/N 
exhibits  an  optimum  value  for  a  pinhole  radius  of  0.63  Airy  units 
(yd  =  2.42)  for  two-photon  fluorescence,  and  0.64  Airy  units  (yd  = 
2.44)  for  three-photon  fluorescence.  The  value  of  S/N  is  then  a 
factor  of  1.6  or  1.9  better  than  for  a  system  with  a  large  pinhole. 


FIGURE  22.9.  Signal-to-noise  ratio  for  two-photon  and  three-photon  fluores¬ 
cence  microscopes  (noise  model  N2:  background  from  autofluorescence  from 
specimen  bulk). 


for  two-photon  and  three-photon  fluorescence,  respectively.  These 
results  are  quantitatively  in  agreement  with  experimental  observa¬ 
tions  (Wang  and  Fraser,  1997). 

DESIGNS  OF  CONFOCAL  MICROSCOPES 

The  basic  form  of  the  confocal  microscope  uses  illumination  from 
a  point  source,  coupled  with  a  photodetector  spatially  limited  by  a 
pinhole.  Scanning  can  be  achieved  by  either  moving  the  specimen 
stage  relative  to  a  stationary  light  spot,  or  by  scanning  the  beam. 
The  most  common  method  of  beam  scanning  uses  galvomirrors, 
which,  as  is  also  the  case  for  stage  scanning,  is  limited  to  compar¬ 
atively  slow  scan  rates,  up  to  a  few  frames  per  second.  Resonant 
scanners  can  achieve  faster  scan  rates,  and  acousto-optic  deflection 
is  even  faster  but  results  have  other  problems  for  fluorescence 
imaging,  notably  the  fact  that  the  fluorescent  signal  cannot  be  de- 
scanned  in  a  simple  manner  because  of  the  Stokes  shift.  A  solution 
to  this  problem  is  to  use  a  slit-shaped  detector  aperture  and  to 
descan  only  in  one  direction  (Draaijer  and  Houpt,  1988). 

An  alternative  approach  for  increasing  scan  speed  is  to  scan 
simultaneously  with  an  array  of  points.  The  earliest  example  of 
this  method  is  the  tandem-scanning  microscope,  in  which  a  rotat¬ 
ing  disk  with  an  array  of  holes  is  used  (Petran  et  al.,  1968).  It  is 
also  possible  to  use  an  array  of  holes,  a  slit,  or  an  array  of  slits.  In 
the  line-illumination  microscope,  such  as  the  Zeiss  5  Live,  a  line 
of  the  specimen  is  illuminated  (Koester,  1980;  Sheppard  and  Mao, 
1988;  Benedetti  et  al.,  1992;  Brakenhoff  and  Visscher,  1992)  and 
the  image  of  the  whole  line  can  be  recorded  using  a  linear  CCD 
array. 

Figure  22.10  shows  schematic  diagrams  for  three  different 
systems  using  line-illumination  and/or  linear  CCD  detectors.  In 
Figure  22. 10(A)  a  scanning  spot  is  deflected  over  the  specimen  and 
partially  descanned  onto  a  CCD  detector.  In  Figure  22.10(B)  the 
scanning  spot  is  again  partially  descanned,  but  on  to  a  single  point 
detector  with  a  slit  aperture.  In  Figure  22.10(C),  the  object  is  illu¬ 
minated  by  a  line  of  light,  and  the  image  along  the  line  recorded 
by  a  CCD  detector.  By  including  a  coupled  display  v-scan,  the  2D 
image  can  be  observed  directly  or  recorded  using  a  2D  CCD  detec¬ 
tor.  These  various  systems  have  been  compared  by  Awamura  and 
Ode  (1992),  in  which  they  claim  that  arrangement  (A)  is  superior 
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FIGURE  22.10.  Schematic  diagrams  of  various  optical  arrangements  using 
line-illumination  and/or  linear  CCD  detectors. 


in  resolution  to  either  (B)  or  (C).  Arrangement  (A)  also  has  the 
major  advantage  for  metrological  work  that  geometrical  distortion 
is  absent.  In  (C),  speckle  noise  can  be  a  problem  in  BSL  imaging 
unless  the  spatial  coherence  of  the  illumination  is  destroyed. 
However,  (C)  has  the  major  advantage  for  fluorescence  imaging 
that  a  complete  line  of  information  can  be  recorded  simultaneously 
with  the  result  that  data  can  be  acquired  rapidly  and  with  less 
danger  of  fluorescence  saturation. 


SAMPLING 

For  WF  fluorescence  imaging,  the  Nyquist  sampling  spacing  is  vN 
=  jt/2.  As  the  first  zero  of  the  Airy  disk  occurs  at  v  =  3.83,  corre¬ 
sponding  to  1  Airy  unit  (AU),  the  Nyquist  sampling  spacing  is  vN 
=  jt/(2  x  3.83)  =  0.41  AU  =  1  AU/2.44.  If  the  image  is  sampled  at 
the  Nyquist  spacing,  or  more  closely,  in  the  absence  of  noise,  the 
exact  image  profile  can  be  recovered.  When  sampling  further  apart 
than  the  Nyquist  spacing,  aliasing  occurs.  Note  that  for  fluores¬ 
cence  imaging  with  sampling  at  the  Nyquist  spacing,  the  image 
can  be  recovered,  but  for  bright-field  imaging  sampling  must  be 
closer  than,  rather  than  equal  to,  the  Nyquist  spacing. 

For  a  detector  array  with  elements  each  of  length  2v*  to 
achieve  Nyquist  sampling  we  have  vd  =  tc/4  (in  optical  units).  With 
line  illumination  and  a  linear  detector  array,  imaging  along  the 
array  has  the  same  spatial  frequency  cut-off  as  for  WF  imaging  so 
that  sampling  at  the  Nyquist  rate  is  appropriate.  If  there  are  n 
samples  in  a  one-dimensional  (ID)  image,  the  field  of  view  is 

2vs  =  rat/2.  (24) 


If  the  number  stored  as  the  intensity  value  for  a  given  pixel  is  pro¬ 
portional  to  the  total  number  of  photons  striking  a  specific  area  of 
the  CCD,  then  the  resolution  of  the  image  data  is  degraded  some¬ 
what  compared  with  that  produced  by  digitizing  a  continuous, 
bandwidth-limited  signal  at  equally  spaced  time  points  (i.e.,  inte¬ 
gral  vs.  point  sampling).  The  resultant  point  spread  function  is  the 
one  without  sampling,  convolved  with  the  detector  sensitivity  dis¬ 
tribution  for  a  single  picture  element.  In  the  absence  of  noise,  the 
more  frequent  the  sampling,  the  closer  the  behavior  tends  to  the 
ideal  response. 


COMPARATIVE  PERFORMANCE  OF 
FLUORESCENCE  MICROSCOPES 

Bleaching-Limited  Performance 

The  comparative  performance  with  regard  to  S/N  of  various 
designs  of  the  fluorescence  microscope  can  be  considered  by  cal¬ 
culating  the  appropriate  signal  and  background  levels.  In  this 
section  we  consider  the  case  when  S/N  is  limited  by  bleaching  of 
the  fluorophore.  Thus,  a  finite  number  of  fluorescent  photons  can 
be  emitted  before  the  fluorophore  bleaches.  (This  assumes  that 
photobleaching  rate  is  independent  of  illumination  intensity.)  We 
term  this  regime  Rl. 

We  have  considered  a  wide  range  of  different  configurations 
including  confocal  microscopes  with  circular,  square,  or  slit  detec¬ 
tor  apertures,  and  point  source  or  finite-sized  source;  line  illumi¬ 
nation  with  coherent  or  incoherent  lines  having  vanishingly  small 
width,  or  finite  width;  disk-scanning  systems  with  arrays  of  aper¬ 
tures  of  various  geometries;  and  WF  microscopy  with  a  finite 
source  and  a  detector  array. 

Figure  22. 1 1  shows  five  different  configurations  of  microscope 
that  we  designate  as  follows: 

Cl.  Confocal  microscope  with  a  point  source  and  a  finite-sized 
detector  radius  vd. 

C2.  Line  illumination  with  a  coherent  line  source  length  2vs 
and  a  linear  array  of  square  detector  elements,  each  with 
dimensions  2vd. 

C3.  Confocal  microscope  with  a  point  source  and  a  slit  detector 
length  2y,  and  breadth  2vd. 

C4.  Disk-scanning  microscope  with  a  square  array  of  incoherent 
sources,  separation  vh,  radius  vd  and  a  similar  array  of 
detectors. 

C5.  WF  microscope  with  a  square  source,  sides  2vs,  and  a  2D  array 
of  square  incoherent  detectors  dimensions  2vd 

In  order  to  model  these  different  configurations  of  microscope  we 
need  to  know  the  relative  values  of  signal  in  the  various  systems. 
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FIGURE  22. 11.  Five  different  configurations  of  microscope. 
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FIGURE  22.12.  Four  different  illumination  models. 


We  consider  the  illumination  models  shown  in  Figure  22.12.  Great 
care  must  be  taken  in  obtaining  the  correct  normalization  for  the 
various  different  microscope  configurations.  The  appropriate 
assumption  is  that  the  total  power  incident  on  the  specimen  in 
forming  a  complete  2D  image  is  kept  constant: 

II.  A  point  source.  Normalizing  the  total  power  in  the  focal  plane 
to  unity,  the  image  of  a  point  source  in  the  object  plane  is  an 
Airy  disk 


relative  background  is  thus  7i:vJ:  4vdvs:  4vJ  for  circular,  slit,  and 
square  detectors,  respectively.  The  total  energy  that  illuminates  the 
sample  in  forming  a  complete  image  in  a  confocal  microscope  of 
a  section  is  proportional  to  n2. 

For  the  line-scanning  microscope,  a  whole  line  of  the  speci¬ 
men  is  illuminated  at  a  time.  Thus,  the  background  is  increased 
by  a  factor  n.  Thus  for  13  and  a  square  detector  (C2),  instead  of 
Eq.  8, 

Bm  =  nnv2.  (28) 

Similarly,  for  a  conventional  microscope  a  whole  field  is  illumi¬ 
nated  at  one  time  and  the  background  is  increased  by  a  factor  n2. 
For  12  and  a  square  detector  (C5) 

B  =  4  n2v2d.  (29) 

For  the  disk-scanning  microscope,  the  distance  between  the 
samples  is  taken  as  vh.  If  ns  is  the  number  of  samples  needed  to 
form  a  line  image  ns  =  vhl2vd.  We  also  introduce  the  number  of 
points  in  a  line  illuminated  simultaneously  nh  =  2 vjvh  so  that  n  = 
n,nh.  Again  the  total  energy  incident  in  forming  an  image  of  a 
section  is  n2.  The  background  for  the  disk-scanning  microscope  is 

B  =  nnfv2.  (30) 

For  line  illumination,  C2,  we  have 

S/N  =  -^2  (31) 


j_  1 

4jtI  v  J 


(25) 


so  that  the  intensity  at  the  focal  point  is  1/4tc. 

12.  The  pixel  is  illuminated  uniformly  with  intensity  l/4vi  This 
normalization  is  taken  so  that  the  total  exposure  in  recording  a 
complete  section  is  the  same  as  for  II,  as  a  conventional  micro¬ 
scope  images  the  whole  section  simultaneously. 

13.  The  image  of  a  long  line  source  gives  in  the  focal  plane. 


1  = 


1  f  sinv  V 
2nvd  V  v  J 


(26) 


The  line  illumination  microscope  images  a  whole  line  at  a  time. 

14.  The  image  of  a  circular  source  radius  vd,  for  vd  large  enough 
that  diffraction  effects  can  be  neglected.  This  will  be  approxi¬ 
mately  true  for  vd  larger  than  about  3. 


In  Figure  22.13,  the  dashed  line  is  a  plot  of  this  value  against  vd 
for  N  =  512.  Unlike  the  confocal  microscope  with  a  slit  detector 
(C3)  as  shown  in  Figure  22.5,  the  S/N  decreases  monotonically  for 
increasing  detector  element  size.  This  is  because  the  length  of  the 
detector  array  is  proportional  to  vd  if  the  detector  elements  are 
square.  For  vd  =  tc/4,  satisfying  the  Nyquist  criterion  the  S/N  is 
0.012. 

For  the  disk-scanning  microscope.  C4,  the  S/N  for  large  vd, 
using  14,  is 

S/N  =  (32) 

-Jnv-dnh 

On  the  other  hand,  for  illumination  by  an  array  of  points,  which  is 
a  good  approximation  for  small  values  of  vd, 

S/N  =  F^Vd)  (33) 

-^nvdnh 


For  the  alternative  illumination  schemes,  the  signal  is  scaled 
by  the  appropriate  intensity  at  the  focal  point.  In  particular,  for  12 
and  a  square  detector  (C5),  for  13  and  a  square  detector  (C2),  and 
for  14  and  a  circular  detector  (C4) 

5  =  (n/v2)F3(vdU2/vd)F3 (vrf),(4/v2)f<  (v,),  (27) 

respectively. 

The  comparative  behavior  has  been  investigated  for  back¬ 
ground  originating  either  from  the  optical  components  (noise 
model  Nl),  or  from  the  3D  bulk  of  the  specimen  (noise  model  N2). 
We  take  as  the  worst  case  a  weak  signal  compared  with  the  back¬ 
ground  (a  >  1 ),  so  that  the  noise  from  the  signal  fluctuations  is  neg¬ 
ligible  compared  with  that  of  the  background. 

When  using  noise  model  Nl,  the  background  is  proportional 
to  the  area  of  the  detector  aperture.  For  the  confocal  system,  the 


Diameter  or  width  (Airy  units) 

FIGURE  22.1 3.  Signal-to-noise  ratio  for  line  illumination  C2,  noise  model  Nl 
(dotted),  the  disk  scanning  microscope  C4,  noise  model  Nl  (lower),  and  the 
conventional  microscope  C5,  noise  model  Nl  (upper). 
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S/N 


FIGURE  22.14.  The  variation  with  detector  element  size  in  S/N  for  incoher¬ 
ent  line  illumination  (noise  model  N2,  width  of  line  equal  to  detector  element, 
512  samples). 


Let  us  assume  that  v,„  the  hole  spacing,  is  40,  corresponding  to  Nh 
=  20.1  for  N—  512  and  sampling  at  the  Nyquist  rate.  Then  the  vari¬ 
ation  in  S/N  is  shown  as  the  lower  line  in  Figure  22.13.  The 
optimum  S/N  of  -0.006  occurs  at  about  0.52  Airy  units  {vd  -  2). 
The  S/N  decays  quickly  with  increasing  hole  diameter  as  the 
source  hole  illuminates  a  larger  region  of  the  object  as  well  as  the 
detector  hole  collecting  more  background. 

For  the  WF  microscope,  C5, 

S/N  =  ]^A  (34) 

2v]n 

which  is  plotted  as  the  upper  line  in  Figure  22.13.  Again,  the  S/N 
decays  quickly  with  vd  as  the  area  of  illumination,  that  is,  the  held 
of  view,  increases  as  the  area  of  a  detector  element  is  made  larger. 
At  the  Nyquist  sampling  rate  the  signal  to  noise  ratio  is  -0.001. 

The  thickness  of  the  object  is  assumed  infinite,  so  that  for  noise 
model  N2,  finite  dimensions  for  the  length  of  line  illumination,  for 
the  finite  source  in  WF  microscopy,  or  for  the  array  in  the  disk¬ 
scanning  microscope  are  assumed.  These  are  assumed  to  be  just 
large  enough  for  Nyquist  sampling  of  an  n  X  n  image  (Eq.  23). 

As  an  example  of  S/N  for  noise  model  N2  for  a  line-illumina¬ 
tion  system,  with  an  incoherent  line  source  of  width  2 vd  and  of 
length  2 nvd  and  a  detector  element  that  is  square  with  sides  2vd,  is 
shown  in  Figure  22.14  for  n  =  512.  The  S/N  reaches  an  optimum 
value  at  about  0.29  Airy  units  (vd  =  1.1).  For  a  very  narrow  line 
illumination  the  optimum  width  of  detector  element  is  slightly 
larger  and  S/N  falls  off  more  slowly  with  vd.  The  value  for  the 
optimum  S/N  is  much  the  same  in  either  case. 

Table  22.1  summarizes  results  for  S/N  according  to  noise 
models  N 1  and  N2  for  the  various  forms  of  microscope.  Roughly, 
compared  with  a  confocal  microscope  with  optimum  pinhole 
diameter,  line-illumination  or  slit-detection  systems  are  a  factor  of 
2  worse  for  noise  model  N2,  while  conventional  microscopes  are 
about  an  order  of  magnitude  worse. 

Saturation-Limited  Performance 

If  the  behavior  of  the  various  systems  is  limited  by  saturation, 
rather  than  bleaching,  their  relative  merits  are  very  different.  In 
this  case,  which  we  call  regime  R2,  the  limit  to  the  number  of 
photons  detected  per  pixel  is  set  not  by  the  total  energy  incident 
per  pixel,  but  by  the  illumination  intensity.  For  a  given  illumina¬ 
tion  power  striking  the  specimen,  the  line-illumination,  disk¬ 
scanning,  or  WF  systems  have  the  advantage  that  numerous  pixels 


are  imaged  simultaneously.  For  line  illumination  n  pixels  are 
imaged  in  parallel,  so  that  for  a  given  illumination  power  the  back¬ 
ground  in  forming  a  complete  image  is  reduced  by  a  factor  n,  so 
that  S/N  is  increased  by  a  factor  Vn  compared  with  a  point  source. 
Similarly,  for  a  WF  microscope,  n 2  pixels  are  imaged  in  parallel, 
and  S/N  is  multiplied  by  n.  The  relative  S/N,  assuming  all  are  oper¬ 
ated  at  the  onset  of  saturation,  are  shown  in  Table  22. 1 .  Under  these 
conditions,  the  line-illumination  microscope’s  behavior  is  superior 
to  the  confocal  microscope  with  a  circular  pinhole. 

Effects  of  Scanning  Speed 

If  our  specimen  is  unchanging  with  time,  we  can  in  principle  take 
as  much  time  as  we  like  in  forming  an  image.  Thus,  we  can  always 
scan  slowly  enough  that  saturation  effects  are  insignificant.  The 
S/N  behavior  is  then  limited  by  bleaching  (regime  Rl),  and  the 
confocal  microscope  with  a  point  source  and  a  circular  pinhole  of 
optimum  diameter  is  clearly  the  best  performer.  If  we  choose  to 
speed  up  the  scan  so  that  a  complete  2D  image  is  recorded  in  some 
particular  finite  time,  then  the  confocal  microscope  with  a  point 
source  will  reach  saturation  of  the  fluorophore  before  the  other 
systems  because  the  dwell  time  on  each  pixel  must  be  shorter.  We 
can  define  a  characteristic  time  Tc  as  the  time  per  pixel  needed  to 
generate  a  2D  image  while  just  avoiding  saturation.  The  charac¬ 
teristic  time  Tc  is  a  property  of  the  particular  fluorophore  that 
depends  critically  on  its  environment.  It  is  thus  the  energy  per  pixel 
that  can  be  incident  on  the  pixel  before  photobleaching  of  the  flu¬ 
orophore  (saturation  to  the  1/e  point)  divided  by  the  maximum 
power  that  can  be  incident  on  the  pixel  without  significant  satura¬ 
tion  (the  power  needed  to  give  an  output  fluorescent  intensity  that 
is  half  its  saturated  value).  The  frame  scan  time  for  onset  of  satu¬ 
ration  is  reduced  by  a  factor  ViV  for  a  line-scanning  system  and 
by  N  for  a  WF  microscope.  In  the  regime  R2,  where  behavior  is 
limited  by  saturation,  S/N  is  proportional  to  the  square  root  of  the 
imaging  time.  The  S/N  behavior  is  thus  as  shown  in  Figure  22.15. 

Let  us  consider  a  particular  example.  Suppose  the  fluorophore 
is  such  that  an  incident  power  of  1  mW  is  just  enough  to  cause  sat¬ 
uration,  and  that  bleaching  occurs  after  10  frames  when  scanning 
at  3  s/frame  in  a  confocal  microscope  with  point  source.  Then  the 
best  S/N  is  achieved  if  all  10  frames  are  averaged  to  collect  the 
maximum  possible  number  of  photons.  The  dwell  time/pixel/frame 
is  about  lOps,  so  that  the  total  dwell  time  (per  pixel)  for  10  frames 
is  100  |is.  In  this  case,  the  characteristic  time  Tc  for  the  fluorophore 
in  the  confocal  microscope  is  also  lOOps.  For  collection  of  10 
frames  at  3  s/frame  (dwell  time  100  (is),  the  point-source  confocal 
microscope  is  clearly  the  best  performer. 


TABLE  22.1.  Signal-to-Noise  Ratio 


N1 

Noise  model 

N2 

Regime  Rl 

Confocal 

1.0 

1.0 

Line  illumination 

0.07 

0.5 

Slit  detector 

0.07 

0.4 

WF 

0.006 

0.08 

Regime  R2 

Confocal 

1.0 

1.0 

Line  illumination 

1.6 

11.0 

Slit  detector 

0.07 

0.4 

WF 

3.0 

40.0 

WF,  widefield. 
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FIGURE  22.15.  Effect  of  pixel  dwell  time  on  S/N:  (A)  Noise  model  Nl;  (B) 
noise  model  N2. 


As  discussed  in  a  later  section,  these  10  frames  can  be  10  dif¬ 
ferent  axial  sections,  and  the  noise  still  tends  to  average  out  when 
forming  a  projection  as  the  total  number  of  photons  per  display 
pixel  is  increased.  When  forming  a  single  image  at  3  s/frame,  the 
line-illumination  system  achieves  slightly  better  S/N  according  to 
noise  model  N2  [Fig.  22.10(B)],  whereas  the  WF  microscope  still 
exhibits  inferior  S/N.  For  ten  frames  at  television  rates,  the  line- 
illumination  performs  best  with  respect  to  S/N,  while  for  a  single 
television-rate  frame  both  the  line-illumination  and  the  WF  micro¬ 
scope  have  better  S/N  than  for  the  point  source  confocal  micro¬ 
scope.  (Note  that  it  is  not  possible  to  read  out  the  cooled-CCD 
detector  we  have  assumed  for  the  WF  case  at  video  rates  without 
degrading  its  performance.)  However,  for  noise  model  Nl,  where 
the  background  comes  from  autofluorescence  of  the  optical  com¬ 
ponents,  the  confocal  microscope  maintains  its  superior  S/N  per¬ 
formance  even  for  high  scan  speeds. 

3D  IMAGING 

Once  a  3D  data  set,  representing  the  intensity  variations  in  the 
image,  is  stored  in  the  computer  imaging  system,  in  many  cases  it 
is  necessary  to  generate  a  projection  through  the  3D  object  rather 
than  a  single  section.  There  are  a  number  of  ways  of  doing  this, 
but  the  most  usual  are  the  mean-projection  algorithm  (Wilson  and 
Hamilton,  1982;  Sheppard  et  al.,  1983),  and  the  peak-projection 


algorithm  (Cox  and  Sheppard,  1983).  These  two  methods  result  in 
substantially  similar  images,  with  the  mean-projection  method 
giving  lower  apparent  noise  levels,  but  lower  levels  of  contrast  as 
well,  which  makes  it  particularly  sensitive  to  the  correct  setting  of 
the  zero  signal  level.  Generation  of  two  projections  in  different 
directions  gives  a  stereo-pair.  The  process  of  generating  a  projec¬ 
tion  improves  its  noise  performance  compared  with  a  single 
section  (Roy  and  Sheppard,  1993).  This  is  the  case  for  both  mean 
and  peak  projections,  although  the  mean-projection  method  is 
more  effective  at  reducing  noise.  The  mean-projection  method 
increases  S/N  as  the  square  root  of  the  number  of  sections 
processed  that  have  significant  signal:  only  four  such  sections  are 
required  to  increase  S/N  by  a  factor  of  2. 2 

The  S/N  behavior  of  the  peak- projection  method  depends  on 
the  statistical  properties  of  the  signal  and  the  display  lookup  table. 
For  large  numbers  of  photons  per  pixel,  the  noise  level  is  reduced 
by  a  factor  of  2  by  processing  24  sections,  while  for  just  two 
photons  per  pixel,  thousands  of  sections  are  required  to  double  the 
S/N. 

In  general,  in  order  to  produce  a  projection  with  a  given  S/N, 
it  is  not  necessary  for  each  section  to  have  that  S/N.  The  mean- 
projection  algorithm  reduces  noise  as  a  direct  result  of  the  aver¬ 
aging  inherent  in  the  method,  no  averaging  of  the  individual 
sections  being  necessary.  There  will  still  then  be  noise  on  the  depth 
information  in  a  stereo-pair  presentation,  but  this  is  a  second-order 
effect  and  is  expected  to  be  much  less  important  perceptually. 

SUMMARY 

•  Care  is  necessary  in  selecting  the  various  system  parameters, 
such  as  pinhole  diameter,  type  of  detector,  pixel  size,  illumi¬ 
nation  intensity,  and  scan  rate,  in  order  to  optimize  the  perfor¬ 
mance  of  the  microscope  system. 

•  Optimum  operation  will  also  depend  on  the  shape  of  the  stain 
in  the  object,  how  the  image  is  to  be  processed,  and  what  infor¬ 
mation  is  eventually  required  from  the  image. 

•  Usually  a  relatively  small  setting  for  pinhole  diameter  results 
in  improved  S/N,  and  a  circular  confocal  aperture  of  about  0.63 
times  the  diameter  of  the  first  dark  ring  of  the  Airy  disk  is  a 
good  benchmark  for  optimum  performance. 

•  Even  if  a  single  frame  looks  noisy,  the  noise  may  be  reduced 
by  the  generation  of  projections,  especially  using  the  mean- 
projection  algorithm  and  an  even  greater  improvement  can  be 
gained  by  deconvolving  the  dataset  in  3D. 

•  The  S/N  performance  of  a  point-source  confocal  microscope 
is  most  noticeably  better  than  other  forms  in  the  presence  of 
background  from  autofluorescence  from  the  cements  and  coat¬ 
ings  of  the  optical  elements. 

•  Point-source  confocal  microscopes  are  preferable  for  genera¬ 
tion  of  high-quality  3D  data  sets  when  there  is  no  limit  on  the 
time  available  for  a  single  frame. 

•  Line-scanning  systems,  or  even  WF  microscopes  with  cooled- 
CCD  detector  arrays,  can  give  superior  S/N  performance  for 
fast  scanning,  particularly  in  the  presence  of  background  con¬ 
sisting  of  autofluorescence  from  around  the  stained  object. 

•  In  both  these  cases,  the  illumination  field  should  be  reduced 
so  that  it  is  not  much  larger  than  the  observed  region  to 
increase  S/N. 


2  As  the  process  of  calculating  the  mean  value  at  each  pixel  reduces  the  bright¬ 
ness,  the  resulting  image  may  be  hard  to  see  unless  the  contrast  is  expanded 
using  a  simple  lookup  table.  Doing  so  does  not  decrease  the  S/N. 
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Comparison  of  Widefield/Deconvolution  and  Confocal 
Microscopy  for  Three-Dimensional  Imaging 

Peter  J.  Shaw 


INTRODUCTION 

The  biggest  limitation  inherent  in  optical  microscopy  is  its  lateral 
spatial  resolution,  which  is  determined  by  the  wavelength  of  the 
light  used  and  the  numerical  aperture  (NA)  of  the  objective  lens. 
Another  important  limitation  is  the  resolution  in  the  direction  of 
the  optical  axis,  conventionally  called  z,  which  is  related  to  the 
depth  of  field.  The  presence  of  a  finite  aperture  gives  rise  to  unde¬ 
sirable  and  rather  complicated  characteristics  in  the  image.  In 
essence,  the  depth  of  field  depends  on  the  size  of  structure  or 
spatial  frequency  being  imaged.  Fine  image  detail,  which  is  gen¬ 
erally  of  most  interest,  has  a  small  depth  of  field,  and  only  features 
within  a  small  distance  of  the  focal  plane  contribute  to  the  image. 
On  the  other  hand,  large  structures  —  low  spatial  frequency  com¬ 
ponents  —  have  a  relatively  large  depth  of  field,  and  contribute  to 
the  detected  image  seen  at  distant  focal  planes.  This  is  very  notice¬ 
able  in  dark-field  imaging  modes,  such  as  epi-fluorescence,  and 
means  that  the  fine  image  detail  may  be  swamped  by  low  resolu¬ 
tion  “out-of-focus”  light  and  thus  either  lost,  or  visualized  with 
very  much  reduced  contrast. 

The  principal  advantage  of  confocal  microscopy  for  biologi¬ 
cal  imaging  is  that  the  optical  arrangement  has  the  effect  of  elim¬ 
inating  much  of  the  out-of-focus  light  from  detection,  therefore 
improving  the  fidelity  of  focal  sectioning  (and  hence  the  three- 
dimensional  imaging  properties),  and  increasing  the  contrast  of 
the  fine  image  detail.  But  the  rejection  of  the  out-of-focus  light 
necessarily  means  that  a  proportion  of  the  light  emitted  by  the 
specimen  is  intentionally  excluded  from  measurement.  All  illumi¬ 
nation  of  the  specimen  has  deleterious  effects  —  bleaching  of  the 
fluorochrome  or  phototoxicity  to  living  cells.  These  specimen- 
dependent  factors  are  the  ultimate  limitation  to  the  quality  of  the 
image,  and  inevitably  confocal  imaging  does  not  detect  much  of 
the  emitted  light. 

An  alternative  way  of  removing  the  out-of-focus  light  involves 
recording  images  at  a  series  of  focal  planes  using  a  conventional 
microscope,  often  called  widefield  (WF)  to  distinguish  it  from 
confocal  imaging,  and  then  using  a  detailed  knowledge  of  the 
imaging  process  to  correct  for  it  by  computer  image  processing. 
This  procedure  is  called  deconvolution,  and  its  application  to  bio¬ 
logical  problems  actually  preceded  the  widespread  introduction  of 
biological  confocal  microscopes  (Castleman,  1979;  Agard  and 
Sedat,  1983;  Agard  et  al,,  1989).  In  contrast  to  confocal  imaging, 
up  to  30%  of  the  total  fluorescent  light  emitted  by  the  specimen 
can  be  recorded  (i.e.,  all  the  light  that  can  be  collected  by  a  single, 
high-NA  objective).  This  chapter  examines  the  question:  Is  it 
better  to  record  all  the  light  emitted  and  process  the  WF  images  to 


redistribute  the  out-of-focus  light  to  produce  a  more  accurate  three- 
dimensional  (3D)  image,  or  to  exclude  the  out-of-focus  light  from 
measurement  in  the  first  place  by  confocal  optics  and  then  decon¬ 
volve  the  confocal  data? 


THE  POINT  SPREAD  FUNCTION:  IMAGING  AS 
A  CONVOLUTION 

In  order  to  derive  a  soundly  based  description  of  the  degradation 
introduced  by  an  optical  microscope,  especially  if  any  attempt  is 
to  be  made  to  reverse  this  degradation,  it  is  necessary  to  be  able 
to  describe  the  relation  between  the  specimen  and  its  optical  image 
in  mathematical  terms.  We  shall  give  here  a  very  condensed  expla¬ 
nation  —  the  interested  reader  is  referred  elsewhere  for  more  rig¬ 
orous  mathematical  derivations  (Agard  et  al.,  1989;  Shaw,  1993; 
Young,  1989;  and  Chapters  20,  21,  22,  24,  and  25,  this  volume). 
Within  some  quite  general  limitations,  the  object  (specimen)  and 
image  are  related  by  an  operation  known  as  convolution.  In  a  con¬ 
volution,  each  point  of  the  object  is  replaced  by  a  blurred  image 
of  the  point  having  a  relative  brightness  proportional  to  that  of  the 
object  point.  The  final  image  is  the  sum  of  all  these  blurred  point 
images.  The  way  each  individual  point  is  blurred  is  described  by 
the  point  spread  function  (PSF),  which  is  simply  the  image  of  a 
single  point.  This  is  illustrated  diagrammatically  in  Figure  23.1. 

The  conditions  that  must  be  met  for  an  imaging  process  to  be 
described  as  a  convolution  are  that  it  should  be  linear  and  shift 
invariant  (Young,  1989).  Imagine  cutting  the  specimen  into  two 
parts  and  imaging  each  part  separately  with  the  microscope.  If 
adding  these  two  subimages  together  produces  the  same  result  as 
imaging  the  whole  specimen,  and  does  this  irrespective  of  how  the 
specimen  is  cut  up,  then  the  imaging  is  said  to  be  linear.  If  the 
imaging  is  indeed  linear,  then  the  specimen  can  be  imagined  cut 
up  into  smaller  and  smaller  pieces,  until  the  size  of  each  piece  is 
well  below  the  resolution  limit,  and  can  be  considered  to  be  simply 
a  point.  The  image  is  then  the  sum  of  the  images  of  each  of  the 
points,  each  multiplied  by  a  function  corresponding  to  the  amount 
of  light  coming  from  that  point.  The  multiplication  and  summing 
is  represented  mathematically  by  an  operation  called  convolution. 
Shift  invariance  simply  means  that  the  imaging  characteristics  and 
thus  the  PSF  are  the  same  over  the  whole  field  of  view,  and 
knowing  one  PSF  is  enough  to  characterize  the  imaging  properties 
of  the  microscope  (Agard  et  al.,  1989;  Shaw  and  Rawlins,  1991a). 

Although  imaging  modes  such  as  phase  contrast  and  differen¬ 
tial  interference  contrast  (DIC)  are  not  linear,  because  their  con¬ 
trast  depends  on  differences  of  refractive  index  within  the  object, 
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FIGURE  23.1 .  Diagram  showing  how  a  single  point  is  imaged  as  the  PSF  by 
a  microscope,  and  thus  that  the  image  of  an  extended  object  is  the  convolution 
of  the  object  with  the  PSF. 


both  WF  and  confocal  epi-fluorescence  microscopy  are  linear  and 
shift  invariant  processes  to  a  good  approximation  (Young,  1989; 
Wilson,  1993). 

Restating  the  foregoing  discussion  mathematically,  we  will 
denote  the  object  as  a  function  of  position  /(r)  and  the  resulting 
image  as  g(r),  and  represent  the  imaging  operation  by  I: 

I[/(r)]  =  g(r)  (1) 

Mathematically,  linearity  means  that  a  linear  combination  of 
objects  produces  the  same  linear  combination  of  images: 

WiM + *2/2  W +•••]  =  WiW] + WzW] +••• 

=  ^l(r)+^2(r)+-"  (2) 

where  kt  and  k2  are  constants,  that  is,  the  imaging  operation  is 
applied  individually  to  each  component  in  the  sum. 

If  linearity  and  shift  invariance  hold,  the  image  of  an  object 
may  be  described  as  the  sum  of  the  images  of  its  parts.  We  may 
approximate  the  object  as  closely  as  we  like  as  the  sum  of  points 
of  suitably  weighted  intensity.  We  denote  a  set  of  regularly  spaced 
sampling  points  by  the  “comb"  function  5(r  -  s,)  —  a  set  of  spikes 
at  the  points  s„  spaced  at  As.  Then  the  object  can  be  represented 
by  the  sum: 

f(r)  =  g/(si)5(r-si)As  (3) 

or,  in  the  limit: 

+00 

/'(«•)=  J/(s)5(r-s)fife  (4) 

The  image  of  each  point  5(r)  is  simply  the  point  spread  function 
—  we  shall  call  it  o(r):  That  is, 

I[5(r)]  =  o(  r) 

then,  I[5(r-s)]  =  o(r-s)  (5) 

and 


I  [/'(r)]  =  / 


J  /(s)6(r-s)r/s 


=  J/(s)[S(r-s)]ds 

+00 

=  J/(s)o(r-s)<fe 

=  g0) 


(6) 


because  I  can  be  taken  inside  the  integral  from  Eq.  2,  and  sub¬ 
stituting  from  Eq.  5.  Thus,  the  image  is  the  convolution  (®)  of  the 
object  with  the  point  spread  function.  More  concisely: 

g( r)  =  /( r)  ®  o(r)  (7) 

Convolutions  are  often  more  easily  handled  as  Fourier  trans¬ 
forms  (FT).  The  convolution  theorem  shows  that  the  FT  of  a  con¬ 
volution  of  two  functions  is  simply  the  product  of  the  individual 
transforms  of  the  functions.  Thus,  the  FT  of  the  image  is  given  by 
the  multiplication  of  the  FT  of  the  object  by  the  FT  of  the  PSF 
(usually  called  the  optical  transfer  function  or  OTF), 

G(S)  =  F(S)0(S)  (8) 

where  F( S),  G(S),  and  O(S)  are  the  Fourier  transforms  of/(r),  g(r), 
and  o(r),  respectively.  Thus  the  FT  of  the  image  is  the  transform 
of  the  object  multiplied  point-by-point  by  a  weighting  function.  An 
image  processing  operation  of  this  type  is  often  termed  a  filter. 

The  difference  in  the  images  produced  by  WF  and  confocal 
imaging  can  be  regarded  simply  as  a  difference  in  their  respective 
PSFs.  In  the  WF  case,  both  theory  (Stokseth,  1969;  Castleman, 
1979)  and  measurement  using  subresolution  fluorescent  beads 
(Hiraoka  et  al.,  1988),  show  that  the  PSF  has  the  form  of  concen¬ 
tric  cones  diverging  from  either  side  of  the  focal  plane;  they  inter¬ 
sect  the  focal  plane  to  give  the  Airy  pattern  of  concentric  rings. 
Figures  23.2(A)  and  23.3(A)  show  an  example  of  the  widefield 
PSF  for  a  high  numerical  aperture  objective  (Leitz  plan  apochro- 
mat,  63x,  NA  1.4),  determined  from  optical  sections  of  subresolu¬ 
tion  fluorescent  beads.  Figure  23.2(A)  shows  some  of  the  original 
WF  optical  sections  through  a  single  fluorescent  bead.  The  Airy 
disk  can  be  seen  expanding  into  a  series  of  concentric  rings.  In 
Figure  23.3(A),  the  raw  data  have  been  cylindrically  averaged  and 
smoothed  by  fitting  3D  spline  functions  to  it.  A  central  section 
through  the  3D  PSF  parallel  to  the  optical  axis  is  shown.  The  rings 
constitute  a  diverging  series  of  cones,  which  can  be  seen  in  section 
as  subsidiary  maxima  diverging  away  from  the  central  focal  plane. 
In  fact,  the  out-of-focus  rings  can  often  be  detected  extending 
many  micrometers  on  either  side  of  the  central  maximum  if  the 
data  are  sufficiently  accurate  (see  Hiraoka  et  al.,  1988,  1990). 

In  the  ideal  case,  the  total  integrated  intensity  at  each  out-of¬ 
focus  plane  is  the  same  as  that  at  the  focal  plane,  but  in  practice  the 
intensity  level  of  a  typical  point  object  such  as  a  small  fluorescent 
bead  drops  below  the  noise  level  of  a  charge-coupled  device  (CCD) 
camera  a  few  micrometers  away  from  the  focus  plane.  In  the 
absence  of  aberrations,  the  PSF  has  rotational  symmetry  about 
the  optical  axis,  and  reflection  symmetry  about  the  plane  of  focus. 
The  PSF  shown  here  is  clearly  far  from  being  symmetrical  in  the 
axial  direction.  This  is  due  to  spherical  aberration  and  is  very 
common  when  imaging  typical  biological  fluorescence  specimens. 
Microscope  objective  lenses  are  designed  to  image  optimally  a 
specimen  immediately  beneath  a  coverslip  of  the  correct  thickness. 
Typical  specimens  used  in  3D  biological  microscopy  are  rather 
thick,  and  so  there  is  an  additional  optical  path  length  through  a 
layer  of  water,  glycerol,  or  other  mounting  medium  that  leads  to 
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FIGURE  23.2.  Sections  through  measured  WF  (A)  and  confocal  (B)  PSFs.  The  objective  lens  was  a  planapochromat  63x,  NA  1.4  (Leitz).  Confocal  detector 
aperture  size  was  0.7  Airy  units.  In  order  to  emphasize  the  fainter  components,  the  square  root  of  the  intensity  is  shown.  Spacing  between  sections  (z)  is  0.4  pm. 
Bar  =  1  pm. 


increased  spherical  aberration  (Chapters  7  and  20,  this  volume). 
Hiraoka  and  colleagues  (1990)  have  shown  that  this  can  be  com¬ 
pensated  for,  to  some  extent,  by  modifying  the  refractive  index  of 
the  immersion  oil,  and  computer-controlled  “add-on”  optical 
systems  are  now  available  for  the  correction  of  this  error. 

Departures  from  rotational  symmetry  have  also  been  noted 
(Hiraoka  et  al.,  1988),  due  either  to  misalignment  of  the  illumina¬ 
tion  system  with  the  microscope  optical  axis  or  to  misalignment 
of  the  optical  surfaces  within  the  objective  itself.  (See  Chapters  11 
and  20,  this  volume,  for  a  more  complete  discussion  of  the  effect 
of  various  aberrations  on  confocal  imaging.) 

The  usual  way  to  measure  a  WF  image  is  by  means  of  con¬ 
ventional  epi-fluorescence  optics.  The  entire  specimen  is  illumi¬ 


nated,  ideally  by  filling  the  back  aperture  of  the  objective  with  light 
emanating  from  an  extended,  perfectly  uniform  light  source.  The 
importance  of  the  light  source  has  received  much  less  attention  in 
epi-fluorescence  imaging  than  in  transmitted  light  imaging  such  as 
DIC  (Inoue,  1986),  but  it  is  equally  important  to  obtain  a  uniform 
PSF  (see  Chapter  6,  this  volume).  The  resulting  image  is  projected 
by  suitable  relay  optics  onto  an  image  detector.  Currently,  scien¬ 
tific-grade,  cooled  CCD-array  cameras  are  the  best  image  acquisi¬ 
tion  devices  for  this  type  of  microscopy  (Hiraoka  et  al,  1988; 
Aikens  et  al,  1989).  It  has  also  been  shown  that  a  type  I  scanning 
microscope,  in  which  only  a  single  aperture  is  used,  is  equivalent 
to  the  WF  arrangement  (Sheppard  and  Choudhury,  1977).  In 
theory,  opening  the  detector  aperture  of  a  confocal  laser  scanning 


FIGURE  23.3.  The  PSFs  shown  in  Figure  23.2  have 
been  fitted  by  3D  cubic  splines  and  cylindrically  aver¬ 
aged.  In  each  case  a  central  x,z  section  is  shown.  The 
square  root  of  the  intensity  is  shown,  and  false  gray  level 
contouring  has  been  used.  The  direction  of  the  optical 
axis  is  vertical.  The  divergent  cones  are  clearly  seen  in 
the  WF  data,  but  much  reduced  in  the  confocal  data.  (A) 
widefield  PSF,  (B-D)  confocal  PSFs  with  different 
detector  apertures;  (B)  4.3  Airy  units,  (C)  2.5  Airy  units, 
(D)  0.7  Airy  units.  The  optical  axis  is  vertical,  the  radial 
axis  is  horizontal. 
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microscope  (CLSM)  infinitely  wide  should  produce  the  same 
imaging  as  WF  optics.  In  practice,  it  is  not  possible  to  have  a  large 
enough  effective  detector  aperture  to  obtain  WF  behavior;  if  this 
were  possible,  the  comparison  between  WF  and  confocal  imaging 
for  a  given  instrument  would  be  much  easier.  Currently,  only  the 
non-descanned  detectors,  commonly  found  on  single-beam  multi¬ 
photon  excitation  fluorescent  microscopes,  closely  mimic  the 
optics  of  an  infinite  detector  diameter. 

The  form  of  the  widefield  PSF,  in  particular,  the  diverging 
cones  of  subsidiary  maxima  on  either  side  of  the  focal  plane,  shows 
why  the  out-of-focus  components  of  the  image  extend  a  long  way 
either  side  of  the  focal  plane;  each  point  in  the  specimen  is  replaced 
in  the  image  by  a  suitably  weighted  copy  of  the  PSF. 

The  degradation  of  the  image  can  also  be  understood  by  con¬ 
sidering  the  OTF,  which  is  the  FT  of  the  PSF.  The  spatial  frequency 
components  of  the  specimen,  that  is,  the  specimen  Fourier  trans¬ 
form  components,  are  multiplied  by  the  OTF  to  give  the  spatial  fre¬ 
quencies  transferred  to  the  image.  For  example,  where  the  OTF  is 
small  at  high  spatial  frequencies,  the  components  of  the  specimen 
that  correspond  to  these  frequencies  are  greatly  attenuated  in  the 
image.  The  widefield  OTF  has  the  form  of  a  torus  with  a  “missing 
cone,”  which  corresponds  to  the  attenuation  of  low  spatial  fre¬ 
quencies  along  the  direction  of  the  optical  axis,  and  represents  the 
lack  of  resolution  in  this  direction,  that  is,  closely  spaced,  hori¬ 
zontal,  planar  features  in  this  direction  are  not  resolved. 

Figure  23.4(A)  shows  a  central  section  through  the  OTF 
derived  from  the  widefield  PSF  shown  in  Figures  23.2(A)  and 
23.3(A).  The  optical  axis  is  vertical,  and  the  full  3D  OTF  is  the 
torus  produced  by  rotating  the  function  shown  about  this  axis. 
Deconvolution  seeks  to  reverse  this  attenuation  and  to  restore  as 
far  as  possible  the  image  spatial  frequency  components  to  their  true 
values.  This  has  the  effect  of  removing  the  out-of-focus  contami¬ 
nation  of  the  focal  plane,  giving  truer  focal  sections,  and  also 
restoring  the  in-plane  high  frequency  attenuation  suggested  by  the 
form  of  the  OTF.  In  addition,  deconvolution  strongly  suppresses 
any  details  that  have  spatial  frequencies  outside  the  OTF  and  that 
may  have  arisen  because  of  the  effects  of  Poisson  noise  on  the 
intensity  measurements  in  individual  pixels. 

In  an  ideal  confocal  microscope,  light  from  a  small  illuminated 
aperture  is  focused  to  a  diffraction-limited  spot  in  the  specimen, 
in  effect  producing  a  light  distribution  equivalent  to  the  widefield 
PSF.  However,  it  should  be  noted  that  the  focused  spot  from  a  laser 
beam  is  not  identical  to  the  focused  spot  from  a  conventional  light 
source  because  of  the  Gaussian  intensity  profile  of  the  laser  source 
(Self,  1983;  Chapter  5,  this  volume ).  The  light  emerging  from  the 
specimen  is  then  spatially  filtered  through  a  second  aperture  —  the 
detector  aperture  or  pinhole  —  that  is  also  in  a  plane  conjugate  to 
the  focal  plane  and  to  the  illuminating  aperture.  The  effect  of  this 
is  to  apply  the  PSF  twice;  thus  the  combined,  confocal  PSF  is  the 
square  of  the  widefield  PSF.  This  reduces  the  intensity  of  the  sub¬ 
sidiary  maxima  very  considerably,  particularly  away  from  the 
plane  of  focus.  Therefore,  the  out-of-focus  flare  is  substantially 
reduced,  and  the  images  are  much  cleaner  optical  sections.  Figure 
23.2(B)  shows  confocal  bead  data  and  Figure  23.3(B-D)  shows 
confocal  PSFs  for  the  same  objective  as  Figures  23.2(A)  and 
23.3(A),  measured  using  three  different  settings  of  the  detector 
aperture  (4.3,  2.5  and  0.7  Airy  units  respectively;  see  Shaw  and 
Rawlins,  1991b).  The  rejection  of  the  light  from  the  out-of-focus 
rings  is  seen  clearly  by  comparison  with  the  widefield  PSF  [but 
note  that  the  confocal  data  in  Fig.  23.2(B)  is  noisier  than  the  WF 
data,  complicating  the  comparison].  The  difference  is  also  clearly 
seen  in  the  OTFs  (Fig.  23.4).  The  “missing  cone”  is  largely  filled 
in,  although  in-plane  high  frequency  attenuation  is  still  apparent, 


FIGURE  23.4.  Central  sections  through  the  WF  (A)  and  confocal  (B-D)  OTFs 
derived  from  the  PSFs  shown  in  Figure  23.3.  The  optical  axis  is  vertical,  the 
radial  axis  horizontal.  The  3D  OTFs  are  the  solids  of  revolution  about  the 
optical  axis.  In  the  WF  case  this  gives  a  torus;  in  the  confocal  case,  an  approx¬ 
imate  ellipsoid,  the  size  of  which  varies  inversely  with  pinhole  diameter. 


and  the  resolution  in  the  direction  of  the  optical  axis  (vertical  in 
Fig.  23.4)  is  3  to  4  times  worse  than  the  in-plane  resolution  even 
with  the  smallest  pinhole  setting. 

In  principle,  the  ideal  confocal  PSF  shows  a  narrower  peak  in 
the  focal  plane,  and  so  should  give  slightly  better  in-plane  resolu¬ 
tion.  However,  the  increase  in  in-plane  resolution  requires  a  very 
small  detector  aperture  and  the  consequent  loss  of  light  makes  it 
difficult  to  obtain  good  images  from  even  the  brightest  fluorescent 
biological  specimens.  In  practical  biological  confocal  micro¬ 
scopes,  the  detector  aperture  is  generally  adjustable.  The  larger  it 
is  made,  the  more  light  is  detected,  but  the  more  the  imaging  tends 
towards  WF  optics.  On  the  other  hand,  opening  the  aperture  a  little 
way  greatly  increases  the  detected  light  and  reduces  the  in-plane 
resolution  virtually  to  the  WF  case  but  still  maintains  much- 
improved  optical  sectioning.  For  given  assumptions  about  the 
thickness  and  fluorochrome  distribution  in  a  sample,  it  is  possible 
to  show  that  there  is  an  optimum  size  that  maximizes  the  signal- 
to-noise  ratio  (S/N)  obtained.  (See  Sandison  et  al.,  1993;  and 
Chapters  2  and  22,  this  volume.)  The  resolution  implied  by  a  PSF 
may  be  quantified  by  measuring  the  peak  width  at  half-maximum 
peak  height  (FWHM),  either  in-plane  (d„)  or  along  the  optical  axis 
( d, ).  For  the  confocal  PSF  shown  in  Figure  23.4(D),  dxy=  0.23 |im, 
and  dz  =  0.8  pm.  These  values  agree  reasonably  well  with  theory 
and  other  measurements. 
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Limits  to  Linearity  and  Shift  Invariance 

We  need  to  bear  in  mind  that  deconvolution  procedures  are  criti¬ 
cally  dependent  on  the  assumptions  of  linearity  and  shift  invari¬ 
ance.  It  is  difficult  to  measure  the  extent  to  which  linearity  is 
obeyed  in  a  real  imaging  experiment.  The  most  obvious  reason  that 
might  cause  deviation  from  linearity  in  fluorescence  microscopy 
would  be  strong  absorption  effects;  the  image  seen  for  a  given 
point  might  then  depend  on  the  way  the  incident  beam  interacted 
with  other  parts  of  the  specimen.  Departures  from  shift  invariance 
are  much  easier  to  assess.  They  may  arise  across  a  field  of  view  in 
a  microscope  because  of  lack  of  flatness  of  field,  or  other  image 
plane  aberrations.  An  example  of  a  large  field  of  view  in  a  confo¬ 
cal  microscope  showing  different  bead  images  at  different  x,y  posi¬ 
tions  (i.e.,  different  PSFs  at  different  positions)  is  shown  in  Figure 
23.5.  The  PSF  also  often  changes  with  depth  within  a  biological 
specimen.  High  NA  objectives  are  usually  designed  to  image  spec¬ 
imens  through  exactly  the  correct  thickness  of  cover  glass,  as  men¬ 
tioned  above.  However,  in  many  3D  biological  specimens  there  is 
an  additional  layer  of  mounting  medium  between  the  cover  glass 
and  the  specimen,  and  this  introduces  some  spherical  aberration. 


The  deeper  into  the  specimen  the  image  plane  is,  the  thicker  the 
intervening  layer,  and  the  more  the  spherical  aberration. 

Departures  from  linearity  would  be  difficult  to  deal  with  com¬ 
putationally,  but  departures  from  shift  invariance  are  not  so  hard. 
One  simple  approach,  which  has  been  used,  is  to  divide  the  orig¬ 
inal  3D  image  into  small  blocks  within  which  the  PSF  does  not 
vary  appreciably,  and  then  deconvolve  each  subimage  with  the 
appropriate  PSF,  finally  recombining  the  subimages.  A  more 
elegant  solution  would  be  to  determine  the  PSF  as  a  function  of 
x,y  and  z,  and  use  this  function  to  determine  the  degraded  image 
in  a  restoration  algorithm.  However,  as  the  degraded  image  would 
no  longer  be  a  simple  convolution,  and  it  could  not  be  calculated 
by  the  efficient  Fourier  transform  methods,  the  computational  cost 
would  be  very  high. 

DECONVOLUTION 

Convolution  of  the  object  with  the  PSF  is  a  simple  mathematical 
transformation  that  is  reversible  in  principle  —  a  procedure  called 
deconvolution.  This  is  particularly  clear  in  the  formulation  in  terms 


FIGURE  23.5.  Confocal  image  of  a  field  of  beads.  The  focal  plane  has  been  set  away  from  the  central,  in-focus,  section  to  show  some  of  the  out-of-focus  parts 
of  the  PSF.  The  lack  of  shift  invariance  is  evident  in  this  image,  as  the  defocused  bead  structure  is  different  in  different  parts  of  the  field  of  view.  The  insets 
show  enlargements  from  different  parts  of  the  field.  The  image  is  also  dimmer  in  the  cental  area  due  to  field  curvature.  (Objective  used  was  Leitz  Planapo,  NA 
1.4,  oil  immersion.) 
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of  FTs  (Eq.  8).  Because  the  image  transform  is  simply  the  object 
transform  multiplied  by  the  OTF,  it  should  be  possible  to  recover 
the  desired  object  transform  by  dividing  the  image  transform  point- 
by-point  by  the  OTF. 

G(S)  =  F(S)0(S)  =>  F(S)  =  G(S)/<9(S)  (9) 

The  undegraded  object  would  then  be  obtained  by  an  inverse  FT 
of  the  result.  Unfortunately,  this  simple  approach  is  made  impos¬ 
sible  by  the  inevitable  presence  of  noise  in  any  real  image.  There 
are  regions  where  the  OTF  becomes  very  small  or  zero  (see  Fig. 
23.4);  however  these  regions  of  the  image  transforms  still  have  a 
noise  component.  Thus,  dividing  by  the  very  small  OTF  values 
will  boost  the  noise  component  of  the  data  to  a  level  where  it  dom¬ 
inates  the  final  reconstruction,  rendering  it  meaningless. 

The  problem  of  restoring  noisy  data  is  one  common  to  many 
fields  in  spectroscopic,  optical,  and  medical  imaging.  The  most 
powerful  methods  of  solution  apply  “constraints”  to  the  solution, 
typically  requiring  the  result  to  be  positive  and  smooth  —  both 
physically  reasonable  requirements.  A  simple  way  to  visualize  this 
is  to  consider  fitting  a  curve  to  some  noisy  data  points.  If  the  fitted 
curve  is  allowed  to  have  many  parameters,  it  can  pass  right  through 
all  the  points  but  may  well  contain  wild  and  meaningless  oscilla¬ 
tions  in  regions  away  from  the  points.  If  we  put  some  constraints 
on  the  curve,  such  as  preventing  physically  meaningless  values  or 
ensuring  a  certain  degree  of  smoothness,  the  curve  may  not  pass 
exactly  through  any  point,  but  will  be  near  them  all,  and  therefore 
a  more  reasonable  and  “believable”  solution.  This  is  the  basis 
of  constrained  deconvolution  methods.  They  are  invariably  more 
complex  and  time  consuming  to  compute  than  the  simpler 
methods,  and  usually  require  multiple  rounds  of  iterative  approx¬ 
imation  to  the  reconstructed  image.  However,  their  power  and  the 
rapidly  increasing  speed  and  decreasing  cost  of  computers  make 
them  the  most  attractive  option  for  3D  digital  image  deconvolu¬ 
tion.  There  are  many  different  methods  for  this  type  of  restoration, 
differing  in  their  degrees  of  rigor  and  their  computing  require¬ 
ments.  The  various  deconvolution  methods  are  discussed  in  more 
detail  in  Chapters  24  and  25.  The  examples  shown  here  are  all  cal¬ 
culated  by  the  constrained,  iterative  method  developed  by  Jansson 
and  others  (Jansson  et  al,  1970;  Agard  and  Sedat,  1983;  Agard 
et  al,  1989;  Shaw,  1993).  We  believe  that  the  Jansson  method 
represents  a  good  compromise  between  rigor  and  computational 
efficiency. 

PRACTICAL  DIFFERENCES 

A  few  practical  differences  relating  to  the  implementation  of  WF 
or  confocal  imaging  should  be  mentioned. 

Temporal  Resolution 

With  a  single-beam  confocal  microscope,  each  focal  section  gen¬ 
erally  requires  image  averaging  to  obtain  an  acceptable  S/N,  and 
this  may  take  several  seconds.  Some  confocal  microscopes  are 
capable  of  scanning  at  video  rates  or  faster,  but  the  image  quality 
is  then  limited  by  the  amount  of  light  measured  during  each  scan. 
If  the  specimens  are  very  bright,  good  time  resolution  is  possible, 
but  in  many  (perhaps  most)  biological  applications,  the  faster  scan 
rate  will  simply  mean  that  more  frames  must  be  accumulated  and 
averaged.  Higher  speed  is  possible  in  disk-scanning  confocals  in 
which  the  maximum  possible  data  rate  is  increased  by  the  use  of 
hundreds  of  beams  striking  the  specimen  simultaneously,  as  dis¬ 
cussed  below. 


However,  even  the  slower  scan-rate  CLSMs  are  capable  of 
very  good  time  resolution  in  some  respects;  the  scanning  beam 
only  passes  over  each  pixel  for  a  very  short  time  and  therefore 
takes  a  frozen  snapshot  of  that  pixel.  It  is  possible  to  scan  a  very 
small  area  or  a  single  line,  and  to  obtain  very  good  temporal  res¬ 
olution  from  the  restricted  set  of  pixels.  A  single  CCD  image  from 
a  typical  biological  specimen  may  require  only  a  fraction  of  a 
second  to  record  and  a  little  longer  for  image  readout,  but  decon¬ 
volution  is  much  more  time  consuming  on  current  computers,  and 
may  take  seconds  or  even  hours  depending  on  the  data-set  size. 
CCD  imaging  thus  has  reasonably  good  time-resolution  for  record¬ 
ing  images,  but  it  takes  much  longer  to  produce  deconvolved  focal 
sections.  In  either  case  specimen  motion  during  data  collection  will 
produce  artifacts.  As  WF  deconvolution  can  only  obtain  optical 
section  data  by  first  collecting  and  then  processing  a  complete  3D 
data  stack,  confocal  clearly  is  faster  when  information  about  only 
a  single  optical  section  is  needed. 

Combination  of  Charged-Coupled  Device  and 
Confocal  Imaging 

A  fundamental  difference  between  CCD  capture  of  WF  images  and 
imaging  by  a  conventional  spot-scanning  confocal  microscope  is 
that  the  image  is  scanned  one  pixel  at  a  time  by  the  confocal  micro¬ 
scope,  whereas  all  the  pixels  for  the  whole  WF  image  are  recorded 
simultaneously  by  the  CCD  detector.  The  CLSM  could  be  regarded 
as  a  serial  device,  whereas  the  CCD-WF  microscope  is  a  parallel 
device.  If,  for  the  sake  of  argument,  the  image  contains  106  pixels, 
then  to  excite  the  specimen  with  the  same  number  of  photons  in 
the  CLSM  will  require  each  pixel  to  be  illuminated  with  106  higher 
light  intensity,  although  for  10-6  as  long,  compared  to  the  contin¬ 
uous  illumination  in  WF.  This  may  be  impossible  in  practice, 
because  the  excited  state  of  the  fluorochrome  molecules  present  in 
the  focused  laser  spot  may  become  saturated.  Even  at  lower  light 
levels,  the  photodamage  related  to  producing  a  fixed  number  of 
excitations  using  the  high  instantaneous  intensity  of  laser  light  may 
be  more  than  that  produced  by  the  much  lower,  sustained  illumi¬ 
nation  of  the  WF  microscope  (see  Chapter  39,  this  volume).  The 
only  solution  with  a  single-beam  CLSM  may  be  to  reduce  the  illu¬ 
minating  intensity,  and  either  scan  for  correspondingly  longer 
times,  or  accept  a  noisier  image. 

A  solution  that  combines  parallel  image  detection  with  confo¬ 
cal  imaging  is  to  scan  the  specimen  with  an  array  of  light  spots, 
instead  of  the  single  spot  used  in  most  CLSMs  (see  Chapter  10, 
this  volume).  One  of  the  first  confocal  microscopes,  designed  by 
Petran  and  colleagues  (1968)  used  this  principle.  The  excitation 
light  passed  through  a  Nipkow  disk,  which  contained  thousands  of 
pinholes  in  a  spiral  pattern,  and  the  emitted  light  was  passed  back 
through  the  same  disk  to  produce  confocal  optics.  The  disk  was 
spun  so  that  the  spiral  pinholes  scanned  the  entire  specimen.  In  its 
original  design,  this  microscope  had  extremely  low  sensitivity 
because  the  pinholes  occupied  only  a  very  small  proportion  of 
the  area  of  the  disk,  and  so  most  of  the  excitation  light  was  lost. 
Yokogawa  introduced  a  radical  improvement  in  the  design  by 
adding  a  microlens  above  each  pinhole,  increasing  the  illumina¬ 
tion  efficiency  by  more  than  an  order  of  magnitude.  The  Yokogawa 
spinning  disk  can  be  combined  with  image  detection  using  a  CCD 
camera  to  produce  a  very  sensitive  confocal  microscope,  and 
instruments  of  this  design  are  now  made  by  a  number  of  compa¬ 
nies.  Initial  results  suggest  that  this  type  of  instrument  may  be 
much  better  than  single-beam  CLSMs  for  low  light  level  and  live¬ 
cell  imaging,  even  though  the  confocal  optics  is  not  quite  so  good, 
and  the  image  contains  somewhat  more  out-of-focus  light.  When 
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coupled  with  one  of  the  new  electron-multiplier  CCDs  (EM- 
CCDs)  having  quantum  efficiency  (QE)  and  noise  specifications 
very  like  those  of  the  PMT,  this  combination  can  collect  10-plane 
3D  images  fast  enough  for  use  in  high  content  screening  applica¬ 
tions  (see  Chapter  46,  this  volume). 

Integration  of  Fluorescence  Intensity 

It  is  sometimes  important  to  obtain  values  for  the  total  fluorescence 
intensity  from  all  or  part  of  a  specimen.  For  example,  one  may  wish 
to  obtain  a  value  for  the  DNA  content  of  a  cell  by  integrating  the 
fluorescence  emission  from  a  DNA-binding  stain.  In  the  WF  case, 
this  can  be  obtained  from  any  single  optical  section  because  the 
total  integrated  fluorescence  is  almost  the  same  in  each  optical 
section  (as  long  as  the  NA  is  not  too  high).  The  partial  confocality 
of  the  WF  mode  (Hiraoka  et  al.,  1990)  means  that  if  a  small  field 
aperture  is  used,  the  focus  plane  of  the  optical  section  should  be 
reasonably  close  to  the  center  of  the  object  of  interest  (a  few 
microns).  In  the  confocal  case,  it  is  necessary  to  sum  the  contribu¬ 
tions  from  each  section  of  a  focal  series,  and  the  section  spacing 
has  to  be  comparable  to  or  less  than  the  confocal  depth  of  field  (<1 
pm  with  a  2  Airy  pinhole).  This  means  that  WF  imaging  is  much 
more  efficient  for  such  measurements,  providing  that  the  objects  to 
be  quantified  do  not  overlap  significantly,  even  when  out  of  focus. 

RESOLUTION,  SENSITIVITY,  AND  NOISE 

Leaving  aside  the  practical  differences  mentioned  above,  given  a 
fluorescent  specimen,  what  is  the  optimal  method  to  excite  the  flu¬ 
orescence,  and  then  to  extract  the  maximum  amount  of  useful 
structural  information  from  the  resulting  fluorescent  light? 

Fluorescence  Excitation 

Laser-scanning  confocal  and  WF  microscopy  differ  markedly  in 
the  way  the  fluorochrome  molecules  in  the  specimen  are  excited 
(see  Table  23.1).  In  WF  microscopy,  each  and  every  plane  of  the 
specimen  is  evenly  illuminated  while  its  image  is  recorded.  In  a 
scanning  confocal  microscope,  the  illuminating  beam  rapidly  tra¬ 
verses  the  specimen,  giving  very  high  light  intensity  at  the  center 
of  the  focal  spot  and  rapidly  decreasing  intensity  over  a  broad 
region  above  and  below  this  spot.  The  instantaneous  light  distri¬ 
bution  in  the  CLSM  is  given  approximately  by  the  form  of  the 
widefield  PSF  (although  the  focused  laser  beam  has  a  somewhat 
different  detailed  distribution).  As  discussed  above,  at  excitation 
levels  above  ~lmW  the  light  intensity  can  easily  saturate  the  flu- 
orochromes  at  the  center  of  the  focal  spot,  and  the  need  to  avoid 
this  in  turn  limits  the  usable  excitation  intensity.  When  the  laser 


light  intensity  is  reduced  enough  to  avoid  saturation,  the  amount 
of  emitted  light  recorded  is  very  small:  10  to  20  photons/pixel/ 1  s 
scan  in  the  stained  areas  of  most  fluorescent  biological  specimens. 
Therefore,  it  is  necessary  to  sum  the  light  from  many  scans.  Thus, 
each  part  of  the  specimen  is  illuminated  by  a  succession  of  high- 
intensity  pulses  of  light.  The  difference  in  behavior  between 
multiple  short  exposures  of  high-intensity  light  and  continuous 
exposure  to  much  lower  light  levels  may  yield  important  differ¬ 
ences  in  the  lifetime  of  fluorochrome  before  fading  or  phototoxic¬ 
ity  in  living  cells.  To  date,  experiences  vary  with  very  sensitive 
specimens  such  as  living  cells  loaded  with  fluorescent  labels.  Some 
investigators  have  used  confocal  imaging  successfully  on  such 
specimens  (Zhang  et  al.,  1990);  others  have  found  that  the  only 
feasible  method  is  low  light  level  recording  with  WF  optics 
(Hiraoka  et  al.,  1989),  and  others  have  used  two-photon  excitation 
(Chapter  28,  this  volume).  It  is  probable  that  the  behavior  of  dif¬ 
ferent  types  of  cells  and  of  different  fluorochromes  varies  (see 
Chapters  38  and  39,  this  volume). 

Fluorescent  Light  Detection 

The  overall  performance  of  any  3D  light  microscope  depends 
strongly  on  the  capabilities  of  the  photodetectors  used.  We  con¬ 
sider  here  the  relative  photon  detection  efficiency  and  associated 
measurement  noise  for  the  most  usual  detectors  in  WF  microscopy 
(and  in  the  spinning  disk  CLSM)  and  in  the  single-beam  CLSM. 
See  Chapter  12  for  a  more  detailed  discussion  of  image  detection, 
and  Pawley  (1994)  for  a  detailed  assessment  of  the  sources  of 
noise.  The  best  area/image  detector  for  most  epi-fluorescence 
microscopy  is  a  scientific  grade,  cooled  CCD  camera.  This  has 
excellent  geometrical  and  photometric  linearity,  a  wide  dynamic 
range,  and  good  photon  detection  efficiency.  Possibly  most  impor¬ 
tant,  the  image  can  be  accumulated  on  the  chip  for  an  arbitrarily 
long  time,  which  means  that,  as  long  as  they  do  not  move,  speci¬ 
mens  can  be  imaged  with  very  low  photon  emission  fluxes  (Aikens 
et  al.,  1989). 

Rather  than  repeating  a  more  general  discussion  well  covered 
in  Chapter  12,  we  shall  illustrate  this  by  taking  the  CCD  camera 
in  use  in  our  laboratory  as  an  example.  This  is  a  Photometries 
camera  electrically  cooled  to  about  -40°C.  The  detective  QE  (in 
effect,  the  proportion  of  photons  reaching  the  faceplate  of  the  CCD 
camera  that  are  converted  to  electrons  in  the  charge  wells)  ranges 
between  20%  in  the  ultraviolet  (UV)  to  50%  at  650  nm.  The  charge 
is  converted  by  a  12-bit  A-D  converter,  giving  a  maximum  value 
of  4096,  which,  according  to  the  manufacturer’s  specifications  for 
the  maximum  amplifier  gain  setting,  corresponds  to  about  6700 
electrons  in  the  CCD  pixel  or  13,400  photons  reaching  the  CCD 
faceplate,  assuming  the  wavelength  for  maximum  QE.  The  relay 
optics  are  arranged  to  give  a  pixel  spacing  without  binning  of  about 


TABLE  23.1.  Summary  of  Pros  and  Cons  of  3D  Microscopy  Methods 


Widefield  Deconvolution 

Single  Spot  Confocal 

Scanning-Disk  Confocal 

Two-Photon 

Effective  detector  QE. 

60%-80%  (CCD) 

3%-12%  (PMT) 

60%-80%  (CCD) 

3%-12%  (PMT) 

Detector  noise  (rms  e/pixel) 

4-12 

<1 

4-12  (<1  for  EM-CCD) 

<1 

Peak  signal  (photons/pixel) 

>30,000 

20-100 

-5000 

20-100 

Acquisition  time  (s/frame) 

Depends  on  CCD 

0.2-10 

Depends  on  CCD 

0.2-10 

readout  (>0.05) 

readout  (>0.05) 

Peak  excitation  intensity/(im2 

lOnW 

lmW 

lpW 

10W 

Excitation  wavelengths 

Hg  arc  350-650  nm 

Available  laser  lines 

Available  laser  lines 

Ti:Sa  700-900  nm 
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0.07  pm  at  the  specimen,  which  is  somewhat  better  than  required 
by  the  Nyquist  criterion,  assuming  -0.2  pm  data.  If  specimen 
fading  or  phototoxicity  is  not  a  problem,  it  is  usual  to  collect  for 
a  time  that  nearly  fills  the  wells  at  the  brightest  points.  Typically 
for  a  bright  specimen  that  is  easily  visible  to  the  eye  through  the 
microscope,  this  takes  an  exposure  of  a  fraction  of  a  second.  Thus, 
in  CCD  imaging,  it  is  quite  common  to  collect  images  with  about 
10,000  photons  at  the  brightest  pixels  and  many  times  this  number 
from  the  several  pixels  that  cover  the  image  of  a  point  object. 

However,  this  can  be  reduced  substantially.  The  limit  to  sen¬ 
sitivity  is  determined  by  the  system  measurement  noise  and  the 
intrinsic  photon  (Poisson)  shot  noise.  The  sources  of  system  noise 
are  primarily  dark  current  and  readout  noise.  The  dark  current  in 
our  CCD  is  0.33  electrons/min/pixel,  which  is  negligible.  The 
readout  noise  is  much  larger  —  ±13  electrons  root  mean  squared 
(RMS).  This  means  the  Poisson  noise,  given  by  the  square  root  of 
the  number  of  electrons,  will  be  less  than  the  readout  noise  in  any 
pixel  having  an  image  intensity  of  less  than  169  electrons.  The  QE 
cannot  be  improved  much  —  a  factor  of  2  at  most  —  but  the 
readout  noise  can  be  much  lower.  In  many  CCD  cameras  ±6  elec¬ 
trons  (RMS)  is  typical,  and  slow-readout  cameras  optimized  for 
low-noise  operation  can  have  values  down  to  ±3  electrons  (RMS) 
readout  noise.  This  is  comparable  with  the  Poisson  noise  at  a  signal 
level  of  about  9  electrons. 

In  the  case  of  single-beam  CLSM,  the  image  detector  is  gen¬ 
erally  a  photomultiplier  (PMT;  see  Chapters  2  and  12,  this  volume). 
It  is  much  harder  to  estimate  the  light  levels  being  measured 
(photons/pm2),  but  they  are  probably  substantially  lower  than  in 
CCD  imaging.  [This  should  not  be  confused  with  the  photon  fluxes 
(photons/pnr/s),  which  are  much  higher,  but  each  pixel  only  emits 
the  photons  for  a  very  short  time.]  It  is  common  for  the  brightest 
pixel  intensity  in  a  single  scan  of  a  CLSM  to  represent  only  10 
detected  photons  (Pawley  and  Smallcomb,  1992;  Pawley,  1994). 
One  might  integrate  30  scans  for  an  image  plane  in  a  weak  spec¬ 
imen,  giving  perhaps  300  detected  photons  and  an  associated 
Poisson  noise  of  about  17  photons.  This  is  much  larger  than  the 
typical  measurement  noise  in  a  PMT,  which  can  be  reduced  to 
extremely  low  levels,  especially  by  photon  counting  (see  Chapter 
12,  this  volume).  However,  the  QE  for  a  PMT  is  worse  than  for  a 
cooled  CCD;  perhaps  only  15%  of  the  photons  reaching  the  detec¬ 
tor  faceplate  produce  signal.  In  this  case,  the  300  detected  photons 
would  represent  2000  photons  going  through  the  confocal  pinhole. 

Another  way  of  comparing  the  two  detection  methods  is  to 
consider  the  number  of  gray  levels  into  which  the  recorded  data 
can  be  reliably  placed.  The  conventional  way  to  do  this  is  to  space 
the  gray  levels  by  one  standard  deviation.  Where  the  only  source 
of  noise  is  Poisson  noise,  this  gives  N/  V]V  or  a/AT  levels  for  N 
detected  photons  (see  Chapter  4,  this  volume).  Where  other  sources 
of  noise  are  present,  the  situation  is  more  complicated.  (See 
Pawley,  1994,  for  a  full  discussion.)  We  can  get  an  estimate  by 
assuming  the  additional  noise  is  purely  additive,  random,  and 
signal  independent  (e.g.,  readout  noise  in  the  CCD).  If  we  want  an 
image  with  10  statistically  significant  gray  levels,  the  brightest 
pixel  would  require  100  electrons  in  the  absence  of  readout  noise. 
This  would  require  200  photons  reaching  the  camera  faceplate 
at  50%  QE.  For  our  CCD  camera  with  13  electrons/pixel 
readout  noise,  10  gray  levels  would  require  (10  +  13)2  =  529 
electrons/pixel,  or  1058  photons.  If  the  CCD  camera  was  equal  to 
the  best  available  with  3  electrons/pixel  readout  noise,  10  gray 
levels  would  require  (10  +  3)2  =  169  electrons,  or  338  photons.  If 
we  had  an  optimally  sensitive  camera  with  80%  QE,  then  211 
photons  would  be  enough  for  10  gray  levels.  In  the  confocal  case, 
the  PMT  noise  is  less  than  1  count,  so  10  gray  levels  would  require 


(10  +  l)2  =  121  counts.  However,  because  the  QE  of  the  PMT  is 
only  about  15%,  this  would  require  806  photons.  The  PMT  multi¬ 
plicative  noise  can  be  reduced  to  extremely  low  levels  by  photon 
counting.  Perhaps  then  100  counts  would  be  enough  to  define  10 
gray  levels,  but  with  15%  efficiency,  this  would  still  require  666 
photons. 

Clearly,  as  long  as  we  assume  that  we  still  need  10  gray  levels, 
low  QE  is  a  limitation  in  current  single-beam  confocal  systems. 
However,  in  the  confocal  approach,  only  in-focus  photons  are 
counted  and  they  represent  the  final  data,  while  in  the  WF  approach, 
more  gray  levels  need  to  be  recorded  because  some  may  be  lost 
when  the  data  is  processed  to  remove  out-of-focus  light,  and  thus  it 
could  be  argued  that  a  statistically  less  well-defined  confocal  image 
would  be  sufficient  for  direct  interpretation  than  would  be  required 
for  reliable  WF  deconvolution.  Ultimately,  there  will  probably  not 
be  a  great  deal  of  difference  in  the  effective  QE  possible  with  the 
two  detection  technologies.  The  noise  in  the  PMT  is  considerably 
more  complicated  than  this  brief  discussion  would  imply  but  ulti¬ 
mately  it  can  be  reduced  below  that  attainable  in  conventional  CCD 
cameras.  However,  even  if  the  detection  can  be  made  optimally  effi¬ 
cient  in  both  cases,  for  specimens  where  fluorochrome  fading  is  not 
a  problem,  it  will  be  easier  to  record  images  with  very  low  statisti¬ 
cal  noise  with  the  CCD  because,  as  a  parallel  device,  it  can  accu¬ 
mulate  data  more  rapidly  than  a  single-beam  CLSM.  For  this  reason, 
the  combination  of  multiple-spot  scanning,  such  as  the  spinning 
disk,  with  CCD  detection,  currently  probably  has  the  best  mix  of 
sensitivity  and  overall  speed  for  confocal  imaging. 

Gain  Register  Charge-Coupled  Devices 

As  discussed  above,  CCD  cameras  suffer  from  relatively  high 
readout  noise,  compared  with  PMTs.  Even  with  the  slowest 
readout  rates,  it  becomes  very  difficult  to  reduce  the  readout  noise 
below  about  3  electrons/pixel.  This  limits  their  usefulness  at  low 
photon  levels.  A  recent  development  in  CCD  technology  signifi¬ 
cantly  improves  CCD  detector  performance  at  low  light  levels.  The 
new  CCD  chips,  called  gain  register  CCDs  or  EM-CCDs  (for  elec¬ 
tron  multiplier),  incorporate  an  on-chip  amplification  register, 
which  amplifies  the  electrons  in  the  charge  wells  by  a  small  amount 
every  time  the  charge  is  moved  from  one  well  to  the  next  during 
the  readout  phase.  By  repeating  this  amplification  hundreds  of 
times  the  accumulated  charge  packet  can  be  amplified  by  very 
large  overall  factors  before  digitization.  In  effect  this  means  that 
the  charge  can  be  increased  to  the  extent  that  the  readout  noise 
becomes  negligible,  and  this  makes  the  CCDs  usable  at  extremely 
low  photon  levels,  and  even  in  a  photon-counting  mode.  The 
penalty  is  that  the  effective  QE  is  reduced  by  a  factor  of  2  by  the 
amplification  noise.  Dark  current  is  also  amplified,  and  so  needs 
to  be  reduced  to  very  low  levels  by  cooling.  At  high  photon  fluxes, 
the  gain  can  be  reduced  to  1,  and  the  chips  behave  as  conventional 
CCD  chips. 

Cameras  incorporating  these  new  chips  will  probably  be  most 
useful  in  applications  where  intensified  CCDs  or  photon-counting 
cameras  have  been  used  in  the  past  because  the  EM-CCD  over¬ 
comes  many  of  the  problems  of  these  cameras.  The  EM-CCDs 
probably  do  not  have  any  advantages  for  most  CCD  WF  imaging 
when  the  image  intensity  is  greater  than  -100  photons/pixel. 
However,  they  may  well  be  ideal  for  spinning-disk  confocal  CCD 
applications,  where  a  single  camera  could  cover  the  range  from 
photon-counting  levels  up  to  many  thousands  of  photons/pixel. 
The  new  CCD  chips  are  at  present  manufactured  by  E2V  (Enfield, 
UK)  and  Texas  Instruments  (Houston,  TX).  All  the  major  CCD 
manufacturers  now  offer  cameras  using  EM-CCD  technology. 
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Measuring  the  sensitivity  of  light  detection  systems  is  not 
straightforward  and  requires  specialized  equipment  not  generally 
available  in  a  biological  laboratory.  Even  with  a  CCD  camera, 
where  the  observed  counts  should  be  simply  related  to  the  incident 
photons,  we  must  rely  on  data  supplied  by  the  manufacturer  to 
make  the  conversion,  and  assume  that  the  electronics  is  working 
optimally.  There  are  also  sources  of  variability  in  the  PMT  and 
associated  electronics.  It  would  be  a  great  help  to  have  standard 
intensity  specimens,  in  addition  to  resolution  specimens,  which 
could  be  checked  on  a  regular  basis  in  any  imaging  laboratory  (see 
Chapters  2,  and  36,  this  volume,  for  more  discussion).  It  would 
then  be  easy  to  determine  whether  the  noise  in  the  image  of  such 
a  specimen  was  at  a  level  consistent  with  the  quoted  measurement 
noise  and  Poisson  statistics. 

Out-of-Focus  Light 

The  main  effect  of  the  confocal  pinhole  is  to  exclude  most  of  the 
out-of-focus  light  from  measurement,  and  thus  to  produce  cleaner 
optical  sections.  Clearly,  if  the  out-of-focus  component  is  regarded 
purely  as  unwanted  noise,  then  it  is  better  not  to  measure  it;  the 
total  number  of  photons  measured,  and  thus  the  Poisson  noise,  are 
increased  without  increasing  the  in-focus  “signal.”  Conversely,  to 
the  extent  that  the  out-of-focus  light  carries  useful  information,  it 
should  be  measured  and  computational  deconvolution  should  be 
used  to  process  the  information  and  obtain  accurate  focal  sections. 
The  relative  merits  of  the  two  approaches  must  depend  on  how 
much  useful  information  the  out-of-focus  light  carries  and  the 
characteristics  of  the  noise  associated  with  this  light.  The  amount 
of  useful  information  contained  in  the  out-of-focus  light  depends 
on  the  specimen  in  question. 

The  respective  OTFs  for  confocal  and  WF  imaging  provide  an 
alternative  way  to  look  at  this  problem  (see  Fig.  23.4).  The  toroidal 
form  of  the  WF-OTF  means  that  image  components  at  low  spatial 
frequencies  away  from  the  jc,y-plane  are  highly  attenuated  —  the 
region  of  the  “missing  cone”  —  whereas  the  confocal  OTF  has  an 
appreciable  value  in  these  regions.  Thus,  it  is  the  spatial  frequen¬ 
cies  that  define  the  depth  of  held  for  large-scale  structure  that  are 
much  better  determined  by  the  CFSM.  There  is  much  less  differ¬ 
ence  between  WF  and  confocal  images  in  the  higher  spatial  fre¬ 
quencies  away  from  the  focus  plane.  The  difference  in  imaging 
therefore  depends  on  the  importance  in  particular  objects  of  the 
contrast  components  in  the  missing  cone  region,  and  on  how 
the  noise  is  distributed  between  the  different  spatial  frequencies. 
The  noise,  in  turn,  determines  the  ultimate  accuracy  to  which  the 
missing  and  attenuated  components  can  be  restored. 

Model  Specimens 

It  is  possible  to  devise  specimens,  real  or  imagined,  which  are  far 
better  imaged  by  confocal  optics.  For  example,  a  uniform,  infinite, 
flat  layer  of  fluorescent  dye  has  spatial  frequency  components  only 
in  the  z-direction.  Because  the  integrated  intensity  at  each  plane  of 
the  ideal  wideheld  PSF  is  the  same,  this  specimen  would  give 
exactly  the  same  light  level  no  matter  where  the  plane  of  focus  was 
and  would  therefore  be  completely  unresolved  by  a  WF  micro¬ 
scope,  while  a  confocal  microscope  would  genuinely  resolve  this 
specimen  in  z.  It  should  be  noted,  however,  that  an  actual  WF 
microscope  would  resolve  such  a  specimen  to  a  limited  extent, 
giving  a  very  broad  maximum  to  the  light  intensity  at  the  relevant 
focal  plane.  This  is  because  the  effective  field  aperture  in  the  WF 
optical  system  is  not  infinite,  and  so  there  is  always  some  degree 


of  “confocality.”  Hiraoka  and  colleagues  (1990)  have  measured 
this  effect  experimentally  and  discussed  it  in  some  detail.  This  is 
one  reason  why  it  is  a  good  idea,  in  WF  microscopy,  to  use  the 
field  diaphragm  of  the  epi-illumination  system  to  limit  the  field  of 
illumination  to  the  area  of  interest  in  the  specimen  and  to  collect 
the  PSF  data  using  this  same  illumination. 

On  the  other  hand,  the  ability  to  image  an  “infinite  plane”  spec¬ 
imen  is  of  little  practical  importance.  A  more  interesting  case  is  a 
very  thin  specimen  (comparable  to  or  less  than  the  depth  of  field) 
containing  structure  of  interest  within  the  plane.  In  this  case,  the 
WF  image  focused  on  the  appropriate  plane  should  be  a  true  rep¬ 
resentation  of  the  light  emitted  because  there  is  no  out-of-focus 
structure  to  contaminate  it.  The  confocal  image  should  be  identi¬ 
cal,  except  that  some  of  the  light  is  excluded  by  the  detector  aper¬ 
ture,  and  the  overall  image  resolution  should  be  slightly  increased. 
Now  consider  collecting  a  through-focal  series  of  this  specimen  by 
WF  microscopy.  Several  of  the  images  will  presumably  contain 
out-of-focus  information  well  above  the  noise  level,  although  the 
high  spatial  frequencies  will  be  attenuated  according  to  the  OTF. 
Thus,  to  some  extent,  this  set  of  images  must  be  equivalent  to  mul¬ 
tiple  measurements  of  the  specimen  plane,  and  a  reconstruction 
scheme  should  be  capable  of  using  them  to  improve  the  statistical 
reliability  of  the  in-focus  image. 

Figure  23.6  shows  some  images  from  a  real  biological  speci¬ 
men  (immunofluorescent  labeling  of  a  very  thin  plant  cytoskele- 
ton  “footprint”)  that  attempts  to  model  this  situation.  Three 
confocal  images  taken  from  a  through-focal  series  starting  with  the 
in-focus  image  and  spaced  by  2  pm  are  shown  in  Figure  23.6(A). 
An  equivalent  WF  series  of  images  of  another  footprint  on  the 
same  slide  is  shown  in  Figure  23.6(B).  Given  the  specimen  thick¬ 
ness  (probably  well  under  1  pm),  there  should  be  no  out-of-focus 
light  to  contaminate  the  WF  image.  The  in-focus  WF  image  should 
be  little  different  from  the  confocal  image.  (A  63x,  NA  1.4  objec¬ 
tive  was  used,  and  the  pinhole  was  set  at  approximately  3  Airy  disk 
diameters.)  There  is  surprisingly  little  difference  between  these 
two  focal  series.  This  implies  that  relatively  little  of  the  out-of¬ 
focus  data  is  eliminated  with  a  detector  aperture  of  this  size  in  a 
specimen  such  as  this,  because  it  consists  of  thin  fibers  arranged 
in  a  plane  and  produces  large  components  at  high,  in-plane  spatial 
frequencies. 

The  Best  Solution:  Deconvolving  Confocal  Data 

The  main  effect  of  the  confocal  optics  should  be  to  remove  the 
very  low  spatial  frequency  out-of-focus  components.  Conversely, 
this  means  that  with  a  detector  aperture  in  this  size  range,  there  is 
still  residual  degradation  by  the  out-of-focus  high  frequency  com¬ 
ponents  from  nearby  planes  in  just  the  same  way  as  for  WF  optics. 
Unless  a  very  small  detector  aperture  can  be  used,  confocal  optics 
do  not  provide  a  complete  solution  to  this  “high  frequency  out-of¬ 
focus  problem."  Because  it  is  almost  invariably  impossible  to  use 
such  very  small  detector  apertures,  deconvolution  offers  a  practi¬ 
cable  way  to  improve  the  fidelity  of  the  high  spatial  frequencies  in 
both  WF  and  confocal  3D  images.  These  considerations  as  well  as 
the  practical  examples  given  below  suggest  that  it  will  often  be 
helpful,  or  even  necessary,  to  apply  deconvolution  to  confocal  data 
to  obtain  the  best  resolution  of  3D  structures.  Furthermore,  in 
images  where  the  pixel  spacing  is  smaller  than  the  Nyquist  fre¬ 
quency,  any  spatial  frequencies  present  in  the  image  beyond  the 
Nyquist  frequency  should  be  removed  before  the  image  is  dis¬ 
played.  This  is  because  any  such  high  spatial  frequencies  cannot 
contain  meaningful  information,  and  may  confuse  interpretation  of 
the  image. 
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FIGURE  23.6.  A  series  of  confocal  (A)  and  WF  (B)  optical  sections  of  a  plant  cytoskeleton  “footprint"  fluorescently  labeled  with  anti-tubulin.  The  specimen 
thickness  is  less  than  1  pm.  The  confocal  detector  aperture  size  was  approximately  3  Airy  units.  The  spacing  between  the  optical  sections  was  2  pm.  Field  width, 
37  pm.  (Specimen  courtesy  of  Dr.  Clive  Lloyd.) 


As  another  way  of  looking  at  this  aspect  of  the  comparison 
between  WF  and  confocal  imaging,  consider  again  the  respective 
PSFs  and  OTFs  (see  Figs.  23.2-23.4).  The  confocal  PSF  has  much 
reduced  subsidiary  maxima  (see  Figs.  23.2  and  23.3),  largely  elim¬ 
inating  the  low  spatial  frequency  out-of-focus  light.  This  corre¬ 
sponds  to  the  missing  cone  region  of  the  WF-OTF  being  “filled 
in”  in  the  confocal  OTF.  This  reduction  of  the  out-of-focus  con¬ 
tribution  is  one  effect  of  deconvolution  on  the  WF  data.  However, 
in  addition  to  this,  both  WF  and  confocal  OTFs  imply  an  attenua¬ 
tion  of  higher  spatial  frequencies  both  in-plane  and  more  markedly 
in  the  z-direction.  Deconvolution  has  the  effect  of  compensating 
for  this  attenuation,  in  effect  sharpening  the  image. 

For  most  specimens,  it  seems  unlikely  that  the  low  spatial  fre¬ 
quency  out-of-focus  light  can  be  regarded  as  carrying  much  if  any 
useful  information  (i.e.,  any  useful  information  that  is  present  for 
a  given  focal  section  is  likely  to  be  restricted  to  focal  sections  very 
close  in  z:  1-2  pm).  The  net  result  of  the  light  that  is  more  out  of 
focus  being  present  in  WF  images  must  be  to  increase  at  least  the 
Poisson  noise  associated  with  the  measurement  of  a  given  in-focus 
signal. 

Although  the  previous  example  may  seem  somewhat  atypical, 
it  has  features  in  common  with  many  real  specimens.  For  example, 
a  specimen  may  consist  of  isolated  regions  of  bright  labeling  that 
are  thin  compared  to  the  depth  of  field  and  that  occur  at  signifi¬ 
cantly  different  depths.  We  might  describe  such  a  specimen  as 
“punctate”  and  an  example  might  be  fluorescent,  in  situ  hybridized 
chromosomes,  which  typically  contain  only  a  few  bright  spots  at 
or  near  the  resolution  limit  and  at  various  focal  planes.  Unless  there 


is  a  substantial  real  background  originating  from  other  focal 
planes,  widefield  CCD  imaging  is  likely  to  be  better  than  confo¬ 
cal  imaging  for  this  type  of  specimen  with  currently  available 
equipment.  There  is  relatively  little  out-of-focus  light  at  low  spatial 
frequencies,  so  in  these  circumstances,  the  QE  of  the  CCD  camera 
would  probably  override  the  disadvantage  that  it  detects  low- 
frequency  out-of-focus  light.  Similar  considerations  would  apply 
to  punctate  labeling  at  the  membrane  of  a  cell  or  organelle. 

If,  in  addition  to  the  thin  plane  of  interest,  the  specimen  had 
uniform  background  intensity,  B,  throughout  the  rest  of  its  volume, 
the  WF  result  would  become  worse  as  the  specimen  becomes 
thicker,  and  confocal  imaging  would  become  better  than  WF 
imaging.  This  is  because  the  intensity  from  each  additional  slice 
of  the  background  is  added  in  its  entirety  to  all  of  the  WF  images. 
Thus,  the  S/N  decreases  linearly  with  specimen  thickness  in  the 
WF  case,  whereas  the  background  is  excluded  from  the  confocal 
image,  and  the  S/B  is  largely  independent  of  specimen  thickness. 
Sandison  and  colleagues  (1993)  have  calculated  the  effect  on  S/N 
and  signal  contrast  of  the  confocal  out-of-focus  light  exclusion. 
Sandison  and  colleagues  (1993)  and  Inoue  (1986)  define  contrast 
as  the  S/B.  Ignoring  other  sources  of  noise,  the  Poisson  noise  for 
Accounts  is  -JW  and  the  S/N  =  SN.  In  the  presence  of  background, 
N  =  S  +  B  and  S/N  =  S/^(S  +  B).  As  B  is  increased  for  a  given 
signal,  both  the  contrast,  S/B,  and  the  S/N  decrease.  There  comes 
a  point  when  the  in-focus  signal,  S,  becomes  lost  in  the  Poisson 
shot  noise  [V(S  +  5)]  produced  by  both  in-focus  and  out-of-focus 
light.  According  to  this  argument,  in  the  WF  case  the  background, 
B,  is  increased,  while  the  in-focus  “signal”  S  remains  the  same. 
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Sandison  and  colleagues  have  calculated  for  model  specimens  how 
S,  «J(S  +  B),  and  B  are  affected  by  the  size  of  the  detector  aper¬ 
ture,  how  confocal  imaging  compares  with  WF  imaging,  and  how 
to  determine  the  detector  aperture  size  which  maximizes  detected 
S/N.  This  is  also  covered  in  Chapter  22. 

The  results  of  this  type  of  analysis  clearly  depend  on  the 
assumptions  made  about  the  distribution  of  fluorescent  material  in 
the  specimen;  similar  analyses  need  to  be  made  for  various  differ¬ 
ent  specimen  geometries.  Furthermore,  the  detector  aperture  size 
that  maximizes  S/N  will  not  in  general  give  the  maximum  resolu¬ 
tion;  deconvolution  can  be  used  to  improve  this  situation. 

PRACTICAL  COMPARISONS 

We  have  undertaken  some  comparisons  of  CLSM  and  CCD 
imaging  of  some  of  the  specimens  we  are  using  in  our  work  — 
fluorescent  in  situ  labeling  of  plant  root  tip  tissue  slices  ~50|im 
in  thickness  (Shaw  and  Rawlins,  1991b;  Highett  et  al.,  1993a,b). 
Deconvolution  can  be  equally  well  applied  to  confocal  data,  using 
the  confocal  OTF,  to  reverse  as  far  as  possible  the  attenuation 
implied  by  these  PSF  measurements  and  to  substantially  reduce 
Poisson  noise.  Figure  23.7  shows  the  confocal  bead  data  used  to 
derive  the  confocal  OTF  both  before  [Fig.  23.7(A,C)]  and  after 
[Fig.  23.7(B,D)]  deconvolution. 

In  both  data  sets,  deconvolution  clearly  sharpens  the  bead 
image,  particularly  in  z,  and  gives  some  confidence  in  applying  the 
procedure  to  3D  confocal  data  sets.  In  Figure  23.8(A-C),  the  same 
portion  of  a  specimen  was  imaged  with  three  different  settings  of 
the  detector  aperture  (0.7,  2.5,  and  4  Airy  units;  Shaw  and  Rawlins, 
1991b).  In  each  case,  a  3D  focal  section  stack  of  about  30  sections 
separated  by  0.2  pm  was  collected.  A  single  equivalent  section 
from  these  data  sets  is  shown  in  Figure  23.8.  The  increase  in  aper¬ 
ture  size  clearly  degrades  the  resolution.  This  must  be  attributed 
primarily  to  contamination  of  the  optical  sections  with  out-of¬ 
focus  light  from  nearby  parts  of  the  specimen  that  were  too  near 
for  the  detector  aperture  to  eliminate.  In  each  case,  the  data  was 


deconvolved  by  the  iterative  Jansson  method,  using  the  appropri¬ 
ate  PSF  measured  with  the  relevant  aperture  settings.  The  equiva¬ 
lent  sections  are  shown  after  deconvolution  in  Figure  23.81D-F). 
The  deconvolution  has  restored  each  image  to  an  equivalent  result. 
Provided  the  image  data  has  been  measured  with  sufficient  accu¬ 
racy,  deconvolution  can  restore  the  different  degrees  of  image 
degradation  given  by  different  sizes  of  detector  aperture.  Figure 
23.8(G-L)  shows  enlargements  of  the  small  area  boxed  in  Figure 
23.8(A).  These  enlargements  show  the  effect  of  the  deconvolution 
algorithm  in  reducing  the  noise  in  the  images. 

Whatever  the  relative  merits  of  confocal  and  WF  imaging,  it 
is  clear  that  deconvolution  can  make  good  confocal  data  look  even 
better.  Figure  23.9  shows  another  example  of  this  from  our  work 
(Beven  et  al.,  1996).  In  this  image  deconvolution  clearly  resolves 
the  labeling  of  transcription  sites  within  a  plant  nucleolus  to  show 
that  they  are  composed  of  many  closely  packed  foci. 

Figure  23.10  shows  a  specimen  imaged  by  both  WF  [Fig. 
23.10(A)]  and  confocal  [Fig.  23.10(C)]  microscopy.  The  agree¬ 
ment  between  the  deconvolved  WF  data  [Fig.  23.10(B)]  and  the 
raw  confocal  data  [Fig.  23.10(C)]  is  remarkably  good,  and  gives 
confidence  that  either  method  will  give  satisfactory  results.  The 
deconvolved  WF  data  seems  somewhat  better  than  the  confocal 
data  in  both  x,y  and  x,z  sections.  However,  the  deconvolution 
applied  to  the  confocal  data  gives  the  clearest  image  of  all  [Fig. 
23.10(D)], 

In  Figure  23.11,  the  same  data  sets  are  resectioned  parallel  to 
the  optical  axis  (x,z)  and  lead  to  very  similar  conclusions.  It  should 
be  noted  that  diverging  cones  from  very  bright  features  are  still 
visible  after  deconvolution  of  the  WF  data.  This  is  because  the  PSF 
used  in  deconvolution  is  not  exactly  right  for  the  conditions  under 
which  the  data  set  was  collected.  There  is  clearly  considerable 
spherical  aberration,  producing  an  asymmetrical  PSF,  possibly 
because  the  cell  was  deep  within  a  tissue  slice.  Obtaining  accurate 
PSFs  for  a  given  specimen  is  often  a  substantial  problem.  The  ideal 
solution  would  be  to  determine  the  PSF  from  the  data  set  itself 
before  deconvolution  or  perhaps  to  use  “blind”  deconvolution  as 
described  in  Chapter  24. 


FIGURE  23.7.  Deconvolution  of  confocal  bead  data.  (A)  x,y  sections  before  deconvolution.  (B)  x,y  sections  after  deconvolution.  (C)  x,z  sections  before  decon¬ 
volution.  (D)  x ,z  sections  after  deconvolution.  (Reproduced  from  Shaw  and  Rawlins,  1991b,  J.  Microsc.  163:151-165.) 
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FIGURE  23.8.  Single  equivalent  sections  from  confocal  data  sets  collected  with  different  detector  aperture  settings  before  and  after  deconvolution  with  the 
appropriate  OTF.  (The  specimen  was  a  pea  root  tissue  slice  labeled  by  fluorescence  in  situ  hybridization  with  a  probe  to  the  rDNA.)  (A-C)  Unprocessed  con- 
focal  data.  (A)  Detector  aperture,  0.7  Airy  units.  (B)  Detector  aperture,  2.5  Airy  units.  (C)  Detector  aperture,  4.3  Airy  units.  (D-F)  Equivalent  sections  to  (A-E) 
after  deconvolution.  Although  the  image  degradation  due  to  increasing  the  detector  aperture  is  clear,  the  reconstructed  results  are  all  very  similar  to  each  other. 
(G-L)  Enlargements  of  the  equivalent  areas  from  (A-F)  to  the  boxed  area  in  (A),  showing  the  effect  of  deconvolution  on  the  noise  in  the  image.  Field  width, 
12.3pm.  (Reproduced  from  Shaw  and  Rawlins,  1991b,  J.  Microsc.  163:151-165.) 


FIGURE  23.9.  Stereo  projections  of  a  confocal  data  set 
before  (A)  and  after  (B)  deconvolution.  The  data  is  from  a 
pea  root  slice  labeled  by  fluorescence  in  situ  hybridization 
with  a  probe  to  the  external  transcribed  spacer  of  the  ribo- 
somal  RNA  transcript,  which  marks  the  transcription  sites 
within  the  nucleolus.  Deconvolution  clearly  shows  these 
sites  as  closely  packed  foci  within  a  subregion  of  the  nucle¬ 
olus,  each  of  which  corresponds  to  an  active  gene  copy.  Bar 
=  2  pm. 
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FIGURE  23.10.  Comparison  of  WF  and  confocal  images  from  the  same  specimen  (a  nucleolus  from  a  pea  root  tissue  slice  labeled  by  fluorescence  in  situ 
hybridization  with  probe  to  the  rDNA).  (A)  Four  consecutive  optical  sections  from  a  3D  data  set  imaged  by  WF  microscopy.  Data  collected  on  a  cooled  CCD 
camera,  using  Leitz  63x,  1.4  NA  objective.  (B)  Equivalent  sections  from  the  same  data  set  as  in  (A)  after  deconvolution.  The  background  flare  is  eliminated  and 
the  contrast  of  the  fine  image  detail  is  greatly  improved.  (C)  Equivalent  confocal  sections  from  the  same  specimen.  The  same  objective  was  used  and  the  con¬ 
focal  detector  aperture  was  set  to  2.5  Airy  units.  The  images  are  remarkably  similar  to  the  deconvolved  WF  ones,  although  the  contrast  of  the  fine  detail  is 
perhaps  a  little  worse.  (D)  Equivalent  sections  from  the  same  data  set  as  (C)  after  deconvolution.  The  clarity  of  the  fine  detail  is  clearly  best  in  this  data.  (E-H) 
Enlargements  of  the  equivalent  areas  to  the  area  boxed  in  (A).  Spacing  between  sections  is  0.25  pm.  Field  width,  8  pm. 
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FIGURE  23.1 1 .  The  same  data  as  in  Figure  23.8  in  the  same  order,  resectioned  to  show  x,z  sections.  Although  the  poor  z-resolution  compared  to  the  in-plane 
resolution  is  very  apparent,  the  comparison  between  the  four  data  sets  leads  to  the  same  conclusions  as  Figure  23.8;  the  deconvolved  WF  and  unprocessed  con- 
focal  data  are  fairly  similar,  whereas  deconvolution  of  the  confocal  data  gives  the  best  result.  Note  also  the  residual  out-of-focus  cone  around  the  bright  feature 
remaining  after  deconvolution  of  the  WF  data.  This  is  due  to  a  mismatch  between  the  actual  PSF  and  the  PSF  used  for  the  deconvolution,  probably  because  the 
data  was  recorded  deep  inside  a  tissue  slice,  and  spherical  aberration  caused  the  PSF  to  be  asymmetric  at  this  depth.  Spacing  between  sections  is  0.25  pm.  Field 
width,  8  pm. 


CONCLUSION 

Speed,  convenience,  and  ease  of  use  have  until  recently  favored 
confocal  microscopy  over  WF/deconvolution  approaches. 
However,  widefield  CCD  microscopes  and  associated  operation 
and  deconvolution  software  are  now  much  more  user  friendly,  and 
limitations  in  computer  power  for  deconvolution  much  less  of  a 
problem.  This  has  led  to  much  more  widespread  use  of  widefield 
CCD/deconvolution  imaging.  Aside  from  these  considerations, 
various  factors  are  important  in  assessing  the  relative  image  quality 
achievable  by  the  two  methods. 

•  Effective  QE  is  currently  better  with  a  cooled  CCD  camera 
than  with  either  the  PMTs  used  in  most  single-beam  CLSMs 
or  the  EM-CCDs  likely  be  used  with  disk  scanners. 

•  Low  spatial  frequency,  out-of-focus  light  or  flare,  correspond¬ 
ing  to  the  missing  cone  region  in  the  widefield  OTF  is  largely 
eliminated  with  confocal  microscopy  using  even  a  relatively 
large  detector  aperture.  Other  things  being  equal,  it  should 
always  be  preferable  to  eliminate  this  light  because  it  can 
almost  never  be  considered  to  contain  useful  information  and 


its  inclusion  will  increase  Poisson  noise,  decreasing  the  signal 
contrast  and  hence  the  S/N  of  the  data.  In  some  cases,  partic¬ 
ularly  with  thick  specimens,  the  low  frequency  out-of-focus 
light  can  completely  swamp  the  in-focus  signal,  and  confocal 
microscopy  then  has  an  overwhelming  advantage.  However, 
deconvolution  of  WF  data  does  a  good  job  of  removing  mod¬ 
erate  levels  of  out-of-focus  flare. 

•  Attenuation  of  higher  spatial  frequencies  is  approximately  the 
same  with  either  WF  or  confocal  imaging  unless  a  very  small 
confocal  detector  aperture  can  be  used.  This  is  almost  invari¬ 
ably  impossible  with  the  signal  levels  encountered  in  biologi¬ 
cal  microscopy.  This  high  frequency  attenuation  can  be 
corrected  by  deconvolution  of  WF  data  and  means  that  decon¬ 
volved  WF  data  may  often  show  higher  effective  resolution 
than  unprocessed  confocal  data. 

•  However,  a  genuine  comparison  should  also  include  deconvo¬ 
lution  of  the  confocal  data,  and  in  the  cases  we  have  shown, 
deconvolved  confocal  data  shows  even  better  resolution  than 
deconvolved  WF  data. 

•  Spinning-disk  confocal  microscopy  combined  with  CCD 
image  detection  in  many  ways  gives  the  best  of  both 
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approaches.  The  confocal  optics  eliminates  much  of  the  out- 
of-focus  light,  and  its  associated  noise,  and  the  CCD  camera 
provides  sensitive  and  accurate  image  detection.  Parallel 
imaging  at  correspondingly  lower  peak  light  levels  seems  to 
have  definite  advantages,  particularly  for  live  cells  and  photo¬ 
sensitive  specimens.  Subsequent  deconvolution  should  have 
the  effect  of  increasing  the  clarity  of  the  image,  as  it  does  with 
data  from  a  single-beam  instrument.  This  situation  can  be 
expected  to  improve  with  the  widespread  use  of  EM-CCDs. 

SUMMARY 

•  The  relative  merits  of  confocal  and  WF  microscopy  depend  on 
the  amount  and  spatial  frequency  spectrum  of  the  out-of-focus 
light. 

•  The  amount  of  out-of-focus  light  depends  on  the  specimen,  the 
depth  distribution  of  the  significant  fluorescent  intensity,  the 
level  of  generally  distributed  non-specific  background,  and 
the  thickness  of  the  specimen. 

•  For  very  thin  specimens,  or  specimens  with  high  fluorescent 
intensities  restricted  to  thin  shells  within  thick  specimens  of 
low  background  intensity,  there  is  little  out-of-focus  light  to  be 
excluded.  In  these  cases,  the  CCD  cameras  in  use  in  WF 
imaging  give  better  QE  and  therefore  statistically  better- 
defined  images  for  a  given  dose  of  illuminating  photons. 

•  Image  deconvolution  should  be  applied  to  confocal  images  as 
well  as  to  WF  images  if  the  highest  resolution  is  needed  and 
to  remove  image  noise. 
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Blind  Deconvolution 

Timothy  J.  Holmes,  David  Biggs,  and  Asad  Abu-Tarif 


INTRODUCTION 

Purpose  of  Deconvolution 

Deconvolution1  is  a  process  most  often  applied  to  three- 
dimensional  (3D)  data  sets  from  widefield  fluorescence  (WFF) 
microscopy.  Additionally,  it  can  be  applied  to  3D  data  for  trans¬ 
mitted  light  bright-held  (TLB)  and  other  widefield  modalities.  It 
improves  resolving  power  and  signal-to-noise  ratio  in  data  from 
confocal  laser-scanning  microscopes  (CLSM)  and  confocal 
spinning-disk  microscopes,  and  it  provides  a  transformation  to  data 
from  differential  interference  contrast  (DIC)  images  to  effectively 
extract  a  3D  image  of  refractive  index  distribution. 

With  WF  microscopes,  3D  data  are  collected  by  optical  sec¬ 
tioning  (Castleman,  1979;  Agard,  1984,  1989).  As  illustrated  in 
Figures  24.1  and  24.2,  an  automatic  focus  controller  and  digital 
camera  record  a  sequence  of  images  of  a  thick  specimen.  The  focus 
controller  drives  either  a  stepper  motor  focus  or  a  piezo  crystal 
embedded  within  a  customized  objective  lens  mount.  When  a 
microscope  having  a  narrow  depth  of  held  (DOF)  is  focused  on  a 
plane,  the  image  contains  both  sharp,  in-focus  features  originating 
from  the  plane  of  focus  and  blurred  features  originating  from 
planes  above  and  below.  After  storing  an  image,  the  microscope  is 
focused  to  an  adjacent  plane,  which  is  about  one  DOF  away,  and 
another  image  is  recorded.  This  process  is  repeated  until  the  entire 
specimen  is  scanned.  The  result  is  a  3D  data  set  that  contains  the 
out-of-focus  blur.  Deconvolution  eliminates  the  blur  to  restore 
clarity  of  the  image.  An  example  is  shown  in  Figure  24.3.  After 
deconvolution,  the  image  may  be  rendered  for  3D  viewing  and 
quantitation  by  methods  discussed  in  Chapters  14  and  15. 

When  using  confocal  microscopes,  including  both  the  CLSM 
and  the  spinning-disk  microscopes  (such  as  the  Yokogawa  CSU22, 
PerkinElmer  Life  Sciences,  Boston,  MA;  the  CARV-2  from  Atto 
Biosciences,  Rockville,  MD;  or  the  DSU  from  Olympus),  images 
of  adjacent  planes  are  optically  sectioned  in  a  similar  way,  except 
that  each  frame  is  collected  using  either  a  single,  scanned  light  spot 
and  a  photodetector  or  a  structured  array  of  scanned  pinholes  that 
serve  as  both  light  spots  and  point  detectors.  Each  confocal  image 
is,  by  itself,  already  partially  deconvolved  because  the  optics  reject 
most  of  the  out-of-focus  light.  Most  of  the  features  recorded  in  a 


1  Deconvolution  is  a  mathematical  process  that  is  the  opposite  to  convolution. 
Convolution  is  described  in  the  Appendix  at  the  end  of  this  chapter.  Convo¬ 
lution  is  useful  because  it  can  mimic  the  blurring  effects  of  diffraction  and 
geometrical  optics.  Deconvolution  is  important  because,  to  the  extent  that  the 
blurring  effect  can  be  described  mathematically,  it  can  reverse  this  blurring 
effect. 


3D  confocal  data  set  are  in  focus.  However,  the  CLSM  has  its  own 
limitations.  While  it  rejects  most  of  the  out-of-focus  light,  it  does 
not  reject  all  of  it.  Some  haze  remains.  In  addition,  the  3D  data  set 
has  axial  smearing.  Point  objects  look  like  American  footballs  ori¬ 
ented  along  z.  As  fewer  photons  are  detected  compared  to  WFF, 
the  raw  data  contains  much  quantum  or  Poisson  noise.  This  noise 
not  only  obscures  real  structures  in  the  image,  as  explained  in 
Chapter  4,  it  can  actually  create  impossible  features.  As  Poisson 
noise  affects  the  intensity  recorded  in  each  pixel  separately, 
random  positive  excursions  can  give  the  appearance  of  small  bright 
features  that  are  only  one  pixel  big.  Such  a  feature  is  impossible 
because,  assuming  that  the  data  was  recorded  using  Nyquist 
sampling,  no  real  feature  should  be  smaller  than  4  to  5  pixels 
across.  Because  deconvolution  suppresses  data  corresponding  to 
features  smaller  than  the  point-spread  function  (PSF),  it  eliminates 
such  noise  features  from  the  data  and  thereby  meets  the  Nyquist 
reconstruction  criterion.  Simply  put,  properly  sampled,  noisy  3D 
data  sets  are  almost  always  improved  by  deconvolution.  Decon¬ 
volution  reduces  or  eliminates  all  four  of  these  problems  and 
thereby  improves  the  confocal  image.  An  example  is  shown  in 
Figure  24.4. 

To  a  first  approximation,  DIC  contrast  is  produced  by  a  set  of 
polarizers  and  Wollaston  prisms  that  form  an  image  that  is  the  dif¬ 
ferential  in  the  direction  of  shear  of  the  refractive  index  of  the  spec¬ 
imen  at  the  focus  plane.  The  image  contains  information  about  the 
3D  refractive  index  distribution,  although  this  information  is  con¬ 
volved  and  non-linearly  distorted  by  the  objective  lens,  polarizers, 
and  differential  interference  optics.  Deconvolution  can  be  used  to 
invert  this  process  and  to  produce  an  image  whose  intensity  is  pro¬ 
portional  to  the  3D  refractive  index  distribution.  An  example  is 
shown  in  Figure  24.5. 

Advantages  and  Limitations 

Early  deconvolution  methods  used  a  calculated  or  measured  PSF 
(Hiraoka  et  al.,  1990;  see  also  Chapter  25,  this  volume ).  The  PSF 
was  measured  by  collecting  a  3D  image  of  a  subresolution  fluo¬ 
rescent  microsphere.  The  accuracy  of  this  process  is  limited  by  the 
fact  that  the  biological  sample  itself  is  part  of  the  optics  of  the 
system.  It  has  its  own  heterogeneous  refractive  index,  and  this 
changes  the  PSF.  In  principle,  the  only  way  to  measure  the  correct 
PSF  is  to  measure  it  within  the  biological  sample  itself,  by  placing 
microspheres  in  the  sample  at  the  depth  being  imaged.  The  depth 
is  important  because  the  blurring  caused  by  spherical  aberration  is 
the  most  prominent  factor  that  changes  in  the  PSF,  and  spherical 
aberration  changes  with  depth  (Gibson,  1991;  Visser,  1992;  see 
also  Chapter  20,  this  volume).  Although  people  have  succeeded  at 
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FIGURE  24.1 .  Optical  sectioning  schematic.  This  setup  is  used  with  the  wide- 
held  modalities,  including  WFF,  TLB,  DIC,  and  polarized  light. 
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FIGURE  24.2.  Hardware  components  needed  for  optical  sectioning. 


inserting  microspheres  into  some  samples,  it  is  impractical  to  do 
so  routinely. 

The  approach  highlighted  in  this  chapter  does  not  use  a  calcu¬ 
lated  or  measured  PSF  and  is  therefore  called  blind  deconvolution 
(Ayers  and  Dainty,  1988;  Bhattacharyya  et  al.,  1996;  Holmes, 
1992;  Krishnamurthi  et  al.,  1995).  The  actual  PSF  is  degraded  by 
any  heterogeneity  in  the  refractive  index  of  the  specimen  (Gibson 
and  Lanni,  1991).  These  complications  are  compounded  in  CLSM 
fluorescence,  where  light  levels  are  very  low  and  the  effect  of 


refractive  index  variations  are  even  more  severe  (Visser  et  al., 
1992;  Chapter  20,  this  volume). 

Blind  deconvolution  uses  an  iterative  process  to  produce  both 
a  reconstructed  estimate  of  the  PSF  as  well  as  the  deconvolved 
data.  Realistic  mathematical  constraints  on  the  PSF  are  used. 
Figures  24.6  and  24.7  demonstrate  a  blind  deconvolution  of  a  data 
set  with  severe  spherical  aberration,  including  the  PSFs  at  the 
beginning  and  end  of  the  reconstruction.  Figures  24.6(A)  and 
24.7(A)  show  xz  views  that  have  a  non-symmetrical  flare  along  z 


FIGURE  24.3.  (A)  Two  slices  from  an  unprocessed  WFF  stack.  (B)  Deconvolved  slices  from  the  blind  deconvolution.  (Image  courtesy  of  M.  Schechter,  Leica 
Microsystems.) 


470  Chapter  24  •  T.J.  Holmes  et  al. 


FIGURE  24.4.  (A)  Two  sections  from  a  confocal  stack. 
(B)  After  deconvolution.  Resolving  power  along  z, 
signal-to-noise  ratio  and  contrast  are  improved.  Chro¬ 
mosomes  are  now  apparent  (1)  and  vimentin  fibers 
are  seen  above  and  below  the  nucleus  (2),  as  they 
should  be,  but  are  not  seen  within  the  nucleus  [left 
frame  of  (B)].  (Image  courtesy  of  M.  Hoppe,  Leica 
Microsystems.) 


FIGURE  24.5.  (A)  Two  optical  sections  from  a  DIC 
stack.  (B)  After  deconvolution.  The  deconvolved 
images  appear  like  CLSM  fluorescent  samples.  No 
stain  is  used,  but  the  brightness  is  proportional  to  the 
refractive  index  of  the  sample,  resulting  in  a  virtual 
refractive  index  map. 
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B  C  D  E 


FIGURE  24.6.  (A)  xz  slice  through  a  3D  WF  fluorescent  data  set  collected  with  a  water-immersion  lens,  showing  spherical  aberration.  (Image  courtesy  of  D. 
Kube,  Case  Western  Reserve  University.)  (B)  xz  sum  projection  of  the  first  guess  of  the  PSF  used  to  seed  the  first  iteration  of  the  blind  deconvolution.  (C)  xz  sunt 
projection  of  the  PSF  produced  by  the  blind  deconvolution  procedure.  (D)  Theoretically  calculated  PSF  with  a  spherical  aberration  coefficient  chosen  to  match 
the  flare  seen  in  Figure  24.6(A).  (E)  Theoretically  calculated  PSF  without  spherical  aberration,  for  comparison. 


that  is  characteristic  of  spherical  aberration.  This  condition  is 
especially  evident  with  water-immersion  lenses,  when  the  collar 
on  the  objective  has  not  been  properly  adjusted  for  the  coverslip 
thickness.  The  PSF  reconstruction  is  shown  in  Figure  24.6(B,C) 
and  is  discussed  later  in  the  section  entitled  “More  Examples  — 
Blind  Deconvolution  and  Spherical  Aberration.” 

A  non-blind  version  of  this  algorithm  (Holmes  and  Liu,  1991) 
is  available  for  cases  where  the  PSF  can  be  accurately  measured 
(Hiraoka  et  al.,  1990)  or  calculated  (Gibson  and  Lanni,  1991). 


The  approach  highlighted  in  this  chapter  uses  a  mathematical 
tool  called  maximum  likelihood  estimation  (MLE).  As  this 
approach  includes  quantum  photons  as  an  underlying  assumption, 
it  results  in  a  reduction  in  the  noise  that  is  caused  by  Poisson  noise. 
Because  of  diffraction,  any  microscope  represents  a  bandlimited 
system.  As  explained  later,  this  means  that  any  “good”  signal  lies 
inside  the  bandlimit.  Much  of  the  undesirable  noise  energy  (i.e., 
noise  signal)  lies  outside  the  bandlimit.  The  algorithm  recognizes 
this  and  subsequently  rejects  the  out-of-band  noise  automatically. 


FIGURE  24.7.  ( A)  xz  maximum  projection  of  the  data  set  shown 
in  Figure  24.6(A).  Spherical  aberration  is  apparent  by  the  asym¬ 
metry  in  the  amount  of  flare  above  and  below  the  sample.  (B) 
Blind  deconvolution  result,  showing  the  elimination  of  spherical 
aberration.  The  PSF  shown  in  Figure  24.6(B)  was  used  as  the 
PSF  first  guess,  and  the  PSF  shown  in  Figure  24.6(C)  was  pro¬ 
duced  by  the  blind  deconvolution  procedure. 
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It  also  constrains  the  deconvolved  image  to  have  only  non¬ 
negative  values,  a  process  that  cannot  be  implemented  with  a 
simple  bandlimiting  filter.  Best  results  are  achieved  when  the  pixel 
or  optical  section  spacing  is  finer  than  required  by  Nyquist 
sampling  (Chapters  2  and  4,  this  volume). 


PRINCIPLES 

Data  Collection  Model 

The  data  collection  is  modeled  (Holmes,  1989,  1992;  Holmes  and 
Liu,  1989,  1991;  Holmes  et  al.,  1991)  according  to  the  following 
equations: 

M-(x,  y,  z)  =f(x,  y,  z)  *h(x,  y,  z),  (1) 

lL(x,  y,  z)  =  p(x,  y,  z)  +  b(x,  y,  z),  (2) 

and 

|i„(x,  y,  z)  =  |i*(x,  y,  z)  +  N(x,  y,  z),  (3) 

where  x  and  y  are  the  in-plane  coordinates,  z  is  the  axial  coordi¬ 
nate,  |x()  is  the  data  collected  under  ideal  conditions  where  there 
is  no  background  level  or  noise,  /()  is  the  fluorescent  dye  concen¬ 
tration  as  a  function  of  position,  *  represents  the  convolution 
operation,  hQ  is  the  PSF,  |i,,()  is  the  ideal  measured  intensity  with 
background  and  without  noise,  b()  is  a  background  level  mainly 
due  to  non-specific  staining  and  the  dark  current  in  the  image 
sensor  (Janesick  et  al.,  1987;  Snyder  et  al.,  1993),  |i„()  is  the 
collected  image  data  that  contains  noise,  and  NQ  is  noise,  pri¬ 
marily  due  to  quantum  variations  in  the  number  of  photons 
recorded. 

Deconvolution  produces  an  estimate,  or  restoration,  of  /(). 
The  background  level  b{)  is  calibrated  as  discussed  later  and  is 
addressed  by  precorrecting  the  raw  data  (Chapters  3  and  13,  this 
volume). 


MAXIMUM  LIKELIHOOD  ESTIMATION 

Maximum  likelihood  estimation  is  a  mathematical  optimization 
strategy  (Van  Trees,  1968)  used  for  producing  best  estimates  of 
quantities  from  data  corrupted  by  random  noise.  Because  of  the 
randomness  of  quantum  photon  emissions,  the  collected  image 
represents  data  having  a  certain  likelihood  of  having  been  pro¬ 
duced  among  a  host  of  other  data  that  might  have  been  recorded. 
MLE  produces  the  estimate  of/()  that  has  the  highest  likelihood 
of  having  given  rise  to  the  specific  data  collected. 

In  mathematical  terms,  we  are  formulating  a  log-likelihood 
function  that  represents  a  measure  of  the  likelihood  that  one  would 
collect  the  specific  noisy  data  that  were  actually  collected.  This 
function  is  actually  a  functional  (i.e.,  a  function  of  a  function)  of 
/(),  h(),  and  jj.„().  Its  formulation  is  based  on  the  random  nature 
of  the  quantum  photon  noise.  This  functional  is  solved  iteratively 
to  produce  its  maximum  value.  In  blind  deconvolution,  specific 
quantities  of  /()  and  h()  are  solved  using  an  iterative  search 
(Dempster  et  al.,  1977).  The  specific  quantities  of/()  and  h()  that 
produce  this  maximum  become  the  reconstructed  image  and  the 
reconstructed  PSF.  With  non-blind  deconvolution,  li()  is  known 
from  a  prior  measurement  and  the  deconvolution  algorithm  solves 
only  for/(). 


Algorithms 

WFF  and  CLSM  Fluorescence 

The  flowchart  for  the  blind  deconvolution  is  shown  in  Figure  24.8. 
In  Step  1,  first  guesses  of  /<0)()  and  ft(0)()  are  made.  Step  2  is 
designed  from  the  expectation-maximization  (EM)  algorithm 
(Dempster  et  al.,  1977;  Holmes,  1989,  1992;  Holmes  and  Liu, 
1989,  1991;  Holmes  et  al.,  1991).  Step  3  enforces  the  constraints 
placed  on  the  solution  of  the  PSF.  The  details  of  Step  3  are  out¬ 
lined  in  Figure  24.9. 

The  system  may  use  a  quantitative  criterion  for  stopping 
the  iterations.  Several  quantitative  stopping  criteria  have  been 
studied  for  applications  in  nuclear  medicine  (Veklerov  and  Llacer, 
1987;  Llacer  and  Veklerov,  1989).  We  prefer  methods  that  monitor 
the  progress  in  the  likelihood  value  and  then  stop  the  iterations 
when  the  improvement  in  this  functional  between  the  last  two 
iterations  drops  below  a  selected  threshold. 

The  design  of  the  flowchart  in  Figure  24.9  is  modeled  after  the 
Gerchberg-Saxton  approach  (Gerchberg  and  Saxton,  1974).  Step 
1  enforces  a  unit  summation  constraint.  Step  2  constrains  the 
energy  of  the  PSF  to  lie  within  an  hourglass  region  bh  that  is  illus¬ 
trated  in  Figure  24.10.  This  constraint  prevents  portions  of 
the  background  intensity  b{)  from  being  mistakenly  assigned  to  the 
PSF.  Step  3  forces  the  3D  optical  transfer  function  [OTF,  the 
Fourier  transform  (FT)  of  the  PSF]  to  be  bandlimited.  The  3D 
OTF  has  a  frequency  band  (BWJ  wherein  it  may  be  non-zero, 
with  a  missing  cone  region  illustrated  in  Figure  24.11.  Step  4 
ensures  non-negativity.  This  step  is  needed  because  Step  3  would 
otherwise  cause  negative  numbers  to  appear  in  the  result,  but  light 
intensity  is  never  negative  (these  negative  pixel  values  are  set  to 
zero).  In  the  two-dimensional  (2D)  diffraction-limited  case,  the 
OTF  frequency  band  is  described  by  the  interior  of  a  circle  having 
a  radius  of  BW^,  following  the  same  formula  as  in  the  caption  of 
Figure  24. 1 1 . 

The  CLSM  fluorescence  algorithm  is  similar  to  the  WFF  algo¬ 
rithm.  The  hourglass  region  of  Figure  24.10  is  replaced  by  a  cylin¬ 
der  of  radius  r,  and  the  bandlimit  region  of  Figure  24. 1 1  is  replaced 
by  a  cylinder  with  twice  the  bandwidth  along  x,y  and  z  and  with 
no  missing  cone  region.  The  bandwidths  are  doubled  because  the 
confocal  PSF,  when  theoretically  determined  according  to  Fourier 
optics  (Conchello  et  al.,  1994),  is  approximated  by  a  multiplica¬ 
tion  of  two  widefield  PSFs.  By  the  convolution  theorem,  when  any 
two  functions  are  multiplied  their  bandwidths  are  added.  In  real 
life,  however,  the  truly  confocal  condition  is  only  met  if  the  detec¬ 
tor  pinhole  is  infinitely  small,  and  the  signal  level  is  so  high  that 
Poisson  noise  can  be  ignored.  Clearly  this  can  never  happen  and 
a  smaller  diameter  of  BW„  is  appropriate. 

Transmitted  Light,  Bright-field  (TLB) 

The  deconvolution  of  TLB  images  of  absorbing  stains  are  pro¬ 
duced  using  a  variation  on  the  WFF  algorithm  (Holmes  et  al., 
1995;  Holmes  and  O’Connor,  2000).  The  main  requirement  is  that 
the  condenser  aperture  must  be  larger  than  the  objective  lens  aper¬ 
ture.  This  variation  of  the  algorithms  depends  on  the  approxima¬ 
tion  that  the  raw  TLB  data  p()  [(with  noise  N()  and  background 
bQ  ignored]  follows  a  simple  linear-systems  model  according  to: 

p(x,  y,z)  =  B-  h(x,  y,  z)*f{x,  y,  z),  (4) 

where  B  is  the  bright  background  intensity,  and  /()  is  the  absorp¬ 
tion  coefficient.  This  approximation  follows  from  Macias-Garza 
and  colleagues  (1989)  when  geometrical  optics  are  considered, 
from  Streibl  (1984;  Eq.  28)  and  when  the  phase  contribution  to  the 
intensity  in  the  image  is  ignored  (i.e.,  the  refractive  index  of  the 
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FIGURE  24.8.  Flowchart  of  the  MLE  blind  deconvolu¬ 
tion  algorithm:  f(k\x,  y,  z)  is  the  reconstructed  dye 
concentration  of  the  kth  iteration,  h(k\x,  y,  z)  is  the  recon¬ 
structed  PSF  of  the  kth  iteration,  jlp  is  the  optically 
sectioned  image  data,  after  being  corrected  for  non- 
uniform  sensitivity  and  shutter-speed  instability,  Nj-k\x, 
y,  z)  is  the  temporary  data  array  that  stores  an  uncon¬ 
strained  form  of  the  PSF  estimate.  Figure  24.9  shows 
details  of  the  constraints  in  Step  3. 


End 


specimen  is  the  same  as  that  of  the  mounting  medium),  from 
Erhardt  and  colleagues  (1985)  when  light  emanating  from  the 
specimen  is  approximated  as  being  fully  incoherent,  and  from 
Sheppard  and  Gu  (1994)  when  the  specimen  is  considered  to  be  a 
weakly  scattering  object  (i.e.,  structures  not  stained  enough  to 
make  them  opaque).  We  noticed  that  by  preprocessing  this  noise- 
free  raw  data  according  to: 

PpCr,  y,z)  =  B-  |i(.«,  y,  z)  (5) 

we  have  a  model  expressed  according  to: 

Mp(*>  y,  z)  =fix,  y,  z)*h(x,  y,  z),  (6) 

which  is  identical  to  Eq.  1. 

The  deconvolution  is  carried  out  as  follows:  First,  the  raw  data 
are  precorrected  for  background,  flat-field  non-uniformities  and 
shutter-speed  fluctuations  (Cooper  et  al.,  1993;  to  arrive  at  p,(.r,  y, 
z),  which  is  an  estimate  of  the  p(x,  y,  z)  shown  in  Eq.  4.  Then,  p,(jc, 
y,  z)  is  preprocessed  according  to  Eq.  5  to  produce  an  estimate  of 
\lp(x,  y,  z),  which  is  denoted  in  Figure  24.8  as  \ip{x,  y,  z).  Finally, 
the  flowchart  of  Figure  24.8  is  executed  on  jlp(x ,  y,  z). 

Figure  24.12  shows  stereo-pairs  both  before  and  after  a  TLB 
deconvolution  of  bright-field  data. 


Differential  Interference  Contrast  (DIC) 

The  objective  of  DIC  deconvolution  is  to  produce  a  3D  image  of 
the  refractive  index  distribution.  The  DIC  image  formation  model 
is  nonlinear  and  uses  Fourier  optics  principles  (Holmes  and  Levy, 
1987;  Hansen  and  Conchello,  1988;  Preza  et  al.,  1998).  A  rough 
conceptual  schematic  of  the  components  in  a  DIC  microscope  is 
shown  in  Figure  24.13.  Two  orthogonally  polarized  beams  are 
produced  by  Wollaston  prisms  and  pass  through  the  specimen  as 
separate  ray  bundles  slightly  displaced  from  each  other  in  the 
shear  direction.  The  image  represents  a  coherent-light  interference 
pattern  that  is  formed  at  the  image  sensor  when  the  two  beams  are 
combined.  Image  contrast  is  produced  by  inhomogeneity  of  the 
sample’s  refractive  index  (RI)  distribution  near  the  plane  of  focus. 

The  observed  image  g  is  modeled  by  a  linearized  approxima¬ 
tion  according  to  the  following  equation; 

g  =  s(K  ■/*  /?,  *  h2)  +  n  (7) 

where  /is  the  original  refractive  index  distribution,  hi  is  the  theo¬ 
retical  DIC  PSF  that  reduces  to  two  delta  functions  (van  Munster 
et  al.,  1997),  h2  is  the  PSF  that  accounts  for  the  resolution  limit  of 
the  optics,  n  is  Gaussian  noise,  and  *  is  the  convolution  operator. 
K  is  a  constant  that  converts  the  change  in  RI  to  a  phase  differ- 
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From  Fig.  24.8  Step  2 


To  Fig.  24.8  Step  4 

FIGURE  24.9.  Flowchart  of  constraints  applied  in  WFF  and  TLB.  These 
blocks  make  up  Step  3  in  Figure  24.8.  The  term  is  the  complement  of  bh 
(see  Fig.  24.10)  and  is  the  region  wherein  h()  is  near  zero.  H^M\u,  v,  w)  is  the 
Fourier  transform  of  Z  is  the  region  of  HQ  which  is  zero.  FFT  and 

FFT  1  indicate  the  forward  and  inverse  fast  Fourier  transforms,  respectively. 
These  constraints  are  carried  out  iteratively. 


FIGURE  24.1 0.  Hourglass  region  used  in  constraining  the  widefield  PSF.  The 
terms  h/,  and  b[  denote  the  regions  inside  and  outside  the  hourglass,  respec¬ 
tively.  The  radius  r  is  set  to  be  a  number  of  Airy  disk  widths  according  to 
r  =  0.61  c/./NA,  where  /.  and  NA  are  the  wavelength  and  numerical  aperture, 
respectively.  We  use  c  =  1.  The  acceptance  angle  0  is  found  from  0  = 
sin  (NA/r|),  where  rj  is  the  refractive  index. 


medium,  the  data  are  processed  using  a  penalty  on  positive  values. 
Finally,  for  proper  reconstruction,  the  intensity  data  should  be 
converted  to  phase  data  by  reversing  the  effect  of  the  nonlinear 
function  ,s(). 

The  result  of  the  reconstruction  is  a  map  of  RI,  or  optical  path 
length,  from  a  DIC  image,  which  is  similar  in  nature  to  that  pro¬ 
duced  by  a  transmitted  light  interference  microscope  (Inoue  and 
Spring,  1997,  p.  86;  Dunn,  1988).  The  technique  also  requires  only 
one  image  per  focal  plane,  compared  to  the  several  images  per 
focal  plane  that  are  required  by  the  rotational  diversity  technique 
(Preza  et  al.,  1998).  While  reconstructed  DIC  data  bear  a  superfi¬ 
cial  resemblance  to  darkfield  (DF)  images,  it’s  important  to 
remember  that  it  is  impossible  to  know  if  a  bright  feature  in  a  DF 
image  represents  a  structure  having  an  RI  that  is  higher  or  lower 
than  its  surroundings. 


ence.  The  function  s()  represents  the  transformation  of  the  phase 
differences  to  intensity  values  that  occurs  when  the  two  waves  are 
combined  at  the  image  sensor. 

The  deconvolution  process  uses  an  iterative  optimization  algo¬ 
rithm  that  is  complicated  by  several  factors.  First,  DIC  produces 
contrast  only  from  changes  in  RI  that  occur  in  the  shear  direction. 
As  a  result  the  differential  PSF  has  a  significant  null  spectral  com¬ 
ponent  in  the  direction  perpendicular  to  the  shear,  and  this  must  be 
accounted  for.  Otherwise,  streaking  artifacts  will  appear  parallel 
to  the  orientation  in  which  the  beams  are  separated  (Kam,  1998). 
Second,  one  cannot  assume  nonnegativity  in  the  RI  change,  rela¬ 
tive  to  the  background.  Instead,  negative  values  are  only  penal¬ 
ized.  This  approach  provides  a  soft  constraint  where  the  negative 
values  are,  in  a  sense,  minimized  but  not  prevented  altogether.  This 
is  a  reasonable  assumption  because,  typically,  biological  structures 
have  a  higher  RI  than  the  aqueous  embedding  medium.  In  cases 
where  the  specimen  refractive  index  is  less  than  the  embedding 


FIGURE  24.1 1 .  Illustration  of  the  region  wherein  the  widefield  3D  OTF  may 
be  non-zero.  The  3D  coordinates  u,  v,  and  w  are  the  frequencies  with  respect 
to  x,  y,  and  Z-  respectively.  The  radial  bandlimit  follows  the  formula:  BW„  = 
2  NA/X.  The  axial  bandlimit  follows  the  formula  BW.  =  (NA)2/(2q7.).  0  follows 
the  same  formula  as  shown  in  the  caption  of  Figure  24.10.  The  biconic  region 
is  the  “missing  cone.”  The  region  of  the  OTF  where  it  is  known  to  have  all 
zero  values,  Z,  is  the  union  of  this  missing  cone  with  the  region  outside  of  the 
cylinder. 
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B 


FIGURE  24.12.  (A)  Stereo-pair  of  a  TLB  stack.  (B)  After  deconvolution. 


^  X'  camera  plate 


DIFFERENT  APPROACHES 
3D 

The  MLE  algorithms,  both  blind  and  non-blind,  are  in  a  category 
of  algorithms  known  as  the  iterative  constrained  algorithms 
(Wallace  et  al.,  2002).  They  are  called  iterative  because  they  repeat 
the  same  steps  to  update  /*(),  as  shown  in  Figure  24.8.  They 
are  called  constrained  because  the  bandwidth  is  limited  and 
the  values  of  f(k\)  are  allowed  only  to  be  nonnegative.  Among 
other  approaches,  described  below,  these  iterative  constrained 
approaches  are  considered  to  be  the  most  quantitatively  accurate; 
that  is,  the  summation  of  the  intensity  in  all  the  pixels  of  the  image 
is  preserved.  The  total  number  of  photons  in  the  final  reconstruc¬ 
tion  is  the  same  as  was  collected  from  all  the  charged-coupled 
device  (CCD)  images.  As,  in  some  sense,  this  is  a  normalization, 
one  might  expect  that  any  deconvolution  approach  can  yield  a 
similar  result,  simply  by  following  the  deconvolution  with  a  nor¬ 
malization  operation.  However,  what  is  different  here  is  that  the 
normalization  is  inherent  within  the  deconvolution  operation  itself. 
It  is  an  intrinsic  fallout  of  the  MLE  process  and  not  something  that 
must  be  done  afterwards. 

One  reason  these  iterative  constrained  algorithms  are  more 
accurate  is  that  they  are  capable  of  recovering  Fourier  transform 
information,  such  as  the  missing  cone  information,  which  was 
removed  by  the  optics.  Details  about  the  missing  Fourier  compo¬ 
nents  and  how  they  are  recovered  may  be  found  in  Holmes  and 
Liu  (1989).  The  inverse  filter  and  nearest  neighbor  approaches 
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Table  24.1.  Processing  Times  and  Memory  Requirements  for 
Implementation  on  a  Windows  2000/NT/XP  3  GHz 
Computer 


10  Iterations,  Blind 

Image  Size 

Deconvolution 

Inverse  Filter 

256  x  256  x  10 

13s 

1.3s 

1024  x  1024  x  50 

20  min 

2  min 

Requirements  for  other  data-set  sizes  can  be  approximated.  Processing  time  will  be 
proportional  to  M  log  M  (. M  =  number  of  voxels)  which  is  the  proportionality  of  an 
FFT  (Oppenheim  and  Schafer,  1975).  Most  of  the  processing  time  is  taken  by  cal¬ 
culating  the  FFTs.  Extra  memory  is  not  needed  for  large  image  sizes  because  the 
image  may  be  decomposed  into  subregions  that  are  deconvolved  separately.  For 
future  computers  with  faster  chip  sets,  processing  speed  is  roughly  proportional  to 
the  Pentium  computer  chip  speed  (e.g.,  3  GHz). 


described  below  do  not  recover  this  information.  Other  iterative 
constrained  approaches  include  the  Jansson—van  Cittert  algorithm 
(Holmes  and  Liu,  1992b)  and  least  squares  (Carrington,  1990) 
algorithms.  Iterative-constrained  algorithms  are  generally  much 
slower  to  compute  than  the  inverse  filter  or  nearest-neighbor  algo¬ 
rithms.  However,  as  computer  speed  has  increased,  this  issue  has 
been  mitigated  significantly  in  recent  years  (Table  24.1).  Today, 
typical  iterative  deconvolutions  require  only  a  few  seconds  to  a 
few  minutes  on  a  personal  computer  (PC). 

A  variant  on  Gold’s  ratio  method  (Agard  et  al.,  1989)  is  another 
iterative-constrained  algorithm.  It  is  similar  to  the  algorithm  shown 
in  Figure  24.8,  except  that  the  convolution  of  h(—x,  —y,  —z)  is 
removed.  It  has  been  developed  by  Sibarita  and  colleagues  (2002) 
to  provide  high-throughput  of  hundreds  of  volumes  in  a  short  time. 
The  advantage  of  this  approach  is  that  of  speed,  while  some  noise 
reduction  is  sacrificed. 

The  nearest-neighbor  algorithm  is  the  fastest  algorithm.  It  is 
a  2D  algorithm  that  works  by  sharpening  each  image  in  the  stack 
one  at  a  time.  Mathematical  details  are  in  Holmes  (1992)  and 
Agard  and  colleagues  (1989).  The  two  adjacent  images  in  the  stack 
(above  and  below)  are  reblurred  by  convolving  them  with  an 
approximation  to  the  PSF.  These  two  reblurred  images  are  then 
multiplied  by  a  coefficient  (<1)  and  subtracted  from  the  image 
being  sharpened.  This  operation  is  known  generically  as  unsharp 
masking  (Russ,  1995).  This  algorithm  is  not  quantitative.  Image 
intensities  are  not  preserved.  A  summation  of  the  processed  image 
is  just  a  fraction  of  the  summation  of  the  original  image.  This  algo¬ 
rithm  is  really  an  image  enhancement  method,  although  it  is  some¬ 
times  controversially  called  a  deconvolution.  A  more  generally 


accepted  term  is  deblurring  algorithm  (Wallace  et  al.,  2002).  Its 
main  advantage  is  that  it  runs  in  a  fraction  of  a  second  on  a  PC. 
Figure  24.14  shows  a  nearest-neighbor  processed  image  of  the 
optical  section  shown  in  Figure  24.3. 

The  inverse  filter  is  another  fast  approach.  The  Fourier  trans¬ 
form  of  Eq.  1  is  expressed  as: 

Mp(u,  v,  w)  =  F(u ,  v,  w)H(u,  v,  w)  (8) 

where  FQ  and  H( )  are  the  Fourier  transforms  of /()  and  h(),  respec¬ 
tively.  Ideally,  /()  ought  to  be  recoverable  from  Mp()  by  taking  the 
inverse  Fourier  transform  of  the  following  operation: 

A(m,  v,  w)  =  Mp{u,  v,  w)H,(u,  v,  w),  (9) 

where 


Hr{u,v,w) 


1 

H(u,v,w) 


(10) 


In  reality,  FFu,  v,  w)  contains  zeros,  so  Eq.  10  is  mathe¬ 
matically  unstable  and  this  approach  does  not  work.  Instead,  a  reg¬ 
ularized  variant  of  Hr(u,  v,  w)  is  used,  according  to  the  expression: 


Hr(u,v,w) 


//*(«,  v,w) 
\H(u,v,w)\2  +k 


(11) 


where  the  *  superscript  denotes  complex  conjugate  and  the  term  k 
is  a  very  small  number  that  is  empirically  chosen.  Equation  11  is 
a  regularized  approximation  to  Eq.  10  that  is  based  on  the  princi¬ 
ples  of  the  Wiener  filter  (Castleman,  1979).  It  approximately 
equates  to  Eq.  10,  except  where  Fl(u,  v ,  w)  is  near  zero. 

The  inverse  filter  is  fast  to  execute  (Table  24.1),  yet  preserves 
reasonable  quantitative  accuracy.  Figure  24.15  shows  the  optical 
slice  shown  in  Figure  24.3  after  application  of  an  inverse  filter. 

The  inverse  filter  works  well  with  phase  objects  imaged  under 
a  TLB  microscope  with  a  small  condenser  aperture.  Iterative  con¬ 
strained  methods  will  not  work  well  to  deconvolve  such  an  image 
because  it  does  not  follow  the  mathematical  model  where  |i,,()  must 
be  nonnegative  and  constructive  interference  causes  areas  in  the 
specimen  to  be  brighter  than  the  background.  An  inverse  filter 
works  because  it  does  not  rely  on  p,p(j  being  nonnegative.  Figure 
24.16  shows  summed  intensity  projections  of  an  unprocessed  stack 
and  a  processed  stack. 


2D  Image  Filtering 

The  nearest-neighbor  algorithm  has  a  variant,  called  the  no¬ 
neighbors  algorithm,  which  operates  only  on  a  single  2D  plane  of 


FIGURE  24.14.  Optical  section  shown  in  Figure  24.3(A)  after  the 
nearest-neighbor  algorithm. 
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the  image  being  sharpened.  Instead  of  reblurring  adjacent  images, 
the  image  being  sharpened  is  reblurred,  multiplied  by  the  coeffi¬ 
cient,  and  subtracted  from  itself.  Because  this  no-neighbors  variant 
operates  on  2D  data  alone,  it  does  not  require  3D  data,  so  it  is  a 
much  simpler  technique  to  use  and  requires  inexpensive  equip¬ 
ment.  The  complete  system  shown  in  Figure  24.2  is  no  longer 
needed  to  take  advantage  of  2D  deblurring.  Any  microscope  with 
a  digital  camera  will  benefit. 

There  is  also  a  purely  2D  variant  of  the  MLE  blind  deconvo¬ 
lution  algorithm  (Biggs,  1998)  which  shares  some  of  these  2D 
advantages.  As  with  the  no-neighbors  algorithm,  any  microscope 
with  a  digital  camera  will  benefit.  This  2D  MLE  algorithm  works 
well  when  the  sample  is  inherently  2D,  as  with  flat  samples  that 
fit  within  the  depth  of  field,  with  objectives  having  a  large  depth 
of  field  or  with  total  internal  reflection  (TIRF)  microscopy.  It 
works  surprisingly  well  even  in  many  cases  where  the  sample  is 
inherently  3D,  as  long  as  high-NA  optics  are  used  to  reduce  depth 
of  field.  However,  in  these  inherently  3D  cases,  it  is  more  accurate 
to  think  of  this  algorithm  as  a  nonlinear  sharpening  filter  than  as 
a  deconvolution  because  the  algorithm  assumes  that  the  imaging 
process  is  2D  and  this  is  an  inaccurate  assumption. 

DATA  CORRECTIONS 

Images  recorded  using  common  intensified  and  standard  CCD 
digital  cameras  can  be  deconvolved.  Peltier-cooled  sensors  (-70° 
or  lower)  are  preferred  because  they  have  the  lowest  noise  level, 
the  best  linearity,  the  highest  dynamic  range  (16  bits),  and  the 
lowest  background  levels.  Image  data  collected  from  any  cooled- 
CCD  camera  must  be  calibrated  for  the  background  level  b(x,  y, 
z),  non-uniform  pixel  sensitivity,  |3(x,  y),  and  the  camera  exposure¬ 
time  instability  T(z)  such  that  the  raw  collected  data  \ir(x,  y,  z)  are 
composed  of: 

j XQc,  y,  z)  =  $(x,  z)7Xz)|iCT,  y,  z)  +  b(x,  y,  z).  (12) 

The  exposure  time  T(z)  varies  with  z  as  the  shutter  may 
fluctuate  by  several  milliseconds  between  frames.  When  non¬ 
mechanical  shutters  are  used,  the  exposure  time  does  not  vary,  but 
the  illumination  varies  because  arc  lamps  wander  and  power 
variations  affect  incandescent  lamps.  The  T(z)  term  accommodates 
these  variations.  The  correction  method  is  outlined  in  Cooper  and 
colleagues  (1993). 

The  corrected  data  set  £t()  is  an  estimate  of  p,().  For  TLB,  this 
estimate  is  further  processed  by  Eq.  5,  and  for  WFF  it  is  substituted 
directly  for  |4,,()  in  Figure  24.8. 


In  CLSM  fluorescent  data,  (3()  and  T(z)  are  irrelevant,  and  only 
b()  must  be  determined.  To  estimate  b,  a  background  region  of 
interest  is  chosen  and  the  average  background  level  is  calculated 
from  this  data.  Alternatively,  a  histogram  of  intensity  levels  is  cal¬ 
culated  on  the  image,  and  the  background  level  is  detected  auto¬ 
matically  by  identifying  the  peak  in  the  histogram  that  is  close  to 
zero. 


LIGHT  SOURCE  AND  OPTICS  ALIGNMENT 

Deconvolution  is  forgiving  about  microscope  misalignment,  but 
only  to  a  limit.  Misalignment  causes  degradation  in  the  PSF.  When 
the  optics  are  misaligned,  the  PSF  is  less  symmetric,  and  deviates 
from  ideal  theory  in  other  ways.  The  non-blind  algorithms  and  the 
inverse  filter  algorithm  that  use  a  PSF  measurement,  adapt  to  this 
assymetry,  to  a  degree,  because  the  assymetry  occurs  in  both  the 
measured  and  actual  PSF.  The  blind  deconvolution  will  adapt  to 
misalignments  too,  much  in  the  way  that  it  adapts  to  spherical  aber¬ 
ration  as  shown  in  Figure  24.6(C). 

On  the  other  hand,  deconvolution  follows  the  proverbial 
“garbage-in  garbage-out”  principle.  Circular  averaging  is  often 
performed  to  reduce  the  noisiness  of  the  measured  PSF  and  this 
causes  the  calibrated  PSF  to  deviate  from  the  true,  assymetric  PSF. 
The  best  images  are  obtained  with  proper  Kohler  alignment  (see 
Chapter  36,  this  volume).  For  non-blind  deconvolution,  this  limit 
is  partly  because  the  PSF  measured  with  a  fluorescent  bead  is  not 
the  same  PSF  that  exists  in  the  biological  sample.  As  the  sample 
itself  is  a  heterogeneous  refracting  medium  that  is  part  of  the 
optics,  when  the  sample  changes,  the  PSF  changes.  Although  blind 
deconvolution  adapts  to  PSF  changes,  the  closer  the  true  PSF  is  to 
the  theoretical  PSF,  the  better  the  results. 

Another  consideration  in  obtaining  a  well-behaving  PSF, 
which  works  well  for  immersion  lenses,  is  to  adjust  the  refractive 
index  of  the  medium  that  is  between  the  objective  lens  and  the  cov- 
erslip.  The  technique  described  in  Hiraoka  and  colleagues  (1990) 
uses  finely  adjusted  immersion  media  to  minimize  the  spherical 
aberration  (see  Chapter  20,  this  volume).  Oils  having  a  different 
RI  are  placed  onto  and  removed  from  the  coverslip,  one-by-one, 
with  RI  differences  of  0.002.  The  refractive  index  that  provides  the 
most  symmetrical  PSF  along  the  z  axis  is  chosen.  It  is  straight¬ 
forward  to  see  the  symmetry  by  focusing  up  and  down  on  a  bead 
and  noticing  if  the  out-of-focus  bead  seen  while  focusing  up  looks 
the  same  as  the  out-of-focus  bead  seen  while  focusing  down  (see 
also  Fig.  20.3,  this  volume).  It  is  necessary  to  match  the  oil  in  this 
same  way  with  the  specimen.  A  fine  structure  needs  to  be  identi- 


FIGURE  24.1 5.  Optical  section  shown  in  Figure  24.3(A)  after  the 
inverse  filter. 
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FIGURE  24.1 6.  Summed  projections  of  a  phase  object  obtained  by  TLB.  (A) 
Unprocessed;  (B)  deconvolved.  (Image  courtesy  of  G.  Eddlestone,  Empire 
Imaging  Systems.) 


fled  in  the  sample,  such  as  a  microtubule,  or  anything  that  is  small, 
and  then  focusing  up  and  down  while  noticing  if  the  out-of-focus 
object  looks  the  same  while  focusing  up  compared  to  focusing 
down.  This  procedure  minimizes  the  spherical  aberration,  which  is 
the  most  severe  aberration.  Alternatively,  the  microscope  manu¬ 
facturers  produce  special  lenses  with  adjustable  “correction 
collars”  that  affect  a  correction  for  spherical  aberration. 

In  polarizing  and  TLB  imaging,  non-uniformity  in  the  light 
source  can  also  cause  problems  (see  Chapter  6,  this  volume).  The 
theoretical  PSF  is  cylindrically  symmetric  because  it  is  formulated 
under  the  assumption  that  light  enters  the  sample  with  equal  inten¬ 
sity  at  every  angle  within  the  numerical  aperture.  In  fact,  light 
enters  the  sample  at  different  intensities  depending  upon  the  angle 
because  the  lamp  is  not  uniform.  A  point  in  the  plane  of  the  lamp 
corresponds  to  an  angle  of  parallel  light  rays  in  the  specimen.  The 
intensity  at  the  center  of  the  lamp  is  proportional  to  the  intensity 
of  parallel  rays  passing  through  the  sample  at  zero  angle.  The 
intensity  at  a  distance  from  the  center  of  the  lamp  enters  the  sample 
at  an  angle  that  is  proportional  to  this  distance.  A  mercury-arc  lamp 
is  brightest  within  its  arc  of  light,  so  angles  that  correspond  to 
points  in  this  arc  have  the  brightest  intensity  impinging  on  the 
sample.  As  a  result  the  PSF  is  not  circularly  symmetric.  Although 
deconvolution  is  forgiving  of  an  asymmetric  PSF,  it  still  works  best 
with  a  symmetric  one.  Ellis  designed  an  illuminator  using  a  vibrat¬ 
ing  fiber-optic  that  provides  a  uniform  light  source  and  thereby 


provides  a  symmetic  PSF  (Ellis,  1979;  Inoue  and  Spring,  1997,  p. 
160).  These  non-uniform  lamp  effects  do  not  exist  with  the  WFF 
microscope.  In  this  case,  the  symmetric  PSF  relies  on  having 
spatially  uniform  excitation  light,  which  is  guaranteed  by  Kohler 
illumination,  and  on  having  well-aligned  emission  optics.  The 
WFF  PSF  does  not  rely  on  having  uniform  intensity  with  respect 
to  the  angle  of  illumination. 

The  partial  confocal  effect  also  causes  PSF  assymetry  (Hiraoka 
et  al.,  1990).  The  field  diaphragm  behaves  like  a  confocal  detec¬ 
tor  aperture  of  sorts  because  it  is  conjugate  to  the  specimen  plane. 
As  it  is  stopped  down,  it  causes  the  WFF  PSF  to  look  ever  more 
confocal-like  and  the  intensity  of  the  PSF  along  the  optic  axis 
decays  more  quickly  than  WFF  theory  predicts.  Furthermore,  this 
decay  is  assymetric  and  makes  it  more  difficult  to  match  the  mea¬ 
sured  PSF  (with  the  bead  present)  to  the  true  PSF  (with  the  spec¬ 
imen  present).  This  effect  makes  it  more  difficult  to  match  the 
assumptions  that  are  used  in  blind  deconvolution.  Therefore,  it  is 
best,  for  deconvolution,  to  keep  the  field  aperture  wide  open.  The 
partial  confocal  effect  still  exists,  but  it  is  minimal. 

NEWEST  DEVELOPMENTS 
Subpixel 

Images  may  be  deconvolved  to  produce  results  that  have  a  pixel 
resolution  different  from  the  original  data  (Carrington  et  al.,  1995; 
Conchello  and  McNally,  1997;  Wang,  2002).  The  idea  is  illustrated 
in  Figure  24.17.  Unobserved  pixels  are  inserted  into  the  mathe¬ 
matical  model  of  the  image.  They  are  recovered  by  treating  them 
as  “unknown  data”  regions  as  outlined  in  Holmes  (1989). 

Figure  24.18(A)  shows  an  image  slice  from  a  data  stack 
(Wang,  2002).  This  is  an  image  of  a  centrosome  labeled  with  green 
fluorescent  protein  (GFP)-gamma-tubulin.  The  length  of  the  cen¬ 
trosome,  along  its  long  axis,  is  ~l|im.  Figure  24.18(B)  shows  a 
deconvolved  result  made  by  increasing  the  pixel  resolution  by  a 
factor  of  3  along  x  and  y.  A  verification  of  the  structure  is  shown 
in  Figure  24.18(C).  This  figure  shows  a  TEM  image  of  the  same 
sample  taken  afterwards  and  verifies  the  recovered  structure. 

Khodjakov  and  colleagues  (2002)  discovered  that  these 
centrosomes  are  sometimes  formed  de  novo ,  without  association 
with  a  pre-existing  organelle.  Deconvolution  was  used  to  observe 
changes  in  the  shape  of  the  pericentriolar  material  cloud  in  living 
samples.  Only  TEM  can  record  such  shapes  directly,  but  not  in 
living  cells. 


Pixels  that  were  collected  by  the  camera  and  are  part  of  the  set 
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FIGURE  24.1 8.  A  subpixel  deconvolution.  (A)  Optical  section  taken  from  a  stack.  The  sample  is  a  centrosome  labeled  with  GFP-gamma-tubulin.  It  is  just  over 
1  pm  long.  (B)  Subpixel  deconvolution.  The  pixel  size  was  reduced  by  3  in  both  directions.  (C)  EM  micrograph  of  the  same  sample,  verifying  the  structure 
shown  in  (B).  (Image  courtesy  of  A.  Khodjakov,  Wadsworth  Laboratories.) 


Polarized  Light 

It  is  possible  to  deconvolve  3D  optical  section  data  of  birefringent 
samples  made  using  a  polarized  light  microscope.  Strictly  speak¬ 
ing,  the  image  formation  model  for  a  polarized-light  microscope 
is  nonlinear  (Hansen  and  Conchello,  1988),  so  from  a  pure  stand¬ 
point.  a  PSF  model  such  as  the  one  used  for  WFF  ought  not  to 
apply.  However,  from  a  practical  standpoint,  it  is  common  engi¬ 
neering  practice  (Close  and  Frederick,  1993)  to  introduce  a  linear 


approximation  to  make  any  nonlinear  model  tractable.  From  this 
perspective,  Eqs.  1  to  3  may  apply,  and  can  be  considered  to  be  a 
linearized  approximation  for  the  polarized-light  microscope. 
Furthermore,  optically  sectioned  images  of  birefringent  samples 
sometimes  appear  similar  to  WFF  samples  as  is  the  case  shown  in 
Figure  24.19(A),  which  shows  a  maximum-value  projection 
stereo-pair  of  the  raw  data.  Figure  24.19(B)  shows  the  stereo-pair 
after  deconvolution.  This  deconvolution  should  work  as  well  with 
the  Pol-Scope  developed  by  Oldenbourg  and  Mei  (1995),  which  is 


FIGURE  24.1 9.  (A)  Stereo-pair  of  a  polarized  light 
stack.  This  stack  was  acquired  from  an  ice-cream 
sample  as  part  of  a  study  of  food  production  quality. 
The  object  is  a  fat  crystal.  (B )  After  blind  deconvo¬ 
lution.  (Image  courtesy  of  K.  Baker,  University  of 
Guelph.) 
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FIGURE  24.20.  Sections  from  an  SLO  in  the  reflected  light  mode.  The  optic 
nerve  region  is  shown  with  large  vasculature  (1),  capillaries  (2),  the  lamina 
cibrosa  (3),  and  the  scleral  ring  (4).  These  structures  are  clearer  after  decon¬ 
volution.  Structures  are  isolated  better  to  their  proper  location  along  z,  as  seen 
by  the  improved  discrimination  between  the  slices.  Before  deconvolution,  the 
scleral  ring  and  lamina  cibrosa  appear  in  most  slices.  After  deconvolution,  they 
are  properly  isolated  to  their  appropriate  locations.  (Data  collected  using  the 
Heidelberg  Retinal  Tomograph  (HRT),  Heidelberg  Engineering,  Heidelberg, 
Germany.  Image  courtesy  of  D.  Garway-Heath,  Moorfields  Eye  Hospital.) 

a  special  type  of  polarized  light  microscope  designed  for  sensitiv¬ 
ity  and  quantitative  accuracy.  It  yields  an  image  in  which  the  inten¬ 
sity  is,  to  a  first  approximation,  proportional  to  the  magnitude  of 
the  birefringence.  Strictly  speaking,  there  will  be  a  positive,  non¬ 
linear  dependence  on  the  scalar  birefringence. 


Live  Imaging 

Two  important  issues  arise  with  live  imaging:  (1)  when  tissue  is 
monitored  over  time,  many  deconvolutions  (sometimes  hundreds) 
must  be  performed.  Such  applications  benefit  from  processing 
times  on  the  order  of  seconds.  (2)  The  sample  moves  during  the 
image  acquisition.  The  first  issue  is  addressed  in  “Future 
Directions.” 

An  example  of  the  second  issue  is  shown  in  Figure  24.20, 
which  shows  confocal  sections  from  a  scanning  laser  ophthalmo¬ 
scope  (SLO;  Bartsch  and  Freeman,  1994;  shown  schematically  in 
Fig.  24.21).  Reflected  light  confocal  sections  were  acquired  during 
an  outpatient  procedure.  Although  the  motion  of  the  eye  is  miti¬ 
gated  by  having  the  patient  fixate  on  a  blinking  light,  the  eye  still 
moves  between  frames  and  during  the  1  /20th  of  a  second  that  each 
frame  is  acquired.  There  is  an  x,y  translation  and  a  rotation  as  well 
as  geometric  distortion  between  frames  (O’Connor  et  al.,  1998). 
The  two  lines  overlaid  on  Figure  24.20(A)  show  that  there  is 
motion  of  the  blood  vessel  shown  by  the  solid  arrow  and,  as  a 
result,  there  is  geometric  distortion  between  frames. 

The  alignment,  rotation,  and  geometric  distortion  are  corrected 
by  preprocessing  the  optical  sections  with  a  software  motion- 
detection  algorithm  that  is  based  on  a  mathematical  cross¬ 
correlation.  Figure  24.20(B)  shows  the  deconvolved  optical 
sections.  The  algorithm  described  in  the  section  “Algorithms  — 
WFF  and  CLSM  Fluorescence”  was  used.  The  geometric  distor¬ 
tion  is  removed.  Improved  optical  sections  and  better  discrimina¬ 
tion  between  slices  are  produced.  The  scleral  ring,  and  other  known 
anatomical  structures  are  better  isolated  to  a  slice,  while  blood 
vessels  and  capillaries  are  clearer.  Figure  24.22  shows  a  stereo-pair 
of  the  deconvolved  stack. 


MORE  EXAMPLES 

Blind  Deconvolution  and  Spherical  Aberration 

An  example  of  a  blind  deconvolution  was  presented  earlier  in  the 
section  entitled  “Advantages  and  Limitations.”  The  first  guess  of 
the  PSF  [Fig.  24.6(B)],  used  to  seed  the  first  iteration  of  the  blind 
deconvolution  of  Step  2  in  Figure  24.8,  was  calculated  as  the  auto¬ 
correlation  of  the  raw  data.  The  autocorrelation  operation  may  be 
found  in  the  book  by  Gaskill  (1978).  It  is,  in  essence  (although  not 
precisely),  the  convolution  (see  the  Appendix  for  this  chapter)  of 
the  PSF  with  itself.  More  precisely,  it  is  expressed  according  to 

hi0\x,  y,  z)  =  N(x,  y,  z)  *  N(-x,  -y,  -z),  (13) 

where  /t(0'()  is  the  first  guess  of  the  PSF. 


FIGURE  24.21.  Schematic  of  the  SLO. 
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FIGURE  24.22.  Stereo-pair  of  the  optic  nerve 
data  set  shown  in  Figure  24.20. 


The  PSF  that  resulted  after  running  the  blind  deconvolution 
starting  from  this  first  guess  is  shown  in  Figure  24.6(C).  Note  that 
it  contains  the  same  flare  seen  in  the  raw  data  of  Figure  24.6(A). 
For  comparison,  Figure  24.6(D)  shows  a  theoretically  calculated 
PSF  with  spherical  aberration  that  matches  the  flare  seen  in  Figure 
24.6(A).  Figure  24.6(E)  shows  a  theoretically  calculated  PSF 
without  spherical  aberration.  From  these  comparisons  it  is  obvious 
that  the  predominant  flare  character  seen  in  raw  data  of  Figure 
24.6(A)  and  in  the  recovered  PSF  of  Figure  24.6(C)  is  due  to  spher¬ 
ical  aberration. 

Widefield  Fluorescence  Simulation 

Simulated  deconvolutions  have  been  carried  out  to  guide  devel¬ 
opment  of  deconvolution  programs.  Though  not  exhaustive,  these 
simulations  are  instructive  because,  unlike  a  biological  specimen, 
the  original  “test  object”  is  known.  Simulations  provide  insight 
into  the  advantages  and  limitations  of  different  deconvolution 
approaches.  Figures  24.23  and  24.24  show  a  WFF  simulation. 

A  computer  program  created  the  object  shown  in  Figures 
24.23  and  24.24(A).  The  theoretical  widefield  PSF,  like  the  one 
shown  in  Figure  24.6(E),  was  created  by  a  computer  program 
that  used  the  diffraction  theory  as  explained  by  Streibl  (1984).  This 
3D  PSF  was  convolved  with  the  3D  object  using  Fourier  trans¬ 
forms  (Gaskill,  1978),  and  the  Fast-Fourier-Transform  algorithm 
(Oppenheim  and  Schafer,  1975).  Once  the  diffraction-limited  3D 
data  set  was  created  in  this  way,  quantum-photon  noise  was 
simulated  by  software  written  to  create  random  numbers  that 
follow  Poisson  statistics  (Mortensen,  1987). 

Figure  24.23  shows  a  schematic  of  the  simulated  test  object 
(Markham  and  Conchello,  1998).  It  has  the  shape  of  wagon  wheel 
pasta.  When  the  out-of-focus  haze  is  removed  by  the  deconvolu¬ 
tion,  the  spokes  on  the  wagon  wheel  and  the  surface  of  the  wheel 
are  restored,  except  for  those  parts  that  are  horizontally  planar.  The 
inability  of  deconvolution  to  restore  horizontal  planes  is  a  well- 
known  limitation  of  widefield  deconvolution  and  is  caused  by  the 
missing  cone  region  (Holmes,  1992;  Holmes  et  al.,  1995;  Streibl, 
1984)  (Fig.  24.11)  which  removes  all  of  the  information  necessary 
to  restore  a  horizontal  plane.  Fortunately,  very  few  biological 
structures  have  perfectly  horizontal  planar  components  labeled  by 


the  dye.  Those  that  do  are  better  imaged  with  a  confocal  micro¬ 
scope.  Note  that  the  same  horizontal  spokes  that  are  not  restored 
in  Figure  24.24(C)  are  restored  in  Figure  24.24(D),  where  the 
horizontal  parts  are  at  the  edge  of  the  cylinder,  so  they  are  not 
planar.  Here,  the  necessary  Fourier  components  needed  to  restore 
the  horizontal  parts  reside  outside  of  the  missing  cone,  and  there¬ 
fore  the  deconvolution  algorithm  has  sufficient  information  to 
recover  them. 

Spinning-Disk  Confocal 

Figure  24.25  shows  a  deconvolved  data  set  from  a  spinning-disk 
confocal  microscope.  An  advantage  of  the  blind  deconvolution  is 
that  it  is  capable  of  adapting  to  many  modalities  because  the  PSF 
does  not  need  to  be  known  perfectly. 

Two  Photon 

Figure  24.26  shows  optical  sections  made  with  a  two-photon 
microscope,  before  and  after  deconvolution.  The  algorithm  used  is 
identical  to  that  used  with  the  confocal  microscope,  because  the 
PSF  of  the  two-photon  microscope  is  similar  to  that  of  the  confo¬ 
cal  microscope.  The  signal-to-noise  level  and  optical  sectioning 
discrimination  are  improved  by  the  deconvolution.  Although  not 
clear  from  this  example,  the  in-plane  resolving  power  may  also  be 
improved  (see  also  Chapter  25,  this  volume). 


position  of 
cross  section 
of  Figure  24.24C 

position  of 
cross  section 
of  Figure  24.24D 


FIGURE  24.23.  Schematic  of  the  simulated  test  object. 
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FIGURE  24.24.  (A)  yz  slice  of  the  test  object  (Fig.  24.23).  (B)  yz  slice  of  the  simulated  WFF  data.  (C)  Deconvolution  of  the  central  cross-section.  (D)  Decon¬ 
volution  of  the  cross-section  taken  at  the  end  of  the  cylinder. 


SPEED 

Prior  speed  limitations  (Holmes  et  al.,  1995)  are  now  less  of  a 
problem.  Twelve  years  ago  deconvolutions  were  performed  on  a 
mainframe  supercomputer  and  now  they  are  performed  on  a  PC. 
Numbers  of  iterations  used  to  be  in  the  100s  and  now  they  are  on 


the  order  of  10.  Deconvolutions  are  performed  in  seconds  rather 
than  hours.  Even  so,  there  will  always  be  demand  for  higher  speed. 
The  operator  needs  fast  visual  feedback  during  four-dimensional 
(4D)  live  imaging,  especially  now  that  the  electron  multiplier  CCD 
(EM-CCD)  makes  it  possible  to  see  faint  signals  at  high  frame  rates 
with  little  readout  noise.  Current  deconvolution  speeds  are  listed 
in  Table  24.1,  see  p.  476. 


FIGURE  24.25.  (A)  Orthogonal  maximum  projections  of  a  fluorescent  sample  taken  from  a  Yokogawa  spinning-disk  microscope  (Ichihara  et  al.,  1996).  (Image 
courtesy  of  Baggi  Somusundaram,  PerkinElmer  Life  Sciences.)  (B)  Deconvolved  result. 
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FIGURE  24.26.  Sections  of  a  two-photon  data  set.  (A-C)  Unprocessed.  (D-F)  Deconvolved.  (Image  courtesy  of  M.  Ellisman,  University  of  California,  San 
Diego.) 


FUTURE  DIRECTIONS 

Parametric  modeling  of  more  aberrations  for  the  calculation  of  the 
PSF  during  the  blind  deconvolution  by  Markham  and  Conchello, 
(1999)  will  make  the  deconvolution  more  robust  against  aberra¬ 
tions.  These  aberrations  include  astigmatism,  coma,  and  various 
types  of  chromatic  aberration. 

The  blind  deconvolution  approach  provides  many  efficient 
ways  to  utilize  the  reconstructed  PSF  obtained  from  one  deconvo¬ 
lution  to  help  make  the  next  deconvolution  converge  more  rapidly. 
One  way  is  to  use  the  PSF  arrived  at  from  a  given  data  set  as  the 
first-guess  PSF  when  processing  other  data  collected  under  similar 
conditions.  Another  way  is  to  use  the  PSF  that  was  arrived  at  from 
a  given  subfield  (or  sub-volume)  in  a  data  set  as  a  first  guess  when 
deconvolving  a  neighboring  subfield.  The  actual  PSF  varies  across 
a  large  field  because  the  wavefront  aberrations  are  dependent  upon 
the  x-,  y-,  and  ^-locations  (see  Fig.  23.5,  this  volume).  Presently, 
blind  deconvolution  adapts  to  these  positional  changes  by  break¬ 
ing  up  the  data  set  into  smaller  sub-volumes  and  reconstructing  an 
independent/adaptive  PSF  for  each  sub-volume. 

Although,  in  a  strict  sense,  every  pixel  ought  to  have  its  own 
PSF,  the  amount  of  computation  necessary  makes  doing  this 
extremely  impractical.  Having  subfields  with  constant  PSFs  within 
each  field  is  a  compromise  that  works  because,  even  though  the 
PSF  varies,  it  varies  slowly  across  the  field. 

Because  camera  sizes  and  the  number  of  channels  will 
increase,  and  live,  time-series  imaging  will  become  ever  more 
common,  processing  speed  improvements  will  always  be  a  require¬ 
ment  of  advancing  work.  Now  that  we  can  achieve  deconvolutions 
in  seconds  instead  of  minutes,  we  would  like  to  have  them 
done  in  milliseconds.  High-throughput  pharmeceutical  screening 
requires  deconvolving  large  3D  data  sets  in  real  time  (see  Chapter 
46,  this  volume).  To  measure  ion  concentrations  in  3D  and  in  real 
time  requires  millisecond  deconvolutions. 

There  are  two  main  approaches  to  meet  these  speed  demands. 
The  algorithmic  approach  involves  refining  the  mathematical 
models  and  designing  trade-offs  so  that  the  algorithm  converges 
mathematically  at  a  faster  rate.  A  second  approach  is  to  utilize 
distributed  processing  and  distributed  computing.  Distributed 


processing  is  a  term  used  to  describe  the  decomposition  of  com¬ 
putations  into  independent  modules  and  sending  each  of  these 
modules  to  a  different  CPU  within  the  same  computer.  The 
Windows  platform  uses  a  programming  technique  called  multi¬ 
threading,  which  provides  the  tools  that  allow  programmers  to 
carry  out  distributed  processing  in  the  Windows  environment. 
Many  of  the  Apple  and  Unix-based  computers  also  provide  dis¬ 
tributed  processing  capabilities.  In  distributed  computing,  which  is 
different  from  distributed  processing,  these  self-contained  parts  are 
sent  over  the  local  network  to  different  computers  (rather  than  to 
different  CPUs  on  the  same  computer)  and  then  returned  to  the 
host  computer  after  the  tasks  are  completed.  Deconvolution  pro¬ 
cedures  can  easily  be  decomposed  by  breaking  up  the  data  set  into 
subvolumes  (Bhattacharyya  et  al.,  1996). 

SUMMARY  OF  MAIN  POINTS 

•  The  MLE  blind  deconvolution  approach  to  deconvolution  has 
a  number  of  important  advantages  —  no  measurement  of  PSF 
is  needed  and  inherent  noise  suppression. 

•  There  are  many  other  types  of  deconvolution,  each  one  having 
different  speed  and  performance  advantages.  Generally,  speed 
is  traded  for  image  quality  (resolution  and/or  noise). 

•  Deconvolution  algorithms  exist  for  many  microscopic  contrast 
modes:  WFF,  CLSM  fluorescence,  SLO,  spinning-disk  confo- 
cal,  TLB,  DIC,  polarized  light,  two-photon. 

•  Although  past  limitations  in  speed  have  been  surpassed,  the 
need  to  process  data  from  larger  cameras  more  rapidly  means 
that  the  processor  is  still  a  limit.  Distributed  processing  and 
algorithm  improvements  will  meet  this  increasing  demand. 
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APPENDIX 

Convolution:  An  Introduction  For  Those  Who 
Don't  Remember  Second-Year  Calculus 

James  B.  Pawley  and  Timothy  J.  Holmes 

The  word  “deconvolution”  has  achieved  a  mantra-like  status 
among  many  light  microscopists.  It  is  often  translated  as  the  magic 
process  that  starts  with  blurry,  hazy  3D  data  and  produces  sharp, 
contrasty  3D  structures.  While  this  is  a  powerful  capability,  and,  as 
is  stated  throughout  this  volume,  3D  data  sets  are  almost  always 
improved  by  being  deconvolved,  it  is  important  to  recognize  that 
deconvolution  is  just  the  reverse  of  a  mathematical  operation  called 
convolution.  In  the  context  of  3D  light  microscopy,  convolution  can 
be  thought  of  as  a  function  that  mimics  the  effect  of  the  blurring 
caused  by  diffraction  and  the  addition  of  out-of-focus  light. 

The  purpose  of  this  Appendix  is  to  demystify  deconvolution 
by  explaining  (albeit  in  a  simplified,  heuristic  way)  the  essential 
features  of  the  mathematical  process  called  convolution. 

Let  us  start  with  the  mathematical  process  of  multiplication. 
We  all  understand  multiplication:  You  take  one  number  (a  con¬ 
stant)  and  add  it  to  itself  as  many  times  as  the  values  of  some 
second  number.  Integration  is  similar,  except  that  one  of  the  things 
you  are  “multiplying”  is  a  variable,  usually  called  the  function  and 
plotted  in  the  y-direction.  The  integral  of  a  function  is  the  area 
under  the  plot  of  the  function  between  some  upper  and  lower 
“limits”  along  the  x  axis.  So  if  you  imagine  dividing  this  space  up 
into  vertical  strips,  all  of  the  same  width  (the  constant),  but  of  vari¬ 
able  length  (where  the  length  is  the  value  of  the  function  for  this 
value  of  x),  then  the  total  area  can  be  thought  of  as  multiplying  a 
constant  by  a  variable. 

Convolution  takes  this  process  a  step  further.  It  is  a  special  type 
of  integration  in  which  one  multiplies  one  function  (or  variable) 
by  another  function  (or  variable),  and  to  make  things  even  more 
fun,  the  function  can  be  a  variable  in  more  than  one  dimension 
(i.e.,  in  x,  y,  and  rather  than  just  x  as  above). 

I  can  hear  your  eyes  glazing  over! 

Before  you  panic  about  functions,  you  don’t  have  to  think 
about  equations  here.  Any  set  of  paired  numbers  will  do.  For 
example,  the  pairing  of  a  set  of  intensity  numbers  with  locations 
in  an  xy  raster  is  a  function.  In  other  words,  a  digital  image  is  a 
function  of  x  and  y. 

Perhaps  some  pictures  can  help  show  how  this  works,  at  least 
for  a  2D  function  (i.e.,  a  sort  of  image)  as  shown  in  Figure  24. Al. 

We  will  start  with  two  very  simple  2D  functions: 


•  A  point  object  (shown  in  green):  all  the  pixels  equal  0  except 
the  one  in  the  middle,  which  equals  1 . 

•  A  simple  blurring  function  (shown  in  red):  a  2D  blob  about 
the  shape  of  the  central  peak  of  an  Airy  disk,  sampled  with 
Nyquist-sized  pixels. 

To  convolve  the  red  function  with  the  green  one,  first  imagine 
setting  the  center  pixel  of  the  red  function  over  the  pixel  at  the  top- 
left  corner  of  the  green  one  and  then  multiplying  the  value  in  each 
red  pixel  by  the  value  in  the  green  pixel  that  it  overlaps.  Then  add 
up  all  these  products  and  enter  the  sum  in  the  top  left  pixel  of  a 
new  “result”  raster  (in  black).  As  initially  all  the  multiplications 
involve  a  zero,  all  the  products  are  zero  and  so  is  their  sum,  so  you 
enter  a  zero  in  the  top-left  pixel  (long  arrow).  Now  step  the  red 
mask  one  step  to  the  right  and  repeat.  Another  zero  (short  arrow). 
Eventually,  however,  as  you  proceed  to  step  the  red  mask  across 
and  down,  a  time  will  come  when  the  non-zero  pixel  in  the  green 
mask  will  coincide  with  a  non-zero  pixel  in  the  red  mask  and  a 
non-zero  sum  of  products  will  be  entered  in  the  black  raster  (long, 
pink  arrow.  Fig.  24.A2). 

It  now  becomes  obvious  that  this  slow,  boring  process  will 
make  the  result  look  just  like  the  red  function.  In  fact,  we  have  just 
mimicked  the  process  of  making  a  blurry  image  of  a  point  object 
and  our  black  result  is,  therefore,  the  infamous  point-spread  func¬ 
tion,  or  PSF.  To  see  that  this  process  is  a  little  more  useful  than 
just  copying  the  red  function,  suppose  now  that  the  object  is  com¬ 
posed  of  two  point  objects,  separated  by  five  “zero  pixels,”  as 
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FIGURE  24. Al .  First  two  steps  of  convolution  operation  between  a  point  and 
a  mask. 
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FIGURE  24. A2.  Convolution  of  a  point  with  a  mask,  continued. 


shown  in  the  green  raster  of  Figure  24. A3.  When  this  object  is  con¬ 
volved  with  the  mask,  the  result  (black  raster)  resembles  a  blurred 
image  of  two  points. 

To  the  extent  that  the  mask  mimics  a  2D,  diffraction-limited, 
point-spread  function,  the  result  in  the  black  raster  will  look  like 
a  diffraction-limited  image  of  an  object  composed  of  two  dark 
points  on  a  white  background.  As  we  have  chosen  a  pixel  size  close 
to  half  of  the  Rayleigh  resolution,  two  points  separated  by  six 
pixels  will  be  represented  by  two  discernible  features  in  the  black 
raster.  You  may  be  more  convinced  of  this  if  you  look  at  Figure 
24.A3  from  the  other  side  of  the  room. 

The  discussion  so  far  has  implied  that  convolution  is  a  process 
that  occurs  in  just  two  dimensions.  Of  course,  in  the  case  of 
microscopy,  this  is  not  true.  Because  the  Airy  figure  is  a  3D  struc¬ 
ture,  the  blurring  it  causes  occurs  in  three  dimensions.  The  3D 
blurring  in  the  near-focus  region  is  difficult  to  diagram  in  2D. 
However,  as  one  moves  away  from  this  plane,  it  is  a  fair  approx¬ 
imation  to  assume  that  the  light  passing  through  the  objective 
appears  as  a  circle.  This  circle  is  the  cross-section  of  a  cone 
centered  at  the  plane  of  the  object.  The  point  object  defines  the 
apex  of  the  cone  and  the  objective  NA  defines  its  angle  (Fig. 
24.A4). 

Three-dimensional  convolution  requires  considering  the  out- 
of-focus  light  in  the  vertical  direction  as  well  as  blurred  light  from 
the  in-focus  plane.  The  problem  becomes  more  complex  when  two 
points  are  located  at  different  heights  in  the  specimen  (see  Fig. 
24.A5). 
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FIGURE  24. A3.  Convolution  of  a  mask  and  a  two-point  object. 
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FIGURE  24. A4.  Three-dimensional  blurring  function  applied  to  a  point  object 
in  widefield  fluorescence. 


It  is  important  to  remember  that  the  total  number  of  photons 
leaving  the  point  and  passing  through  the  objective  remains  virtu¬ 
ally  constant  no  matter  where  the  objective  is  focused.  Therefore, 
the  image  of  an  in-focus  point  appears  very  intense,  while  the 
image  of  an  out-of-focus  point  is  much  dimmer  because  now  the 
same  number  of  photons  are  being  spread  over  a  larger,  blurred 
circle.  It  can  be  instructive  to  see  just  how  important  this  effect 
can  be. 

About  Out-of-Focus  Light 

Suppose  that  under  given  widefield  illumination  conditions,  a  CCD 
with  Nyquist-sized  pixels  (0.1  |dm  referred  to  the  sample)  records 
a  total  of  10,000  photons  from  a  particular  point  object  in  each 
image  in  the  stack.  When  the  point  object  is  located  at  the  focus 
plane,  as  illustrated  in  Figure  24. A6,  about  80%  of  the  10,000 
photons  are  distributed  between  the  12  bright  pixels  centered  on 
the  object  or  about  650  photons/pixel.  Assuming  a  high-NA  lens 


to  objective 


images  recorded  from  each  focus  plane. 


FIGURE  24.A5.  Three-dimensional  blurring  function  applied  to  two  point 
objects  in  different  planes  in  widefield  fluorescence. 
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Area  of  focused 
spot  -0.126  pm2 
or  about  1 2  pixels 


If  diam.  of  Airy  dark  ring  is  0.4  pm, 
CCD  pixels  should  be  0.1  pm  square, 
Pixel  area  =  0.01  pm2. 


plane 


Just  1  pm  above  the  focus 
plane,  the  beam  area  is 
9.4  pm2  or  about  940  pixels. 
As  this  is  ~80x  more  pixels 
than  at  the  focus  plane,  the 
light  is  ~80x  less  intense. 


FIGURE  24. A6.  Geometrical  optics  used  to  calculate  the  rapidity  with  which 
the  signal  from  a  point  object  diminishes  with  distance  from  the  focus  plane  in 
widefield  microscopy. 


(a  =  60°)  and  simple  geometrical  optics,2  by  the  time  the  feature 
is  only  1  pm  out  of  focus,  these  photons  will  be  spread  over  a  circle 
having  an  area  of  9.4  pm2  or  940  pixels.  If  the  intensity  is  spread 
evenly  over  this  circle,  then  the  detected  signal  level  from  the 
now  out-of-focus  feature  drops  to  <7  photons/pixel.  As  this  is  only 
slightly  more  than  the  RMS  noise  level  of  a  good,  slow-scan  CCD, 
it  is  hard  to  see  how  the  detection  of  this  out-of-focus  light  can 
provide  much  useful  structural  information. 

The  result  is  that,  when  the  NA  is  high,  the  signal  intensity 
recorded  per  pixel  from  a  point  source  drops  off  very  fast  as 
the  point  goes  out  of  focus,  even  in  widefield  fluorescence 
microscopy.  As  noted  in  Chapter  23,  signals  emanating  from  small 
features  located  much  more  than  2  pm  from  the  focus  plane  are 
more  likely  to  add  Poisson  noise  than  to  add  useful  structural 
information. 

Convolution,  Fourier  Transforms,  and  the  FFT 

The  tedious  process  of  overlapping,  multiplying,  adding,  and 
replacing  would  not  be  of  much  use  were  it  not  for  the  fact  that 
the  process  can  be  greatly  simplified  if  we  first  apply  a  mathe¬ 


matical  operation  called  the  Fourier  transform  (FT)  to  the  3D  data 
sets  representing  the  PSF  and  the  object.  The  FT  of  an  object  looks 
a  lot  like  its  diffraction  pattern.  Like  the  “real-space”  object,  the 
“Fourier-space  object”  is  a  3D  distribution  of  values  but,  whereas 
the  real  object  has  only  a  single  value  at  each  3D  location  (the 
intensity),  the  FT  of  the  object  has  both  an  intensity  and  a  phase 
angle.  The  phase  angle  information  is  used  by  the  inverse  FT 
process  to  convert  the  FT  data  back  into  a  real-space  object.  To 
perform  the  process  of  convolution  using  FTs,  we  must: 

1.  Calculate  the  3D  FT  of  the  object,  creating  a  3D  array  of  inten¬ 
sity  and  phase  values. 

2.  Calculate  the  3D  FT  of  the  PSF,  creating  a  second  3D  array  of 
intensity  and  phase  values. 

3.  Simply  multiply  these  two  arrays,  point  by  point,  creating  a 
new  3D  intensity/phase  array. 

4.  Calculate  the  inverse  Fourier  transform  of  this  product  array. 

This  may  all  sound  very  obscure,  but  fortunately  there  is  a 
very  fast  and  helpful  computer  algorithm  called  the  fast  Fourier 
transform  that  can  rapidly  perform  both  the  3D  transform  and  its 
inverse! 

Coincidentally,  calculating  the  FT  of  an  object  exactly  dupli¬ 
cates  the  effect  that  a  perfect  lens  has  on  light  that  diffracts  at  its 
focus  plane  and  then  proceeds  to  its  back-focal  plane  (BFP).  That 
is,  given  a  perfect  lens,  the  phase  and  intensity  pattern  of  the  light 
passing  through  the  BFP  is3  the  2D  FT  of  the  object  at  the  focus 
plane.  Unfortunately,  we  cannot  just  record  the  image  of  the  BPF 
and  then  apply  the  inverse  FT  to  reconstruct  the  2D  object  because 
we  can  only  record  the  intensity,  not  the  phase,  of  this  pattern. 

X-ray  diffraction  studies  confront  the  same  problem  but  in  this 
case,  the  ordered,  crystalline  nature  of  the  specimen  makes  it  pos¬ 
sible  to  make  shrewd  guesses  about  the  phases.  As  a  result,  success 
in  discerning  molecular  structure  depends  on  first  measuring  the 
intensity  of  the  diffraction  pattern  and  then  on  guessing  and  iter¬ 
ating  ever-more-accurate  estimates  of  the  phases. 


2  An  admitted  oversimplification  that  works  better  as  the  distance  from  the 
focus  plane  increases. 


3  Strictly  speaking,  it  is  the  magnitude  and  the  phase  of  E-field  in  the  BFP  that 
is  the  FT  of  the  E-field  and  phase  in  the  object  plane.  The  light  intensity  in 
each  plane,  is  the  square  of  these  magnitudes.  But,  this  is  getting  us  into  tech¬ 
nical  details  that  require  their  own  chapter. 
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Image  Enhancement  by  Deconvolution 

Mark  B.  Cannell,  Angus  McMorland,  and  Christian  Soeller 


INTRODUCTION 

In  this  chapter  we  will  try  to  provide  a  more  intuitive  (and  less 
mathematical)  insight  into  image  formation  and  practical  image 
restoration  by  deconvolution  methods.  The  mathematics  of  image 
formation  and  deconvolution  microscopy  have  been  described  in 
greater  detail  elsewhere  (see  Chapters  11,  22,  23,  and  24),  so  we 
will  limit  our  discussion  to  fundamental  issues  and  gloss  over  most 
of  the  mathematics  of  image  restoration.  We  will  also  focus  on  prac¬ 
tical  ways  of  assessing  microscope  performance  and  getting  the 
best  possible  data  before  applying  more  sophisticated  image  pro¬ 
cessing  methods  than  are  usually  seen  in  the  literature.  Before  we 
lose  the  interest  of  the  confocalist,  we  would  point  out  there  is  no 
such  thing  as  a  real  widefield  microscope  because  real  microscopes 
have  limited  field  of  view  and  other  apertures  within  the  optical 
train.  These  apertures  introduce  a  limited  degree  of  confocality 
even  into  widefield  microscopes  that,  as  we  will  see,  improves  the 
behavior  of  the  microscope  in  a  way  that  makes  quantitative  image 
restoration  both  possible  and  very  worthwhile.  The  methods  (and 
software)  that  improve  images  from  conventional  widefield  (WF) 
microscopes  will  also  significantly  improve  confocal  images  and 
our  practical  experience  shows  that  the  output  of  both  confocal  and 
multi-photon  microscope  systems  benefit  from  application  of 
image  restoration  methods.  For  those  who  wish  to  know  the  con¬ 
clusion  of  this  chapter  without  further  reading,  it  is  simply:  decon¬ 
volve  all  data  (if  you  can).  To  stress  this  point,  we  suggest  that  even 
a  properly  adjusted  confocal  microscope  will  not  give  the  best  pos¬ 
sible  confocal  image  and  the  application  of  appropriate  deconvo¬ 
lution  methods  will  increase  contrast,  reduce  noise,  and  even 
improve  resolution.  That  such  worthwhile  effects  are  made  possi¬ 
ble  by  computation  resides  in  the  fact  that  during  deconvolution  the 
information  content  of  the  image  can  be  increased  by  the  addition 
of  information  about  the  imaging  process  itself  as  well  as  that  about 
the  statistics  of  photon  capture.  Thus,  in  our  opinion,  the  applica¬ 
tion  of  deconvolution  methods  can  always  increase  image  quality, 
regardless  of  the  source  of  the  image. 

BACKGROUND 

The  resolution  of  a  measurement  instrument  is  always  limited  so 
that  the  recorded  image  of  the  sample  is  completely  dependent  on 
the  properties  of  the  measurement  device.  Microscopy  is  simply  a 
measurement  process  used  to  quantify  information  about  small 
objects,  and  the  limits  to  resolution  in  microscopy  are  manifest  as 
blurring  in  the  acquired  images.  Blurring  arises  from  the  intensity 


of  signal  at  any  point  in  the  image  being  some  weighted  sum  of 
intensities  from  all  points  in  the  object  space.  With  detailed  under¬ 
standing  of  the  weighting  function,  we  can  develop  ways  of  cor¬ 
recting  the  image  for  the  limiting  behavior  of  the  microscope.  In 
addition,  almost  all  microscopic  samples  are  inherently  three- 
dimensional  while  the  detectors  [such  as  the  eye,  charge-coupled 
device  (CCD)  camera,  or  photographic  film]  are  two-dimensional. 
Because  it  is  well  known  that  two-dimensional  representations  of 
three-dimensional  objects  lead  to  artifacts  and  ambiguity,  one 
should  expect  fundamental  imaging  problems  in  microscopy. 

A  number  of  phenomena  contribute  to  the  blurring  process  in 
light  microscopy.  First,  the  wave  nature  of  light  dictates  that  as  it 
passes  through  apertures  (e.g.,  the  optical  components  making  up 
the  microscope),  diffraction  occurs,  which  results  in  spreading  and 
merging  of  light  rays  in  the  image  (Abbe,  1873;  see  Chapter  22). 
Second,  samples  can  scatter  light  thereby  removing  the  corre¬ 
spondence  between  the  source  of  a  light  ray  and  its  final  location 
in  the  image  plane.  This  problem  makes  high-quality  imaging  in 
turbid,  thick  samples  (such  as  brain  slices)  highly  problematic.  We 
should  also  note  that  this  scattering  also  prevents  certain  light  rays 
from  passing  through  the  optical  train  and  thereby  prevents  some 
information  from  the  object  reaching  the  image.  Similarly,  a  dense 
object  can  obscure  an  (in-focus)  object  behind  it  and  prevent  infor¬ 
mation  from  it  reaching  the  image.  Finally,  refractive  index  mis¬ 
matches  in  the  light  path  (e.g.,  between  the  immersion  medium  and 
sample)  and  imperfections  in  the  optical  components  (and  their 
alignment)  can  add  to  aberrations  that  will  also  cause  light  rays  to 
be  misdirected. 

The  conventional  WF  microscope  is  designed  to  correctly 
image  a  sample  at  the  focal  plane  of  the  objective;  however  any 
light  source  within  the  conical  volume  of  light  collected  by  the 
objective  lens  can  contribute  light  to  the  image  plane.  While  this 
is  not  blurring,  it  will  introduce  a  loss  of  contrast  for  in-plane 
objects  and  contribute  to  the  perception  of  a  blurred  image.  Objects 
outside  the  plane  of  sharp  focus  are  also  seen  in  the  image  plane 
as  blurred  images  that  also  reduce  contrast  for  in-focus  objects.  (It 
is  the  reduction  of  this  effect  by  confocal  microscopes  that  makes 
them  so  useful.)  Finally,  the  limited  aperture  of  the  microscope 
system  (i.e.,  its  limited  light  and  information  gathering  ability)  and 
the  limited  wavelength  of  light  itself  lead  to  blurring  of  objects. 
With  so  many  limiting  factors,  it  is  not  surprising  that  real  micro¬ 
scope  imaging  always  represents  a  trade-off  between  various  prob¬ 
lems.  For  example,  increasing  the  numerical  aperture  of  the  system 
may  decrease  blurring  and  increase  signal  from  in-focus  objects 
but  can  also  lead  to  a  loss  of  image  contrast  due  to  the  acceptance 
of  more  scattered  light  from  thick  samples.  In  addition,  high 
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numerical  aperture  objectives  have  limited  working  distances  that 
may  limit  their  ability  for  deep  sectioning. 

Image  Formation 

The  transformation  of  information  from  a  real  object  to  blurred 
image  can  be  expressed  mathematically  using  an  operation  called 
convolution.  Deconvolution  is  the  reverse  operation,  whose 
purpose  in  microscopy  is  to  remove  the  contribution  of  out-of¬ 
focus  objects  from  the  image  plane  as  well  as  (partially)  reverse 
aberrations  arising  from  imperfections  in  the  optical  train.  It  should 
be  intuitive  that,  by  using  information  about  the  imaging  process, 
deconvolution  techniques  should  be  able  to  improve  the  quality  of 
the  image  above  that  which  could  be  achieved  by  any  other  method 
which  does  not  provide  extra  information  (beyond  that  contained 
in  the  image  plane  itself).  The  transformation  earned  out  by  the 
microscope  on  data  from  the  object  can  be  defined  by  a  mathe¬ 
matical  function  called  the  point  spread  function  (PSF).  The  PSF 
is  the  multi-dimensional  image  of  a  point  in  space  and  it  can  be 
practically  measured  by  imaging  very  small  objects  (which  must 
be  less  than  the  wavelength  of  light  in  size)  or  computed  from  the 
physical  and  optical  properties  of  the  imaging  system  (see  Fig. 
25.1).  Given  imperfections  in  the  imaging  system  (or  put  another 
way,  limited  knowledge  of  the  real  optical  system)  the  latter 
approach  is  inherently  limited,  but  assumptions  about  the  proper¬ 
ties  of  the  PSF  can  help  some  blind  deconvolution  methods. 

Put  simply: 

i(x,  y,  z,  t)  =  o(x,  y,  z,  t)  ®  psfix,  y,  z,  t)  (1) 

where  x,  y,  z,  t  are  the  dimensions  of  space  and  time  and  i  is  the 
recorded  image,  o  the  actual  underlying  object.  The  ®  operator  is 
convolution  (see  below  and  Fig.  25.2).  If  we  take  the  Fourier  trans¬ 
form  (F{ })  of  this  equation,  the  ®  is  replaced  by  multiplication: 


F{i(x,  y,  z,t)}=  F{o(x,  y,  z,,  t)}  x  F{psflx,  y,  z,  t)}  (2) 

This  simple  equation  shows  that  image  restoration  should  be 
possible  and  might  be  achievable  by  dividing  the  Fourier  trans¬ 
form  of  the  image  by  the  Fourier  transform  of  the  PSF  and  then 
taking  the  inverse  Fourier  transform.  Before  exploring  this  idea 
further,  it  should  be  noted  that  these  equations  remind  us  of  the 
importance  of  correct  data  sampling  (image  and  PSF)  and  the 
behavior  of  the  PSF  over  time  and  space.  For  example,  although 
we  generally  assume  that  the  microscopes  do  not  change  proper¬ 
ties  as  we  focus  through  the  sample  and  from  day  to  day,  is  this 
really  so?  When  objectives  are  damaged  by  lack  of  care  (such  as 
from  a  collision  with  slide/stage  or  accumulation  of  dirt),  the 
gradual  loss  of  imaging  performance  may  go  undetected.  This  is 
particularly  a  problem  for  confocal  microscopes  whose  pinhole 
can  effectively  hide  a  poor  or  deteriorating  PSF  at  the  expense  of 
signal  strength.  Also  note  that  coverslips  (which  are  an  integral 
part  of  the  optical  system)  vary  in  thickness  from  batch  to 
batch  and  therefore  introduce  variable  amounts  of  spherical 
aberration. 

From  here  on,  we  assume  that  everything  is  time  invariant. 
(This  is  not  a  trivial  assumption  as,  in  reality,  samples  are  never 
time  invariant,  they  move  —  especially  if  alive,  they  bleach,  the 
microscope  focus  drifts  with  temperature,  etc.)  So,  from  Eq.  2,  if 
one  can  measure  or  compute  the  PSF  with  sufficient  accuracy,  why 
can  images  not  be  restored  to  the  point  where  any  desired  resolu¬ 
tion  is  achieved?  The  major  problem  resides  in  the  noise  that  is 
always  present  in  a  physical  measurement.  The  convolution  or 
blurring  of  the  object  by  the  PSF  is  spatial  frequency-dependent 
and  attenuates  high-frequency  components  (provided  by  small  fea¬ 
tures  and  edges)  more  than  low  frequency  (large,  smooth)  objects. 
Hence,  deconvolution  must  boost  high  spatial  frequency  compo¬ 
nents  more  than  low  spatial  frequency  components.  Because  noise 


1 .25  NA,  widefield 


0.90  NA,  widefield 

FIGURE  25.1.  Theoretical  PSFs  calculated  using 
widefield  equations.  On  the  left,  sections  at  0.5  pm 
--intervals  from  the  plane  of  sharp  focus  are  shown 
while  xz  sections  are  shown  on  the  right.  Panel  (A) 
shows  PSFs  for  two  typical  lenses.  Note  the  effect 
of  reducing  NA  on  the  axial  extent  of  the  PSF.  Panel 
(B )  shows  equivalent  PSFs  for  the  above  lenses  in  a 
confocal  configuration.  Note  that  the  confocal  PSF 
looks  very  similar  to  the  widefield  PSF  with  the  out¬ 
lying  wings  removed.  Again,  the  improved  axial 
response  of  the  higher  NA  lens  is  quite  clear  in  these 
images.  The  gray  scale  is  shown  on  the  right  and  is 
logarithmic. 
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FIGURE  25.2.  Schematic  diagram  demonstrating  the  convolution  (®)  opera¬ 
tion  with  a  6  X  6  pixel  object  and  a  3  X  3  pixel  blurring  kernel.  The  profiles 
above  show  the  maximum  projection  of  the  two-dimensional  grids  as  would 
be  seen  looking  across  the  planes  from  above.  Note  how  the  contrast  of  the 
peaks  in  the  image  is  reduced  and  smeared  across  the  image. 


is  present  at  all  frequencies  in  the  image  right  up  to  the  spatial  sam¬ 
pling  frequency  which  is  l/(pixel  size),  noise  components  are 
boosted  at  all  frequencies  during  deconvolution.  At  more  than  half 
the  sampling  frequency  of  the  image,  there  is  no  real  object  data 
retained  (from  the  Nyquist  sampling  theorem)  but  the  ever-present 
noise  receives  an  even  larger  boost  during  deconvolution. 

Put  another  way,  because  noise  does  not  come  directly  from 
the  object,  but  is  introduced  during  the  imaging  process,  and 
because  noise  is  rich  in  high-frequency  components,  deconvolu¬ 
tion  (which  seeks  to  boost  high-frequency  content)  can  amplify 
noise  to  the  point  where  it  masks,  and  renders  useless,  any  infor¬ 
mation  in  the  resulting  deconvolved  image.  The  addition  of  noise 
also  generates  ambiguity  in  the  image  restoration  (deconvolution) 
process  so  that  more  than  one  optimal  restoration  solution  exists 
with  no  a  priori  method  of  determining  which  solution  best  rep¬ 
resents  the  real  object.  Current  deconvolution  algorithms  work 
around  some  of  these  limitations,  often  by  making  a  number  of 
reasonable  assumptions  about  the  object  (such  as  smoothness  and 
nonnegativity)  and  include  extra  information  about  the  noise 
process  itself. 

Having  introduced  image  restoration  by  deconvolution,  we 
will  now  describe  the  convolution  and  deconvolution  processes  in 
more  detail  and  give  a  guide  to  some  of  the  practical  issues  regard¬ 
ing  image  deconvolution.  We  will  also  discuss  how  some  popular 
deconvolution  algorithms  vary,  and  assess  the  utility  of  image 
restoration.  We  will  show  that  deconvolution  can  substantially 
improve  WF  microscopy,  to  the  point  where  it  can  be  used  as  a 
viable  and  practical  alternative  to  confocal  laser  scanning 
microscopy  (CLSM)  and  can  also  be  used  to  good  effect  to 
improve  the  quality  of  confocal  and  multi-photon  microscope 
images. 

FORWARDS:  CONVOLUTION  AND 
THE  IMAGING  SYSTEM 

Before  it  is  possible  to  develop  an  algorithm  for  reversing  the 
defects  induced  during  the  imaging  process,  it  is  necessary  to 
develop  a  basis  for  image  formation.  This  section  will  give  a  short¬ 
ened  version  of  that  formalization  and  more  detailed  descriptions 
can  be  found  in  Agard  and  colleagues  (1989)  and  Young  (1989). 


The  notation  used  generally  follows  the  conventions  of  Press  and 
colleagues  (1992). 

The  starting  point  for  this  formulation  requires  two  assump¬ 
tions  regarding  the  imaging  process:  linearity  and  shift  invariance. 
The  principle  of  linearity  is  met  if  the  sum  of  the  images  of  two 
separate  objects  is  the  same  as  the  image  of  the  combined  objects. 
The  assumption  of  linearity  is  generally  safe  in  fluorescence 
microscopy  provided  that  detectors  are  linear  and  we  avoid  self¬ 
quenching  and  self-absorption  by  the  fluorophores  (Chapters  16 
and  17,  this  volume).  Shift  invariance  implies  that  the  image  of  an 
object  will  be  the  same  regardless  of  where  in  the  field  of  view 
that  object  lies.  (While  no  real  imaging  system  meets  the  require¬ 
ment  of  shift  invariance,  it  is  a  reasonable  assumption  for  a  high- 
quality  research  microscope,  subject  to  our  being  aware  of  the 
potential  complication  introduced  by  changing  aberrations  across 
the  field  and  with  focus.)  Now,  because  any  object  can  be  repre¬ 
sented  by  a  superposition  of  many  delta  functions  (point  light 
sources)  whose  individual  images  are  simply  shifted  copies  of  the 
PSF,  it  follows  from  the  principles  of  linearity  and  shift  invariance 
that  the  whole  image  is  made  from  the  superposition  (sum)  of 
appropriately  scaled  and  shifted  instances  of  the  PSF.  For  clarity, 
we  can  express  this  in  one  dimension  as  an  equation: 

+CO 

i(x)  =  J  o(x  -  x')psf(x')dx'  (3) 

which  is,  in  fact,  mathematically  the  same  as  the  convolution  of 
the  object  with  the  PSF.  In  symbolic  form: 

i(x)  =  o(x)  ®  psfix)  (4) 

which  gives  a  basis  for  the  first  general  equation  given  above.  As 
noted  above,  convolution  in  the  spatial  domain  is  equivalent  to 
multiplication  in  the  frequency  domain  so  we  can  write  (for  three- 
dimensional  objects): 

F{ i(x,  y,z)}  =  F{ o(x,  y,  z) }  x  OTF(q,  ¥,  Q  (5) 

where  OTF()  is  the  optical  transfer  function  and  is  the  Fourier 
transform  of  the  PSF  (see  Figs.  25.3  and  25.4),  \|/,  and  £  are  the 

spatial  frequency  coordinates  of  the  OTF  in  Fourier  space,  derived 
from  the  PSF  sampled  in  x,  y,  and  z,  respectively.  The  OTF  is  intro¬ 
duced  here  because  convolutions  are  most  quickly  computed  in  the 
frequency  domain  by  using  the  fast  Fourier  transform  (FFT). 
Because  the  PSF  is  assumed  to  be  shift  invariant,  only  a  single 
Fourier-transformed  PSF  is  used  for  computations  and  the  values 
of  the  PSF  in  the  spatial  domain  per  se  have  no  practical  value  for 
deconvolution.  Thus,  it  is  more  convenient  to  describe  the  PSF  of 
a  microscope  in  terms  of  its  OTF  rather  than  its  PSF.  The  OTF 
directly  describes  how  blurring  affects  the  various  frequency  com¬ 
ponents  making  up  an  object  and  immediately  gives  insight  into 
the  resolving  power  of  the  microscope.  Although  we  usually 
measure  the  PSF  in  the  image  plane  (rather  than  at  the  rear  aper¬ 
ture  of  the  objective  where  the  OTF  exists)  and  most  researchers 
are  more  familiar  with  an  ideal  PSF  and  can  identify  microscope 
defects  by  inspecting  it  (see  below),  the  OTF  is  actually  more 
useful.  Put  another  way,  because  the  performance  of  the  entire 
microscope  is  the  result  of  the  combination  of  the  OTFs  of  each 
optical  lens  and  aperture,  it  is  easier  to  immediately  determine  the 
effect  of  an  aperture  or  lens  in  a  microscope  from  its  OTF.  As  an 
example,  because  a  circular  aperture  reduces  the  amplitude  of  the 
OTF  with  increasing  spatial  frequency,  it  is  immediately  clear  that 
the  ultimate  determinant  of  resolution  must  be  the  most  limiting 
aperture  in  the  system  [usually  the  numerical  aperture  (NA)  of  the 
objective  lens]. 


Image  Enhancement  by  Deconvolution  •  Chapter  25 


491 


Spatial 

domain 


Frequency 

domain 


Widefield 


Confocal 


* 


FIGURE  25.3.  The  relationship  between  the  PSF  and  OTF  in  confocal  and  widefield  modes  of  a  microscope.  For  simplicity,  we  assume  that  the  pinhole  func¬ 
tion  leads  to  a  simple  squaring  of  the  PSF,  a  reasonable  approximation  for  an  optimal  pinhole  with  similar  excitation  and  emission  wavelengths.  The  convolu¬ 
tion  of  the  widefield  OTF  with  itself  in  the  frequency  (Fourier)  domain  is  the  equivalent  of  squaring  in  the  spatial  domain.  Note  that  the  inversion  of  the 
convolution  (inverted  convolution  symbol)  is  difficult  in  the  frequency  domain  and  is  more  easily  accomplished  in  the  spatial  domain  (by  taking  the  square  root). 
This  illustrates  that  some  operations  are  more  simply  carried  out  by  operating  in  the  appropriate  domain;  convolution  is  easier  in  the  frequency  domain  while 
multiplication  is  simpler  in  the  spatial  domain.  Such  interchangeability  between  spatial  and  frequency  domains  is  central  to  deconvolution  methods. 


FIGURE  25.4.  Volume  rendered  illustrations  of  the  widefield  PSF  (A)  and  OTF  (B).  The  PSF  extends  to  infinity  as  a  cone  with  ripples  in  the  intensity  in  all 
directions.  The  OTF  appears  smaller  being  closer  to  a  toroidal  shape  except  that  the  center  values  are  non-zero.  This  leads  to  a  cone  of  low  values  in  the  center 
of  the  toroid.  The  toroid  width  in  both  directions  is  limited  by  diffraction.  The  gray  scale  shading  is  proportional  to  the  logarithm  of  the  intensity. 
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PROPERTIES  OF  THE  POINT 
SPREAD  FUNCTION 

Because  the  PSF  of  a  microscope  determines  how  it  creates  images 
of  an  object,  the  shape  of  the  PSF  determines  the  way  in  which 
images  of  objects  blur  into  each  other  in  the  final  image.  At  the 
focal  plane  of  a  WF  microscope,  the  PSF  [see  Fig.  25.1(A)]  exists 
as  a  central  Airy  disk,  surrounded  by  dimmer  concentric  rings  that 
are  the  result  of  diffraction  occurring  as  light  passes  through  the 
circular  apertures  of  the  microscope.  Away  from  the  plane-of- 
focus,  the  PSF  spreads  outwards,  forming  two  apex-opposed  cones 
with  diffraction  ripples  within  them  [Fig.  25.4(A)].  This  pattern 
reflects  the  fact  that  WF  microscopes  form  images  from  cones  of 
light  gathered  by  an  objective  lens,  with  the  angle  of  the  cone 
and  thus  the  shape  of  the  PSF  being  determined  primarily  by 
the  NA  of  the  objective  lens  (Chapters  11,  22,  23,  and  24,  this 
volume).  The  axial  extent  of  the  concentric  cones  of  the  widefield 
PSF  at  "-levels  well  removed  from  the  focal  plane  show  how  sig¬ 
nificant  the  out-of-focus  contribution  is  to  a  WF  image.  This  is  not 
the  case  for  the  CLSM  PSF  whose  PSF  approaches  background 
levels  with  distance  [Fig.  25.1(B)],  a  point  which  will  be  revisited 
later. 

It  is  important  to  note  that,  unlike  theoretically  derived  wide¬ 
field  PSFs,  measured  widefield  PSFs  are  rarely  symmetric  around 
the  focal  plane,  nor  radially  symmetrical  about  the  optical  axis. 
Asymmetry  along  the  optical  axis  (z-axis)  is  commonly  due  to 
spherical  aberration,  which  may  result  from  refractive  index  mis¬ 
matches  between  the  objective,  immersion  medium,  and  sample  or 
tube  length/coverslip  thickness  errors.  Radial  asymmetries  in  the 
PSF  are  commonly  the  result  of  misalignment  of  optical  compo¬ 
nents  about  the  z-axis,  either  as  tilt  or  decentration  (see  Cagnet  et 
al. ,  1962;  Keller,  1995)  (see  Fig.  25.5). 


The  extent  of  the  PSF  determines  the  resolution  of  the  micro¬ 
scope.  PSF  size  is  often  expressed  in  terms  of  the  width  at  which  it 
is  half  the  maximum  intensity  (full-width  half-maximum,  or 
FWHM).  A  typical  FWHM  for  the  widefield  PSF  of  a  high  power 
immersion  objective  lens  might  be  <0.4  pm  in  the  focal  plane.  The 
axial  extent  of  a  true  widefield  PSF  should  be  infinite  with  constant 
energy  at  all  planes,  but  real  microscopes  lose  signal  energy  with 
distance  because  additional  apertures  (beyond  the  entrance  pupil)  in 
the  system  lead  to  the  more-distant  wings  of  the  widefield  PSF  being 
clipped  (or  vignetted).  Nevertheless,  the  extended  axial  response  of 
the  WF  microscope  results  in  a  massive  loss  of  in-plane  contrast  for 
extended  specimens  which  are  either  fluorescent  or  scattering. 


QUANTIFYING  THE  POINT  SPREAD  FUNCTION 

Most  deconvolution  methods  (i.e.,  not  blind  ones)  require  knowl¬ 
edge  of  the  PSF  relevant  to  the  particular  imaging  conditions 
encountered.  PSFs  can  be  estimated  either  mathematically  or  from 
direct  measurement.  In  blind  deconvolution  methods  an  estimate 
of  the  PSF  is  also  produced  by  the  deconvolution  algorithm.  (Note 
that  if  the  object  and  image  are  known,  the  OTF  is  given  by  the 
Fourier  transform  of  the  image  divided  by  the  Fourier  transform 
of  the  object;  see  Eq.  5.)  While  it  is  relatively  easy  to  quantify  the 
PSF  of  a  microscope  under  controlled  conditions  (see  Fig.  25.5), 
matching  the  conditions  encountered  during  an  experimental 
imaging  situation  is  often  impractical,  and  commonly  some 
approximation  to  the  likely  real  PSF  is  used. 

Most  methods  for  calculating  theoretical  PSFs  are  based  on 
equations  in  the  definitive  work  of  Born  and  Wolf  (1980).  A  good 
description  of  the  calculations  for  a  confocal  PSF  is  given  by  van 
der  Voort  and  Brakenhoff  (1990),  in  which  the  PSF  is  calculated 
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FIGURE  25.5.  Measuring  the  widefield  PSF.  (A) 
shows  the  appearance  of  a  0.2  pm  bead  with  varying 
degrees  of  defocus.  Note  that  although  the  most 
focused  image  is  reasonably  circular,  with  defocus 
extra  structure  appears  in  the  Airy  disk.  These 
uncontrolled  aberrations  need  to  be  measured  for 
correct  deconvolution.  Note  also  that  the  aberra¬ 
tions  are  not  symmetrical  about  the  point  of  sharp 
focus  so  that  enforcement  of  symmetry  in  blind 
deconvolution  would  be  problematic.  (B)  shows  a 
through-focus  series  with  spherical  aberration. 
Asymmetry  above  and  below  best  focus  is  evident. 
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from  the  NA  of  the  objective,  the  illuminating  and  emitted  wave¬ 
lengths,  and  the  refractive  index  of  the  immersion  medium  in  either 
(simpler)  paraxial  forms  or  with  WF  integrals  (see  Fig.  25.6). 
Numerically  derived  expressions  for  the  PSF  give  an  indication  of 
the  best  possible  resolution  for  a  given  objective  but  these  limits 
are  not  achievable.  The  real  world  limitations  in  alignment  of  the 
optical  path  and  lens  aberrations  inevitably  lead  to  a  real  PSF  that 
is  larger  and  has  more  asymmetric  structure  than  predicted  by 
mathematical  approaches.  Particular  attention  should  be  paid  to 
whether  a  paraxial  approximation  is  appropriate  because  there  are 
considerable  differences  between  the  results  of  paraxial  and  WF 
calculations  for  the  high-NA  objectives  which  are  often  used  in 
experiments  (Fig.  25.6).  The  problem  is  confounded  by  software 
packages  that  do  not  reveal  what  method  is  used  to  calculate  the 
PSF.  In  our  experience,  real  PSFs  are  typically  >20%  bigger  than 
calculated  versions  (based  on  the  stated  NA)  and  their  shapes  are 
rarely  perfectly  symmetrical  about  the  optical  axis  and  focal  plane, 
even  when  we  have  tried  to  control  refractive  index  mismatches. 
It  should  be  intuitively  obvious  (from  the  preceding  discussion) 
that  the  accuracy  of  the  deconvolved  image  can  be  no  better  than 
that  of  the  PSF  that  was  used  for  the  deconvolution.  Errors  in  the 
PSF  can  produce  bizarre  artifacts  in  the  image  (such  as  hollow 
cones  or  slanting  tails  around  bright  objects)  as  well  as  the  algo¬ 
rithm  not  properly  converging  to  minimum  noise  (see  Fig.  25.7). 

The  PSF  of  a  microscope  can  be  measured  by  taking  images 
of  commercially  available  fluorescent  beads  of  <0.2  pm  diameter. 
These,  being  smaller  than  the  diffraction  limit  of  visible  light,  act 
like  point  light  sources,  and  their  imaged  size  and  shape  therefore 
represent  the  microscope  PSF.  It  is  a  common  misconception  that 
even  smaller  beads  give  better  estimates  of  the  PSF.  This  is  not  the 
case  as  the  imaging  process  is  a  convolution  and  the  convolution 
is  dominated  by  the  larger  of  the  two  objects  being  convolved.  In 
addition,  the  beads  become  much  less  bright  as  smaller  beads  are 
used  (note  that  the  signal  intensity  will  fall  with  the  cube  of  the 
diameter)  so  precise  measurement  of  the  entire  PSF  becomes  very 
difficult,  especially  as  the  PSF  is  recorded  away  from  the  focal 
plane.  For  those  who  feel  uncomfortable  using  0.2  pm  spheres,  it 
is  possible  to  use  deconvolution  of  the  PSF  image  using  a  0.2  pm 
sphere  (the  actual  object)  to  yield  the  precise  PSF  (from  Eq.  5), 
but  in  practice  it  is  rarely  necessary  to  go  to  such  lengths.  In  our 
experience,  the  difference  in  the  final  results  of  deconvolution 
using  bead  images  and  PSFs  derived  by  deconvolution  of  bead 
images  with  spherical  sources  is  negligible. 

When  imaging  beads  for  the  purposes  of  PSF  measurement, 
the  image  data  should  be  obtained  at  as  high  a  resolution  as  pos¬ 
sible.  In  this  case,  Nyquist’s  criterion  of  using  twice  the  desired 


resolution  is  insufficient,  and  pixel  resolutions  0.05  pm  (or  even 
better)  are  desirable.  This  has  some  advantages:  (1)  The  fine  data 
spacing  facilitates  data  resampling  to  enable  the  PSF  resolution  to 
match  that  of  the  recorded  image.  It  is  often  the  case,  especially 
when  confocal  variable  zoom  controls  are  used,  that  the  final  image 
resolution  may  be  different  from  that  used  to  acquire  the  PSF.  (2) 
Generous  over-sampling  also  allows  a  greater  dynamic  range  to  be 
recorded  as  the  received  photons  are  recorded  in  more  voxels. 
Even  if  the  detector  is  only  8  bit.  over-sampling  by  a  factor  of  2 
in  all  three  dimensions  raises  the  effective  bit  resolution  to  1 1  bits. 
This  increase  in  effective  dynamic  range  is  very  useful  as  it  enables 
both  the  bright  center  of  the  PSF  as  well  as  more  dim  distant  fea¬ 
tures  to  be  recorded.  For  deconvolution,  low  noise  PSFs  are 
required  and  this  can  be  achieved  by  averaging  a  number  of  indi¬ 
vidual  bead  images,  possibly  from  different  beads  (because  the 
principle  of  shift  invariance  requires  that  the  bead  image  —  the 
PSF  —  is  not  affected  by  the  position  of  the  bead).  This  approach 
requires  an  algorithm  for  aligning  the  centers  of  the  image  of  each 
bead  so  that  the  averaging  operation  can  combine  data  points  in 
each  bead  image.  The  coordinates  of  the  center  of  the  3D  bead 
image  can  be  derived  by  calculating  the  center  of  signal  mass  of 
an  image  that  has  been  subjected  to  a  thresholding  procedure  to 
completely  remove  any  background  signal.  The  most  distal  parts 
of  the  image  should  also  be  removed  by  the  chosen  threshold  to 
prevent  the  estimate  of  the  bead  centroid  depending  on  the  posi¬ 
tion  of  the  bead  within  the  sampled  data  volume. 

Perhaps  the  most  problematic  aspect  of  measuring  PSFs  using 
fluorescent  beads  is  matching  the  experimental  imaging  conditions 
to  which  the  PSF  is  to  be  applied.  First,  to  account  for  the  effect 
of  any  chromatic  aberrations  (as  well  as  the  direct  effect  of  wave¬ 
length  on  resolution),  PSFs  should  be  measured  from  beads  that 
emit  light  of  a  similar  wavelength  to  the  experimental  fluorophore 
being  imaged.  (Beads  are  available  with  a  range  of  emission  wave¬ 
lengths  from  415  to  680  nm.)  Errors  in  refractive  index,  either 
between  the  objective  and  immersion  medium  or  between  immer¬ 
sion  and  bathing/mounting  media,  need  to  be  consistent  for  correct 
replication  of  the  aberrated  PSF.  Because  experimental  samples  are 
often  less  well  controlled  than  calibration  bead  samples,  we  often 
add  a  few  beads  to  critical  experimental  samples  to  provide  a  check 
of  the  actual  PSF  in  the  biological  sample.  These  beads  are  readily 
recognized  in  images  as  they  usually  stick  to  the  outside  of  cells 
(in  live-cell  imaging)  and  do  not  bleach  as  rapidly  as  the  experi¬ 
mental  sample.  Although  it  is  possible  to  remove  the  bead  from 
the  experimental  image  digitally,  we  have  never  had  to  do  this  in 
practice  as  it  is  usually  possible  to  find  a  suitable  field  of  view  free 
of  beads. 


FIGURE  25.6.  Comparison  of  widefield  and  paraxial  com¬ 
putations  of  the  PSF.  The  left  panel  shows  the  computed  PSF 
for  a  1 .25  NA  objective  using  the  integrated  field  equations 
(van  der  Voort  and  Brakenhoff,  1990).  The  right  panel  shows 
the  equivalent  results  for  the  paraxial  equations  which 
combines  the  electric  vectors  using  the  assumption  that 
sin(0)  =  0.  The  paraxial  assumption  leads  to  greater  ringing 
in  the  response  but  a  narrower  Airy  disk. 
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FIGURE  25.7.  Image  restoration  by  Wiener  filter¬ 
ing.  Panel  (A)  shows  the  original  test  image  and  the 
PSF  used  to  blur  it.  The  resulting  blurred  image  has 
lost  contrast  for  all  small  features.  Panel  (B)  shows 
the  results  of  image  restoration  with  variable 
amounts  of  noise  added  to  the  blurred  image  for 
various  values  of  k  (see  Eq.  10).  On  the  left  side, 
without  noise  the  inverse  operation  perfectly  restores 
the  data.  However,  even  with  1%  noise  ( N  =  0.01) 
inverse  filtering  yields  fewer  data  than  even  the  orig¬ 
inal  blurred  image.  When  k  =  10-4,  the  effect  of  noise 
is  reduced  but  at  the  expense  of  high  frequency  per¬ 
formance.  The  loss  of  high  frequency  components 
leads  to  ringing  near  sharp  edges  and  can  be  seen  in 
the  small  colored  circles.  When  k  =  5  •  10-2,  noise  is 
even  better  controlled  but  the  loss  of  high  frequency 
information  is  more  severe.  This  can  be  clearly  seen 
by  comparing  the  centers  of  the  spoked  targets.  Note 
that  the  image  where  k  =  10"4  and  N  =  0.01  is  com¬ 
parable  in  appearance  to  the  image  k  =  5  •  10-2  and  N 
=  0.1.  Thus  increasing  k  controls  increasing  noise  at 
the  expense  of  reduced  spatial  resolution. 


THE  MISSING  CONE  PROBLEM 

As  noted  above,  while  the  PSF  provides  information  about  the 
spatial  throughput  of  a  microscope,  its  Fourier  transform  (the  OTF) 
gives  information  about  the  frequency  response  of  a  microscope 
directly.  While  the  NA  of  the  microscope  limits  the  highest  spatial 
frequency  that  can  be  recorded,  extended  sources  (i.e.,  lower 
spatial  frequency  objects)  lose  signal  strength  with  defocus 
because  source  light  rays  must  originate  in  the  finite  field  of  view 
of  the  microscope.  Thus,  the  OTF  of  the  WF  microscope  looks  like 
a  toroid  [Fig.  25.3  and  25.4(B)]  with  so-called  missing  cones  above 
and  below  the  focal  plane.  These  missing  cones  of  reduced  signal 
reflect  the  cone  of  light  acceptance  of  the  objective  (assuming  it  is 
the  limiting  aperture  in  the  microscope).  The  width  of  the  torus  in 
all  planes  is  limited  by  the  numerical  aperture  of  the  objective 
giving  rise  to  a  bandlimit  equal  to  2NA/X  (as  required  from 
Nyquist  sampling  of  the  object  using  the  spatial  frequency  of  light 
in  the  immersion  medium).  The  axial  band  limit  is  NA2/2riA,  while 
the  angle  of  the  missing  cone  is  arcsin(NA/ri),  where  T|  is  the  lim¬ 
iting  refractive  index  of  the  mounting  media  between  objective  and 
specimen. 

In  a  CLSM,  the  pinhole  aperture  restricts  the  ability  of  light 
from  out-of-focus  objects  to  reach  the  detector.  This  means  the 
light-focusing  pattern  of  the  objective  is  used  to  determine  both 
the  illumination  and  emission  light  paths,  and  so  the  PSF  of  a 
CLSM  is  approximately  the  square  of  the  widefield  PSF  (assum¬ 
ing  that  the  ratio  of  excitation  to  emission  wavelength  is  close  to 
1).  The  PSF  of  a  two-photon  microscope  is  similarly  related  to  the 


power  of  the  widefield  PSF  because  the  probability  of  near  simul¬ 
taneous  multiple  photon  absorptions  is  proportional  to  the  illumi¬ 
nating  intensity  raised  to  the  nth  power,  where  n  is  the  nearest 
integer  ratio  of  illumination  to  excitation  wavelengths  (Denk  et  al., 
1990).  As  a  result,  the  PSFs  [Fig.  25.2(A-C)]  are  ovoids  with  the 
long  axis  corresponding  to  the  optical  axis  of  the  microscope.  The 
corresponding  OTF  does  not  have  missing  cones  and  is  closer  to 
an  ovoid  (Fig.  25.3).  This  can  be  explained  by  the  fact  that  the  con- 
focal  PSF  is  the  product  of  the  illumination  and  detection  PSFs 
(Wilson,  1990)  and  in  Fourier  space,  multiplication  becomes  con¬ 
volution.  Convolution  of  one  toroid  by  another  tends  to  fill  in  the 
center  void. 

As  pointed  out  earlier,  deconvolution  can  be  thought  of  as 
the  division  of  the  Fourier  transform  of  the  image  by  the  OTF  of 
the  microscope.  The  low  values  of  the  OTF  in  the  neighborhood 
of  the  missing  cone  and  near  the  bandlimit  greatly  amplify  any 
noise  in  the  image  and  this  severely  limits  the  ability  of  deconvo¬ 
lution  methods  to  restore  images.  This  problem  is  partly  amelio¬ 
rated  by  the  apertures  in  the  optical  train  that  limit  the  ability  of 
the  microscope  to  capture  light  from  distant  objects  (whose  in¬ 
plane  components  would  have  low  OTF  values);  their  attenuation 
prevents  the  massive  amount  of  signal  (and  noise)  in  the  integrated 
far  field  from  completely  swamping  in-focus  data.  Put  another 
way,  blurring  of  data  from  nearby  objects  is  the  biggest  problem, 
while  signal  from  more  distant  objects  is  attenuated  because 
real  microscopes  are  not  true  WF  instruments.  This  problem  is 
reduced  in  the  CLSM  because  the  missing  cone  does  not  exist, 
however,  the  loss  of  photons  due  to  the  pinhole  reduces  the  signal- 
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to-noise  ratio  in  the  image  and,  during  deconvolution,  all  noise  is 
amplified. 

NOISE 

Photon  noise  is  always  present.  The  quantal  nature  of  light  means 
that  a  given  illumination  intensity  is  associated  with  statistical  vari¬ 
ation  in  the  time  at  which  photons  are  detected.  The  number  of 
photons  observed  over  any  period  behaves  as  a  Poisson  process 
whose  variance  is  equal  to  the  mean.  In  fluorescence  microscopy, 
the  number  of  photons  that  can  be  collected  is  limited  by  photo¬ 
damage  (bleaching)  and  the  signal-to-noise  ratios  are  quite  low.  In 
addition,  although  the  PSF  and  object  are  generally  considered 
to  be  three-dimensional,  the  bleaching  processes,  fluctuations  in 
illumination  intensity,  and  even  movement  of  the  object  make 
real  objects  four-dimensional.  To  remove  this  fourth  dimension 
we  need  to  take  a  three-dimensional  snapshot  to  freeze  time- 
dependent  changes.  This  implies  limited  exposure  periods  must  be 
used,  which  of  course  further  limits  light  collection. 

Photons  need  to  be  converted  to  electrical  signals  for  quantifi¬ 
cation  and  this  process  also  introduces  noise.  Even  in  the  absence 
of  photons,  all  detectors  give  spurious  signals  (dark  current)  while 
undetected  photons  lead  to  a  reduced  signal-to-noise  ratio.  Over  a 
range  of  wavelengths,  average  detector  efficiencies  are  generally 
below  50%  and  this  figure  is  further  degraded  by  significant  losses 
in  the  optical  components  in  front  of  the  detector  (such  as  barrier 
filters  and  mirrors).  It  is  not  unusual  to  see  overall  detection  effi¬ 
ciencies  well  below  5%  in  real  microscope  systems  at  some  wave¬ 
lengths  and  in  the  face  of  such  signal  losses  the  photon  (shot)  noise 
in  the  system  can  be  severe.  In  addition,  the  detector  may  add  elec¬ 
trical  noise  to  the  signal  before  digitization. 

A  third  source  of  noise  resides  in  the  digitization  of  the  signal. 
In  most  systems,  the  detector  provides  an  analog  signal  that  is  dig¬ 
itized  into  discrete  levels,  and  8-  to  12-bit  detectors  are  most 
common.  There  is,  of  course,  a  one-half-bit  uncertainty  in  the  dig¬ 
itization  of  any  such  signal  and  this  typically  leads  to  between 
0.2%  to  0.01%  noise.  Although  this  may  seem  small  (compared  to 
photon  noise),  this  noise  is  also  amplified  during  data  processing. 
(To  appreciate  the  potential  contribution  of  this  component  we 
should  recall  that  the  original  in-focus  data  may  represent  <5%  of 
the  total  signal.)  The  digitization  noise  does  not  apply  to  all  digital 
photon-counting  systems.  Despite  the  concerns,  photon  noise 
generally  remains  the  dominant  noise  source  (Mullikin  etal.,  1994; 
Pawley,  1995;  van  Kempen  et  al.,  1997)  in  microscopic  fluores¬ 
cence  images,  at  least  when  using  PMTs.  When  using  CCD  detec¬ 
tors,  other  sources  of  noise  are  added.  For  a  very  good  CCD 
with  a  readout  noise  of  ±10  photoelectrons,  readout  noise  will 
dominate  until  100  photons  are  detected  (at  which  point  these  noise 
sources  are  equivalent).  For  cameras,  another  source  of  noise  is 
added  due  to  pixel-to-pixel  variation  in  gain  (and/or  sensitivity). 
This  flat-field  noise  is  age  and  temperature  sensitive,  it  increases 
in  direct  proportion  to  signal  strength,  and  it  might  be  equivalent 
to  photon  noise  at  a  signal  strength  of  -5000  photons.  At  high 
light  levels  this  noise  source  dominates  but  can  often  be  partially 
controlled  by  correcting  the  image  for  the  gain/sensitivity  varia¬ 
tion  (by  normalizing  the  image  to  a  reference  image  of  a  white 
field). 

Finally,  the  thousands  of  computations  required  during  decon¬ 
volution  may  lead  to  computational  errors  due  to  truncation.  For¬ 
tunately,  modern  computers  have  quite  long  word  lengths  so  this 
problem  is  largely  disappearing,  but  errors  in  programming  such 
as  inappropriate  typecasting  can  lead  to  truncation  errors  propa¬ 


gating.  Computational  errors  are  often  hard  to  diagnose  as  they 
usually  appear  as  general  noise  in  the  image. 

DECONVOLUTION  ALGORITHMS 

In  the  previous  discussion  we  have  shown  that  in  fluorescence 
microscopy  (in  all  its  modes  including  widefield,  confocal, 
and  multi-photon)  the  imaging  process  can  be  mathematically 
described  by  a  convolution.  Convolution  consists  of  replacing  each 
point  in  the  original  object  with  its  blurred  image  (the  PSF)  in  all 
three  dimensions  and  summing  together  overlapping  contributions 
from  adjacent  points  to  generate  the  resulting  three-dimensional 
image.  As  pointed  out  above,  the  inverse  operation  or  deconvolu¬ 
tion  can  be  performed  (at  least  in  principle)  by  dividing  the  Fourier 
transform  of  the  recorded  image  by  the  Fourier  transform  of  the 
PSF  (this  argument  followed  from  Eq.  6).  Although  mathemati¬ 
cally  correct,  this  inverse  filter  algorithm  does  not  work  well  in 
practice  because  the  inevitable  noise  in  the  recorded  data  can  be 
so  strongly  amplified  as  to  render  the  deconvolved  data  useless.  It 
is  this  problem  that  more  sophisticated  deconvolution  algorithms 
have  been  developed  to  overcome  and  more  robust  methods  are 
well  established.  In  this  section  we  will  describe  the  ideas  behind 
these  improved  methods  without  too  much  mathematics.  Note 
that  when  using  the  term  image  here,  we  mean  the  full  three- 
dimensional  data  set,  often  called  an  image  stack.  If  desired,  the 
reader  can  consult  one  of  the  Numerical  Recipes  in  X  texts  (where 
X  may  be  Fortran,  C  or  C++;  see  also  http://www.nr.com  for  online 
access)  for  more  detailed  discussion  of  some  of  the  fundamental 
mathematical  concepts  involved  (i.e.,  maximum  likelihood, 
minimization  of  functions,  etc.). 

Nearest-Neighbor  Deconvolution 

Deconvolution  algorithms  vary  greatly  with  respect  to  computa¬ 
tional  intensity,  that  is,  how  long  it  actually  takes  to  compute  the 
deconvolved  image.  Among  the  simplest  algorithms  that  are  of 
practical  use  are  the  nearest-neighbor  approaches.  Such  algorithms 
are  based  on  the  simplifying  assumption  that  the  loss  of  contrast 
in  any  given  plane  is  only  due  to  signal  arising  from  objects  in 
planes  immediately  above  and  below  it.  To  restore  the  data  with 
this  assumption,  we  can  subtract  appropriately  blurred  versions  of 
the  object  planes  above  and  below  the  current  focal  plane  from  the 
recorded  image  data.  The  blurring  of  the  adjacent  planes  is,  of 
course,  determined  by  the  PSF.  For  example,  the  signal  contribu¬ 
tion  from  object  structures  in  the  plane  immediately  above  can  be 
obtained  by  a  two-dimensional  convolution: 

Oj+i  ®  psfj+i  (6) 

where,  assuming  that  image  planes  are  recorded  a  spacing  Az  apart, 
psfj+ 1  denotes  the  values  of  the  PSF  in  the  next  image  plane,  Az 
from  the  focus.  To  calculate  this  contribution  we  would  need  to 
know  Oj+ 1  (in  addition  to  the  three-dimensional  PSF),  but  we  only 
have  image  data  at  this  point.  We  therefore  make  a  second  approx¬ 
imation  (which  is  no  worse  than  the  first)  that  o;+1  =  ij+l.  Thus,  a 
fraction  of  this  blurred  nearest-neighbor  data  is  then  subtracted 
from  the  data  recorded  in  the  current  image  plane,  or,  as  an 
equation: 

Oj  =  [ij  -  c(iH  ®  psfj-i  +  iJ+ 1  ®  psfj+ 1)]  (7) 

where  the  index  j  identifies  the  current  plane,  /  -  I  and  j  +  1  the 
planes  immediately  below  and  above,  o  and  i  are  the  deconvolved 
object  and  recorded  image,  respectively,  and  c  is  a  constant  <1. 
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This  algorithm  increases  in-plane  contrast  and  blurring  in  the  focal 
plane  can  also  be  reduced  by  applying  a  2D  inverse  filter  of  the 
in-focus  PSF  ( psff ').  For  further  details  see  Agard  and  colleagues 
(1989): 

Oj  =  [ij  -  c(ij. i  <E>  psfH  +  ij+1  ®  psfj+ 1)]  ®  psfj1  (8) 

Note  that  all  the  convolutions  are  two-dimensional,  making 
this  method  suitable  for  real-time  implementation  with  FFTs.  On 
the  other  hand,  one  should  not  be  surprised  that  the  enhancement 
obtained  with  this  method  is  only  moderate  and  (most  important) 
not  quantitative.  These  limitations  reside  in  the  subtractive  nature 
of  the  process  which  will  add  noise  and  alter  signal  amplitudes. 
Nevertheless,  where  qualitative  imaging  at  high  speed  is  required, 
useful  increases  in  in-plane  contrast  can  be  achieved  with  this 
method.  For  better  results  one  needs  to  resort  to  fully  three- 
dimensional  algorithms. 

An  even  faster  implementation  can  be  made  by  assuming  that 
the  data  in  adjacent  planes  is  the  same  as  that  in  the  image  plane, 
that  is,  ij± i  =  ij.  In  this  no-neighbors  method,  the  PSF  is  used  to 
subtract  a  blurred  contribution  of  the  image  plane  from  itself. 
Strictly  speaking,  this  is  not  a  deconvolution  method  at  all  but  is 
formally  equivalent  to  an  unsharp-mask  high-pass  filter.  Clearly, 
the  gross  assumption  underlying  this  approach  limits  its  applica¬ 
bility  to  very  sparse  three-dimensional  objects  and  any  intensity 
quantitation  would  be  out  of  the  question. 


Wiener  Filtering 

Turning  now  to  full  three-dimensional  approaches,  as  noted  above 
the  object  can  be  deconvolved  by  inverse  filtering,  that  is,  direct 
division  of  the  Fourier  transform  of  the  data  by  the  OTF: 


F{o}  = 


m 

OTF 


(9) 


In  this  equation,  the  low  values  of  the  OTF  in  the  region  of  the 
missing  cone  will  increase  noise  in  the  restored  image.  Put  another 
way,  the  data  which  is  lost  in  the  missing  cone  cannot  be  replaced 
unless  we  supply  some  other  information  that  acts  as  a  constraint 
on  the  restoration.  Nevertheless,  while  the  amplification  of  noise 
makes  direct  inversion  nearly  useless  (at  least  for  microscopic 
images),  it  can  be  improved  by  a  simple  modification  (Shaw  and 
Rawlins,  1991): 


Nonlinear  Constrained  Iterative 
Deconvolution  Algorithms 

Why  use  nonlinear  methods?  The  problem  resides  in  the  fact  that 
linear  methods  cannot  guarantee  that  the  deconvolved  object  is 
positive.  Rather,  in  the  presence  of  noisy  image  data,  some  pixels 
in  the  deconvolved  object  will  assume  negative  values,  a  result 
which  is  obviously  incorrect.  For  fluorescent  images  all  photon 
fluxes  must  be  positive  so  nonlinear  algorithms  that  ensure  no  neg¬ 
ative  data  values  in  the  reconstructed  object  are  clearly  preferable. 
Such  positivity  constraints  can  be  easily  enforced  within  iterative 
algorithms.  In  iterative  algorithms,  the  deconvolved  object  is  not 
computed  in  a  single  pass  (as  is  the  case  in  nearest-neighbor  or 
Wiener  filtering)  but  the  object  is  calculated  in  a  series  of  steps 
that  are  then  repeated.  Each  step  results  in  a  new  estimate  of  the 
object  that  should  be  somewhat  closer  to  the  actual  object.  During 
this  process,  positivity  can  be  enforced  by  explicitly  setting  nega¬ 
tive  values  in  the  current  estimate  to  zero.  The  small  error  that  is 
introduced  by  this  clipping  procedure  should  be  corrected  in  the 
next  iteration  and,  with  a  well-behaved  algorithm,  the  estimate 
converges  to  a  nonnegative  solution  that  closely  approximates  the 
true  object.  After  a  certain  number  of  steps,  the  process  is  termi¬ 
nated  by  the  user  to  yield  the  deconvolved  data. 

All  iterative  algorithms  start  from  an  initial  estimate  and  we 
will  discuss  reasonable  starting  choices  briefly  later.  The  algorithm 
then  enters  the  iterative  loop  in  which  a  new  estimate  is  calculated 
from  the  current  estimate  and  the  recorded  image  data.  The  posi¬ 
tivity  constraint  is  then  enforced  by  setting  all  negative  pixels  in 
the  new  estimate  to  zero.  New  estimates  are  calculated  repeatedly 
using  this  scheme  until  the  user  (or  the  software  implementation 
that  he  uses)  decides  to  terminate  the  computation  and  accept  the 
last  calculated  estimate  as  the  deconvolved  object. 

Expressed  as  a  piece  of  pseudo  code: 

1 .  Pick  a  starting  estimate  o° 
a.  Calculate  OTF  (if  blind) 

2.  Compute  an  initial  new  estimate  (ok+l)  from  previous  estimate 
ok,  recorded  image  data  i  and  OTF 

3.  Set  all  negative  pixels  in  ok+l  to  zero  to  give  the  current  best 
estimate  and  apply  any  other  constraints 

a.  If  blind,  estimate  a  new  OTF  from  F(ij/F(ok+lj  applying  any 
constraints  such  as  NA,  symmetry  and  nonnegativity. 

4.  Go  to  step  2  unless  a  stop  criterion  is  reached 


F{o }  =  F{i}  x 


OTF 

OTF2  +  K 


(10) 


where  K  is  an  additive  correction  factor  (see  Figure  25.7).  By 
adding  K,  noise  amplification  will  be  limited  where  the  OTF  is 
small.  It  can  be  seen  that  if  K  =  0,  the  equation  reduces  to  the  orig¬ 
inal  inverse  filter  (Eq.  9).  The  impact  of  K  can  be  reduced  in  the 
low  frequency  domain  (where  noise  is  less  of  a  problem)  by 
making  it  depend  on  the  spectral  characteristics  of  noise  and  object 
data.  In  this  case,  K  can  be  mathematically  optimized  and  this 
optimal  filter  is  called  the  Wiener  filter.  Unfortunately,  Wiener 
filters  do  not  generally  provide  optimal  image  quality  because:  (1) 
suppression  of  any  part  of  OTF  during  restoration  affects  the  entire 
image,  (2)  where  the  OTF  is  small,  noise  will  be  amplified,  possi¬ 
bly  to  an  unacceptable  degree,  and  (3)  small  errors  or  uncertain¬ 
ties  in  the  OTF  will  lead  to  large  errors  in  the  reconstruction. 
Generally,  for  microscopic  imaging  purposes,  Wiener  and  similar 
linear  filters  have  been  superseded  by  nonlinear  iterative  decon¬ 
volution  algorithms  that  have  become  more  applicable  with 
increasing  computer  power. 


Differences  between  iterative  algorithms  primarily  arise  from 
the  way  in  which  the  new  estimate  is  calculated  (step  2). 

To  calculate  a  better  estimate  of  the  object,  we  must  have  some 
way  to  find  out  if  our  current  object  guess  (ok)  is  good  (or  not). 
Ideally,  the  estimate  blurred  by  the  PSF  would  be  the  same  as  our 
recorded  image  i  except  for  some  small  difference  due  to  the  noise 
in  the  image  data.  Following  this  idea,  we  can  calculate  an  error 
signal  as  the  difference  between  our  blurred  guess  and  the  actual 
image  data.  So  to  update  the  estimate  we  can  take  a  fraction  of 
error  signal  and  add  it  to  the  current  estimate  (a  Newton  iterative 
approach).  This  approach  underlies  the  Jansson-van  Cittert  algo¬ 
rithm  (Jansson  et  al.,  1976).  As  an  equation: 

ok+1  =  ok  +  y(i  -  ( ok  ®  psf))  (11) 

and  with  a  good  choice  for  y  (which  can  vary  across  the  image) 
this  scheme  should  lead  to  successively  better  estimates.  However, 
in  early  implementations  the  choice  of  y  was  rather  ad  hoc 
(Jannson,  1970;  Agard  et  al.,  1989)  and  noise  leads  to  uncertainty 
as  to  whether  the  algorithm  actually  achieves  the  optimal  solution. 


Image  Enhancement  by  Deconvolution  •  Chapter  25  497 


Ill  more  recent  iterative  algorithms,  the  updating  scheme  is 
derived  from  more  stringent  mathematical  reasoning.  One  class  of 
algorithms  that  work  well  in  practice  are  called  maximum  likeli¬ 
hood  algorithms.  The  idea  behind  this  approach  is  based  on  the 
notion  of  the  likelihood  of  a  certain  object  o  to  underlie  our 
recorded  data  i.  In  this  context,  we  can  view  the  deconvolution 
problem  as  a  multi-dimensional  fitting  problem.  In  this  view  we 
anticipate  that  for  the  correct  choice  of  fitting  parameters  (actually 
the  pixels  values  in  o),  there  should  be  a  finite  probability  that  our 
data  should  have  been  observed.  Thus,  we  have  to  try  to  find  the 
object  o  which  maximizes  the  likelihood  that  our  data  would  have 
occurred  given  a  specific  noise  model  (which  is  required  to  math¬ 
ematically  calculate  these  probabilities).  Put  simply,  the  idea  is  to 
fit  a  function  (the  blurred  object  guess)  to  the  data  with  appropri¬ 
ate  weighting  for  noise  (which  should  be  a  concept  familiar  to  all 
experiment  scientists);  see  also  the  discussion  in  Press  and  col¬ 
leagues  (1992,  section  15.1). 

To  deal  with  noisy  fluorescence  data,  we  can  therefore  calcu¬ 
late  the  probability  that  our  data  would  have  occurred,  assuming 
the  noise  follows  a  Poisson  (or  some  other  noise  model)  distribu¬ 
tion.  Having  found  an  explicit  mathematical  expression  for  this 
probability  we  then  proceed  to  find  the  object  o  that  maximizes  this 
probability.  Instead  of  trying  to  find  this  maximum  directly  we  can 
make  life  a  bit  easier  by  looking  for  the  maximum  of  the  logarithm 
of  the  probability.  Due  to  the  properties  of  logarithms,  this  log- 
likelihood  is  maximal  at  the  same  location  that  the  likelihood  itself 
is  maximal,  however,  the  mathematical  expressions  generally 
become  easier  to  handle  after  taking  the  logarithm.  To  actually 
find  the  maximum,  mathematicians  use  the  observation  that  at  a 
maximum  the  derivative  of  the  log-likelihood  will  be  zero.  Cutting 
through  the  mathematical  derivation,  one  finally  arrives  at  an 
expression  for  the  updating  step  in  the  presence  of  Poisson  noise: 

oM  =okx  IF{  f\  — - - 1  x  OTF'l  (12) 

l  \ok®psf\  J 

where  IF{ }  is  the  inverse  Fourier  transform  and  OTF'  is  the 
complex  conjugate  of  the  OTF.  (The  Fourier  transform  yields  real 
and  imaginary  parts  so  the  OTF  is  complex.)  By  definition,  the 
complex  conjugate  of  a  complex  number  x  +  iy  is  x  -  iy.  Multipli¬ 
cation  by  the  complex  conjugate  in  Fourier  space  is  known  as  cor¬ 
relation.  Note  that  in  this  algorithm  we  do  not  use  the  additive  error 
7—  ok®  psf  but  rather  the  relative  difference  H(ok®  psf).  This  algo¬ 
rithm  is  also  known  as  the  Richardson-Lucy  algorithm  after  the 
authors  who  first  derived  this  scheme  for  astronomical  applications 
(Richardson,  1972;  Lucy,  1974).  For  an  example  of  the  image 
enhancement  possible  with  this  algorithm  see  Figures  25.8  and 
25.9.  Assuming  Gaussian  noise  in  the  data  (for  large  photon  fluxes, 
the  Poisson  noise  model  approaches  Gaussian  noise)  we  arrive  at 
an  alternative  updating  step  that  uses  the  additive  error  signal: 

ok+ 1  =ok  +  IF{F{i  -  ok® psf}  x  OTF'}  (13) 

With  both  types  of  maximum  likelihood  algorithms,  we  should 
stop  after  a  certain  number  of  iterations  for  best  results  as,  even¬ 
tually,  the  error  and/or  noise  starts  to  increase  again. 

This  brings  us  back  to  the  start  and  stopping  choices  that 
must  be  made  with  iterative  deconvolution  algorithms.  Reasonable 
choices  for  the  first  estimate  are  (a)  the  recorded  image  itself,  (b) 
a  constant  estimate  with  every  value  set  to  the  mean  of  the  recorded 
image,  and  (c)  a  smoothed  version  of  the  recorded  image.  Using 
the  Richardson-Lucy  maximum-likelihood  algorithm  (Eqs.  12, 
13)  choices  (b)  or  (c)  are  recommended.  A  more  detailed  discus¬ 
sion  can  be  found  in  van  Kempen  and  van  Vliet  (2000).  We  also 


need  a  criterion  of  when  to  stop  the  iterative  procedure  and  accept 
the  current  estimate  as  the  deconvolved  object.  A  reasonable 
approach  involves  monitoring  the  relative  change  between  esti¬ 
mates  as  the  iteration  proceeds.  As  the  relative  difference 

°k+i~°k  (14) 

°k 

between  subsequent  estimates  falls  below  a  chosen  threshold  (that 
generally  depends  on  the  amount  of  noise  in  the  images)  the  iter¬ 
ations  should  be  stopped.  For  example,  with  some  of  the  data 
shown  in  this  chapter,  thresholds  of  -lO”4  were  used. 

There  are  other  issues  that  should  be  considered  in  a  serious 
implementation  of  constrained-iterative  deconvolution  methods, 
for  example,  background  estimation  (van  Kempen  and  van  Vliet, 
2000)  and  prefiltering  of  data  and  PSF  (van  Kempen  et  al. ,  1997). 
The  other  approach  to  iterative  deconvolution  is  to  use  a  regular¬ 
ization  term,  often  the  Tikhonov  functional,  as  set  out  by  Tikhonov 
and  Arsenin  (1977),  which  aims  to  limit  the  generation  of  large 
intensity  oscillations  in  the  deconvolved  image  (which  will  grow 
as  the  loss  of  high  frequency  information  in  the  OTF  becomes  lim¬ 
iting).  A  number  of  similar  methods  based  on  this  approach  exist, 
and  these  vary  primarily  in  the  approach  taken  to  find  the  solution 
to  the  regularization  term  which  minimizes  the  signal  oscillations 
or  roughness  in  the  deconvolved  image.  One  such  solution  is  the 
iterative  constrained  Tikhonov-Miller  (ICTM)  algorithm,  which 
uses  a  minimization  strategy  based  on  conjugate  gradients  to  min¬ 
imize  the  Tikhonov  functional.  Tikhonov-regularized  forms  of  the 
Richardson-Lucy  algorithm  also  exist,  representing  a  consolida¬ 
tion  of  these  two  classes  of  solution. 

Comparisons  of  a  number  of  deconvolution  algorithms  (van 
Kempen  et  al .,  1997;  van  Kempen,  1999)  show  that  there  is  rela¬ 
tively  little  difference  between  their  various  deconvolved  results. 
The  regularized  algorithms  tend  to  be  less  sensitive  to  noise  than 
Richardson-Lucy-based  methods,  and  reach  an  appropriate  stop¬ 
ping  point  sooner.  However  the  RL  algorithm  provides  slightly 
better  deconvolution  results,  as  quantified  by  the  mean  square  error 
between  the  image  and  the  PSF-blurred  estimate  of  the  decon¬ 
volved  object.  A  useful  improvement  in  the  Richardson-Lucy 
method  can  be  achieved  by  prefiltering  the  image  and  PSF  with  a 
Gaussian  smoothing  filter  which  reduces  high  frequency  noise 
contributions  without  affecting  the  deconvolution  of  lower  fre¬ 
quency  image  components. 

One  other  variant  of  the  general  deconvolution  algorithm  is 
blind  deconvolution  (Holmes  and  O’Connor,  2000  and  Chapter  24, 
this  volume ),  where  the  deconvolution  is  performed  in  the  absence 
of  an  independently  measured  (or  calculated)  PSF.  Instead,  the  PSF 
is  iteratively  estimated  along  with  the  experimental  data  during  the 
deconvolution  process  simultaneously.  Proponents  of  blind  decon¬ 
volution  suggest  this  method  is  superior  to  using  PSF-dependent 
calculations  with  a  PSF  rendered  incorrect  by  sample-induced 
aberrations  and  scattering.  On  the  other  hand,  one  can  argue  that 
by  supplying  less  information  to  the  reconstruction,  the  final  result 
must  be  inferior,  if  the  PSF  was  known  to  sufficient  precision.  In 
addition,  if  the  blind  method  makes  use  of  symmetry  in  the  PSF 
and/or  the  NA  of  the  objective,  artifacts  will  be  produced  if  lens 
aberrations  are  present  and/or  the  assumed  NA  is  modified  by,  for 
example,  improper  illumination  of  the  rear  aperture  of  the  lens. 

Comparison  of  Methods 

A  common  question  is:  What  method  works  best?  Unfortunately, 
there  is  no  single  answer  because  all  imaging  methods  represent  a 


498 


Chapter  25  •  M.B.  Cannell  et  al. 


FIGURE  25.8.  Maximum  likelihood  deconvolution  of  the  cardiac  /-system.  (A)  shows  a  two-photon  image  of  a  single,  rat  cardiac  cell  in  dextran-linked  fluo¬ 
rescein.  The  fluorescein  has  filled  the  /-tubules  which  are  <~300nm  in  diameter.  (B)  shows  an  enlarged  view  from  a  part  of  (A).  Note  the  relatively  low  signal- 
to-noise  present  in  this  image  although  tubules  can  be  clearly  seen.  (C)  shows  the  restored  image.  Note  the  improvement  in  signal-to-noise  ratio  as  well  as  the 
increased  contrast.  There  has  also  been  a  useful  but  moderate  increase  in  spatial  resolution.  The  fine  structure  of  the  /-system  can  now  be  appreciated.  (D)  shows 
the  effect  of  deconvolving  the  image  with  an  incorrect  PSF.  At  first  sight,  the  results  appear  reasonable,  but  closer  inspection  reveals  the  presence  of  artifactual 
data  when  compared  to  (C).  In  view  of  the  general  problem  of  controlling  spherical  aberration,  without  good  controls  such  artifact  could  easily  lead  to  erroneous 
conclusions. 


FIGURE  25.9.  Enhancement  of  two-photon  images  by  deconvolution.  The  panel  at  the  left  shows  a  neuronal  dendrite  in  a  living  brain  slice.  The  limited  photon 
yield  leads  to  little  fine  structure  being  visible  in  the  volume-rendered  image.  After  maximum  likelihood  deconvolution,  synaptic  spines  are  clearly  visible  and 
even  appear  to  have  necks  (arrows).  In  this  case,  deconvolution  has  been  much  more  useful  to  control  noise  rather  than  increase  optical  resolution. 
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trade-off  between  the  three  fundamental  limits  in  information 
transfer  processes:  resolution  versus  speed  versus  noise.  Thus, 
improvements  in  one  area  will  usually  compromise  another  and  it 
is  up  to  the  experimenter  to  consider  what  is  actually  desired  of 
the  imaging  method.  This  principle  is  generally  limiting  even 
within  a  chosen  restoration  method,  because  once  resolution  has 
been  increased  to  a  certain  point,  further  gains  can  only  be  made 
by  compromising  noise  and/or  speed.  As  a  guide,  we  suggest  that 
deconvolved  confocal/multi-photon  images  provide  the  ultimate 
image  in  terms  of  resolution  (see  Figs.  25.8  and  25.9).  In  some 
cases,  deconvolved  WF  images  with  iterative  methods  can  achieve 
comparable  resolution  to  confocal/multi-photon  methods.  Nearest- 
neighbor  methods  provide  a  lower  level  of  image  improvement 
that  can  help  control  loss  of  contrast  in  WF  images  but  suffer  from 
their  non-quantitative  nature  (i.e.,  one  should  not  compare  bright¬ 
ness  between  different  parts  of  the  image).  Deconvolution  of  WF 
images  may  provide  a  resolution  approaching  that  of  confocal 
methods  but  would  be  problematic  in  scattering  specimens  or  in 
the  face  of  spherical  aberration. 

In  terms  of  speed,  nearest-neighbor  methods  are  fastest  with 
execution  times  for  interative-constrained  methods  being  compa¬ 
rable  regardless  of  whether  the  source  images  are  WF  or  confo¬ 
cal/multi-photon.  This  pecking  order  also  seems  to  apply  in  terms 
of  signal-to-noise  ratio  (including  errors  in  absolute  signal  levels) 
with  nearest-neighbor  methods  being  worst  and  iterative  con¬ 
strained  methods  being  best.  In  terms  of  noise,  WF  methods  are 
initially  superior  until  resolution  starts  to  become  limiting,  at 
which  point  differences  between  methods  largely  disappear. 

Finally,  because  even  after  deconvolution,  image  quality  is 
critically  dependent  on  the  quality  of  the  input  data,  we  suggest  20 
tips  for  better  imaging: 

1.  Store  unused  objectives  lens  down,  clean,  and  dry  in  the 
manufacturer’s  case. 

2.  Before  using  an  objective,  inspect  it  with  a  microscope  eye¬ 
piece  for  cleanliness  and  damage  —  note  any  scratches  across 
the  metal  front  of  the  lens  —  possibly  resulting  from  colli¬ 
sions  with  stage  or  specimens.  If  spring-loaded,  check  the 
objective  front  is  not  locked  down. 

3.  Check  the  coverslips  for  thickness  (check  the  coverslip 
requirement  on  the  microscope  barrel,  usually  0.17  mm)  and 
that  they  are  suitable  for  the  chosen  objective.  If  the  objec¬ 
tive  has  a  correction  collar,  use  it! 

4.  Get  a  supply  of  subresolution  fluorescent  beads  in  a  wide 
range  of  excitation  and  emission  wavelengths  to  measure  the 
PSF  and  test  microscope  performance.  Mount  them  in  stan¬ 
dard  slides  (avoid  mountants  with  harsh  solvents);  you  can 
also  purchase  slides  already  made  up. 

5.  Oversample  the  PSF  and  subsequently  resample  to  image  res¬ 
olution  by  applying  a  Gaussian  filter  (to  reduce  noise)  fol¬ 
lowed  by  a  cubic-spline  interpolation.  If  appropriate,  apply  a 
circular-symmetry  constraint  on  the  PSF  to  further  reduce 
noise. 

6.  Add  a  small  quantity  of  beads  to  samples  before  sealing 
the  coverslip  to  enable  visual  inspection  of  the  PSF  in 
widefield. 

7.  Always  seal  wet-mounted  specimens  to  prevent  them  drying 
out. 

8.  Minimize  spherical  aberration  by  appropriate  selection  of 
objective  lenses/mounting  media.  If  the  objective  has  a  cor¬ 
rection  collar,  adjust  it  to  make  the  observed  PSF  as  sym¬ 
metrical  as  possible.  This  can  be  achieved  by  watching  the 
bead  image  as  the  microscope  is  slightly  defocused.  The  Airy 


disk  should  grow  symmetrically  in  both  directions  with 
defocus  (see  Fig.  20.3,  this  volume). 

9.  If  a  high-power  oil  lens  must  be  used  with  a  water-based  spec¬ 
imen,  try  raising  the  refractive  index  of  the  specimen  with 
sucrose,  mannitol,  or  salt  if  osmotic  pressure  is  not  a  problem, 
but  be  careful  not  to  allow  crystal  formation.  It  should  be 
noted  that  an  84%  weight/volume  sucrose  solution  has  the 
same  refractive  index  as  glass  (~1.5).  Alternatively,  use 
immersion  oil  having  a  refractive  index  >1.52. 

10.  In  confocal  applications,  always  open  the  pinhole  slightly  to 
increase  light  flux  while  maintaining  a  reasonable  degree  of 
confocality  to  achieve  the  desired  axial  resolution. 

11.  Use  the  highest  voxel  resolution  possible  in  xy  and  z,  but  try 
to  avoid  bleaching  problems. 

12.  Minimize  bleaching  problems  with  antifade  mountants  and 
latest-generation  fluorochromes. 

13.  If  imaging  multiple  wavelengths  sequentially,  image  the 
longest  wavelength  fluorochromes  first. 

14.  Ascertain  the  minimum  and  maximum  extents  of  the  speci¬ 
men  at  the  lowest  possible  illumination  intensity  (with  the 
pinhole  quite  open  in  confocal  applications).  The  idea  is  not 
to  produce  a  nice  image  at  this  point  but  simply  detect  where 
the  fluorescent  specimen  resides. 

15.  Keep  a  copy  of  the  lens  PSF  with  the  image  data  and  record 
voxel  dimensions,  objective,  mounting  media  (some  imaging 
software  allows  comments  to  be  placed  in  the  image  header). 

16.  Image  stacks  will  usually  fit  within  a  single  CD-ROM  (do  not 
use  any  data  compression  unless  you  are  sure  it  is  lossless) 
so  make  a  CD-ROM  copy  of  the  data  before  leaving  the 
laboratory. 

17.  Avoid  mechanical  artifacts  during  imaging  by  keeping 
acoustic  noise  down,  the  door  closed,  and  avoiding  touching 
any  part  of  the  microscope  system.  Vibration  isolation 
systems  may  be  essential  in  laboratories  with  suspended 
floors.  Air-cooled  laser  system  fans  should  be  mechanically 
decoupled  from  the  microscope  with  a  duct  to  a  separate 
exhaust  fan. 

18.  Use  dimmable  incandescent  lighting  in  the  laboratory  and 
dim  the  lights  while  collecting  data.  On  inverted  microscopes, 
cover  the  specimen  with  a  black  cup. 

19.  Use  a  lookup  table  that  makes  it  easy  to  spot  over-  and  under¬ 
flows  in  the  data.  Make  sure  the  black  level  of  the  imaging 
system  is  set  to  >0;  this  is  even  more  important  than  having 
a  few  pixels  saturated. 

20.  Always  have  sample  slides  containing  beads  on  hand  and 
always  check  the  microscope  image  quality  with  them  before 
starting  imaging,  every  day. 

21.  Deconvolve  everything! 
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Fiber-Optics  in  Scanning  Optical  Microscopy 

Peter  Delaney  and  Martin  Harris 


INTRODUCTION 

Fifteen  years  ago,  exploitation  of  fiber-optics  in  scanning  confo- 
cal  optical  microscopy  was  minimal.  Most  practical  applications 
relied  upon  technology  derived  quite  directly  from  telecommuni¬ 
cations  applications  (particularly  the  types  of  fibers  themselves 
and  the  methods  by  which  they  were  produced,  packaged,  and 
interfaced). 

Since  that  time,  there  has  been  increasing  acceptance  of  fiber 
components  as  illumination  and  detection  elements  in  commercial 
scanning  microscopes,  and  a  marked  increase  in  development  of 
basic  fiber  technologies  for  scanning  microscopy  applications. 
Substantial  progress  has  also  been  made  in  developing  entirely 
new  classes  of  highly  miniaturized  scanning  confocal  microscopes 
made  possible  only  by  fiber  technologies. 

Invention  and  production  of  more  specialized  optical  fibers  and 
fiber-optic  devices  has  not  only  increased  the  practical  opportunity 
to  replace  many  of  the  conventional  optical  components  with  their 
more  flexible  fiber-optic  counterparts,  but  in  many  cases  has 
yielded  completely  novel  and  unique  fiber  components.  Beyond 
the  use  of  fiber-optic  elements  as  efficient,  flexible  light  conduits, 
fibers  are  now  available  that  can  act  as  spectral,  spatial,  and  polar¬ 
ization  filters,  beam-splitters,  lasers,  and  nonlinear  devices  for 
delivering  light  from  pulsed  lasers. 

This  chapter  will  introduce  some  of  the  relevant  theoretical  and 
practical  parameters  of  fiber-optics,  discuss  these  in  the  context  of 
the  role  of  fibers  performing  key  functions  within  scanning  micro¬ 
scopes,  then  move  to  a  system-focused  level  that  reviews  the  types 
of  instruments  that  use  fiber  components  and  their  capabilities  and 
performance. 

KEY  FIBER  TECHNOLOGIES  RELEVANT  TO 
SCANNING  MICROSCOPY 

Optical  fibers  with  properties  relevant  to  confocal  microscopy 
range  in  complexity  from  slight  variants  of  standard,  telecom¬ 
munications  fiber  through  to  novel  fibers  exploiting  complex 
microstructures.  An  understanding  of  fiber  design  and  function  is 
pivotal  to  understanding  their  exploitation  in  scanning  micro¬ 
scopes.  We  now  overview  the  particular  aspects  of  various  fiber 
technologies  that  define  their  roles  in  various  implementations  of 
confocal  microscopy. 


Glass  Made  from  Gas  and  Its 
Transmission  Properties 

Fiber  technology  is  largely  founded  on  production  of  ultra-pure 
glasses.  This  is  achieved  by  creating  glass  from  gas  by  bubbling 
hydrogen,  oxygen,  and  helium  through  containers  of  volatile 
silicon  tetrachloride,  germanium  tetrachloride,  and,  often,  phos¬ 
phorus  and  boron  halide  compounds.  The  vapor  is  burned  to  form 
a  chalky,  white  deposit  and  further  heating  collapses  this  to 
produce  an  ultra-clear  glass  rod  known  as  a  preform.  During  this 
process,  the  composition  of  the  vapor  is  varied  to  produce  a  spe¬ 
cific  cross-sectional  refractive  index  profile.  The  preform  is  then 
melted  and  drawn  down  into  fiber,  jacketed  for  protection,  and 
wound  onto  drums.  Importantly,  the  drawing  process  preserves  the 
cross-sectional  profile  of  composition  and  associated  refractive 
index  (RI),  and  these  are  captured  into  the  glass  preform  during  its 
fabrication.  Thus,  simple  or  complex  RI  profiles  can  be  made  in 
the  macroscopic  domain  (over  many  millimeters)  of  the  preform, 
and  subsequently  become  microscopic  during  the  drawing  process. 

For  the  glass  compositions  most  commonly  used  in  optical 
fibers,  the  purity  ensures  that  there  is  virtually  no  absorption  of 
light  during  transmission  for  almost  the  entire  wavelength  range 
from  350  nm  to  1600nm  (the  transmission  curve  for  silica  glass  is 
shown  in  Fig.  26.1).  The  predominant  loss  mechanism  through  this 
wavelength  range  is  Raleigh  scattering. 

This  transmission  efficiency  can  be  difficult  to  appreciate 
when  working  with  the  relatively  short  lengths  of  fiber  used  for 
microscopy  applications.  Although  at  wavelengths  commonly  used 
in  scanning  microscopy,  losses  in  silica  glass  are  orders  of  magni¬ 
tude  higher  than  at  the  wavelengths  for  which  telecommunications 
fiber  is  commonly  designed,  telecommunications  requires  trans¬ 
mission  over  the  50  to  80km  distances  between  repeater  stations. 
Thus,  even  for  blue  light  over  distances  of  several  meters,  losses 
are  a  fraction  of  a  percentage.  Inefficiencies  in  the  optics  used  to 
launch  the  input  (so-called  insertion  losses)  or  project  the  fiber 
output  (e.g.,  excluding  part  of  the  Gaussian  beam  profile)  are  much 
more  significant.  If  improperly  managed,  these  losses  can  lead  to 
the  perception  that  fiber  propagation  is  lossy. 

Step  Index  and  Gradient  Index  Optical  Fibers 

Some  of  the  simplest  optical  fiber  designs  are  comprised  of  a  core 
of  glass  surrounded  by  a  cladding  of  a  different  glass  having  a 
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FIGURE  26.1.  Transmission  losses  in  silica  glass 
optical  liber  versus  wavelength:  (A)  material  bandgap 
absorption  of  silica;  (B)  material  bandgap  absorption 
of  germania;  (C)  Raleigh  scattering  contributes  loss 
mechanisms,  which  form  bounding  asymptotes  to  the 
overall  shape  of  this  spectrum;  (D)  infrared  (IR) 
photon  interactions;  (E)  absorption  by  residual  water 
(OH  bonds)  in  the  glass  superimpose  several  peaks  in 
the  spectrum;  and  (F)  transition  metal  absorption 
bands. 
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lower  refractive  index  (a  so-called  step  index  optical  fiber).  This 
forms  a  closed,  cylindrical  refractive  index  boundary  that  serves 
to  entrap  and  propagate  light  by  total  internal  reflection. 

One  of  the  properties  of  step  index  optical  fibers  is  that  the 
light  that  converges  onto  the  input  surface  at  higher  angles  must 
traverse  a  greater  effective  path  length  before  it  reaches  the  exit 
surface  than  light  incident  at  lower  angles.  Hence,  any  pulse  of 
light  will  be  temporally  dispersed.  To  overcome  this  limitation, 
gradient  index  optical  fibers  were  developed  (Wilson  and  Hawkes, 
1989a).  Gradient  index  fibers,  as  their  name  suggests,  possess  a 
gradient  of  RI  from  highest  (at  the  center  of  the  core)  to  that  of  the 
cladding  [as  depicted  in  Fig.  26.3(B)].  Rather  than  being  reflected 
at  an  abrupt  core/cladding  boundary,  the  wavefront  of  light  is 
steered  continuously  as  it  traverses  the  RI  gradient.  Light  travels 
faster  through  the  outer,  lower  RI  medium,  and  hence  tends  to 
keep  up  with  light  propagating  in  the  inner  regions  of  the  core, 
resulting  in  less  temporal  dispersion  of  the  wavefronts  of  light 
propagated. 

Modes  in  Optical  Fibers 

A  propagating  wavefront  can  interfere  constructively  or  destruc¬ 
tively  with  totally  internally  reflected  regions  of  the  same  wave- 
front  traveling  across  the  fiber  at  a  slightly  different  angle.  Where 
this  interference  is  constructive,  the  light  propagates  efficiently 
along  the  fiber.  Hence,  propagation  of  wavefronts  at  a  particular' 
wavelength  is  only  supported  for  specific  paths  or  angles  within 
the  fiber.  These  are  called  modes. 

The  modes  that  a  given  fiber  will  support  are  a  function  of  the 
diameter  of  the  fiber  core  d,  the  wavelength  of  light  X,  and  the  RIs 
of  the  core  and  cladding,  as  well  as  the  RI  profile  itself  (in  the  case 
of  gradient  index  fiber).  An  important  parameter  incorporating  all 
these  factors  is  called  the  V-number.  The  larger  the  V-number,  the 
more  modes  a  fiber  will  support,  down  to  a  cutoff  value  of  V  = 
2.405,  at  which  only  a  single,  fundamental  mode  can  be  propa¬ 
gated  by  the  fiber  (Wilson  and  Hawkes,  1989b).  This  mode  can  be 
launched  by  focusing  a  train  of  converging  concentric  wavefronts 
to  an  appropriate  diffraction-limited  spot  on  the  end  of  the  core, 
and  the  light  will  then  essentially  propagate  as  a  train  of  parallel 
wavefronts,  as  depicted  in  Figure  26.2.  The  light  emergent  at  the 
output  of  such  a  fiber  will  also  form  a  coherent  virtual  point  source, 
radiating  a  train  of  diverging,  concentric  wavefronts  analogous  to 
laser  light  diverging  from  the  waist  region  to  which  it  has  been 
focused,  but  with  the  numerical  aperture  (NA)  of  the  fiber  deter¬ 
mining  the  angle  of  divergence.  As  will  be  discussed  later,  this 


ability  of  single-mode  fibers  to  maintain  coherence  has  many 
important  properties  relevant  to  their  use  in  confocal  microscopy. 
Importantly,  when  the  V-number  falls  below  the  single-mode 
cutoff,  the  fiber  still  guides  the  fundamental  mode.  Note  that  V- 
number  is  related  to  wavelength  in  such  a  way  that  a  fiber  is  single 
mode  for  all  wavelengths  longer  than  the  wavelength  associated 
with  the  single-mode  cutoff.  Thus,  a  fiber  can  be  single  mode 
across  a  range  of  wavelengths. 

Modes  can  be  visualized  by  viewing  the  far  field  projection  of 
light  emerging  from  a  fiber  (i.e.,  the  appearance  of  the  light  output 
falling  onto  a  piece  of  paper).  For  example,  a  large-cored  step 
index  fiber  [such  as  depicted  in  Fig.  26.2]  will  be  heavily  multi- 
mode  and  may  support  hundreds  or  thousands  of  modes,  which 
will  appear  as  very  fine  spots  that  move  rapidly  with  any  distur¬ 
bance  to  the  fiber.  Smaller-cored  gradient-index  fiber  may  support 
just  a  few  modes  and  these  will  appear  as  characteristic  lobes.  The 
output  of  single-mode  fiber  appears  as  essentially  a  radially  sym¬ 
metric  Gaussian-distributed  disk. 


Output  if 
all  modes 


FIGURE  26.2.  Diagrammatic  representation  of  key  differences  between  light 
propagation  in  step  index  multi-mode  fiber  and  gradient  index  single-moded 
fiber.  In  a  large  core  step  index  multi-moded  fiber,  the  superset  of  all  mode 
fields  are  confined  to  a  cone  which  defines  the  NA  of  the  fiber.  The  various 
modes  allow  propagation  of  light  involving  low  through  high  angles  of  total 
internal  reflection.  It  is  possible  to  couple  laser  light  into  subsets  of  these 
modes.  In  the  example  depicted  here,  launching  of  laser  light  at  lower  than  the 
maximum  NA  launches  lower  order  modes,  resulting  in  fiber  output  that  is  like¬ 
wise  lower  in  NA  than  that  defined  by  the  full  set  of  modes  supported  by  the 
fiber. 
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Evanescent  Wave  and  Polarization  Effects  in 
Optical  Fibers 

When  light  is  guided  by  total  internal  reflection  from  an  RI  gradi¬ 
ent,  a  portion  of  the  electromagnetic  wave,  called  the  evanescent 
wave,  travels  on  the  “far  side”  of  the  boundary  (i.e.,  in  the 
cladding)  relative  to  where  the  mode  is  traveling.  Correspondingly, 
an  appreciable  amount  of  the  optical  energy  carried  by  a  fiber  is 
transmitted  by  the  evanescent  wave  in  the  cladding  surrounding 
the  core,  even  for  a  step-index  core. 

The  phase  shift  and  lateral  displacement  that  takes  place  when 
light  is  guided  by  the  refractive  index  gradient  is  different  for  the 
two  orthogonal  polarization  planes  of  the  light.  The  distance  that 
the  evanescent  wave  penetrates  into  the  cladding  depends  on  the 
angle  of  incidence  and  the  polarization  state. 

Light  from  lasers  is  usually  linearly  polarized.  Conventional 
optical  fiber  has  radial  symmetry  and  if  this  symmetry  were  perfect 
and  the  fiber  was  held  straight,  then  the  polarization  plane  would 
not  be  changed  during  transmission.  In  practice,  neither  of  these 
two  conditions  holds,  so  that  the  polarization  plane  of  the  emerg¬ 
ing  light  may  be  rotated  or  it  may  be  converted  into  a  left-  or  right- 
handed  circular  or  elliptically  polarized  beam.  This  is  essentially 
uncontrolled  in  circular  cross-section  fiber. 

Polarization-Maintaining  Fibers 

There  are  several  designs  of  optical  fiber  that  allow  polarization  to 
be  maintained.  It  is  either  possible  to  make  a  fiber  with  an  ellipti¬ 
cal  core  cross-section  or  one  with  a  circular  core  that  is  stressed 
by  structures  within  the  fiber  but  outside  of  the  light-carrying  core. 
These  are  made  of  a  different  glass  type,  which  contracts  more  and 
produces  stress  in  the  core  as  the  molten  fiber  cools.  This  stress 
produces  a  birefringence  causing  the  speed  of  light  traveling  in  the 
fiber  in  one  polarization  plane  to  differ  from  the  speed  in  the  other 
direction  (by  about  1  wavelength  every  millimeter  of  travel).  This 
distance,  called  the  beat  length,  is  equivalent  to  a  retardation  in 
crystals  and  wave  plates  of  one  full  wavelength.  The  phase 
between  the  two  orthogonal  polarization  propagation  modes  thus 
changes  back  and  forth  and  the  coupling  of  energy  from  one  polar¬ 
ization  mode  that  occurs  in  one  half  beat  length  is  reversed  in  the 
next  half  beat  (hence  little  net  change  results).  These  so-called  high 
birefringence  (Hi-Bi)  fibers  can  be  bent  without  altering  the  polar¬ 
ization  state  of  the  output  light.  Two  common  designs  of  stressed 
fiber  are  called,  descriptively.  Panda  and  Bow  Tie  fibers  (see  dia¬ 
grams  in  Fig.  26.3).  The  Panda  fiber  is  made  by  drilling  holes  in 
the  preform  on  either  side  of  the  core  and  inserting  rods  of  glass 
having  a  higher  thermal  expansion  coefficient  before  the  fiber  is 
drawn.  The  Bow  Tie  and  elliptical  core  structures  are  made  as 
the  preform  is  deposited  from  the  gas.  When  using  polarization- 
maintaining  fibers  in  some  applications,  it  is  critically  important 


to  ensure  that  the  laser  launch  polarization  is  aligned  with  either 
the  fast  or  the  slow  axis  of  the  fiber.  Rotating  the  fiber  exit  tip  will 
rotate  the  polarization  plane  by  exactly  the  same  angle  and  this  can 
be  employed  to  align  polarization  when  setting  up  experiments.  It 
also  should  be  pointed  out  that  each  of  the  modes  in  a  multimode 
fiber  can  have  its  own  separate  linear  polarization  state.  Failure  to 
perform  alignment  correctly  can  cause  large  variations  (commonly 
from  30%  to  80%,  but  up  to  100%)  in  the  delivery  of  the  desired 
polarization  mode  into  the  microscope.  In  some  cases,  this  can  be 
monitored  by  direct  power  measurements  at  the  objective,  or,  more 
generally,  by  evaluating  the  images  of  a  stable  test  specimen  such 
as  a  mirror.  Note  that  swings  in  polarization  mode  might  not  be 
measurable  directly  at  the  fiber  output,  as  the  light  may  still  be 
coupled  into  a  non-desired  polarization  mode,  and  the  polarization 
change  may  only  become  evident  as  an  intensity  change  after  the 
light  interacts  with  microscope  or  detection  components  that  are 
polarization  dependent.  Any  significant  swings  in  these  observa¬ 
tions  (more  than  a  few  percentage  points)  should  flag  the  possible 
need  to  realign  the  fiber  launch  and/or  alignment  of  the  fiber  output 
to  the  microscope. 

Fused  Biconical  Taper  Couplers:  Fiber-Optic 
Beam-Splitters 

If  two  sections  of  optical  fiber  are  fused  together  and  then  stretched 
while  still  viscous,  the  cores  are  tapered,  thinned  out,  and  brought 
very  close  together.  Each  core  overlaps  significantly  with  the 
evanescent  wave  associated  with  light  guided  by  the  other.  This 
allows  light  energy  to  be  transferred  from  one  core  to  the  other, 
forming  a  type  of  beam-splitter.  Such  devices  are  called  fused 
biconical  taper  couplers  (FBTCs)  [Fig.  26.4(A)],  and  may  fit  into 
a  capsule  the  size  of  a  match.  They  have  been  manufactured  with 
net  optical  power  losses  of  less  than  0.01%.  The  theory  covering 
the  division  of  the  light  is  not  intuitive  and  is  covered  in  Snyder 
and  Love  (1983).  These  couplers  can  produce  a  50 : 50  division  of 
the  the  light  or  any  other  desired  ratio. 

Although  the  split  ratio  of  any  FBTC  varies  with  X,  couplers 
can  be  fabricated  to  be  relatively  ^-independent  or  highly  X- 
dependent.  In  the  latter  case,  they  can  be  used  as  dichroic  beam¬ 
splitters  [Figure  26.4(B)],  They  can  also  be  made  to  split  the 
incoming  light  from  one  fiber  so  that  light  with  orthogonal  polar¬ 
ization  modes  emerges  from  the  two  output  legs  of  the  coupler. 
Alternatively,  they  can  split  light  in  a  ratio  that  is  independent  of 
the  polarization  plane  (Wilson  and  Hawkes,  1989c). 

FBTCs  can  be  used  to  combine  the  light  from  separate  laser 
sources  and  they  can  also  function  as  beam-splitters  in  confocal 
microscopy,  where  they  offer  additional  modularity  by  providing 
separate  fiber  arms  for  laser  launch,  fiber  output,  and  signal  detec¬ 
tion.  They  have  also  been  a  key  component  in  making  optical 
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FIGURE  26.3.  Polarization  maintaining  fiber  core  patterns.  Various  means  are  pictured  whereby  specific  core  profiles  and  stress  structures  are  fabricated  into 
a  fiber  preform  prior  to  drawing.  This  results  in  an  orthogonally  biased  pair  of  modes,  each  favoring  one  linear  polarization  plane. 
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FIGURE  26.4.  Fused  biconical  taper  couplers.  The  general  configuration  is 
shown  in  (A),  whereby  two  fibers  are  melted  together  and  drawn,  forming  a 
waist  region  in  which  the  mode  fields  guided  by  the  cores  overlap  and  exchange 
energy.  Laser  light  coupled  into  any  arm  of  the  coupler  will  be  apportioned  in 
the  two  opposing  arms  in  a  ratio  that  is  dependent  upon  the  wavelength  char¬ 
acteristics  of  the  coupler,  which  can  be  varied  by  design.  An  example  is  given 
in  (B)  for  a  coupler  with  strong  wavelength  dependence  that  could  be  used,  for 
example,  to  provide  a  50 : 50  split  for  laser  illumination  at  488  nm,  while  favor¬ 
ing  the  transmission  of  confocal  green  fluorescence  emission  on  one  particu¬ 
lar  arm.  Thus,  the  coupler  could  be  used  as  for  laser  illumination,  pinhole 
detection,  and  a  dichroic  beam-splitter  in  a  fiber-based  confocal  microscope. 


coherence  tomography  a  useful  in  vivo  imaging  technique,  where 
they  provide  the  beam-splitting  and  recombination  paths  for  the 
requisite  interferometer  (OCT;  see  also  Chapter  34,  this  volume). 
One  of  the  advantages  in  these  applications  is  that  fiber-based 
interferometers  are  not  troubled  by  dust,  as  are  conventional  inter¬ 
ferometers. 


Microstructure  Fibers 

New  designs  of  fibers,  collectively  called  microstructure-core 
fibers  (but  also  referred  to  as  photonic  crystals,  air-clad  fibers,  or 
endlessly  single-mode  fibers,  etc.)  have  recently  been  introduced. 
The  cross-sectional  views  of  several  such  designs  are  presented  in 
Figure  26.5.  Rather  than  exploiting  a  simple  radially  symmetric 
refractive  index  profile  to  guide  light,  these  fibers  consist  of 
complex  patterns  of  air  holes  and  glass  fins. 

These  structures  give  rise  to  unique  optical  properties,  in  par¬ 
ticular  nonlinear  propagation  effects  that  are  not  handled  well  by 
the  conventional  descriptors  of  linear  modes  in  cylindrical  fibers. 
Some  of  these  fibers  [see  Fig.  26.5(A,E)]  suspend  what  is  effec¬ 
tively  a  large  fiber  core  in  a  web  of  glass  supports.  Others  [Fig. 
26.5(B-F)]  have  a  series  of  thin  tubes  around  a  hollow  core. 
Although  this  pattern  appears  to  defy  the  theory  of  conducting  light 
by  total  internal  reflection  (TIR)  in  a  high-RI  core,  they  can  be 
understood  as  Bragg-grating  mirrors/photonic  bandgap  structures, 
surrounding  and  guiding  the  wave  energy.  The  light  is  trapped 
between  mirror  structures  in  the  cladding  formed  by  the  surfaces 
of  the  regular  array  of  holes.  Common  examples  of  intrinsically 
colorless,  transparent,  multi-layer  structures  that  act  as  mirrors 
include  the  reflective  skin  cells  of  fish,  gel  holograms,  and  the  iri¬ 
descent  scales  on  butterfly  wings.  Applications  of  these  fibers  will 
be  discussed  in  the  context  of  optical  fiber  illumination. 


Fiber  Image  Transfer  Bundles 

Coherent  image  transfer  bundles  are  one  of  the  earliest  applica¬ 
tions  of  optical  fibers.  Tens  or  hundreds  of  thousands  of  fibers  are 


FIGURE  26.5.  Images  of  the  profiles  of 
various  microstructured  optical  fiber.  Air 
cored  fibers  (B-E)  exhibit  low  or  negligible 
chromatic  dispersion,  whereas  air  cladding 
structures  (A),  and  in  a  hybrid  with  an  air 
core  (E  and  F)  can  be  pre-reverse  chirped  to 
excite  nonlinear  modes  of  propagation  of 
ultra-short,  high  peak  power  pulses.  The  two 
larger  air  cores  in  (D)  give  this  fiber  polar¬ 
ization-preserving  properties. 
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combined  in  a  bundle,  each  fiber  transferring  one  spot  of  light  from 
a  focused  image  at  one  end  of  the  bundle  to  an  identical  configu¬ 
ration  of  fiber  tips  at  the  opposite  end.  This  process  reconstitutes 
the  image.  Invented  by  Logie-Baird  in  1927,  fiber-optical  bundles 
were  widely  used  for  internal  endoscopy  after  the  1950s,  but  now 
have  largely  been  replaced  by  miniature  solid-state  video  cameras. 
These  devices  present  particular  issues  in  terms  of  the  way  that 
they  sample  the  image  plane  to  be  transferred.  Because  the  fiber 
cores  are  separated  by  the  cladding  regions  (unavoidable  if  the 
cores  are  to  guide  light),  the  image  plane  is  sampled  only  at  dis¬ 
crete  locations.  This  complicates  the  interpretation  of  their  perfor¬ 
mance  and  will  be  discussed  below  in  more  detail  in  the  context 
of  the  microscope  configurations  that  exploit  them. 

Two  processes  are  commonly  used  to  make  coherent  image 
transfer  bundles.  Fibers  or  preform  rods  with  high  RI  centers  can 
be  packed  together  in  a  hexagonal  pattern  and  then  fused  and 
drawn  out.  The  hexagonal  array  and  RI  pattern  are  maintained  in 
miniature  form  during  the  drawing.  This  process  may  be  repeated 
if  a  smaller  core  is  required.  Alternatively,  coherent  bundles  can 
be  formed  by  winding  very  thin  fibers  on  a  mandrel,  gluing 
together  all  the  fibers  in  one  section,  and  then  cutting  through  and 
polishing  both  ends  of  that  section.  Bundles  of  fiber  made  by  the 
latter  method  are  as  flexible  as  a  hank  of  silk.  Fused  bundles  are 
generally  stiffen 


KEY  FUNCTIONS  OF  FIBERS  IN 
OPTICAF  MICROSCOPES 

Fiber  technologies  are  presently  exploited  in  various  instruments 
including  confocal  microscopes,  OCT  scanners,  and  laser  scanning 
nonlinear  microscopy  systems.  Most  of  these  applications  involve 
the  use  of  fibers,  either  for  remote  laser  delivery  and/or  as  a  detec¬ 
tion  aperture  and  some  use  fibers  that  serve  both  these  functions 
simultaneously.  The  major  factors  important  in  such  systems  are 
(1)  the  efficient  launch  of  laser  light  into  the  optical  fiber;  (2) 
proper  handling  of  the  fiber  output  as  a  light  source;  (3)  efficient 
projection  of  the  returning  signal  light  onto  a  fiber  that  is  acting  as 
a  detector  aperture;  and  (4)  transmission  of  short  pulses  suitable 
for  nonlinear  microscopy.  Note  that  these  functions  are  not  neces¬ 
sarily  independent. 


Optical  Fiber  for  Delivering  Fight 

The  use  of  optical  fibers  as  flexible  laser  delivery  systems  has  been 
established  for  many  years,  proving  particularly  useful  in  medical 
laser  delivery  [e.g.,  in  “laser  knife”  surgery  or  laser  ablation  of  skin 
blemishes  (Katzir,  1993)].  Among  the  major  advantages  to  be 
gained  by  fiber-optic  light  delivery  are  the  isolation  of  bulky  laser 
equipment  (including  vibration  isolation)  and  the  flexibility  pro¬ 
vided  by  being  able  to  actually  handle  and  manipulate  the  laser 
output  (e.g.,  in  hand-held  laser  cutting  devices).  The  properties  of 
the  output  depends  both  on  key  fiber  parameters  (e.g.,  RI  gradient, 
V-number)  and  those  of  the  coupled  laser  (modes,  wavelength, 
etc.). 

The  most  common  application  of  optical  fibers  in  confocal 
microscopy  is  the  use  of  a  single-mode  laser  beam  coupled  to  an 
optical  fiber  that  is  single-mode  at  the  laser  wavelength(s). 

Factors  important  to  efficient  coupling  of  laser  light  into  an 
optical  fiber  include:  the  beam  profile  and  polarization  of  the  laser, 
the  mode  profile  of  the  fiber,  its  NA  and,  in  the  case  of  a  pol- 
preserving  fiber,  its  orientation,  and  the  relative  position  of  the 
lens  system  used  to  project  the  laser  output  into  the  fiber  core.  The 


key  requirement  is  to  focus  the  beam  onto  the  fiber  in  a  manner 
that  achieves  maximum  overlap  between  the  distribution  of  energy 
in  the  focused  spot  and  the  modal  field  that  exists  near  the  fiber 
tip,  that  is,  there  must  be  a  complete  overlap  between  the  focused 
beam  and  the  acceptance  cone  of  the  fiber.  In  practical  terms,  this 
means  that  the  beam  to  be  launched  into  the  fiber  must  itself  be 
everywhere  coincident  with  the  light  that  would  emerge  from  the 
fiber  had  it  been  launched  from  the  opposite  end.  Indeed,  actually 
back  launching  a  small  amount  of  light  in  this  manner  serves  as 
a  very  useful  tool  in  the  alignment  of  a  laser/fiber  coupling 
apparatus. 

Many  devices  for  launching  laser  light  into  optical  fibers 
consist  of  a  lens  holder  that  can  be  aligned  to  the  laser  beam  and 
a  means  of  positioning  an  optical  fiber  at  the  focused  spot,  includ¬ 
ing  an  adjustment  of  its  tilt.  Orienting  the  fiber  to  maximize  light 
throughput  will  in  general  correspond  with  translational  alignment 
of  the  spot  onto  the  fiber  core  (Figure  26.6).  However,  Figure  26.6 
also  shows  that  this  will  not  necessarily  be  sufficient  to  ensure 
maximum  coupling  efficiency  as  this  also  requires  both  perfect 
axial  (tilt)  alignment  and  matching  of  the  NA  of  the  fiber  with  that 
of  the  incident,  converging  wavefront  (Fig.  26.6). 

Although  the  Gaussian  beam  profile  of  single-mode  transverse 
electromagnetic  mode  (TEMoo)  lasers  is  typically  well  matched 
(but  not  perfectly)  to  the  modal  field  of  single-mode  optical  fiber 
(Snyder  and  Love,  1983),  the  output  from  a  solid-state  laser  is 
typically  asymmetric  and  astigmatic  unless  corrected  with  dedi¬ 
cated  anamorphic  lenses.  If  the  lens  system  is  very  well  corrected 
for  the  application  and  meticulously  aligned,  coupling  efficiency 
of  approximately  90%  is  possible;  however,  with  commonly  avail¬ 
able  components  (dedicated  lens  systems  that  are  commercially 
available  for  common  laser  and  fiber  types)  60%  to  70%  should 
be  routinely  expected. 

Instabilities  in  the  output  power  can  be  caused  by  instability 
in  any  of  the  parameters  mentioned  above  as  being  important  to 
efficient  launch. 

Consideration  should  also  be  given  to  the  light  that  is  not  prop¬ 
agated  in  the  core  of  the  fiber.  For  an  air-to-fiber  launch,  approxi¬ 
mately  4%  will  be  lost  at  the  air-glass  interface  by  Fresnel 
reflection  (assuming  the  fiber  end  has  no  anti-reflection  coatings). 
Light  that  enters  the  fiber  but  is  not  propagated  in  modes  will  travel 
short  distances  in  the  cladding  (centimeters  to  tens  of  centimeters) 
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FIGURE  26.6.  Single-mode  fiber  launch.  Efficient  coupling  of  a  TEM00  laser 
beam  into  a  single-mode  fiber  requires  optimization  of  multiple  degrees  of 
freedom.  First,  the  beam  must  be  focused  onto  the  core  region.  Second,  the 
beam  NA  must  be  matched  to  the  fiber  NA  and  the  axis  of  the  convergent  cone 
of  light  must  be  aligned  with  the  mode  field  of  the  fiber.  The  incident  beam  in 
the  diagram  that  is  not  overlapping  with  the  mode  field  of  the  fiber  will  not  be 
coupled  into  the  fundamental  mode,  and  will  be  radiated  away  over  a  short  dis¬ 
tance  of  fiber. 
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before  being  entirely  radiated  away,  and  will  generally  have  a  neg¬ 
ligible  impact  on  the  output  of  fibers  having  lengths  typical  of 
those  used  for  microscope  illumination.  However,  the  unguided 
light  can  excite  fluorescence  in  fiber  jacketing  materials,  a  very 
small  proportion  of  which  may  be  propagated  and  contribute  to 
background  in  the  optical  path  of  the  microscope.  Careful  selec¬ 
tion  of  non-fluorescent  materials  for  fiber  jacketing  (particularly 
near  the  launch  end)  can  eliminate  this  problem. 

As  discussed  previously,  the  emergent  light  will  be  a  series  of 
concentric,  diverging  wavefronts,  as  though  from  a  focused  point 
of  laser  light.  These  spherical  waves  can  then  be  re-collimated  to 
provide  a  beam  having  the  characteristics  of  a  pure,  fundamental¬ 
mode  laser  beam.  Note  that  although  collimation  may  only  be 
required  for  certain  applications,  the  preservation  of  this  coherent 
laser  wavefront  is  a  critical  optical  property  that  affects  all  sub¬ 
sequent  handling  and  propagation  of  the  beam.  When  the  output 
is  projected  into  an  optical  system  such  as  a  confocal  microscope, 
the  output  of  the  single-mode  optical  fiber  is  best  considered  to  be 
equivalent  to  a  TEMoo  laser  beam  diverging  from  the  point  at 
which  it  is  focused. 

In  contrast,  if  a  multimode  fiber  is  used  for  laser  delivery,  the 
different  modes  will  act  as  a  series  of  light  sources  or  pinholes, 
each  of  which  will  image  to  a  separate  spot  within  the  specimen. 
Note,  however,  that  in  a  practical  situation,  the  number  of  possi¬ 
ble  modes  that  actually  propagate  light  is  very  sensitive  to  the 
launch  conditions  (it  is  often  possible  to  launch  only  the  funda¬ 
mental  mode  or  a  limited  subset  of  modes  in  a  multimode  fiber 
by  careful  launch  alignment).  Coupling  efficiency  for  launching 
laser  light  into  the  fundamental  mode  of  a  few-mode  fiber  is  com¬ 
parable  to  that  for  single-mode  fiber.  The  treatment  of  single-mode 
optical  fiber  output  as  a  coherent  point  source  cannot  be  accurately 
generalized  to  the  case  of  a  multimode  fiber.  Although  each  of 
the  different  individual  modes  will  still  maintain  coherence,  the 
overall  output  will  be  the  sum  of  all  modes  propagated.  The  most 
important  difference  will  generally  be  that  the  output  can  neither 
be  restored  to  a  single,  parallel  wavefront,  nor  focused  to  a  dif¬ 
fraction-limited  spot. 

Optical  Fiber  as  a  Detection  Aperture 

The  use  of  optical  fiber  as  a  detection  aperture  offers  the  potential 
for  transmitting  the  signal  light  returning  from  the  specimen  to  an 
external  detector.  The  detectors  themselves  may  include  special¬ 
ized  devices  such  as  spectrometers  as  well  as  the  more  common 
photomultiplier  tube.  The  intrinsic  properties  of  the  single-mode 
fiber  allow  it  to  provide  imaging  characteristics  similar  to  those  of 
a  conventional  pinhole  detector.  The  core  size  is  related  to  the  NA 
of  the  beam  entering  it  in  the  same  manner  but  it  is  defined  in  terms 
of  the  soft  edges  of  a  Gaussian  mode  field,  instead  of  the  abrupt 
edges  of  a  physical  pinhole  in  an  opaque  material  (i.e.,  the  edges 
of  the  “virtual"  pinhole  are  fuzzy). 

Single-mode,  few-mode,  and  multi-mode  fibers  can  also  act  as 
coherent  detectors.  This  has  been  demonstrated  for  the  case  of  a 
two-mode  optical  fiber,  for  which  differential  measurement  of  the 
modes  permits  phase-contrast  imaging  of  surfaces  with  very  accu¬ 
rate  height  determination  (Kimura  and  Wilson,  1991;  Wilson  et  al., 
1992). 

The  spatial  distribution  of  modes  within  a  multimode  detec¬ 
tion  fiber  may  also  carry  information  regarding  the  source  of  the 
signal.  For  example,  spatial  filtering  of  light  from  the  near  field  or 
the  far  field  projection  of  the  fiber  can  mimic  the  role  of  a  variable 
detection  aperture.  A  rather  elegant  example  uses  the  difference  in 
RI  coefficients  between  silica  glass  and  the  silicone  elastomer 


jacket  to  form  a  temperature-tuned  variable  pinhole  with  no 
moving  parts.  (Hands  and  Delaney,  patent).  The  variation  in 
pinhole  function  achievable  by  this  approach  is  dependent  upon 
the  number  of  modes  supported  by  the  fiber,  and  the  extent  to 
which  the  coupling  optics  distribute  light  into  these  modes.  It  is 
possible  to  control  the  effective  pinhole  size  continuously  from 
that  of  a  single-mode  fiber  up  to  a  large-core  multimode  fiber, 
or  a  change  of  several  times  in  effective  pinhole  diameter.  This 
approach  is  applicable  to  single  fibers  only  and  not  to  fiber  bundles. 

Same  Fiber  for  Both  Source  and 
Confocal  Detection 

The  use  of  a  single  fiber  as  both  the  source  and  detection  aperture 
best  exploits  the  potential  for  fibers  to  offer  stability  and  robust¬ 
ness  in  alignment,  as  well  as  to  enable  miniaturization  of  the 
imaging  head.  This  made  it  possible  to  construct  compact  scan¬ 
ning  heads  to  mount  on  microscopes  as  well  as  highly  minia¬ 
turized  imaging  heads  (both  in  single-fiber  and  fiber  bundle 
configurations,  see  below). 

One  technical  difficulty  in  implementing  any  approach 
whereby  light  is  detected  from  the  illuminating  fiber  (as  in  a  single- 
optical-fiber  confocal  microscope;  Delaney  et  al.,  1994a)  relates 
to  the  fact  that  both  ends  of  the  fiber  are  in  image  planes  of  the 
microscope.  As  a  result  reflections  of  the  excitation  light  that  are 
generated  at  the  fiber  tip  must  be  suppressed.  Fresnel  reflection  of 
the  incident  laser  at  optical  surfaces  can  direct  a  significant  amount 
of  light  into  the  detection  path.  As  a  result,  careful  management  of 
launch  conditions  becomes  critical  for  reasons  other  than  effi¬ 
ciency,  as  it  represents  a  source  of  background  noise  or  artifacts. 
Both  the  launch  efficiency  itself  and  the  control  of  light  scattering 
at  both  fiber  ends  assume  paramount  importance. 

Standard  methods  commonly  used  to  manage  insertion  losses 
in  telecommunications  fibers  can  be  applied  directly  to  these 
situations: 

•  Angle  polishing  of  fiber  tips  so  that  the  reflected  portion  of  the 
launched  laser  is  reflected  along  a  separate  path  than  the  mode 
field  of  the  fiber  (i.e.,  the  path  that  light  emerging  from  the 
fiber  will  follow).  Any  light  to  be  measured  back  through  this 
fiber  can  thus  be  spatially  separated  from  the  reflected  light. 
Angle  polishing  can  also  be  used  to  suppress  return  loss  from 
the  distal  endface  because  the  light  reflecting  back  from  the 
angled  surface  will  not  be  aligned  with  that  in  the  propagated 
mode. 

•  Index  matching  of  fiber  or  bundle  terminations  (e.g.,  by  filling 
the  space  between  the  launch  optics  and  the  fiber  tip  with  an 
immersion  medium).  This  minimizes  or  eliminates  reflections 
that  occur  at  an  air-glass  refractive  index  boundary,  or  deflects 
them  away  from  the  detection  path. 

•  Anti-reflection  coating  the  fiber  tips. 

Other  sources  of  background  include  Raman  generation  in  the 
fiber  core  and  fluorescence  from  the  polymers  used  in  the  fiber 
coating  and  connectors.  Careful  selection  of  fiber  jacketing  and  the 
materials  used  in  connectors  (including  glues)  is  essential  if  one  is 
to  minimize  sources  of  stray  light.  If  handled  carefully,  background 
levels  5  to  6  orders  of  magnitude  below  the  incident  laser  power 
are  achievable. 

Note  that,  as  polarization  sensitivity  of  modes  may  confound 
efficient  detection  of  the  randomly  polarized  fluorescence  signal 
(e.g.,  from  unbound  fluorescent  probes),  systems  based  on  this 
design  tend  to  use  optical  fibers  that  are  not  polarization-preserving. 
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Fiber  Delivery  for  Nonlinear  Microscopy  with 
Femtosecond  Lasers 

Two-photon,  three-photon,  and  allied  nonlinear  microscopy 
requires  ultra-short  (femtosecond  domain)  pulses  delivered  at  very 
high  peak  optical  powers  to  generate  second-  and  third-order 
effects.  Although  titanium-sapphire  femtosecond  lasers  are  widely 
employed  for  this  purpose  with  direct  beam  delivery,  the  use  of 
fibers  to  transmit  the  pulses  would  provide  greater  convenience  if 
a  number  of  difficult  technical  challenges  can  be  overcome.  Short 
pulses  necessarily  contain  a  small  range  of  wavelengths  (10  nm  at 
lOOfs,  white  light  at  1  fs).  Once  the  pulse  is  in  the  fiber,  chromatic 
dispersion  (due  to  different  wavelengths  propagating  at  different 
speeds)  causes  the  shorter  wavelength  components  to  lag  those 
with  longer  wavelength.  This  makes  the  pulse  last  longer.  For 
example,  a  50  fs  pulse  [which  has  a  spectral  spread  of  around 
20  nm  full- width  half-maximum  (FWHM)]  can  spread  to  several 
picoseconds  in  the  process  of  traversing  a  meter  of  fiber.  Some 
investigators  have  attempted  to  overcome  this  problem  by  using  a 
device  known  as  a  pre-reverse  chirper  to  progressively  delay  the 
longer  wavelengths  relative  to  the  shorter  wavelengths.  Thus,  the 
slower-traveling  blue  components  gain  a  head  start  on  the  faster- 
traveling  red  components.  If  tuned  correctly,  the  faster  red  com¬ 
ponents  catch  up  to  the  slower  blue  just  as  they  reach  the  end  of 
the  fiber,  reconstituting  the  original  short  pulse  at  their  target  des¬ 
tination  in  the  sample. 

Although  elegant,  this  approach  has  two  limitations.  First,  it 
does  not  deal  with  a  secondary  spectrum,  which  is  not  so  easily 
corrected.  Second,  even  if  the  chromatic  dispersion  is  perfectly 
corrected,  the  presence  of  high  peak  powers  concentrated  into  a 
tiny  fiber  core  can  alter  its  function  of,  or  even  destroy  it  (Fork 
et  al.,  1984). 

Although  pre-reverse  chirping  can  result  in  fiber  delivery  of 
pulses  short  enough  to  excite  two-photon  fluorescence,  the  limita¬ 
tions  of  pulse  restoration  and  peak  power  have  prevented  such 
systems  from  imaging  specimens  to  a  greater  depth  than  single¬ 
photon  confocal  microscopy,  thus  negating  the  benefit  of  nonlin¬ 
ear  excitation.  However,  various  microstructure  fibers  now  hold 
much  promise  for  bringing  practical  fiber-optic  illumination  to 
nonlinear  optical  microscopy. 

Some  classes  of  air-core  optical  fiber  have  been  shown  to 
have  virtually  zero  chromatic  dispersion  and  are  also  capable  of 
transmitting  high  power  levels  (Knight  et  al.,  1998;  Bjarklev  et  al., 
2003).  Other  fibers  that  suspend  a  large  central  core  by  a  fine  struc¬ 
ture  of  spokes  have  been  shown  to  be  capable  of  transmitting 
140 fs  pulses  at  high  peak  power  (Ouzounov  et  al.,  2002). 

Some  of  these  nonlinear  effects  are  caused  by  the  fact  that  the 
pulse  intensity  is  so  high  that  it  locally  changes  the  RI  of  the  glass 
and  allows  the  trailing  part  of  the  wave  to  catch  up  in  a  process 
called  self-phase  modulation.  This  can  generate  ultra-short  isolated 
wave  sets  called  optical  solitons,  which  are  able  to  travel  in 
such  fibers  over  considerable  distances  without  spreading.  The 
process  is  analogous  to  that  which  produces  tidal  bores  in  some 
estuaries. 

Not  surprisingly,  microstructured  fibers  demonstrate  different 
properties  for  the  linear  propagation  of  continuous  light  energy 
than  for  the  nonlinear  propagation  of  pulsed  energy.  In  particular, 
some  feature  very  large  core  diameters  that  are  massively  multi- 
mode  in  the  linear  domain.  Thus,  they  may  function  as  a  large 
area  detection  pinhole.  This  may  be  appealing  in  many  nonlinear 
microscopy  applications  because  nonlinear  illumination  can  be 
combined  with  non-confocal  linear  detection  using  the  same 
fiber. 


Large  Core  Fibers  as  Source  or 
Detection  Apertures 

In  addition  to  their  function  as  a  point  source  or  a  detection  aper¬ 
ture,  optical  fibers  can  also  provide  diffuse  illumination  and  large 
area  detection. 

For  example,  very  large  fluid-core  fibers  can  scramble  the 
spatial  or  temporal  structure  of  arc  or  filament  light  sources  with 
the  result  that  the  endface  of  the  core  serves  as  a  uniform,  diffuse 
source  (Ellis,  1985)  (as  described  in  Chapter  6,  this  volume). 

Likewise,  large-core  fibers  can  be  used  to  carry  the  signal  light 
passing  through  a  physical  pinhole  aperture  in  the  scan  head  to  a 
remotely  mounted  photodetector.  When  used  in  this  fashion,  it  is 
important  to  remember  that,  no  matter  how  sophisticated  the  optics 
used  to  magnify  or  demagnify  the  semi-coherent  light  coming 
through  the  pinhole,  one  cannot  demagnify  the  original  source  of 
the  signal  without  losing  photons.  Therefore,  the  distal  end  of 
the  fiber  must  always  be  at  least  as  large  as  the  source  of  light  in  the 
specimen  and,  were  such  a  fiber  to  be  used  as  a  collector  for 
non-descanned  detection  in  nonlinear  microscopy,  its  input  sur¬ 
face  should  be  larger  than  the  area  of  the  raster  scanned  in  the 
specimen. 

BENCHTOP  SCANNING  MICROSCOPES 
EXPLOITING  FIBER  COMPONENTS 

Single-mode  fiber-optic  laser  delivery  is  almost  standard  on 
modern  commercial  laser  scanning  confocal  microscopes.  Over 
several  design  generations,  this  has  enabled  the  design  of  more 
compact  scanning  heads,  eliminated  laser  vibration  from  the 
microscope  stand,  and  increased  the  flexibility  in  laser  delivery, 
although  fused  biconical  taper  couplers  are  not  yet  commonly  used 
for  mixing  the  outputs  from  multiple  lasers. 

Most  systems  require  control  of  the  polarization  state  of  the 
laser  illumination  for  multiple  reasons.  First,  the  quarter-wave- 
plate  antiflex  systems  for  suppressing  specular  reflections  from  lens 
surfaces  are  polarization  dependent  (see  Chapter  7,  this  volume). 
Second,  as  the  split  ratio  of  the  45°  reflecting  surface  in  beam¬ 
splitter  cubes  exhibits  strong  polarization  dependence,  any  change 
in  beam  polarization  can  produce  significant  changes  in  the 
excitation  power  delivered  to  the  specimen.  Third,  differential 
interference  contrast  (DIC)  and  other  contrast  techniques  require  a 
fixed,  linear  polarization  state.  Consequently,  most  laser-scanning 
confocal  microscopes  deliver  the  laser  light  with  polarization- 
maintaining  fibers.  Due  to  their  specialized  core  profiles  (Fig.  26.3), 
polarization-maintaining  fibers  can  support  each  of  two  polariza¬ 
tion-dependant  modes  and  act  as  a  single-mode  fiber  for  each  of 
them.  Hence,  when  properly  coupled  to  a  linearly  polarized  laser 
source,  they  act  as  a  single-mode  fiber. 

At  different  times,  several  systems  have  also  been  produced 
that  utilize  fiber-optics  on  the  detection  path,  enabling  modular¬ 
ization  of  the  detection  scheme. 

The  use  of  one  fiber  for  illumination  and  another  for  detection 
offers  a  high  degree  of  modularity  and  flexibility  (e.g.,  as  exploited 
in  the  Nikon  Cl  system).  This  has  been  employed  in  a  system  for 
confocal  micro-Raman  imaging  (Sharonov  et  al.,  1992),  offering 
great  flexibility  in  laser  sources  and  detection  components. 

Incorporation  of  fiber  delivery  in  multi-photon  microscopes 
has  remained  a  niche  field  for  many  of  the  reasons  described 
above.  Although  there  have  been  numerous  attempts  to  modify 
commercial  laser-scanning  microscopes  to  accept  fiber-delivered 
pulsed  light  sources  using  reverse  pre-chirping  and  a  standard 
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cylindrical  fiber,  these  have  remained  limited  to  applications 
requiring  longer  pulse  lengths  and  lower  peak  powers.  Such 
systems  have  not  provided  the  deep  tissue  imaging  and  flexibility 
in  multi-photon  excitation  cross-section  common  from  conven¬ 
tional  systems. 

Now,  however,  microstructure  fiber  is  rapidly  being  applied  to 
this  area  and  early  data  suggests  exciting  progress.  With  these 
advances,  fiber  delivery  will  be  increasingly  adopted  for  nonlinear 
microscopy  in  experimental  and  commercial  systems.  This  will 
allow  novel  configurations  and  the  development  of  miniaturized 
scanning  systems,  such  as  those  already  available  in  confocal 
microscopy. 

MINIATURIZED  SCANNING  CONFOCAL 
MICROSCOPE  IMAGING  HEADS 

The  most  basic  properties  of  confocal  microscopy  suggest  numer¬ 
ous  applications  involving  microscopic  imaging  of  living  tissue  in 
vivo.  In  particular,  the  transition  of  biomedical  research  from  bench 
to  bedside  faces  numerous  obstacles  in  understanding  biological 
and  morphological  events  in  complex  living  systems  on  a  micro¬ 
scopic  scale.  Until  now,  relatively  few  applications  have  been 
realized  due  in  no  small  part  to  the  cumbersome  nature  of  bench- 
based  microscope  configurations.  The  development  of  miniatur¬ 
ized,  scanning-probe  devices  suitable  for  practical  in  vivo  confocal 
microscopy  has  long  been  a  goal.  The  requirements  of  a  miniature 
confocal  imaging  head  for  in  vivo  microscopy  are  small  diameter 
(one  to  several  millimeters),  short  rigid  length  (millimeters  to 
several  centimeters),  and  flexible  umbilical  connection  to  the  rest  of 
the  instrument.  In  some  applications  (particularly  in  clinical  medi¬ 
cine),  there  are  further  requirements  for  compatibility  with  conven¬ 
tional  endoscope  components  and  for  disinfection  or  sterilization. 

Fiber-optic  technologies  have  played  an  enabling  role  in  these 
developments,  some  of  which  have  led  to  practical  products  and 
medical  devices  that  employ  various  optical  schema,  as  depicted 
in  Figure  26.7  and  described  below. 

Miniature  Confocal  Imaging  Heads  Based  on 
Coherent  Imaging  Bundles 

Coherent  imaging  fiber  bundles  present  an  intriguing  range  of 
trade-offs  between  the  need  for  image  quality  versus  simplicity  of 
miniaturization  and  implementation. 

A  coherent  fiber-optic  imaging  bundle  can  be  used  to  transfer 
the  scanned  image  plane  from  an  open  location  to  one  with  spatial 
constraints  [i.e.,  one  mounted  on  an  endoscope;  Gmitro  and  Aziz, 
1993;  Fig.  26.6(A)].  The  focused  laser  beam  is  scanned  across  the 
proximal  surface  of  the  bundle  injecting  light  sequentially  into 
each  of  the  fibers.  Each  fiber  can  then  act  as  a  confocal  pinhole, 
returning  light  from  the  field  on  which  its  projection  is  focused  in 
the  sample. 

Such  a  microscope  can  operate  in  a  contact  mode  in  which  the 
polished  end  of  the  fiber  bundle  defines  the  image-isolation  plane 
by  directly  touching  the  tissue.  Alternatively,  an  appropriate  lens 
can  be  mounted  distal  to  the  exit  surface  of  the  bundle  to  allow  the 
projected  image  of  the  scanned  spot  or  line  to  be  refocused  onto 
the  specimen  to  obtain  confocal  images.  In  this  case,  any  individ¬ 
ual  fiber  in  the  bundle  functions  in  a  manner  equivalent  to  the 
single  fiber  in  Figure  42. 6B  as  described  in  the  next  section. 
Contact  imaging  provides  the  smallest  practicable  endomicroscope 
but  the  resolution  is  effectively  limited  by  the  physical  spacing  of 
the  fibers  (~5  pm).  The  second  method  can  have  a  higher  resolu- 
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FIGURE  26.7.  Optical  schema  for  miniaturized  confocal  microscopy  systems 
and  heads.  (A)  Scanning  the  proximal  end  of  a  coherent  fiber  bundle  transfers 
the  illumination  sequentially  to  the  fibers  in  the  bundle.  The  light  emerging 
from  one  or  more  fibers  is  coupled  to  the  tissue  either  by  a  lens  system  (main 
diagram)  or  by  direct  tissue  contact  (inset).  In  the  former  case  (main  diagram), 
the  design  of  the  lens  system  dictates  the  resolution  achieved  for  imaging  the 
specimen,  and  in  particular  how  the  discontinuous  nature  of  the  bundle  impacts 
the  optical  sampling  of  the  tissue  (see  main  text).  In  the  latter  case,  light  emer¬ 
gent  from  a  given  core  in  the  bundle  (e.g.,  core  1,  inset)  will  illuminate  its  mode 
field  in  the  tissue.  A  near  field  point  (2,  inset)  in  the  mode  field  will  return  light 
to  core  1,  but  not  to  adjacent  cores.  Slightly  further  from  the  end  of  the  bundle, 
points  illuminated  by  core  1  (point  4,  inset)  can  return  light  both  to  the  illu¬ 
minating  core  1  and  to  adjacent  core  4.  Thus,  the  sampling  of  the  tissue  is  gov¬ 
erned  by  the  NA  of  the  fibers,  scattering  in  the  tissue,  and  the  geometry  of  the 
bundle.  (B)  A  single  fiber  is  used  as  the  illumination  source  and  confocal  detec¬ 
tion  path  aperture,  coupled  to  the  focus  in  the  tissue  by  a  miniature  lens  system. 
Scanning  can  then  be  achieved  by  a  variety  of  means.  The  fiber  can  be  physi¬ 
cally  scanned  relative  to  a  laterally  fixed  lens,  or  the  fiber  can  be  fixed  and  the 
lens  itself  laterally  scanned.  Alternatively,  the  fiber  and  lens  can  be  fixed  and 
scanning  performed  by  micromirror  devices,  or  microlenses  fitted  to  the  tip  of 
the  fiber  itself,  which  are  scanned  together  (not  shown). 


tion  if  the  projected  image  of  the  fiber  bundle  is  de-magnified  by 
the  lens  system  (Fig.  42.7,  insert). 

Although  sampling  artifacts  can  occur  when  the  hexagonal 
pattern  of  the  fibers  is  imaged  onto  the  CCD,  to  reveal  the  black 
cladding  around  each  core,  this  can  be  overcome  by  software  pro¬ 
cessing  to  reconstitute  the  signals  from  the  individual  fibers,  pro¬ 
ducing  an  image  that  appears  continuous  [as  it  does,  for  instance, 
in  the  Mauna  Kea  Technologies  system  (Paris,  France)  (see  Fig. 
42.9]. 
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However,  such  correction  does  not  address  the  potential  under¬ 
sampling  in  the  objective  imaging-plane,  where  the  output  of  each 
fiber  (and  its  corresponding  detected  field)  is  discrete  from  that 
interrogated  by  neighboring  fibers. 

A  successful  optical-sampling  system  would  not  waste  any 
light,  and  would  both  illuminate  the  focus  plane  in  the  object  uni¬ 
formly  and  project  the  image  of  an  in-focus  point  object  so  that  it 
is  at  least  4  core-spacings  wide  when  it  strikes  the  distal  end  of  the 
fiber  bundle  (see  Chapter  4  about  sampling,  this  volume). 

Several  interacting  operational  variables  affect  this  sampling 
problem  and  its  impact  on  photon  efficiency:  packing  the  fibers 
more  closely  at  the  bundle  output  requires  the  use  of  high-RI  glass 
cores  and  this  results  in  the  fiber  having  a  higher  NA  and  there¬ 
fore  emitting  light  into  a  larger  spot  at  the  BFP  of  the  objective.  If 
the  NA  of  the  fiber  is  larger  than  the  back-NA  of  the  objective 
[NAback  =  NAfront]/M,  where  M  is  the  (de)magnification],  then  much 
of  the  excitation  light  emerging  from  the  fiber  core  will  miss  the 
objective.  This  reduces  the  photon  efficiency  in  proportion  to  the 
square  of  the  ratio  of  the  respective  NAs. 

The  objective  lens  system  itself  has  both  a  magnification  and 
an  NA.  As  long  as  the  BFP  is  filled,  a  larger  NA  implies  smaller 
spots  in  the  focus  plane,1  and  consequently  makes  it  more  likely 
that  the  focused  spots  will  not  overlap.  Higher  demagnification 
not  only  implies  a  lower  NAback,  but  also  ensures,  that  for  a  given 
fiber  spacing,  the  spots  at  the  focus  plane  in  the  specimen  will  be 
closer  together. 

The  optimization  depends  heavily  on  the  properties  of  the 
bundle.  The  most  suitable  bundles  are  densely  packed,  and  consist 
of  cores  of  approximately  4pm  diameter  with  a  center-to-center 
fiber  spacing  of  approximately  6pm  (i.e.,  the  cladding  between  is 
~2pm).  This  yields  a  high-density  30,000  element  bundle  that  is 
about  2  mm  in  diameter,  including  the  jacket.  If  the  cores  get  much 
smaller,  they  begin  to  act  as  single-mode  fibers,  especially  at 
longer  wavelengths,  slightly  reducing  their  throunghput  of  inco¬ 
herent  fluorescent  light.  If  the  cladding  gets  much  thinner,  light 
begins  to  leak  to  adjacent  fibers.  To  reduce  this  effect,  the-core-to- 
cladding  RI  difference  must  be  made  as  high  as  possible,  result¬ 
ing  in  a  high  fiber  NA  (0. 3-0.4,  versus  NA  0.1-0.15  for 
communication  fiber).  As  the  NA  of  the  objective  varies  between 
0.4  and  0.5,  operation  at  a  magnification  as  low  as  5x  will  still 
reduce  the  back  NA  to  -0.1,  meaning  that  only  1/16  of  the  light 
emitted  by  a  0.4  NA  fiber  will  strike  it:  lOx  operation  would  be 
4x  worse. 

A  0.5  NA  objective  demagnifying  488  nm  light  emerging  from 
a  bundle  with  5  pm  spacing  by  10: 1  will  produce  spots  of  that  are 
-0.75  pm  at  FWHM  on  a  0.5  pm  spacing  in  the  focus  plane.  These 
spots  should  overlap  fairly  well,  and  when  the  scanning  system 
scans  the  laser  beam  over  the  proximal  surface  of  the  bundle, 
sequentially  illuminating  the  cores  on  the  distal  surface,  the  entire 
focus  plane  will  eventually  be  excited.  However,  any  fluorescent 
or  scattered  light  from  an  in-focus  point  object  that  returns  through 
the  objective,  will  form  an  image  with  a  FWHM  of  only  5  pm  in 
diameter  at  the  fiber  bundle  surface.  This  is  two  times  smaller  than 


1  This  illustrative  calculation  assumes  that  the  light  emerging  from  each  core 
is  fully  coherent  (i.e.,  that  the  fiber  supports  only  a  single  mode).  Because  the 
libers  are  actually  multimode,  the  size  of  spots  in  the  focus  plane  is  not 
directly  proportional  to  the  NA  and  they  are  actually  somewhat  larger  than 
stated.  However,  on  the  return  journey,  correct  fiber  spacing,  not  fiber  size,  is 
the  parameter  required  to  properly  sample  the  magnified  image  of  an  in-focus 
point  object. 


it  should  be  to  be  sampled  by  at  least  2  measurements  (i.e.,  2  cores 
across  the  FWHM,  4  cores  across  its  entire  diameter)  as  is  required 
by  Nyquist. 

As  with  any  form  of  undersampling,  it  can  be  overcome  by 
increasing  the  sampling  frequency  (i.e.,  increasing  the  optical  mag¬ 
nification  or  decreasing  the  core  spacing)  to  meet  the  Nyquist 
criterion.  Alternatively,  one  can  “spoil  the  optical  resolution,” 
perhaps  by  introducing  aberrations  into  the  optics.  As  a  result, 
energy  from  each  fiber  will  be  spread  over  a  larger  area  in  the  spec¬ 
imen  and  the  returning  confocal  signal  will  again  be  further  defo- 
cused  on  its  way  back  to  the  bundle.  This  results  in  some  light 
striking  adjacent  fibers,  achieving  a  similar  result  to  increasing  the 
sampling  rate.  It  was  employed  by  the  authors  in  the  prototype 
constructed  to  obtain  Figures  42. 8E  and  42. 8F. 

Because  any  method  used  to  overcome  this  problem  necessar¬ 
ily  involves  mismatching  the  NAs  of  the  fiber  and  the  optical 
system,  substantial  reductions  (i.e.,  from  90-99%)  in  photon  effi¬ 
ciency  cannot  be  avoided.  The  optimization  and  compromises 
depend  heavily  on  the  properties  of  the  bundle. 

Resolution  and  Optical  Efficiency  of  Bundles 

It  can  be  seen  from  the  above  that  the  resolution  achievable  with 
a  bundle  system  cannot  be  directly  derived  from  the  number  and 
spacing  of  the  fibers  in  the  bundle  and  the  objective  lens  magnifi¬ 
cation  and  NA.  For  a  properly  sampled  objective  space,  several 
fibers  are  required  per  objective  resel.  A  properly  matched  laser 
system  can  achieve  a  70%  launch  efficiency  into  a  single  core,  but 
measurements  show  that  the  light  throughput  drops  to  -20% 
when  the  spot  is  defocused  to  cover  several  fibers  at  the  polished 
end  of  a  bundle.  From  geometrical  arguments,  it  follows  that 
if  Nyquist-sampled  resolution  is  required,  then  a  considerable 
amount  of  excitation  light  (>75%)  will  be  lost  due  to  overfilling 
and  around  70%  of  the  confocal  return  is  lost  at  the  cladding 
interface. 

As  with  any  fiber-based  confocal  system  involving  illumina¬ 
tion  and  detection  via  the  same  fiber,  end-face  reflections  and  flu¬ 
orescence  from  the  fibers  or  associated  packaging  may  give  rise 
to  artifactual  background  signal.  As  with  single-fiber  confocal 
systems,  these  reflections  may  be  minimized  using  the  same  tech¬ 
niques  as  those  exploited  in  telecommunications  (i.e.,  refractive 
index  matching).  Images  from  the  Mauna  Kea  bundle  system  show 
effective  removal  of  the  fiber  bundle  autofluorescence  pattern.  This 
may  be  achieved  by  first  using  a  high  degree  of  over-sampling  to 
properly  capture  the  fine  structure  of  the  pattern,  and  then  sub¬ 
tracting  the  fluorescence  pattern  from  the  bundle  from  the  raw 
tissue  data  sets. 

Bundle  Imagers  for  In  Vivo  Studies  in  Animals 

Despite  the  above  compromises,  the  imaging  probes  that  have  been 
produced  using  bundles  have  produced  the  smallest  confocal 
microscopes.  Probes  range  from  approximately  0.3  to  several  mil¬ 
limeters  in  diameter,  and  produce  images  with  several  thousand  to 
several  tens  of  thousands  of  resels  per  image.  Bundle  probes  are 
thus  well  suited  to  in  vivo  microscopy  in  small  animal  models,  and 
potentially  in  clinical  applications  that  require  access  to  very  small 
orifices.  For  example,  these  devices  enable  colonoscopy  in  mouse 
models  of  early  colorectal  cancer  formation,  permitting  longitudi¬ 
nal  studies  of  individual  tumors  (Cave  et  al.,  2005).  A  range  of 
examples  of  animal  tissues  examined  using  miniature  fiber-bundle 
confocal  microscopes  is  presented  in  Figure  26.9. 
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Simple  filtering 


MKT  approach 


OS  approach 


FIGURE  26.8.  Software  compensation  of  bundle  artifacts.  The  pattern  formed  by  the  structure  of  the  fiber  bundle  can  be  reduced  using  various  software 
approaches.  Simple  spatial  filtering  can  soften  the  bundle  artifact,  but  significantly  degrades  resolution  and  contrast:  (A)  raw  image,  (D)  image  filtered  with  four 
passes  of  a  5  x  5  pixel  Gaussian  blur.  A  better  approach  involves  collecting  a  reference  image  of  the  bundle  pattern  to  subtract  artifacts  such  as  bundle  autoflu¬ 
orescence  (C)  raw  image,  (E)  processed  image.  A  combination  of  background  subtraction,  pixel  dilation,  and  filtering  was  used  to  process  image  (C)  and  to 
produce  image  (E).  Potential  artifacts  of  each  of  these  approaches  must  be  considered  in  the  context  of  how  the  distal  optical  sampling  is  represented  by  the  raw 
data  obtained  by  scanning  the  proximal  end  of  the  fiber  bundle  (particularly  when  this  is  optimized  for  maximum  laser  launch  efficiency,  which  results  in  clear 
resolution  of  the  bundle  elements).  [Images  (B)  and  (E)  courtesy  of  Mauna  Kea  Technologies,  Paris,  France.  Images  (A),  (C),  (D),  and  (F)  courtesy  of  Optiscan, 
Melbourne,  Australia.] 


Scan  Heads  Based  on  Single  Fibers  with 
Miniature  Scanning  Mechanisms 

The  use  of  a  single  fiber  acting  as  both  the  source  and  detection 
aperture  offers  contiguous  sampling  of  the  imaging  plane  compa¬ 
rable  to  that  offered  by  conventional  scanning  mirrors.  However, 
the  requirement  to  miniaturize  a  scanning  mechanism  for  incor¬ 
poration  into  the  probe  tip  presents  substantial  technical  chal¬ 
lenges.  Several  approaches  have  been  taken  and  practical  devices 
have  been  realized,  some  of  which  are  diagrammed  in  Figure  26.7. 
Scanning  mechanisms  include  electromagnetic  or  piezoelectric 
scanning  of  the  fiber  itself,  scanning  of  micro-mirrors,  and  scan¬ 
ning  of  a  miniature  lens. 

Vibrating  the  Fiber  Tip 

In  Figure  26.6(B)  (Delaney  et  al.,  1993a),  the  output  of  a  single¬ 
mode  fiber  is  projected  by  a  lens  to  form  a  collimated  beam.  This 
beam  is  focused  by  a  second  lens  into  an  image  plane  in  the  spec¬ 
imen.  These  components  form  the  confocal  illumination  path  and 


the  same  fiber  acts  as  the  detection  aperture  for  the  return  light. 
Scanning  is  implemented  by  moving  the  fiber  tip,  and  being  con- 
tiuous,  this  system  suffers  from  none  of  the  sampling  problems  that 
complicate  the  design  of  the  fiber-bundle  confocals. 

All  systems  in  which  mechanical  scanning  is  carried  out  at  the 
tip  require  a  counter- vibrating  mass  if  small  size  and  fast  scan  are 
desired.  For  example,  the  Optiscan  system  carries  the  fiber  along 
one  tine  of  a  tuning  fork  to  produce  a  resonant  sinusoidal  scan. 
The  second  tine  moves  in  opposition  balancing  the  reaction  forces, 
which  would  otherwise  move  the  case  and  result  in  visco-elastic 
damping  by  interaction  with  the  tissue  (Harris,  patent).  This 
approach  has  been  used  for  a  scanner  that  is  5  mm  in  diameter  (see 
Fig.  26.10  inset)  and  forms  the  basis  of  a  commercial  clinical 
endomicroscope  for  human  imaging  (see  Fig.  26.13). 

Vibrating  the  Lens  and  Fiber 

An  alternative  to  scanning  the  fiber  tip  in  front  of  static  projection 
optics  is  from  an  endoscope  image  by  mounting  a  very  small  lens 
on  the  tip  of  the  fiber,  so  that  the  fiber  output  is  focused  to  a  point 
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FIGURE  26.9.  Imaging  of  animal  tissue  using  miniature  bundle  systems.  (A)  Colonic  crypts  of  a  mouse  mucosa  after  vital  application  of  cresyl  violet  marking 
the  cytoplasm  and  Sytol3  marking  the  nuclei.  Imaged  endoscopically  using  the  Cell~vizio,  with  a  ProFlex  S-1500,  FOV:  400  x  280  pm.  (B)  Distal  axon  and 
motor  endplates  of  a  neuromuscular  junction  of  a  transgenically  marked  nervous  system  (Thyl-YFP  mouse)  with  skeletal  muscle  fibers  marked  by  topical  appli¬ 
cation  of  Sytol3.  Imaged  with  the  Cell~vizio  and  the  direct  contact  of  a  ProFlex  S-0650.  (C)  Proximal  end  of  a  cut  sciatic  nerve  in  a  transgenic  Thyl-YFP 
mouse,  demonstrating  the  single-axon  resolution.  Imaged  using  a  hand-held  ProFlex  S-0650.  FOV:  400  x  280  p.m.  (D)  Mouse  cremaster  microvasculature  stained 
with  FITC-albumin,  injected  intravenously.  Imaged  by  the  Cell~vizio  and  a  hand-held  ProFlex  S-1500.  FOV:  400  x  280  pm.  (E)  Rat  kidney  imaged  with  a  pro¬ 
totype  bundle  contact  tip,  scanned  using  a  desktop  confocal  microscope  (Optiscan  f900e).  Field  width,  400  pm.  [Images  (A),  (B),  and  (C)  courtesy  of  Igor  Charvet 
and  Paolo  Meda  CMU,  Geneva,  Switzerland,  using  equipment  provided  by  Mauna  Kea  Technologies,  Paris,  France;  (D)  courtesy  of  Elisabeth  Laemmel  and  Eric 
Vicaut,  LEM,  Paris,  France;  and  (E)  courtesy  of  Vladimir  Dubaj  and  Andrew  Wood,  Swinburne  University,  Melbourne,  Australia,  utilizing  equipment  provided 
by  Optiscan,  Melbourne,  Australia.] 


near  the  fiber  tip  (Giniunas  et  al. ,  1991).  Fused-fiber  microlenses, 
gradient  index  lenses,  and  polymer  lenses  formed  by  the  light 
leaving  the  fiber  tip  have  all  been  tried.  While  offering  appealing 
dimensions  in  both  diameter  and  length,  such  systems  are  gener¬ 
ally  impractical  due  to  their  short  working  distance  and/or  narrow 
beam  diameter. 


Scanning  with  Micromirrors 

Dickinsheets  and  Kino  (1996)  have  demonstrated  a  system  using 
two  mirrors  micro-machined  from  crystalline  silicon.  An  optical 
design  by  Olympus  using  a  single  mirror  held  in  a  gimbal  arrange¬ 
ment  with  torsion  strips  to  provide  resonance  has  also  been  demon- 


FIGURE  26.1 0.  Miniature  scanning-fiber  confocal  microscope 
scanner  and  rigid  endoscope.  The  scanner  (pictured  inset  top 
left)  contains  integrated  raster  scanning,  a  z-axis  actuator,  and 
projection  objective  optics  encapsulated  behind  a  fixed  external 
window.  The  scanner  produces  1024  x  1024  images  at  1  frame 
per  second.  The  objective  NA  is  0.55,  providing  lateral  resolu¬ 
tion  of  0.7  pm  and  axial  resolution  of  7  pm.  The  external 
window  is  placed  into  contact  with  the  sample  and  the  z-axis 
actuator  shifts  the  imaging  depth  from  0  to  250  pm  relative  to 
the  contact  surface.  For  in  vivo  microscopy,  the  scanner  is  pack¬ 
aged  for  the  requirements  of  the  application,  in  this  case  as  a 
rigid  arthroscope  for  minimally  invasive  surgical  access  to  inter¬ 
nal  organs,  or  in  the  flexible  endomicroscope  shown  in  Figure 
26.13.  Images  show  a  prototype  made  by  Optiscan  Pty  Ltd, 
Melbourne,  Australia.  A  schematic  representation  of  the  con¬ 
figuration  of  key  components  in  the  scanner  is  shown  inset  at 
lower  right. 
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strated.  The  fiber  emits  light  through  a  hole  in  the  center  of  a 
primary  catadioptic  scanning  mirror.  The  beam  path  is  then  folded 
by  a  secondary  mirror  on  the  surface  of  the  first  lens  element 
(Murakami  et  al.,  2005). 

Scanning  Fiber  Confocal  Microscopes  for 
In  Vivo  Imaging  in  Animals 

The  scanning  fiber  approach,  described  above  and  diagrammed  in 
Figure  26.7,  has  been  implemented  in  a  5  mm  diameter  scanner 
and  packaged  either  as  a  hand-held  probe  or  as  a  rigid  endoscope, 


as  shown  in  Figure  26.10.  The  devices  also  include  an  active 
z-axis  actuator  that  moves  the  entire  scanning  mechanism  and  lens 
system  relative  to  an  imaging  window  incorporated  into  the  end  of 
the  device.  The  window  at  the  tip  of  the  device  is  thus  placed 
directly  against  the  animal  tissue  of  interest,  rather  than  the  tissue 
being  placed  onto  a  stage.  Mechanical  contact  provides  a  high 
degree  of  stability,  and  the  images  are  obtained  interactively,  with 
hand-held  controls  to  adjust  the  imaging  plane  depth  relative  to  the 
window  surface.  Sample  images  are  presented  in  Figure  26.11, 
demonstrating  practical  imaging  of  microvascular,  cellular,  and 
subcellular  structures  in  vivo. 


FIGURE  26.1 1 .  In  vivo  microscopy  using  the  scanning  fiber  confocal  microscope  shown  in  Figure  26.10.  (A)  Mouse  liver  imaged  following  intravenous  admin¬ 
istration  of  FITC-dextran.  (B)  A  region  of  this  image  zoomed  in  software  to  reveal  the  high  information  content  and  stability  achievable  in  vivo.  (C)  Image  of 
mouse  brain  following  intravenous  administration  of  Acriflavine,  highlighting  the  longitudinal  orientation  of  the  nuclei  of  vascular  epithelial  cells  versus  the  cir¬ 
cumferential  orientation  of  nuclei  in  vascular  smooth  muscle  and  muscle.  All  images  are  single  1  s  scans  at  1024  X  1024  pixel  resolution,  ex  =  488nm,  det  = 
505-585  nm,  and  are  courtesy  of  Dr.  Ralf  Kiesslich  and  Dr.  Martin  Goetz,  Mainz  University  Hospital,  Mainz,  Germany. 
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IMPLEMENTATIONS  FOR  CLINICAL 
ENDOMICROSCOPY 

Trends  in  medical  diagnostics  and  therapeutics  are  towards  point- 
of-care  diagnostics  and  ever  more  rapid  delivery  of  diagnostic 
information  for  guiding  timely  medical  interventions.  There  has 
thus  been  growing  interest  in,  and  research  into,  various  techniques 
for  achieving  non-invasive  histologic  imaging  using  miniaturized 
devices  suitable  for  in  vivo  microscopy.  In  the  past  few  years, 
numerous  clinical  applications  have  arisen,  accelerated  by  the  rel¬ 
atively  recent  development  of  prototypes  and  commercial  systems 
certified  for  clinical  investigation. 

Three  areas  have  involved  substantial  research  and  develop¬ 
ment  effort  and  have  been  applied  clinically: 

Skin:  Both  non-fiber-optic  and  scanning  fiber  (Gonzalez  and 
Tannous,  2002;  Swindle  et  al,  2003). 


Cervix:  Both  fiber  bundle  and  scanning  fiber  (Liang  et  al.,  2001; 

McLaren  et  al.,  2003;  Fig.  26.12). 

Gastrointestinal  tract:  The  scanning  fiber  approach  has  resulted  in 
a  certified  commercial  gastrointestinal  endomicroscope  by 
Pentax,  Japan  (Delaney  et  al.,  1994b;  Kiesslich  et  al,  2004; 
see  Fig.  26.14). 

Clinical  studies  in  all  three  areas  have  achieved  diagnostic  sig¬ 
nificance,  suggesting  a  likely  role  for  these  technologies  in  clini¬ 
cal  practice. 


SUMMARY 

Although  to  date  relatively  few  developments  have  resulted  in 
practical  systems  for  clinical  investigation,  numerous  ongoing 
efforts  are  aimed  at  achieving  clinical  confocal  microscopy,  and 


FIGURE  26.12.  Imaging  of  the  human  cervix  using  miniature  confocal  microscope  heads.  (A)  The  in  vitro  images  of  cervical  epithelium  obtained  with  a 
reflectance  confocal  microscope  demonstrates  the  ability  to  resolve  cellular  and  nuclear  structures  in  whole  tissues.  This  epi-illumination  confocal  microscope 
consists  of  an  810nm  continuous-wave  diode  laser,  a  pair  of  galvanometer  scanning  mirrors,  a  scan  lens,  and  a  25  X  0.8  NA  water-immersion  objective  lens. 
The  measured  lateral  and  axial  resolution  of  this  system  are  0.8  and  2  to  3  pm,  respectively.  The  focus  plane  is  50pm  below  the  surface,  the  field  width  -460. 
(B,  C)  In  vivo  fiber-optic  confocal  microscope  backscattered  light  images  of  cervical  epithelium  classified  as  (A)  normal  and  (B)  abnormal,  CIN  II/III  at  histo¬ 
logic  examination.  FOV  -200  X  200  pm.  The  prototype  constructed  for  this  study  scans  1064nm  continuous  illumination  from  a  Nd:YAG  laser  focused  into  a 
30,000-fiber  coherent  imaging  bundle  (average  core  diameter  4  pm,  average  pitch  7  pm,  fiber  NA  0.3).  A  miniature  objective  lens  at  the  distal  tip  has  a  back  NA 
of  0.3  and  an  objective  NA  of  1.0.  Specular  reflection  from  the  fiber  surfaces  is  reduced  by  a  polarizing  beam-splitter,  index-matching  oil  at  both  fiber  bundle 
surfaces,  and  a  7°  angle  polish  on  the  distal  fiber  bundle  surface.  The  measured  lateral  and  axial  resolution  for  this  fiber-optic  confocal  microscope  are  2.1  and 
3  to  6pm,  respectively.  Images  are  obtained  at  15  frames  per  second.  Hydraulic  suction  is  used  to  axially  scan  tissue  through  the  fixed  focal  plane.  The  water 
is  also  used  to  reduce  specular  reflection  from  the  surface  of  the  tissue.  (Images  courtesy  of  Rebecca  Richards- Kortum.)  (D,  E)  Clinical  in  vivo  imaging  of  human 
cervix  using  a  fiber-scanning  confocal  microscope  prototype  similar  to  that  pictured  in  Figure  26.10.  Fluorescence  images  (excitation  488  nm,  detection  above 
505  nm)  following  topical  applications  of  0.05%  Acriflavine  (FOV:  500  X  350  pm).  Nuclei  are  well  resolved,  showing  regular  nuclear  size,  shape,  and  distribu¬ 
tion  in  normal  cervix  (D)  versus  crowded  distribution  and  irregular  size  and  shape  of  nuclei  (E)  in  squamous  cell  carcinoma  (diagnoses  provided  by  conven¬ 
tional  histopathological  assessment  of  biopsies).  (Images  courtesy  of  Dr.  Jeffrey  Tan,  Royal  Women’s  Hospital,  Melbourne,  Australia.) 
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FIGURE  26.1 3.  Clinical  endomicroscope  distal  tip.  The  single-fiber  scanner  shown  in  Figure  26.10  has  been  incorporated  into  a  clinical  endomicroscopy  system 
by  Pentax  Corporation,  Medical  Division  (Tokyo,  Japan).  The  distal  tip  is  12.8  mm  in  diameter,  and  accommodates  the  conventional  endoscope  components  and 
functions  (labeled)  for  macroscopic  imaging,  illumination,  biopsy  suction,  spraying,  and  articulation.  The  confocal  scanner  protrudes  by  several  millimeters,  and 
appears  within  the  corner  of  the  conventional  charge-coupled  device  (CCD)  view,  thus  allowing  the  endoscopist  to  target  it  to  specific  locations  on  the  tissue. 
(Image  courtesy  of  Pentax  Corporation.) 


FIGURE  26.14.  Confocal  imaging  of  the  human  gastrointestinal  tract  in  patients  undergoing  clinical  endoscopy.  (A-C)  Colonic  mucosa.  (A)  Crypt  architec¬ 
ture,  goblet  cells,  and  epithelial  cells  in  the  plane  of  the  superficial  epithlelium.  (B)  Subsurface  microvasculature  surrounded  by  the  collagen  matrix  of  the  lamina 
propria ,  50  (im  beneath  the  tissue  surface,  presenting  the  radial  alignment  of  epithelial  (including  goblet)  cells  lining  the  crypt  lumen.  (C)  Imaging  of  a  rectal 
adenocarcinoma  showing  complete  loss  of  crypt  architecture,  marked  cellular  pleomorphism,  and  patchy  distribution  of  goblet  cells  throughout  the  image  plane. 
(D-F)  Human  upper  gastrointestinal  images.  (D)  Surface  epithelium  and  (E)  hyper- vascularization  of  the  lamina  propria  in  a  66-year-old  male  with  portal  hyper¬ 
tension.  (F)  In  a  62-year-old  patient  with  Barrett’s  esophagus,  a  small  region  of  neoplasia  is  imaged  with  the  endomicroscope.  [Images  (A,B)  courtesy  of  Prof. 
Adrian  Polglase,  Cabrini  Monash  University  Department  of  Surgery,  Melbourne,  Australia.  Images  (C-F)  courtesy  of  Dr.  Ralf  Kiesslich,  Mainz  University  Hos¬ 
pital,  Mainz,  Germany.] 
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even  the  miniaturization  of  multi-photon  microscopes  suitable  for 
in  vivo  microscopy.  Thus,  it  is  reasonable  to  expect  that  many  other 
approaches  will  soon  become  practical  tools  for  clinical  and  labo¬ 
ratory  in  vivo  scanning  optical  microscopy. 

ACKNOWLEDGMENTS 

We  are  grateful  to  Mauna  Kea  Technologies  (Paris,  France)  for 
Figure  26.8(A,B,D,E),  to  Igor  Charvet  and  Paolo  Meda  (CMU, 
Geneva,  Switzerland)  for  Figure  26.9(A,B,C),  to  Vladimir  Dubaj 
and  Andrew  Wood  (Swinburne  University,  Melbourne,  Australia) 
for  Figure  26.9(E),  to  Elisabeth  Laemmel  and  Eric  Vicaut  (LEM, 
Paris,  France)  for  Figure  26.9(D),  to  Dr.  Ralf  Kiesslich  and  Dr. 
Martin  Goetz  (Mainz  University  Hospital,  Mainz,  Germany)  for 
Figure  26.11,  to  Rebecca  Richards-Kortum  (Rice  University, 
Houston,  TX)  for  Figure  26.12(A-C),  to  Dr.  Jeffrey  Tan  (Royal 
Women’s  Hospital,  Melbourne,  Australia)  for  Figure  26.12(D,E), 
and  to  the  Pentax  Conporation  (Tokyo,  Japan)  for  Figures  26.13 
and  26.14.  Dr.  Wendy  McLaren  provided  invaluable  assistance 
with  the  figures. 

REFERENCES 

Bjarklev,  A.,  Broeng,  J.,  and  Bjarklev,  A.S.,  2003,  Photonic  Crystal  Fibers, 
Kluwer  Academic  Publications,  New  York,  pp.  1-298. 

Cave,  C.,  Fourmeaux  du  Sartel,  M.,  and  Abrat,  B.,  2005,  Fibered  confocal  flu¬ 
orescence  microscopy:  A  new  tool  to  perform  colonoscopy  in  mice.  Gas¬ 
troenterology  128(Suppl  2):A-249. 

Delaney,  R,  Harris,  M.,  and  King,  R.,  1993a,  Novel  microscopy  using  fiber¬ 
optic  confocal  imaging  and  its  suitability  for  subsurface  blood  vessel 
imaging  in  vivo,  Clin.  Exp.  Pharmacol.  Physiol.  20:197-198. 

Delaney,  P.,  Harris,  M.,  and  King,  R.,  1993b,  Remote  interrogation  of  objec¬ 
tive  lenses  using  a  novel  fiber-optic  confocal  microscope  reduces  move¬ 
ment  artifacts,  In:  The  1993  International  Conference  on  Confocal 
Microscopy  and  3D  Image  Processing  Proceedings,  Sydney,  Australia, 

p.  66. 

Delaney,  R,  Harris,  M.,  and  King,  R.G.,  1994a,  Fiber-optic  laser  scanning 
confocal  microscope  suitable  for  fluorescence  imaging,  Appl.  Opt. 
33:573-577. 

Delaney,  P.,  King,  R.,  Lambert,  J.,  and  Harris,  M.,  1994b,  Fiber-optic  confocal 
imaging  (FOCI)  for  subsurface  imaging  of  the  colon  in  vivo.  J.  Anat. 
184:157-160. 

Dickensheets,  D.L.,  and  Kino,  1996,  A  micromachine  scanning  confocal  optical 
microscope.  Opt.  Lett.  21:764-766. 

Ellis,  G.W.,  1985,  Microscope  illuminator  with  fiber-optic  source  integrator, 
J.  Cell.  Biol.  101:832a. 

Fork,  R.,  Martinez,  O.E.,  and  Gordon,  J.,  1984,  Negative  dispersion  using  pairs 
of  prisms,  Opt.  Lett.  9:150. 


Gmitro,  A.,  and  Aziz,  D.,  1993,  Confocal  microscopy  through  a  fiber-optic 
imaging  bundle.  Opt.  Lett.  18:565-567. 

Giniunas,  L.,  Juskaitis,  R.,  and  Shatalin,V.,  1991,  Scanning  fiber-optic  micro¬ 
scope,  Electron  Lett.  27:724-726. 

Gonzalez,  S.,  and  Tannous,  Z.,  2002,  Real-time,  in  vivo  confocal  reflectance 
microscopy  of  basal  cell  carcinoma,  J.  Am.  Acad.  Dermatol.  47:869-874. 

Harris,  M.R.,  UK  Patent,  GB  2,340,332.  Granted  16-02-2000. 

Harris,  M.,  and  Delaney  P.M.,  US  Patent,  5,926,592.  Granted  20-07-1999. 

Harris,  M.,  and  Delaney,  R,  US  Patent  no.  5,926,592. 

Katzir,  A.,  1993,  Applications  of  lasers  in  therapy  and  diagnosis,  In:  Lasers  and 
Optical  Fibers  in  Medicine,  Academic  Press,  San  Diego,  California. 

Kiesslich,  R.,  Burg,  J.,  Veith,  M.,  Burg,  J.,Vieth,  M.,  Gnaendiger,  J.,  Enders, 
M.,  Delaney,  P,  Polglase,  A.,  McLaren,  W.,  Janell,  D.,  Thomas,  S.,  Nafe, 
B.,  Galle,  P,  and  Neurath,  M.,  2004,  Confocal  laser  endoscopy  for  diag¬ 
nosing  intraepithelial  neoplasias  and  colorectal  cancer  in  vivo.  Gastroen¬ 
terology  127:706-713. 

Kiesslich,  R.,  et  al.,  2005,  Diagnosing  Helicobacter  pylori  in  vivo  by  confocal 
laser  endoscopy,  Gastroenterology.  Forthcoming. 

Kimura,  S.,  and  Wilson,  T.,  1991,  Confocal  scanning  optical  microscope  using 
single  mode  fiber  for  signal  detection,  Appl.  Opt.  30:2143-2150. 

Kino  and  Dickinsheets,  D.L.,  US  Patent  no.  6,172,789. 

Knight,  J.,  Broeng,  J.,  Birks,  T.,  and  Russel,  R,  1998,  Photonic  bandgap  guid¬ 
ance  in  optic  fibers.  Science  282:1476 

Liang,  C.,  Sung,  K.,  Richards-Kortum,  R.,  and  Descour,  M.,  2001,  Fiber  con¬ 
focal  reflectance  microscope  (FCRM)  for  in-vivo  imaging,  Opt.  Express 
9:821-830. 

McLaren,  W.,  Tan,  J.,  and  Quinn,  M.,  2003,  Detection  of  cervical  neoplasia 
using  non-invasive  fibreoptic  confocal  microscopy,  In:  Eurogin  2003 
(J.  Monsonego,  ed.),  Monduzzi  Editore,  Bologna,  Italy,  pp.  213-217. 

Murakami,  M.,  2005,  A  miniature  confocal  optical  scanning  microscope  for 
endoscopy,  in  MOEMS  display  and  imaging  systems  III,  (H.  Urey  and 
D.L.  Dickensheets,  eds.),  Proc  SPIE.  5721:119-131. 

Ouzounov,  D.,  Moll,  K.,  Forster,  M.,  Zipfel,  W.,  Webb,  W.,  and  Gaeta,  A.,  2002, 
Delivery  of  nanojoule  femtosecond  pulses  through  large-core  microsure 
fibers.  Opt.  Lett.  27:1513-1515. 

Sharonov,  S.,  Morjani,  H.,  and  Manfait,  M.,  Confocal  spectral  imaging  analy¬ 
sis:  A  new  concept  to  study  the  drug  distribution  in  a  single  living  cancer 
cell,  Anticancer  Res.  12: 1 804. 

Snyder,  A.,  and  Love,  J.,  Optical  Waveguide  Theory,  Chapman  and  Hall, 
London  1983. 

Sung,  K.,  Liang,  C.,  Descour,  M.,  Collier,  T.,  Follen,  M.,  and  Richards-Kortum, 
R.,  2002,  Fiber-optic  confocal  reflectance  microscope  with  miniature 
objective  for  in  vivo  imaging  of  human  tissues,  IEEE  Trans.  Biomed.  Eng. 
49:1168-1172. 

Swindle,  L.,  Thomas,  S.,  Freeman,  M.,  and  Delaney,  P,  2003,  View  of  normal 
human  skin  in  vivo  as  observed  using  fluorescent  fiber-optic  confocal 
microscopic  imaging,  J.  Invest.  Dermatol.  121:706-712. 

Wilson,  J.,  and  Hawkes,  J.,  Optoelectronics.  An  Introduction,  2nd  ed., 
Prentice-Hall,  Hemel  Hempstead,  UK  Sects.  8.3.3,  8.3.4,  and  8.5.2. 

Wilson,  T.,  Juskaitis,  R.,  and  Rimvydas,  1992,  Confocal  and  differential 
microscopy  using  optical  fiber  detection.  In:  Proceedings  of  SPIE- 
International  Society  for  Optical  Engineering.  Vol.  1660pt  2. 


27 


Fluorescence  Lifetime  Imaging  in  Scanning  Microscopy 

H.C.  Gerritsen,  A.  Draaijer,  D.J.  van  den  Heuvel,  and  A.V.  Agronskaia 


INTRODUCTION 

Fluorescence,  Lifetime,  and  Quantum  Efficiency 

It  was  not  until  the  1930s  that  measurement  of  the  fluorescence 
lifetime  (y)  and  the  confirmation  of  theory  on  this  phenomenon 
became  possible  (Pringsheim,  1961).  Before  that  time  only  phos¬ 
phorescence  lifetimes  had  been  measured,  evidently  because  this 
phenomenon  is  one  or  more  orders  of  magnitude  slower.  The  dis¬ 
tinction  between  fluorescence  and  phosphorescence  was  first  made 
by  the  observation  of  the  afterglow.  Emission  having  a  noticeable 
afterglow  was  called  phosphorescence.  All  other  processes  that  did 
not  have  a  noticeable  afterglow  were  called  fluorescence.  Later, 
the  distinction  between  the  two  was  based  on  quantum  mechanics. 
In  phosphorescence  a  triplet  to  ground  state  transition  is  involved. 
Quantum  mechanically  such  a  transition  is  spin  forbidden  and 
therefore  has  a  low  probability.  Consequently,  the  rate  at  which 
such  a  process  occurs  is  low  and  the  lifetime  is  long.  The  fluores¬ 
cence  phenomenon  only  involves  singlet  states  with  transition 
probabilities  that  are  much  higher,  making  the  process  faster.  In 
its  simplest  form,  the  fluorescence  or  phosphorescence  intensity 
decay  after  the  excitation  source  is  shut  off  follows  an  exponen¬ 
tial  curve  described  by: 

m  =  V'/T  (1) 

where  I(t)  is  the  fluorescence  intensity  at  time  t ,  I0  is  the  the  fluo¬ 
rescence  intensity  at  time  zero,  and  y  is  the  time  when  the  fluo¬ 
rescence  has  dropped  to  I0/e. 

The  fluorescence  quantum  efficiency  <Ey  (Straughan  and 
Walker,  1976)  is  defined  as  the  ratio  of  the  number  of  photons 
absorbed  to  the  number  of  photons  emitted  as  fluorescence  by  the 
molecule.  It  is  dependent  on  the  relative  magnitudes  of  the  rate 
constants  involved  in  fluorescence  and  other  competing  decay 
processes.  The  processes  involved  in  the  decay  of  a  molecule  with 
singlet  ground  state,  S0,  excited  singlet  state,  Si,  and  excited  triplet 
state,  T!  are  diagrammed  in  Figure  27.1  (Jablonski  diagram)  and 
listed  in  Table  27.1. 

It  can  be  shown  that  <ty  =  k^/Skprocesses  and  y  =  l/Ekf^-esse,,  = 
Oy/kf  where  <6,  is  the  fluorescence  quantum  efficiency,  kf  is  the 
fluorescence  decay  rate  constant,  kISC  is  the  rate  constant  of  inter¬ 
system  crossing,  kiC  is  the  rate  constant  of  internal  conversion,  and 
kRe  is  the  rate  constant  for  reaction  product. 

In  the  case  of,  for  example,  different  microenvironments  or  dif¬ 
ferent  conformations  of  the  fluorescent  molecules,  the  decay  curve 
may  become  multi-exponential. 


Fluorescence  Lifetime  Spectroscopy 

Early  measurements  of  nanosecond  lifetimes  on  bulk  specimens 
were  performed  by  means  of  Pockel  or  Kerr  cells  (Pringsheim, 
1961).  Such  devices  contain  an  optically  transparent,  nonlinear 
crystalline  solid  (Pockel  cell)  or  an  optically  transparent,  nonlinear 
liquid  (Kerr  cell).  The  polarization  direction  of  light  passing 
through  these  cells  is  rotated  by  putting  a  voltage  across  them. 

Because  detectors  and  amplifier  electronics  were  not  yet  fast 
enough  for  direct  measurements,  y  could  only  be  measured  by 
applying  a  time  difference  between  the  excitation  and  emission 
cells.  This  was  accomplished  by  changing  the  distance  between  the 
two  cells  and  measuring  the  average  intensity  as  a  function  of  the 
time  difference  between  the  two  cells.  Light  travels  30cm  in  Ins. 

The  popularity  of  fluorescence  lifetime  spectroscopy  has 
grown  considerably  in  recent  decades  because  of  the  availability 
of  fast  electronics,  detectors,  and  light  sources. 

A  number  of  techniques  are  available  to  measure  the  decay 
curves  of  fluorescence  processes.  In  general,  y  measurements  are 
either  made  in  the  time  domain  or  in  the  frequency  domain.  In  the 
frequency  domain,  one  measures  the  phase  shift  of  the  fluorescent 
light  with  respect  to  the  phase  of  a  modulated  excitation  source. 
Measurements  in  the  time  domain  are  generally  performed  by 
measuring  the  time  dependency  of  the  fluorescence  intensity  with 
respect  to  a  short  excitation  pulse.  It  falls  beyond  the  scope  of  this 
chapter  to  give  a  detailed  overview  of  all  the  numerous  imple¬ 
mentations  and  applications  of  fluorescence  lifetime  spectroscopy. 
The  reader  is  referred  to  the  existing  literature  (Lakowicz,  1983; 
O’Connor  and  Phillips,  1984;  Clark  and  Hester,  1989).  Many 
applications  of  fluorescence  lifetime  spectroscopy  can  be  found 
in  the  fields  of  protein  studies,  membranes  and  cells,  and  energy 
transfer  and  ion  concentration  measurements.  We  will  restrict  our 
consideration  to  microscopic  applications. 

Fluorescence  Lifetime  Imaging  Applications 

Apart  from  measurements  on  bulk  specimens,  fluorescence  life¬ 
time  spectroscopy  equipment  can  be  modified  for  microspec¬ 
troscopy  and  microscopy.  This  makes  it  possible  to  measure  y  of 
small  volumes  in  a  specimen.  Starting  in  the  last  decade  of  the 
20th  century,  fluorescence  lifetime  measurements  were  extended 
into  the  field  of  widefield  imaging  (Morgan  et  al.,  1990,  1992; 
Lakowicz  and  Berndt,  1991;  Ni  and  Melton,  1991;  Wang  et  al., 
1991;  Gadella  et  al.,  1993;  Lakowicz  and  Szmacinski,  1993;  Webb 
et  al.,  2002),  confocal  imaging  (Bugiel  et  al.,  1989;  Buurman  et 
al,  1992;  Sanders  et  al.,  1994),  and  multi-photon  excitation 
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FIGURE  27.1 .  Jablonski  diagram  showing  the  processes  involved  in  the  decay 
of  a  molecule  with  singlet  ground  state  S0,  excited  singlet  state  Si,  and  excited 
triplet  state  1| . 


imaging  (Piston  et  al.,  1992;  So  et  al.,  1996;  French  et  al.,  1998; 
Sytsma  et  al.,  1998).  Fluorescence  lifetime  imaging  (FLIM)  can 
be  used,  for  example,  in  multi-labeling  experiments,  for  measuring 
ion  concentration  or  cell  chemistry,  and  for  measuring  fluorescence 
resonance  energy  transfer  (FRET)  efficiencies.  These  uses  are 
discussed  below. 

Multi-Labeling  with  Dyes 

In  general,  probes  that  have  the  same  emission  wavelength  usually 
do  not  have  the  same  y,  so  they  can  be  distinguished  based  on  their 
lifetime  differences  provided  the  microscope  has  sufficient  lifetime 
resolution.  The  number  of  probes  that  can  be  imaged  simultane¬ 
ously  depends  on  the  lifetime  range  of  the  FLIM  system.  In  the 
spectral  domain,  the  emission  bands  are  broad  and  in  conventional 
fluorescence  microscopes  not  more  than  two  or  three  probes  can 
be  imaged  simultaneously,  with  each  probe  necessarily  having  its 
own  detection  channel.  However,  given  a  typical  y  resolution  in 
the  sub-nanosecond  range  and  a  dynamic  range  of  tens  of  nano¬ 
seconds,  it  should  be  possible  to  simultaneously  image  10  to  20 
probes.1  Recording  probes  with  differing  y  can  be  achieved  with 
only  a  single  detector  and  one  excitation  wavelength,  and  it  does 
not  require  the  use  of  spectral  selection  on  the  emission  side. 
Because  no  bandpass  filters  are  needed,  the  complete  spectral  band 
is  detected  and  detector  sensitivity  is  increased. 

Discrimination  of  the  signals  from  specific  probes  based  on 
their  x,  alone  can  be  done  using  simple  image  processing.  In  com¬ 
bination  with  spectral  selection,  the  number  of  distinct  probes  that 
can  be  imaged  becomes  even  larger. 

With  some  a  priori  knowledge  of  the  y  of  the  probes  used, 
signals  from  mixtures  of  probes  are  easily  deconvolved  and  the 
relative  abundances  calculated  (Verveer  et  al,  2000).  Some  probes 
have  even  been  designed  specifically  for  y  imaging  (Sauer  et  al., 
1993). 

Concentration  Imaging 
(pH,  Ca2+,  K+,  Na+,  02,  etc.) 

For  many  existing  fluorescent  ion  indicators,  the  y  of  the  free  form 
differs  from  that  of  the  ion-bound  form.  In  principle  this  property 
can  be  used  for  the  quantitative  imaging  of  ions  (pH,  Ca2+,  K+,  Na+) 


1  As  long  as  they  do  not  all  occur  in  the  same  location. 


(Lakowicz  and  Szmacinski,  1993;  Lakowicz  et  al.,  1994;  Sanders 
et  al.,  1994,  1995).  Oxygen  and  other  quenching  agents  reduce 
y  (Gerritsen  et  al.,  1997).  This  effect  can  also  be  used  for  imag¬ 
ing  purposes,  making  it  possible  to  image  important  biological 
parameters. 

Fluorescence  lifetime  imaging  has  tremendous  advantages 
compared  to  other  methods  that  are  either  invasive  (micro¬ 
electrodes)  or  require  a  calibration  on  the  specimen  (ratio  imaging). 
Using  y  as  a  parameter  for  measuring  dye  concentrations  makes 
the  measurements  independent  of  intensity  effects  such  as  shading 
in  the  image,  changes  in  laser  intensity,  or  absorption  in  the 
specimen,  and  calibration  is  often  easier  than  calibration  of  X-ratio 
images.  For  several  fluorescent  ion  indicators  it  was  shown  that  no 
in  vivo  calibration  procedure  is  required  (Sanders  et  al,  1995; 
Herman  et  al.,  1997). 

Chemical  Environment 

The  y  of  a  given  probe  is  often  sensitive  to  local  differences  in 
chemical  environment.  This  can  be  exploited  to  obtain  informa¬ 
tion  about  micro-environments  in  general.  For  example,  one 
could  image  hydrophobicity,  viscosity  (molecular  friction  effects), 
mobility,  or  membrane  potential. 

Quantitative  Fluorescence 
Intensity  Measurements 

Most  users  of  a  fluorescence  microscope  measure  the  concentra¬ 
tion  of  a  fluorescent  probe  to  get  information  about,  for  example, 
the  extent  of  DNA  damage  or  the  structure  of  an  object.  However, 
the  intensity  signal  is  only  a  good  measure  of  the  dye  concentra¬ 
tion  if  the  quantum  efficiency  stays  constant  and  the  environment 
often  affects  the  quantum  efficiency.  A  calibration  procedure  is 
required  to  correlate  the  fluorescence  intensity  to  the  probe  con¬ 
centration  and  this  procedure  may  be  suspect  because  the  calibra¬ 
tion  environment  is  different  from  the  specimen  environment. 
Fortunately,  y  is  closely  related  to  the  quantum  efficiency  and  so 
a  measurement  of  y  can  be  used  to  check  the  validity  of  the  cali¬ 
bration  procedure.  If  the  y  measured  during  calibration  is  equal  to 
that  in  the  specimen,  this  indicates  that  the  quantum  efficiency  is 
the  same  in  both  cases  and  thus  the  intensity  can  be  correlated  to 
the  concentration.  If  factors  that  do  not  effect  y,  such  as  photo- 
bleaching,  occur,  making  the  y  measurement  increases  the  reli¬ 
ability  of  the  intensity  measurement.  In  some  cases  it  has  been 
possible  to  correct  for  changes  in  the  quantum  efficiency  by  using 
y  information  (Morgan  et  al.,  1990). 

Fluorescence  Resonance  Energy  Transfer 

One  of  the  more  recent  applications  of  fluorescence  microscopy  is 
the  imaging  of  colocalization  on  a  nanometer  scale  by  means 
of  Forster  resonance  energy  transfer  (Forster,  1946;  Lakowicz, 
1983;  Clegg,  1996).  This  technique  is  also  often  referred  to  as 
fluorescence  resonance  energy  transfer  (FRET).  More  details  on 
FRET  can  be  found  in  Chapter  45.  In  conventional  fluorescence 
microscopy  colocalization  can  be  studied  by  using  two  differently 
labeled  molecules  that  emit  in  two  distinct  emission  bands.  As 
soon  as  signals  from  both  detection  channels  occur  in  the  same 
pixel  the  two  molecules  colocalize  within  the  resolution  limit  of 
the  microscope.  Because  the  resolution  of  the  microscope  is 
much  larger  than  the  size  of  the  molecules,  colocalization 
measured  in  this  way  does  not  prove  that  the  two  molecules 
interact.  FRET  can  be  employed  to  study  colocalization  on  a  scale 
of  a  few  nanometers  and  therefore  this  method  does  yield 
information  about  molecular  interactions.  In  FRET  an  excited 
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TABLE  27.1.  Processes  Involved  in  the  Decay  of  an  Excited  Molecule 


Molecule  S0  +  hv 

_ 

Molecule  Si 

Excitation 

Molecule  Si 

— 

Molecule  S0  +  hv 

Fluorescence  (F) 

Molecule  Si 

Molecule  T, 

Intersystem  crossing  (ISC);  non-destructive  quenching  effect  (triplet  state  build-up)  where  molecule 
will  become  available  again  only  after  returning  to  the  ground  state  (phosphorescence) 

Molecule  Si 

Molecule  S0 

Internal  conversion  (IC);  non-destructive  quenching  effect  (molecule  stays  available) 

Molecule  Si 

Products 

Reaction  (Re);  Destructive  quenching  effect  (molecule  is  lost  for  fluorescence) 

Molecule  Si 

Molecule  S0 

Non-radiative  decay  (NR);  molecule  returns  to  the  ground  state  without  fluorescing 

fluorescent  donor  molecule  directly  transfers  its  excited  state 
energy  to  an  acceptor  molecule.  This  is  a  resonant  process  that 
occurs  through  a  dipole-dipole  interaction  between  the  donor  and 
acceptor.  A  prerequisite  for  FRET  to  occur  is  that  the  emission 
band  of  the  donor  overlaps  with  the  absorption  band  of  the 
acceptor  and  in  addition  the  two  molecules  should  be  within  a 
few  nanometers  of  each  other.  The  presence  of  the  acceptor 
results  in  the  introduction  of  an  additional  decay  channel  for  the 
excited  donor  molecules  and  therefore  the  total  decay  rate  of  the 
donor  molecule  is  increased.  Consequently,  FRET  results  in  a 
reduction  of  both  the  donor  fluorescence  intensity  and  the  fluores¬ 
cence  lifetime  (X/  =  l/EkProcesses).  FRET  can  be  conveniently  quan¬ 
tified  in  terms  of  the  energy  transfer  efficiency  E.  E  can  be 
expressed  in  the  relative  donor  intensity  reduction  E  =  1  -  IDA/[n, 
and  in  the  relative  donor  lifetime  reduction  E  =  1  -  XDA/xD. 
Flere,  IDA  and  ID  are  the  intensities  of  the  donor  in  the  presence 
and  absence  of  the  acceptor  respectively.  xDA  and  xD  are  the  life¬ 
times  of  the  donor  in  the  presence  and  absence  of  the  acceptor, 
respectively. 

FRET  imaging  experiments  can  be  carried  out  by  means  of 
fluorescence  intensity  measurements.  However,  this  approach 
requires  the  use  of  emission  filters  and  quantification  of  E  is  com¬ 
plicated  by,  for  example,  filter  bleedthrough  and  direct  excitation 
of  the  acceptor.  In  order  to  correct  for  these  effects  multiple  refer¬ 
ence  images  need  to  be  recorded. 

FRET  imaging  can  also  be  accomplished  by  means  of 
FLIM  (Wouters  and  Bastiaens,  1999).  Now,  the  energy  transfer 
efficiency  E  can  be  directly  derived  from  a  single  measurement 
of  the  donor  lifetime,  provided  that  the  donor  lifetime  of  dye  in 
the  absence  of  the  acceptor  is  homogeneous  throughout  the 
specimen.  This  approach  is  straightforward  and  at  present  FLIM- 
based  FRET  imaging  seems  to  be  the  preferred  method  to  carry 
out  quantitative  FRET  experiments  on  interactions  between 
molecules. 

From  the  applications  listed  above  it  is  obvious  that  fluo¬ 
rescence  lifetime  contrast  is  a  versatile  and  powerful  tool  in 
microscopy.  In  the  next  section  we  discuss  the  basics  of  frequency- 
and  time-domain  methods  for  implementing  FLIM  and  a  number 
of  applications  of  this  technique  will  then  be  described. 


FLUORESCENCE  LIFETIME  IMAGING 
METHODS 

Introduction 

FLIM  can  be  implemented  in  widefield  microscopes  (WF), 
confocal  laser-scanning  microscopes  (CLSMs),  and  multi-photon 
excitation  microscopes  (MPEMs).  The  use  of  the  point  scanning 
geometry  in  FLIM  is  advantageous.  If  a  diffraction-limited 
volume  element  in  a  specimen  with  a  lifetime  Xi  is  embedded  in  a 


homogeneous  background  with  lifetime  x2  and  imaged  in  a  WF 
microscope,  the  measured  Tf  will  be  averaged  over  the  entire 
excited  volume.  This  will  contribute  a  large  x2  background  fluo¬ 
rescence  component  to  the  measured  decay.  In  the  confocal  setup, 
however,  only  the  small  volume  element  near  the  focal  plane  is 
sampled,  and  therefore  much  less  of  the  x2  background  component 
is  observed. 

At  present,  several  techniques  are  being  employed  for  the  mea¬ 
surement  of  X/ on  bulk  specimens.  Basically  the  techniques  can  be 
subdivided  in  frequency-domain-based  methods  such  as  phase 
fluorometry  (Gratton  and  Limkeman,  1983;  lameson  and  Gratton, 
1983;  Lakowicz,  1983)  and  time-domain-based  methods  such  as 
time-correlated  single-photon  counting  (O’Connor  and  Phillips, 
1984)  and  methods  based  on  time-gated  detection.  In  either 
method,  both  the  light  source  and  the  detection  system  have  to  be 
modified. 

A  major  difference  between  fluorescence  lifetime  measure¬ 
ments  on  bulk  specimens  and  fluorescence  lifetime  imaging  is  the 
number  of  fluorophores  available  in  the  detection  volume.  In  con¬ 
ventional  non-imaging  spectroscopy  experiments,  a  large  number 
of  fluorescent  molecules  is  present  in  the  detection  volume  and 
the  accuracy  of  the  experiments  is,  in  general,  not  limited  by 
the  strength  of  the  fluorescence  signal.  In  a  typical  confocal  or 
multi-photon  imaging  experiment,  only  a  small  volume  is  imaged 
and  in  general  a  low  number  of  fluorescent  molecules  is  present 
in  this  volume.  On  assuming  typical  dye  concentrations  of  tens 
of  micromolar,  we  find  that  on  the  order  of  100  to  200  dye 
molecules  are  present  in  the  focus  of  the  microscope  objective. 
Furthermore,  photobleaching  constrains  the  maximum  number 
of  photons  that  can  be  emitted  per  fluorescent  molecules  to 
values  between  103  to  105  photons  before  photobleaching  takes 
place. 

For  conventional  confocal  and  multi-photon  fluorescence 
intensity  imaging,  images  of  reasonable  quality  can  be  acquired 
with  as  few  as  20  to  30  detected  photons  per  pixel.  FLIM  requires 
much  more  signal  per  pixel  to  produce  acceptable  images.  In 
general,  at  least  one  order  of  magnitude  more  signal  is  required  to 
obtain  images  of  reasonable  quality.  Therefore,  the  efficiency  of 
the  microscope,  the  detector,  and  the  lifetime  acquisition  method 
are  of  crucial  importance.  For  this  reason  we  will  discuss  the 
factors  that  determine  the  sensitivity  of  the  different  lifetime 
imaging  methods  in  detail. 

Lifetime  Sensing  in  the  Frequency  Domain 
The  Phase  Fluorometry  Method 

Phase  fluorometry  is  one  of  the  oldest  methods  used  for  the  deter¬ 
mination  of  y.  The  essence  of  the  method  is  phase-sensitive  detec¬ 
tion  in  combination  with  intensity-modulated  excitation  of  the 
specimen.  Either  a  pulsed  or  a  sinusoidally  modulated  light  source 
is  used  in  this  technique.  In  the  case  of  sinusoidally  modulated 
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excitation,  the  time-dependent  excitation  intensity,  E(t)  (see  Fig. 
27.2)  is  given  by: 

E(t)  =  E0{1  +  M£sin(CGf))  (2) 

with  CO  the  modulation  frequency,  E0  the  average  excitation  inten¬ 
sity,  and  Me  the  modulation  depth  of  the  excitation,  defined  as  the 
ratio  between  the  AC  amplitude  and  the  DC  component  of  the  exci¬ 
tation  signal.  The  delay  between  the  absorption  of  an  excitation 
photon  and  the  emission  of  the  fluorescence  introduces  a  phase- 
shift  cf)  and  demodulation  of  the  emission  F{t)  with  respect  to  the 
excitation  light  (see  Fig.  27.2). 

Fit)  can  be  written  as: 

Fit)  =  F,\  I  +  Mf  sin(cot  +  0)]  (3) 

with  My  the  modulation  depth  of  the  emission.  For  a  simple  mono- 
exponentially  decaying  fluorescence  signal,  the  following  relation 
can  be  derived  relating  xs,  CO  and 

x  ,  =  —  tan(c|))  (4) 

'  co 

The  xf  of  a  specimen  can  be  accurately  determined  by  measuring 
the  phase  difference  between  the  excitation  signal  and  the  fluores¬ 
cence  signal.  Assuming  a  strong  signal,  §  can  usually  be  deter¬ 
mined  to  an  accuracy  of  ±0.2°.  The  phase  measurements  are 
usually  done  at  a  number  of  different  frequencies  ranging  from 
several  hundred  kilohertz  up  to  many  hundred  megahertz.  Using 
this  approach,  fluorescence  decays  containing  more  than  one  decay 
component  can  be  examined.  The  value  of  each  xf  as  well  as  its 
relative  contribution  to  the  total  fluorescence  signal  can  be  deter¬ 
mined  (Gratton  et  at,  1984). 

As  the  ratio  MF/ME  is  affected  by  xf,  this  ratio  can  be  also 
employed  to  obtain  quantitative  lifetime  information.  The  relation 
between  the  relative  modulation  iMF/ME)  and  xf  is: 
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Intensity  modulation  of  the  exciting  light  can  easily  be  accom¬ 
plished  by  using  electro-optical  modulators  (EOM)  or  acousto- 


FIGURE  27.2.  The  excitation  and  emission  signals  in  phase  fluorometry.  The 
emission  is  delayed  and  demodulated  with  respect  to  the  excitation  signal. 


optical  modulators  (AOM)  (see  Chapter  3,  this  volume ).  These 
devices  allow  modulation  over  a  wide  range  of  modulation  fre¬ 
quencies,  up  to  about  1  GFIz,  and  can  be  used  in  combination  with 
comparatively  cheap  CW  lasers. 

Alternatively  a  pico-  or  femtosecond  pulsed  laser  source  can 
be  used  as  a  modulated  light  source.  In  this  case  one  can  use  the 
harmonic  content  of  the  excitation  signal  to  produce  a  range  of 
modulation  frequencies.  Depending  on  the  pulse- width  of  the  laser, 
the  frequency  content  can  extend  up  to  200  GHz,  far  higher  than 
the  best  available  detectors. 

On  the  detection  side,  a  phase-sensitive  detector  is  required. 
This  is  often  realized  by  using  a  gain-modulated  photomultiplier 
tube  (PMT)  or  a  micro-channel-plate  detector,  modulated  at 
exactly  the  same  frequency  (homodyning)  or  slightly  different  fre¬ 
quency  (heterodyning)  as  the  excitation  light  source.  In  practice 
the  detector  system  will  limit  the  maximum  useful  excitation 
frequency.  Cross-modulation  is  normally  used  to  transform  the 
high  frequency  signal  to  a  lower  frequency  domain  because 
this  improves  noise  immunity.  All  the  noise  picked  up  at  high 
frequencies  is  simply  filtered  out.  In  the  case  of  a  fast  PMT,  the 
modulation  frequencies  will  be  limited  to  <1.4  GHz  while 
microchannel  plates  (MCP)  can  be  used  up  to  20  GHz. 

Fluorescence  Lifetime  Imaging  Using 
Phase  Fluorometry 
Widefield,  Slit-Scanning,  and 
Spinning-Disk  Implementations 

In  several  laboratories  a  phase-fluorometry-based  detection 
scheme  has  been  incorporated  in  a  WF  microscope  (see,  e.g., 
Morgan  et  al.,  1990,  1992;  Lakowicz  and  Berndt,  1991;  Marriott 
et  al.,  1991a;  Gadella  et  al.,  1993).  In  order  to  combine  the  phase 
fluorometry  method  with  WF  microscopy,  a  detector  sensitive  to 
phase  and  position  is  required.  One  possibility  is  a  gain-modulated 
image  intensifier  in  combination  with  a  charge-coupled  device 
(CCD)  detector.  The  intensifier  is  modulated  at  the  same  frequency 
as  the  excitation  source,  with  a  fixed  phase  angle  difference  with 
respect  to  the  phase  of  the  excitation  source.  Although  the  gain 
of  the  intensifier  is  modulated  at  very  high  frequencies,  its  output 
signal  does  not  show  high  frequency  components  because  it  is 
time  averaged  by  the  slow  response  of  its  output  phosphor  (about 
1  ms).  Consequently,  the  CCD  detector  records  a  time-averaged 
(low-pass  filtered)  intensity  image  where  the  intensities  depend 
on  the  phase  difference  between  the  modulated  excitation 
source  and  the  modulated  detector  gain.  The  time-averaged  fluo¬ 
rescence  intensity  recorded  in  one  CCD  pixel  is  determined  by  the 
detector  gain  modulation  depth.  MD,  the  modulation  depth  of 
the  fluorescence  emission,  MF,  and  the  phase  difference  between 
the  gain  modulation  signal  and  the  emission  modulation  signal 

_  §F- 

I  -  1  +  0.5  Md  Mf  cos((|)/7  -  (|)D)  (6) 

By  recording  a  number  of  images  at  different  (|)D  -  §E  phase 
angles,  the  phase  angle  and  modulation  depth  of  each  pixel  can 
be  established.  This  set  of  detector-phase-dependent  images  can 
then  be  converted  into  a  xf  map  of  the  specimen.  Because  of  the 
constant  DC  background  excitation  on  which  the  modulated 
fluorescence  signal  is  superimposed,  a  minimum  of  two  images 
must  be  recorded  in  order  to  obtain  xf  information. 

Several  modes  of  operation  have  been  suggested  for  this  type 
of  setup  (Lakowicz  and  Berndt.  1991).  One  possible  mode  of  oper¬ 
ation  is  to  record  one  image  at  a  specific  phase  angle  (|)x.  and  another 
one  with  no  detector  modulation.  After  subtracting  the  unmodu- 
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lated  image  from  the  modulated  image  (/„„),  the  value  in  one 
specific  pixel  will  be: 

4,  -  1^  =  A  0.5  Md  Mf  cos(c|v  -  (J)t)  (7) 


type  of  confocal  microscope.  Here,  the  light  source  can  easily  be 
modulated  using  EOMs  or  AOMs  or  by  using  a  pulsed-laser 
system  and  any  of  the  phase-sensitive  detection  schemes  described 
above  can  be  used. 


From  this  equation  it  can  be  seen  that  for  §F  =  n/2  +  (f)A,  Ion  - 
/^equals  zero.  Effectively  this  means  that  the  signal  arising  from 
fluorescence  with  a  lifetime  1,  =  tan(7t/2  +  (^/co  is  zero  and  thus 
is  in  a  sense  suppressed.  Phase  angles  §F  >  7t/2  +  (x  <  y)  or 
smaller  than  7X/2  +  §x  (x  >  xr)  will  give  rise  to  negative  and  posi¬ 
tive  values  of  Ion  -  I„ff,  respectively.  This  scheme,  however,  is  less 
suitable  for  absolute  y  determinations  because  Ion  -  Ioff  still  con¬ 
tains  constants  related  to  the  quantum  efficiency,  the  concentration 
of  the  fluorescent  material,  and  various  instrumental  parameters. 

This  limitation  can  be  overcome  by  recording  three  images: 
one  with  a  detector  modulation  phase  one  at  +  n/2,  and  one 
with  no  detector  modulation.  After  subtracting  the  unmodulated 
image  from  each  of  the  modulated  images,  the  values  at  one  pixel 
for  the  two  detector  modulation  phases  will  be 

/(M  -  Lff  =  0.5  A  Md  Mf  cos(0f  -  «  (8) 

and 


I(§x  +  n/2)  -  Ioff  =  0.5  A  Md  Mf  cos(c|)f  -  §x  -  n/2)  (9) 

respectively.  After  taking  the  ratio  of  Eqs.  8  and  9  we  obtain 


1  =  — =  tan^F  ~  ^  =  0YCf’x  (10) 
cos(t|) F-tyx-  n/2) 


For  c|)v  =  0,  Eq.  10  only  depends  on  (| >F  and  the  known  modu¬ 
lation  frequency  CO.  Therefore,  this  mode  of  operation  does  provide 
fully  quantitative  y  imaging.  However,  in  the  case  of  multi¬ 
exponential  decay  of  the  fluorescence,  the  measured  y  will  be 
concentration-  and  quantum-efficiency-weighted  average  lifetime 
over  all  the  decay  components.  Consequently,  a  mixture  of  a 
number  of  fluorescent  species  with  different  lifetimes  will  yield 
only  a  single  value,  and  truly  selective  imaging  of  one  particular 
species  is  difficult. 

Alternatively,  a  large  number  of  images  recorded  at  different 
detector  phase  angles  and  modulation  frequencies  can  be  recorded. 
After  a  full  analysis  of  each  pixel,  using  a  nonlinear,  least-squares 
fitting  procedure,  the  y  and  relative  contribution  of  each  individ¬ 
ual  fluorescence  decay  can  be  resolved  (Jameson  and  Gratton, 
1983;  Gratton  et  al,  1984)  and  the  relative  abundance  of  each 
species  can  be  determined  directly.  However,  because  point¬ 
scanning  confocal  data  acquisition  times  are  on  the  order  of  1  to 
10  s/frame,  the  total  data  acquisition  time  may  become  long,  and 
as  the  images  are  recorded  sequentially,  this  method  is  sensitive  to 
photobleaching.  The  effects  of  bleaching  may  be  reduced  by  aver¬ 
aging  a  number  of  interleaved  data  sets,  each  one  recorded  using 
a  different  phase  sequence. 

A  significant  amount  of  computational  power  is  required  for 
the  analyses  of  complete  images,  and  one  must  also  consider  the 
non-uniform  response  of  the  image  intensifier.  At  high  modulation 
frequencies,  a  difference  in  the  modulation  depth  and  the  phase 
shift  is  observed  between  the  center  and  the  edge  of  intensifiers 
(Lakowicz  and  Berndt.  1991).  This  can  be  compensated  for  by  a 
calibration  method  using  a  uniform  reference  specimen  for  which 
y  is  known.  Another  way  to  reduce  this  effect  is  to  use  small 
diameter  intensifiers. 


Disk-Scanning  Implementations 

Implementation  of  y  imaging  based  on  phase  fluorometry  is 
straightforward  when  the  method  is  combined  with  a  spinning-disk 


Point-Scanning  Implementations 

The  phase-fluorometry-based  method  of  y  sensing  can  be  easily 
implemented  in  a  point-scanning  microscope.  An  example  of  a 
phase-fluorometry-based  detection  method  in  a  scanning  two- 
photon  excitation  microscope  is  given  below.  This  work  concerns 
one  of  the  first  lifetime  imaging  scanning  microscopes  (Piston 
et  al.,  1992). 

Here,  a  mode-locked  dye  laser  producing  femtosecond  pulses 
at  a  repetition  frequency  of  75.6 MHz  was  used  as  a  light  source 
and  a  simple  modulated  PMT  detector  was  employed,  but  the 
detection  scheme  was  somewhat  different  from  that  discussed 
above.  The  detector  was  modulated  at  a  frequency  slightly  higher 
than  the  excitation  frequency  and  as  a  result  the  fluorescence  is 
detected  at  A/,  the  difference  frequency  between  the  excitation  and 
detection  modulation  frequencies.  This  cross-correlation  detection 
scheme  (heterodyning)  transforms  the  signal  to  the  low-frequency 
domain  and  assures  high  noise  immunity.  In  order  to  obtain  y 
images,  three  images  are  acquired,  each  one  with  an  additional 
120°  phase  shift. 

Test  measurements  on  rat  basophilic  leukemia  cells,  stained 
with  the  DNA  indicator  Hoechst  33342,  were  recorded  with  a  total 
pixel  dwell  time  of  18ps  (3  X  6|is/pixel).  In  the  original  paper 
describing  this  work,  only  fluorescence  lifetime  line  profiles  are 
shown.  Based  on  the  noise  level  in  the  lifetime  traces,  the  accu¬ 
racy  in  the  lifetime  is  estimated  to  be  10%.  Higher  pixel  dwell 
times  will  result  in  more  accurate  lifetime  images. 

In  Figure  27.3  the  original  lifetime  images  from  which  the  line 
profiles  were  extracted  are  shown  (image  courtesy  Dr.  Dave 
Piston).  Areas  outside  the  nuclei  yield  low  signal  and  therefore 
have  lifetimes  assigned  as  zero.  A  clear  difference  is  visible 
between  the  lifetime  calculated  from  the  demodulation  and  that  of 
the  phase  shift.  This  difference  in  lifetime  is  indicative  of  a  multi¬ 
exponential  decay  of  the  fluorescence. 

Another  example  of  frequency-domain  fluorescence  lifetime 
imaging  can  be  found  in  So  and  colleagues  (1996).  This 
implementation  is  also  based  on  a  multi-photon  microscope. 
However,  these  approaches  could  easily  also  be  implemented  on  a 
conventional  CLSM. 


Fluorescence  Lifetime  Sensing  in  the 
Time  Domain 

Time-Domain-Based  Methods 
Streak  Camera  Implementations 

In  time-domain  methods,  the  fluorescence  intensity  decay  as  a 
function  of  time  is  recorded  after  excitation  with  a  pulsed  light 
source.  A  good  example  of  this  is  the  combination  of  a  picosecond 
pulsed  laser  for  the  excitation  of  the  specimen  coupled  with  a 
streak  camera  (O’Connor  and  Phillips,  1984).  Here,  the  streak 
camera  records  the  whole  fluorescence  decay  curve  after  the  spec¬ 
imen  has  been  excited  with  a  laser  pulse. 

Time  Correlated  Single-Photon 
Counting  Implementations 

A  more  common  way  to  record  y  is  time-correlated  single-photon 
counting  (TCSPC)  (Lakowicz,  1983;  O’Connor  and  Phillips, 
1984).  Here,  the  fluorescent  molecules  are  excited  using  a  brief 
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Fluorescence  Lifetime  In  RBI  Cells 

Hoechst  33342  with  1000  Hz  Cross  Correlation  Frequency 


FIGURE  27.3.  Multi-photon  phase-fluorometry-based  FLIM  image  of  rat  basophilic  leukemia  cells,  stained  with  Hoechst  33342. 


light  pulse  after  which  the  timing  of  single-photon  emission  is 
recorded.  Using  this  approach  the  probability  distribution  for  the 
emission  of  a  single  photon,  and  thus  the  fluorescence  decay  curve, 
is  recorded.  The  high  time  resolution,  25  to  250  ps,  and  wide 
dynamic  range  of  xf  of  this  technique  have  made  it  popular  for 
spectroscopic  applications.  A  typical  experimental  geometry  is 
depicted  in  Figure  27.4.  The  specimen  is  excited  by  the  pulsed  light 
source.  A  trigger  pulse  synchronized  with  the  excitation  light  pulse 
is  used  to  start  a  time-to-amplitude  converter  (TAC).  The  fluores¬ 
cence  emitted  by  the  specimen  is  detected  by  a  PMT,  sent  through 
a  discriminator  (DISC),  and  then  used  to  stop  the  TAC.  The  output 
from  the  TAC  will  be  proportional  to  the  time  difference  between 
the  start  and  stop  pulses.  The  TAC  output  is  now  converted  to  a 
digital  word  by  means  of  an  analog-to-digital  converter  (ADC). 
This  digital  word  is  used  as  a  pointer  to  an  address  in  a  his- 
togramming  memory.  Finally,  the  value  at  this  specific  address  is 
incremented.  After  repeating  this  process  numerous  times,  a  his¬ 
togram  of  the  fluorescence  decay  curve  is  recorded  in  the  memory. 
The  dead-time  of  the  TAC  electronics  is  comparatively  long,  typ¬ 
ically  300  to  1000  ns.  Therefore,  care  must  be  taken  that  the  count 
rate  of  the  experiment  is  sufficiently  low  to  prevent  pulse  pile-up. 
The  TAC  usually  operates  in  the  reversed  start-stop  geometry. 
Here,  the  TAC  is  started  by  the  fluorescence  signal  and  stopped  by 


FIGURE  27.4.  A  schematic  diagram  of  a  TCSPC  setup. 


the  laser  trigger.  In  this  way  the  TAC  is  only  triggered  by  usable 
events,  and  not  by  laser  trigger  pulses  that  do  not  result  in  a 
detected  fluorescence  photon.  Therefore,  this  mode  of  operation 
suffers  less  from  dead-time  effects.  In  the  reversed  start-stop 
geometry,  pile-up  is  minimized  by  reducing  the  excitation  inten¬ 
sity  to  about  1  to  5  detected  photons  per  100  excitation  pulses.  Fur¬ 
thermore,  in  spectroscopy  applications  excitation  frequencies  not 
exceeding  10  MHz  are  employed  to  ensure  that  the  fluorescence 
decay  signal  from  one  excitation  pulse  is  not  affected  by  that  of 
other  excitation  pulses.  Therefore,  the  maximum  count  rate  in  con¬ 
ventional  TCSPC  is  less  than  100  kHz.  The  time  required  to  trans¬ 
fer  the  decay  curve  from  the  histogramming  memory  to  the 
computer  system  can  be  substantial.  Instead  of  the  TAC,  time-to- 
digital  converters  (TDCs)  can  be  used.  Here,  the  timing  of  events 
is  determined  by  means  of  solid-state  delay  lines  and  directly 
translated  into  a  digital  word.  In  practice  TDCs  have  very  similar 
properties  as  TACs. 

In  general,  the  decay  curves  recorded  by  TCSPC  are  fitted  to 
a  (multi)  exponential  decay.  Here,  usually  the  time  response  of  the 
instrument  is  taken  into  account  by  employing  an  iterative  decon¬ 
volution  technique. 

In  general,  the  use  of  conventional  TCSPC  equipment  for 
imaging  results  in  very  long  acquisition  times.  Conventional 
TCSPC  equipment,  however,  has  been  employed  in  CLSM  for 
fluorescence  spectroscopy  on  discrete  microscopic  volumes 
(Ghiggino  et  al.,  1992;  Vanderoord  et  al.,  1995)  and  for  fluores¬ 
cence  lifetime  imaging  at  a  low  acquisition  speed  (Bugiel  et  al., 
1989).  Operating  the  TCSPC  detection  system  at  too  high  detec¬ 
tion  rates,  above  5%  of  the  excitation  frequency,  results  in  distor¬ 
tion  of  the  recorded  decay  curve  (Lakowicz,  1999). 

More  recently,  TCSPC  plug-in  cards  for  PCs  have  been  devel¬ 
oped  that  are  optimized  for  imaging  applications.  These  cards  have 
a  much  lower  dead  time  than  the  conventional  TCSPC  electronics 
intended  for  use  in  spectroscopy  (Becker  et  al. ,  2003;  Kwak  and 
Vanden  Bout,  2003). 
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Time-Gating  Implementations 

Another  approach  to  fluorescence  lifetime  imaging  is  based  on 
combining  pulsed  excitation  with  time-gated  detection.  In  time¬ 
gated  detection,  use  is  made  of  one  or  more  time  channels,  each 
with  a  different  time  offset  with  respect  to  the  excitation  pulse. 
Time  gating  can  be  implemented  using  both  single-photon  count¬ 
ing  and  analog  detection  methods.  Here,  we  will  restrict  ourselves 
to  implementations  based  on  single-photon  counting.  In  time-gated 
photon  counting,  the  dead  time  of  the  detection  electronics  can  be 
very  low  (sub-nanoseconds)  and  consequently  such  a  detection 
system  can  be  operated  at  very  high  count  rates.  In  principle, 
multiple  fluorescence  photons  can  be  detected  for  each  excitation 
pulse.  By  sampling  the  fluorescence  decay  at  different  offsets  with 
respect  to  the  excitation  pulse,  information  about  the  fluorescence 
lifetime  can  be  obtained.  In  the  early  days  of  lifetime  imaging  often 
only  two  time-gated  detection  channels  were  used  (see  Fig.  27.5), 
each  with  a  different  delay  time  with  respect  to  the  excitation 
pulse.  For  a  mono-exponential  fluorescence  decay,  1/  can  be  cal¬ 
culated  from  the  ratio  IA/IB  of  the  intensities  recorded  in  the  two 
time  windows  A  and  B. 


At 

lo§  (Tt/4) 


(11) 


with  At  the  time  difference  between  the  start  of  the  two  time 
windows.  This  equation  is  only  valid  for  windows  of  equal  width 
and  provided  the  excitation  pulse  is  short  compared  to  Xf.  It  must 
be  noted  here  that,  although  the  fluorescence  signal  is  convolved 
with  the  excitation  pulse,  the  intrinsic  fluorescence  decay  is 
observed  after  the  excitation  pulse  has  died  out.  The  optimum  gate 
width  At  for  a  particular  Xf  amounts  to  2.5  Xf  (Ballew  and  Demas, 
1989). 

In  two-channel,  time-gated  detection,  only  one  decay  time  can 
be  determined,  and  a  multi-exponential  fluorescence  decay  yields 
only  a  single  effective  Xf.  This  limitation  can  be  overcome  in  prin¬ 
ciple  by  using  more  detection  windows.  The  time  gates  can  be 
acquired  in  one  single  pass  or  in  multiple  passes.  In  the  case  of  a 
multiple-pass  implementation,  all  photons  arriving  at  the  detector 
when  it  is  gated  off  are  lost.  As  a  result,  sensing  more  channels 
reduces  the  efficiency  of  the  system.  If  the  time  gates  are  all  acti¬ 
vated  sequentially  after  each  and  every  excitation  pulse,  a  much 
better  photon  economy  can  be  realized  even  when  a  large  number 
of  gates  are  employed. 


Time-Domain  Fluorescence  Lifetime  Imaging 
Widefield,  Slit-Scanning,  and  Spinning-Disk 
Microscopy  Implementations 

Several  groups  have  been  working  on  WF  imaging  systems  that 
employ  time-gated  detection  (see,  e.g.,  Ni  and  Melton,  1991;  Wang 
et  al.,  1991;  Schneckenburger  and  Konig,  1992;  Periasamy  et  al., 

1996) .  Often,  gated,  image  intensifiers  are  used  with  minimum 
gate  times  of  several  nanoseconds.  This  limits  the  use  of  these 
imaging  systems  to  measuring  fluorescence  lifetimes  greater  than 
Ins.  After  the  acquisition  of  two  images,  Eq.  11  can  be  used  to 
transform  the  two  intensity  images  into  a  Xf  image.  Because  the 
images  are  acquired  sequentially,  this  approach  is  sensitive  to 
photobleaching  effects.  Much  faster  gated,  image  intensifiers  that 
allow  sub-nanosecond  time  gating  are  also  available  (Scully  et  al., 

1997) .  Using  this  type  of  system,  complete  decays  can  be  sampled 
by  recording  sequences  of  images  at  different  gate  offsets.  The  effi¬ 
ciency  of  such  systems  goes  down  inversely  with  the  number  of 
time  gates.  Again,  averaging  schemes  can  be  employed  to  com¬ 
pensate  somewhat  for  the  effect  of  photobleaching  on  the  mea¬ 
surement  of  xf. 

A  streak-camera-based  lifetime  imaging  system  was  employed 
by,  for  example,  Minami  and  Hirayama  (1990).  More  recently, 
Krishann  and  colleagues  (2003)  employed  a  two-dimensional  (2D) 
streak  camera  in  combination  with  a  slit-scanning  multi-photon 
excitation  microscope.  Time  traces  were  recorded  for  all  the  points 
imaged  onto  the  entrance  slit.  The  system  can  be  employed  for  life¬ 
time  imaging  with  picosecond  time  resolution. 

Point  Scanning  Implementations 

Most  of  the  present  implementations  of  FLIM  in  point-scanning 
microscopes  are  based  on  photon-counting  techniques.  Several 
commercial  suppliers  exist  for  both  TCSPC  PC  plug-in  cards  and 
for  fast  time-gated  photon  counting  detection  systems  (Van  der 
Oord  et  al.,  2001;  Becker  et  al.,  2003;  Kwak  and  Vanden  Bout, 
2003). 

TCSPC  FLIM 

TCSPC  PC  plug-in  cards  have  been  optimized  for  high  through¬ 
put  data  collection.  Some  of  the  cards  have  comparatively  large 
onboard  memory  banks  that  remove  the  memory  transfer  bottle¬ 
neck.  Here,  the  maximum  frame  size  is  restricted  by  the  amount 
of  on-board  memory.  In  other  plug-in  cards,  the  memory  of  the 


FIGURE  27.5.  Time  gating. 
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host  computer  is  used  and  the  timing  histogram  is  built  up  in  the 
computer  memory.  Furthermore,  the  dead  time  of  the  electronics 
has  been  reduced  to  150  to  300  ns.  Therefore,  these  TCSPC  cards 
are  considerably  faster  than  conventional  TCSPC  electronics 
designed  for  spectroscopy  applications  (Becker  et  al.,  2003;  Kwak 
and  Vanden  Bout,  2003). 

The  dead  time  of  the  TCSPC  detection  electronics  is  domi¬ 
nated  by  that  of  the  TAC  or  TDC.  FLIM  systems  based  on  this 
type  of  acquisition  electronics  typically  employ  a  large  number  of 
time  channels  (typically  64-4096)  that  are  comparatively  narrow 
(25-200ps).  In  principle,  these  FLIM  systems  are  still  based 
on  the  concepts  used  in  the  conventional  spectroscopic  TCSPC 
electronics.  Therefore,  they  can  in  principle  provide  spec¬ 
troscopic  information  as  long  as  sufficient  fluorescence  signal  is 
present. 

It  should  be  stressed  though  that  a  large  number  of  detected 
photons  is  required  in  order  to  realize  spectroscopic  accuracy.  In 
spectroscopy  experiments  whole  decay  curves  are  recorded  with 
as  many  as  one  million  counts  in  the  decay.  This  is  far  more  than 
is  available  in  fluorescence  imaging  experiments.  Furthermore, 
care  must  be  taken  to  not  operate  the  TCSPC  electronics  at  too 
high  count  rates.  Operation  of  detection  systems  at  a  rate  of  one 
over  the  dead  time  at  the  input  will  result  in  a  loss  of  50%  of  the 
detection  efficiency  due  to  pile-up.  Moreover,  operation  of  TCSPC 
at  too  high  count  rates  will  result  in  distortion  of  the  decays.  Tree 
spectroscopic  accuracy  can  only  be  realized  when  the  detection 
count  rate  is  below  5%  of  the  excitation  rate  (O’Connor  and 
Phillips,  1984;  Lakowicz,  1999). 

Time-Gated  FLIM 

Our  group  has  implemented  time-gated  detection  in  several  point¬ 
scanning  microscopes.  Originally  we  employed  a  system  based  on 
two  time  gates  (Buurman  et  al.,  1992;  Draaijer  and  Houpt,  1988). 
Here,  the  detection  channel  of  an  existing  CLSM  was  modified  to 
accommodate  the  time-gated  detection  technique  in  combination 
with  single-photon  counting  detection.  A  fast  (0.8  ns  rise  time)  red- 
sensitive  PMT  was  followed  by  a  fast  discriminator  with  a  3  ns 
pulse  pair  resolution.  The  output  signal  from  the  discriminator  was 
fed  to  two  time -gated  counter  circuits  that  were  both  synchronized 
with  the  excitation  source.  All  the  time-critical  electronics  were 
built  using  emitter  coupled  logic  (ECL)  electronics.  This  detection 
circuit  could  be  used  at  count  rates  of  up  to  about  107cps  without 
serious  pile-up  effects  visible  in  the  lifetime.  The  minimum  gate 
width  of  this  system  was  approximately  2  ns  and  using  this  setup 
lifetimes  of  less  than  1  ns  could  be  imaged.  An  important  advan¬ 
tage  of  using  two  time-gated  detection  channels,  which  are  opened 
sequentially  after  every  laser  pulse,  is  the  relative  insensitivity  to 
photobleaching.  Furthermore,  at  the  optimum  gate  width  of  2.5  If, 
the  total  acquisition  time  per  excitation  pulse  amounts  to  5  1/  and 
approximately  99.3%  of  the  total  decay  is  captured  within  this 
period. 

More  recently  we  investigated  the  performance  of  time-gating 
systems  with  4  and  8  channels,  respectively  (Van  der  Oord  et  al., 
2001;  de  Grauw  and  Gerritsen,  2001).  Here,  the  same  design  phi¬ 
losophy  was  used.  Again  all  the  gates  open  sequentially  after  each 
and  every  excitation  pulse.  This  approach  ensures  both  high  photon 
efficiency  and  a  high  maximum  count  rate  capability.  The  dead 
time  of  the  electronics  is  below  1  ns  and  therefore  doesn't  limit  the 
maximum  count  rate  of  the  lifetime  acquisition  system.  In  practice 
the  performance  of  such  a  lifetime  imaging  systems  is  dominated 
by  the  properties  of  the  detector. 


Comparison  of  Confocal  Fluorescence  Lifetime 
Imaging  Methods 

Three  detection  methods  have  been  used  so  far  for  recording 
fluorescence  lifetime  images  in  point-scanning  microscopes: 
time  gating,  TCSPC,  and  phase  fluorometry.  In  this  section  we  will 
attempt  to  compare  these  approaches.  Aspects  that  we  take  into 
consideration  are  the  shortest  observable  lifetimes,  the  possibility 
of  recording  multiple  lifetimes  per  pixel  (multi-exponential 
decays),  the  sensitivity  for  bleaching,  the  photon  economy,  and  the 
acquisition  time.  It  must  be  stressed,  though,  that  the  limitations 
of  all  methods  are  to  a  large  extent  determined  by  the  present  state- 
of-the-art  of  the  technology.  The  properties  of  the  detector  have  a 
profound  influence  on  the  performance  of  the  different  lifetime 
imaging  methods.  Therefore,  a  separate  section  is  included  to  sum¬ 
marize  detector  properties. 

Shortest  Lifetime 

The  shortest  lifetime  that  can  be  imaged  using  phase  fluorometry 
is  limited  by  the  speed  of  the  detector  and  the  modulation  fre¬ 
quency  or  pulse  repetition  frequency  of  the  light  source.  Based  on 
the  experience  in  non-imaging  experiments  using  the  fastest  detec¬ 
tors,  lifetime  imaging  down  to  0.2  ns  should  be  feasible  (Ven, 
personal  communication). 

In  the  case  of  time-gated  detection,  the  minimum  lifetime  is 
determined  by  the  minimum  gate  width,  the  width  of  the  excita¬ 
tion  pulse,  and  the  speed  of  the  detector.  Gate  widths  of  less  than 
1  ns  can  be  realized  by  the  use  of  fast  electronics.  In  combination 
with  the  fastest  PMTs,  the  minimum  observable  lifetime  is  again 
estimated  to  be  about  0.2ns. 

The  TCSPC  systems  can  be  operated  with  channel  widths 
down  to  10  to  40  ps.  Therefore,  the  minimum  observable  lifetime 
in  this  type  of  system  will  be  only  limited  by  the  timing  proper¬ 
ties  of  the  detector  and  the  laser  pulse  width  rather  than  the  per¬ 
formance  of  the  detection  electronics.  In  practice  the  fastest 
available  detectors  (MCP-PMT)  exhibit  a  timing  jitter  (transit  time 
spread,  TTS)  of  about  25  ps  (Hamamatsu,  1997).  Accurate  deter¬ 
mination  of  lifetimes  close  to  or  shorter  than  the  TTS  is  difficult. 
Therefore,  the  limiting  lifetime  for  TCSPC-based  systems  will  be 
in  practice  on  the  order  of  50  to  lOOps  when  a  fast  detector  is 
employed. 

Experience  with  conventional  (TCSPC,  phase  fluorometry) 
fluorescence  lifetime  spectroscopy  has  shown  that  measuring  short 
lifetimes  (<0.5  ns)  is  not  straightforward  and  prone  to  errors.  Mea¬ 
surements  carried  out  using  different  setups  often  produce  different 
results;  differences  of  0. 1  ns  or  larger  are  not  unusual.  Moreover, 
short  lifetimes  are  often  the  result  of  quenching  processes  such  as 
collisional  quenching  and  FRET  that  not  only  reduce  the  lifetime 
but  also  the  quantum  efficiency  of  the  fluorescent  molecules. 
Therefore,  short  lifetimes  are  often  accompanied  by  low  intensi¬ 
ties  that  make  lifetime  imaging  of  very  short  lifetimes  far  from 
trivial. 

Multi-Exponential  Lifetimes 

The  first  frequency-domain  FLIM  systems  were  only  capable  of 
the  recording  one  average  lifetime  per  pixel.  Multiple  lifetimes  per 
pixel  from  multi-exponential  decays  can  in  principle  be  recorded 
by  acquiring  sets  of  images  at  different  excitation  modulation  fre¬ 
quencies.  This  increases  both  the  data  acquisition  time  and  the  dose 
of  light  to  the  specimen  by  an  amount  proportional  to  the  number 
of  modulation  frequencies. 
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For  example.  Squire  and  colleagues  (2000)  recently  demon¬ 
strated  that  multi-exponential  lifetime  imaging  is  in  principle  pos¬ 
sible  using  frequency-domain  FLIM.  Flere,  use  was  being  made  of 
a  WF  microscope  equipped  with  a  laser  modulated  at  a  set  of  har¬ 
monic  frequencies.  Homodyne  detection  at  each  of  these  frequen¬ 
cies  was  earned  out  simultaneously  by  mixing  with  matching 
harmonics  present  in  the  gain  characteristics  of  a  MCP-based 
image  intensifier.  In  principle,  this  approach  should  also  be  usable 
in  scanning  microscopes. 

The  time-gated  detection  scheme  based  on  two  time  gates 
yields  only  an  average  lifetime.  Multiple  lifetimes  per  pixel  can  be 
obtained  by  extending  the  number  of  gates.  This  was  demonstrated 
by,  for  example,  Scully  and  colleagues  (1997)  in  WF  microscopy 
and  by  de  Grauw  and  Gerritsen  (2001)  in  a  time -gated  system 
equipped  with  eight  gates.  In  principle,  this  should  also  be  pos¬ 
sible  by  using  a  system  equipped  with  only  four  time  gates.  For 
the  TCSPC-based  FLIM  systems,  obtaining  of  multiple  lifetimes 
is  straightforward. 

All  of  the  above  FLIM  methods  are  in  principle  capable  of 
resolving  multiple  lifetimes.  However,  at  present  only  very  few 
examples  are  available  of  multi-exponential  analyses  of  FLIM 
images  of  biological  specimens  at  the  single-pixel  level.  The  main 
problem  with  the  multi-exponential  analyses  of  FLIM  images  is 
the  limited  amount  of  signal  available  in  imaging  experiments.  In 
spectroscopy-based  experiments  usually  about  106  counts  are  accu¬ 
mulated  in  the  decay  and,  in  general,  up  to  three  decay  compo¬ 
nents  can  be  resolved.  In  exceptional  cases,  in  particular  when  the 
decay  components  are  far  apart,  more  decay  components  can  be 
separated. 

An  estimate  of  the  total  amount  of  signal  available  from  one 
detected  volume  element  in  the  specimen  shows  that  it  is  difficult 
to  realize  high  signal  levels  in  a  scanning  microscope.  In  practice 
a  few  thousand  detected  photons  per  pixel  can  be  obtained  from  a 
strongly  fluorescing  specimen.  However,  photobleaching  effects 
often  limit  the  amount  of  signal  to  (much)  lower  signal  levels.  By 
spatially  binning  the  amount  of  signal  per  pixel  can  be  increased 
at  the  price  of  a  loss  of  spatial  resolution. 

A  thorough  statistical  analysis  by  Kollner  and  Wolfram  (1992) 
showed  that  about  two  orders  of  magnitude  more  fluorescence 
signal  is  required  for  the  resolving  of  two  lifetimes  differing  by  a 
factor  of  2,  than  acquiring  a  single  lifetime  with  the  same  accuracy. 

Bleaching 

One  of  the  attractive  properties  of  FLIM  is  that  the  fluorescence 
lifetime  is  not  affected  by  fading  due  to  photobleaching.  The 
ability  to  take  advantage  of  this  property  depends  on  the  practical 
implementation  of  the  method. 

In  the  phase-fluorometry-based  methods,  at  least  two  but 
preferably  three  measurements  need  to  be  recorded  at  different 
detection  phase  angles  in  order  to  obtain  the  lifetime  information. 
Because  these  measurements  are  recorded  sequentially,  bleaching 
occurs  between  each  acquisition,  leading  to  a  distortion  of  the 
observed  lifetime.  To  some  extent  this  can  be  compensated  for  by 
averaging  over  all  the  possible  recording  sequences  at  the  differ¬ 
ent  detector  phase  angles.  However,  this  requires  a  highly  stable 
excitation  source,  which  may  be  a  limiting  factor  for  both  the  time 
resolution  and  the  accuracy  in  the  obtained  lifetime. 

In  the  time -gated  method,  the  sensitivity  to  photobleaching 
depends  on  the  implementation.  If  the  signal  is  acquired  in  each 
of  the  time-gated  detection  channels  after  every  excitation  pulse, 
the  method  is  not  sensitive  to  fading  due  to  photobleaching  effects. 
The  TCSPC-based  FLIM  method  is  also  not  affected  by  fading 
due  to  photobleaching. 


It  should  be  stressed  that  photobleaching  may  be  accompanied 
by  other  photodegradation  processes,  including  the  production  of 
fluorescing  photoproducts.  These  photoproducts  may  introduce 
new  decay  components  and  affect  the  quality  of  the  data. 

Photon  Economy 

Unfortunately,  no  experimental  information  is  available  which  can 
be  used  to  make  a  quantitative  comparison  of  the  photon  economy 
of  different  FLIM  methods.  However,  some  theoretical  work  has 
been  carried  out  on  the  statistics  of  time-domain-based  lifetime 
acquisition  (Ballew  and  Demas,  1989;  Kollner  and  Wolfram, 
1992).  In  addition,  computer  simulations  have  been  carried  out 
to  compare  different  methods  (Gerritsen  et  al .,  1996,  2002; 
Periasamy  et  al.,  1999). 

In  order  to  compare  the  different  methods,  a  figure  of  merit  is 
required.  Here,  we  employ  a  figure  of  merit,  F,  which  compares 
the  accuracy  of  the  lifetime  acquisition  method  with  that  of  a 
simple  intensity  measurement  with  the  same  number  of  detected 
photons,  F  =  (Ax/x)/(A//7),  with  x  the  lifetime.  Ax  the  standard  devi¬ 
ation  of  the  lifetime  determination,  I  the  number  of  detected 
photons  in  the  intensity  measurement,  and  A I  the  standard  devia¬ 
tion  of  the  intensity  determination.  On  assuming  Poisson  statistics, 
A HI  is  proportional  to  1/V7  and  the  expression  for  F  can  be 
rewritten  as: 

F  = - V7  (12) 

x 

Now,  F  is  a  measure  for  the  sensitivity  of  the  lifetime  acquisition 
method  that  is  independent  of  the  number  of  counts  in  the  decay, 
provided  that  A x/x  is  also  governed  by  Poisson  statistics.  F  is 
always  larger  than  1  and  a  low  value  of  F  indicates  a  high  sensi¬ 
tivity.  The  calculations  and  simulations  presented  here  concern 
ideal  photon-counting  systems.  Effects  related  to  noise  and  timing 
jitter  of  the  electronics  and  detectors  are  not  included.  The  param¬ 
eters  in  the  simulations  were  chosen  to  yield  the  optimum  sen¬ 
sitivity  at  a  lifetime  of  2  to  3  ns.  Using  Eq.  12,  the  F  values  can  be 
converted  into  the  number  of  counts  (in  the  total  decay)  required 
to  realize  a  specific  accuracy,  I  =  [F/(Ax/x)]2.  Sensitivities  of  two 
configurations  can  be  compared  by  taking  the  square  of  the  ratio 
of  their  figures  of  merit,  (Fl/F2)2. 

The  figure  of  merit  of  phase-fluorometry-based  FLIM  methods 
used  were  investigated  by  simulating  three  time-dependent  fluo¬ 
rescence  signals,  each  with  the  same  limited  number  of  counts. 
Here,  1000  counts  were  used  for  each  of  the  three  simulated 
signals.  Next,  the  fluorescence  signals  were  multiplied  by  the 
detector  response  at  detector  phase  angles  0  and  tx/2  and  with  the 
detector  modulation  switched  off  ( MD  =  0),  respectively.  The 
fluorescence  lifetime  can  be  calculated  from  the  three  simulated 
detector  images  using  Eq.  10.  After  repeating  this  procedure  1000 
times,  the  relative  standard  deviation  from  the  real  1/  was 
calculated. 

The  phase  fluorometry  simulations  were  performed  for  pulsed 
(delta  pulse)  as  well  as  sinusoidally  modulated  excitation  sources. 
In  the  case  of  pulsed  excitation,  a  mono-exponential  decay  curve 
was  simulated  (Eq.  1),  while  in  the  case  of  sinusoidal  modulation, 
a  signal  distribution  proportional  to  Eq.  3  was  simulated.  In  both 
cases,  Md  and  ME  were  set  to  1  (best  case)  while  the  optimum  value 
of  (Of  for  ij  was  used. 

The  results  of  the  computer  simulations  are  summarized  in 
Figure  27.6(A).  In  the  case  of  sinusoidal  modulation  of  the  exci¬ 
tation  source,  the  values  of  F  are  large  (minimum  value  about  6), 
indicating  a  low  sensitivity.  In  the  case  of  pulsed  excitation,  a  much 
lower  value  of  F  is  found  of  about  1.5.  This  comparison  shows 
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FIGURE  27.6.  Figure  of  merit  F  for  frequency-domain-based  lifetime  detection  (A)  and  time-domain-based  lifetime  detection  with  a  variable  number  of  equal 
time  channels  (B).  Low  values  of  F  indicate  high  sensitivity. 


that  phase  fluorometry  using  pulsed  excitation  requires  at  least 
(6/1. 5)2  =  16  times  fewer  photons  to  reach  the  same  accuracy  in 
1f  sensing  compared  with  sinusoidal  modulation.  In  practice  the 
difference  may  even  be  more  dramatic  because  MD  <  1  when 
sinusoidal  excitation  is  being  used. 

The  performance  of  systems  with  gates  of  equal  width  can 
be  calculated  analytically  (Ballew  and  Demas,  1989;  Kollner 
and  Wolfram,  1992).  Here,  we  present  analytical  calculations  on 
systems  with  equal  gate  widths  based  on  the  work  by  Kollner,  but 
now  expressed  in  terms  of  the  figure  of  merit.  F,  defined  earlier. 
For  gates  of  non-equal  width,  such  calculations  are  difficult  or  even 
impossible.  However,  previous  work  by  our  group  on  systems  with 
four  and  eight  gates  of  non-equal  width  shows  great  similarities 
with  the  results  presented  here  (de  Grauw  and  Gerritsen,  2001). 

Here,  we  restrict  ourselves  to  systems  with  2,  4,  8,  16,  and  64 
time  channels  of  equal  width  and  a  total  collection  time  per  laser 
pulse  of  10ns.  A  10  to  12ns  collection  time  per  laser  pulse  is 
typical  for  FLIM  systems  equipped  with  high-repetition-rate  lasers 
such  as  titanium: sapphire  lasers.  The  results  of  the  figure  of  merit 
calculations  are  summarized  in  Figure  27.6(B).  At  all  lifetimes  the 
figure  of  merit  of  the  two  gate  systems  is  significantly  larger  than 
that  of  the  systems  with  four  or  more  gates.  In  particular,  at  life¬ 
times  below  2  ns  the  difference  grows  rapidly.  At  a  lifetime  of  2  ns 
the  difference  in  sensitivity  |(F|/F2)2]  is  about  70%. 

Interestingly,  above  2  ns  no  significant  differences  in  F  are 
found  for  systems  with  four  or  more  gates.  The  differences 
between  systems  with  four  and  more  gates  only  show  up  below 
1 .5  ns  and  they  all  seem  to  be  very  well  usable  down  to  lifetimes 
as  short  as  0.5  ns  or  even  shorter.  The  system  with  64  gates  has  a 
very  low  figure  of  merit  down  to  the  shortest  lifetimes.  In  practice 
the  theoretical  figures  of  merit  cannot  be  realized  for  lifetimes 
of  the  order  of,  or  shorter  than,  the  time  response  of  the  detection 
system.  Usually  the  time  response  is  on  the  order  of  several 
hundred  picoseconds.  Therefore,  for  short  lifetimes  the  sensitivity 
of  real  FLIM  systems  will  be  lower  than  the  theoretical  calcula¬ 
tions  indicate. 

At  long  lifetime  the  sensitivity  of  all  configurations  in  Figure 
27.6  go  down.  Now,  the  total  collection  time  is  short  compared 
with  the  lifetime  and  counts  in  the  tail  of  the  decay  are  not  detected. 
Consequently  the  sensitivity  goes  down.  The  performance  for 
longer  lifetimes  can  be  improved  by  increasing  the  total  gate  width. 


However,  this  will  reduce  the  duty  cycle  of  the  system  and  the 
acquisition  time  will  go  up. 

The  figure  of  merit  of  the  time-domain-based  systems  is 
somewhat  lower  than  of  the  phase-fluorometry-based  systems. 
However,  the  phase-fluorometry-based  systems  equipped  with 
pulsed  excitation  perform  well  at  short  lifetimes.  At  longer  life¬ 
times,  however,  the  figure  of  merit  rises  more  rapidly  than  in  the 
case  of  the  time-domain-based  methods. 

In  the  above  comparison,  only  the  effect  of  Poisson  noise  on 
the  accuracy  in  xf  was  taken  into  account.  In  practice,  other  factors 
such  as  detector  noise  and  dead  time  effects  will  affect  the  photon 
economy  as  well.  Nevertheless,  it  is  clear  from  the  calculations 
that  time-gated  detection,  TCSPC,  and  phase  fluorometry  using 
pulsed  excitation  are  in  principle  all  suitable  for  lifetime  imaging. 

Acquisition  Time 

The  data  acquisition  time  is  governed  by  the  photon  economy  as 
well  as  by  other  factors  such  as  the  duty  cycle  and  the  maximum 
signal  strength  with  which  the  detection  electronics  and  detector 
can  cope.  For  FLIM  methods  employing  pulsed  excitation,  the 
duty  cycle  will  be  determined  by  1f  as  well  as  the  period  T  between 
two  laser  pulses.  In  this  case,  the  ratio  X//T  is  a  good  measure  of 
the  duty  cycle  of  the  experiment. 

When  a  high-repetition  laser  such  as,  for  example,  a  titanium: 
sapphire  laser,  is  being  used,  the  maximum  repetition  frequency  of 
the  light  source  is  about  80  MHz,  and  %fIT  =  0.24  for  Xf  =  3  ns.  For 
longer  lifetimes  the  duty  cycle  goes  up,  however,  the  figure  of 
merit  in  general  also  goes  up.  In  practice  it  will  be  difficult  to 
measure  long  lifetimes  with  high  repetition  rate  light  sources.  Such 
lifetimes  require  lower  repetition  rates  of  the  light  source  and 
therefore  the  acquisition  time  will  go  up. 

For  the  phase  fluorometry  method,  using  pulsed  excitation, 
XfIT  has  a  similar  value  at  optimum  a>f.  The  duty  cycle  in  a  phase 
fluorometry  experiment  using  sinusoidal  excitation  is  about  50%, 
which  somewhat  compensates  for  the  poor  photon  economy. 

Because  both  the  photon  economy  and  the  duty  cycle  of  the 
methods  using  pulsed  excitation  are  about  the  same,  in  theory 
similar  acquisition  times  are  expected.  In  our  experience,  the 
acquisition  time  in  time-gated  lifetime  imaging  can  be  as  short  as 
10|is/pixel  for  strongly  fluorescing  specimen.  Here,  the  maximum 
count  rate  of  the  photon-counting  detector  of  5  to  10  MHz  is  the 
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FIGURE  27.7.  The  effect  of  pile-up  on  the  detection  efficiency  as  a  function 
of  the  incident  count  rate  and  a  dead  time  of  200  ns  (dashed  curve). 
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limiting  factor.  The  dead  time  of  the  detection  electronics  of  1  ns 
is  not  a  limiting  factor. 

In  TCSPC-based  FLIM,  however,  the  acquisition  times  are, 
in  general,  much  longer.  Here,  the  bottleneck  is  not  the  photon 
economy  but  the  maximum  count  rate  of  the  detection  electronics. 
The  dead  time  of  the  TCSPC  electronics  is  comparatively  large, 
on  the  order  of  200  to  300ns.  The  fraction  of  detected  photons,/, 
amounts  to: 


/  = 


1 


l  +  td-  Cl 


(13) 


with  td  the  dead  time  of  the  system  and  C,  the  incident  count  rate. 

This  equation  only  accounts  for  the  number  of  counts  lost  due  to 

pile-up.  Compared  to  a  detector  without  a  dead  time,  a  200  ns  dead 
time  would  result  in  a  50%  loss  of  detected  photons  at  1/200  ns  = 

5  MHz  (see  Fig.  27.7).  At  a  count  rate  of  1  MHz,  about  17%  of  the 
signal  is  lost  due  to  pile-up.  Importantly,  besides  the  loss  of  sen¬ 
sitivity  due  to  pile-up  the  decay  curve  may  be  distorted  when  the 
TCSPC  system  is  operated  at  too  high  a  count  rate  (Demas,  1983). 
In  practice  one  should  ensure  that  the  most  strongly  fluorescing 
areas  in  the  specimen  do  not  show  serious  pile-up  effects.  For  these 
reasons  TCSPC-based  FLIM  should  be  operated  at  low  average 
count  rates  of  the  order  of  several  100  kHz.  Consequently,  the 
acquisition  times  are  comparatively  long. 


The  phase  fluorometry  methods  that  employ  pulsed  excitation 
are  usually  equipped  with  PMTs  operating  as  analog  (non-photon 
counting)  detectors.  In  this  mode  of  operation,  much  stronger 
signals  can  be  detected  than  in  the  photon-counting  mode.  There¬ 
fore,  it  is  expected  that  the  phase-fluorometry  method  using  pulsed 
excitation  can  be  operated  at  higher  acquisition  rates  than  the 
photon-counting-based  methods.  This  is  supported  by  some  recent 
work  carried  out  by  Gratton’s  group  (Gratton  et  al. ,  2003).  In  this 
work  it  is  demonstrated  that  the  (analog)  phase  fluorometry 
method  can  handle  significantly  larger  signal  intensities  than  the 
TCSPC  methods.  Here,  however,  the  shortest  pixel  dwell  time 
amounts  to  50|is.  The  phase  fluorometry  method  will  also  be  better 
able  to  deal  with  large  signals  than  other  photon-counting-based 
methods. 

In  the  case  of  the  phase  fluorometry  method  using  a  sinu¬ 
soidally  modulated  light  source  and  phase  sensitive  detection, 
comparatively  long  acquisition  times  are  expected  due  to  the  low 
sensitivity. 

Costs 

The  costs  of  the  phase  fluorometry  method  using  sinusoidal  exci¬ 
tation  are  mainly  determined  by  the  comparatively  low  costs  of  the 
EOM  or  AOM  and  that  of  the  CW  laser,  if  it  is  not  already  avail¬ 
able.  The  detection  system  for  either  method  is  a  modulated  PMT. 
The  costs  of  the  methods  requiring  pulsed-excitation  sources  have 
always  been  dominated  by  the  price  of  pulsed  lasers.  However, 
comparatively  cheap  pulsed  diode  lasers  have  recently  become 
available  (Pico  Quant  Gmbh).  The  lasers  provide  picosecond 
pulses,  repetition  rates  as  high  as  40  MHz,  and  1  to  2mW  of 
average  power.  At  present  lasers  with  wavelengths  in  the  range  375 
to  445 nm  and  635  to  1550nm  are  available.  Other  wavelengths 
close  to  480  nm  are  expected  to  become  available  soon.  The 
average  power  of  the  lasers  does  not  allow  high  acquisition  rate 
FLIM,  but  suffices  for  the  acquisition  of  good  quality  FLIM  data 
at  moderate-to-slow  speed. 

Table  27.2  summarizes  the  (estimated)  features  of  time-gating, 
TCSPC  and  phase  fluorometry. 

Detectors  for  Fluorescence  Lifetime  Imaging 

The  performance  of  fluorescence  lifetime  imaging  systems  criti¬ 
cally  depends  on  the  detector  properties.  Here,  we  will  restrict  our¬ 
selves  to  detectors  operating  in  photon-counting  mode.  However, 
most  of  the  discussion  below  also  applies  to  the  non-photon- 
counting  mode  of  operation. 

The  properties  considered  here  are  detector  timing  behavior, 
maximum  count  rate,  and  quantum  efficiency.  Side-on  PMTs  are 
often  used  in  scanning  microscopes  because  of  their  high  quantum 
efficiency  of  up  to  about  35%.  However,  the  side-on  tubes 


TABLE  27.2.  Comparison  of  Different  Lifetime  Imaging  Methods 


Phase  Fluorometry 

Time  Gating 

TCSPC 

Pulsed 

Sinusoidal 

Shortest  t/ 

0. 2-0.5  ns 

0.1  ns 

0.2-0. 5  ns 

0.2-0. 5  ns 

Sensitive  to  bleaching 

No 

No 

Yes 

Yes 

Photon  economy 

Good 

Good 

Good 

Poor 

Duty  cycle 

<0.1 

<0.1 

<0.1 

50% 

Acquisition  time  per  pixel 

>10-5s 

>10^*s 

>10~l * * * 5s 

>10““  s 

Costs 

Moderate/high 

Moderate/high 

Moderate/high 

Low 
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employed  in  scanning  microscopes  usually  show  very  poor  timing 
properties  and  transit-time  spreads  (TTS)  of  several  nanoseconds 
are  not  uncommon.  Therefore,  such  detectors  are  not  usable  for 
FLIM.  Fast,  head-on  PMTs  exhibit  a  TTS  as  low  as  several 
hundred  picoseconds  and  are  in  general  much  better  for  timing- 
critical  applications.  Usually  their  quantum  efficiency  does  not 
exceed  10%  and  they  can  operate  at  maximum  count  rates  of  5  to 
10  MHz.  At  higher  count  rates,  pulse  pile-up  degrades  their  per¬ 
formance  and  they  may  even  be  damaged  when  the  maximum 
anode  current  is  exceeded. 

The  shortest  lifetimes  that  can  be  measured  with  a  lifetime 
imaging  system  depend  on  both  the  timing  properties  of  the  detec¬ 
tor  and  of  the  detection  electronics.  PMTs  optimized  for  fast  timing 
applications  have  a  TTS  as  low  as  200  to  400ps.  This  limits  the 
shortest  lifetimes  that  can  be  measured  with  this  type  of  tube  to 
similar  values.  In  order  to  measure  shorter  lifetimes  microchannel 
plate  PMTs  (MCP-PMTs)  (McLoskey  et  al.,  1996)  can  be  em¬ 
ployed.  These  detectors  have  a  quantum  efficiency  of  about  10% 
and  a  TTS  as  low  as  25  ps.  At  present  the  MCP-PMTs  have  the 
best  timing  performance  for  the  measurement  of  fast  decays. 
However,  the  MCP-PMTs  are  expensive  and  vulnerable  and 
their  maximum  count  rate  is  limited.  For  example,  the  popular 
Hamamatsu  R3809U-50  MCP-PMT  has  a  maximum  average 
anode  output  current  of  lOOnA,  corresponding  to  a  maximum 
count  rate  of  100  to  300  kHz.  However,  Hamamatsu  advises 
operating  this  device  at  less  than  a  maximum  count  rate  of  20  kHz 
(Hamamatsu,  1997).  In  general,  all  detectors  exhibit  a  wavelength- 
dependent  time  response  that  complicates  the  accurate  determina¬ 
tion  of  short  decay  times  (Lakowicz,  1999). 

Both  PMTs  and  MCP-PMTs  show  a  rather  poor  quantum 
efficiency  of  about  10%.  Alternatively,  single-photon  counting 
avalanche  photo  diodes  (SPADs  )  or  PMTs  with  a  GaAs  photo 
cathode  can  be  used.  The  SPADs  have  high  quantum  efficiencies, 
about  70%  at  650  nm,  and  can  be  used  for  single-photon  counting 
applications  at  count  rates  of  up  to  1  to  2  MHz.  Their  typical  timing 
jitter  is  specified  by  PerkinElmer  to  be  about  350  picoseconds  and 
no  maximum  value  is  specified.  In  practice  we  found  that  values 
on  the  order  of  700  ps  are  more  common.  Moreover,  the  timing 
properties  of  these  devices  are  count-rate  dependent.  These  prop¬ 
erties  make  the  SPADs  less  suitable  for  lifetime  imaging.  The 
GaAs  photocathodes,  also  have  comparatively  high  quantum 
efficiencies  of  about  40%  in  the  red  part  of  the  spectrum.  Their 


maximum  count  rate  is  limited  to  about  1  to  3  MHz  and  their  timing 
resolution  is  about  350 ps. 

The  above  figures  can  be  used  to  make  an  estimate  of  the 
maximum  frame  rate  for  a  specific  type  of  detector.  Assuming  a 
256  x  256  pixel  image  and  an  average  number  of  counts  per  pixel 
of  250,  we  find  frame  acquisition  times  of  3.2  to  1.6  s,  500  to 
160  s,  16  to  8  s,  and  16  to  5  s  for  the  PMT,  MCP-PMT,  SPAD,  and 
GaAs  PMT,  respectively.  Very  short  lifetimes  of  about  25  ps  can 
only  be  measured  with  a  MCP-PMT  at  the  price  of  long  acquisi¬ 
tion  times.  Fast  acquisition  speeds,  in  combination  with  a  lower 
lifetime  limit  of  a  few  hundred  picoseconds,  can  be  realized  with 
fast  PMTs.  Interestingly,  the  GaAs  PMT  combines  a  lifetime  limit 
comparable  with  that  of  fast  PMTs,  with  high  quantum  efficiency. 
This  makes  this  detector  an  interesting  alternative  for  low  light 
level  FLIM. 

The  acquisition  time  performance  not  only  depends  on  the 
properties  of  the  detector  but  also  on  the  properties  of  the  acqui¬ 
sition  electronics.  The  dead  time  of  the  acquisition  electronics  is 
particularly  important:  large  dead  times  will  result  in  the  loss  of 
counts  at  high  count  rates. 

In  lifetime  imaging,  the  probability  of  a  photon  arriving  at  the 
detector  usually  follows  a  (multi)  exponential  distribution.  There¬ 
fore,  the  probability  of  pile-up  is  non-uniform  in  time  and  peaks 
at  time  equals  zero  of  the  exponential  function.  Consequently, 
when  pile-up  takes  place  the  recorded  fluorescence  decay  is  dis¬ 
torted  and  the  fitted  lifetimes  show  a  count-rate  dependency.  In  our 
experience,  a  pile-up  of  5%  to  10%  does  not  significantly  affect 
the  recorded  fluorescence  lifetimes.  For  a  detection  system  with  a 
dead  time  of  10  ns,  this  corresponds  to  a  maximum  count  rate  of 
5  to  10  MHz.  This  goes  down  to  250  to  500  kHz  for  a  detection 
system  with  a  dead  time  of  200  ns. 


APPLICATIONS 

Multi-Labeling  and  Segmentation 

Fluorescence  lifetime  imaging  is  able  to  discriminate  between 
a  large  number  of  probes  based  on  lifetime  differences  alone.  In 
Figure  27.8,  an  example  of  fluorescence-lifetime-based  discrimi¬ 
nation  of  fluorescent  probes  is  shown.  Here,  a  mixture  of  seven  dif¬ 
ferent  fluorescent  probes  (Yellow  Green  beads,  NYO  beads,  FITC 


FIGURE  27.8.  (A)  Intensity  image  of  a  specimen  containing  a  number  of  different  fluorescent  beads  and  crystals.  (B)  Corresponding  fluorescence  lifetime 
image.  (C)  The  histogram  of  fluorescence  lifetimes. 
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beads.  Bright  Blue  beads,  Nile  Red  beads,  Lucifer  Yellow  solution, 
and  benzopyrylene  crystals)  was  imaged  with  a  homemade 
two-photon  fluorescence  lifetime  microscope  equipped  with  an  8- 
channel  time-gated  detector  (Sytsma  et  al.,  1998).  In  the  lifetime 
image  and  in  the  lifetime  histogram,  six  different  lifetimes  are 
clearly  visible.  The  areas  with  the  highest  intensities  yield  the  nar¬ 
rowest  lifetime  distributions.  The  fluorescence  lifetimes  of  the 
beads  were  determined  by  means  of  TCSPC.  The  yellow-green 
beads  and  the  Lucifer  Yellow  solution  showed  mono-exponential 
decays  with  decay  times  of  4.27  ns  and  4.95  ns,  respectively.  The 
other  beads  and  the  crystal  showed  multi-exponential  decays.  The 
average  lifetimes  of  these  decays  were  2.8  ns  for  NYO  beads,  3.1 
ns  for  FITC  beads,  3.2  ns  for  Bright  Blue  beads,  3.6  ns  for  Nile  Red 
beads,  and  10.8  ns  for  the  benzopyrylene.  The  lifetimes  of  FITC 
and  Bright  Blue  beads  are  separated  by  only  0. 1  ns  and  show  up  as 
one  broadened  peak  in  the  histogram.  The  widths  of  peaks  in  the 
histogram  are  determined  by  statistics  alone.  Here,  the  minimum 
and  maximum  intensity  per  pixel  amounted  to  70  counts  and 
140,000  counts  for  the  Lucifer  Yellow  solution  and  benzopyrylene, 
respectively.  A  higher  signal  per  pixel  improves  the  signal-to-noise 
ratio  and,  therefore,  reduces  the  widths  of  the  lifetime  peaks  in  the 
histogram.  Here,  the  time  between  two  excitation  pulses  amounts 
to  12.2  ns  (82  MHz  Ti:Sa  laser).  Consequently,  the  lifetime  sensi¬ 
tivity  for  the  long  lifetime  benzopyrylene  (10.8ns)  is  poor. 

In  Figure  27.9,  confocal  fluorescence  intensity  and  fluores¬ 
cence  lifetime  images  from  the  alga  Gymnodinium  nagasakience 
are  shown.  The  images  were  recorded  with  a  confocal  microscope 
equipped  with  a  simple  time-gated  detection  system  with  two, 
2ns-wide  gates  (Buurman  et  al. ,  1992).  The  alga  was  stained  with 
an  antibody-FITC  conjugate  against  its  outer  membrane.  The  flu¬ 
orescence  intensity  image  in  Figure  27.9(A)  shows  signal  from 
both  the  antibody-FITC  and  the  autofluorescence  of  the  chloro¬ 
phyll  contained  in  the  chloroplasts  because  no  spectral  discrimi¬ 
nation  was  used  to  separate  the  fluorescence  of  these  two 
substances.  Figure  27.9(B)  shows  a  fluorescence  lifetime  image 
of  the  alga;  the  gray  value  is  a  measure  for  If.  Chloroplasts 
show  up  as  dark  spots  because  for  chlorophyll  1f  ~  0.7  ns,  while 
the  areas  around  the  membrane  are  lighter  because  of  the  FITC 
Xf~  1.1ns. 

The  lifetime  image  can  be  segmented  and  used  as  a  mask  to 
obtain  two  separate  images;  one  with  fluorescence  intensity  repre¬ 
senting  chlorophyll  [Fig.  27. 10(A)]  and  the  other  with  fluorescence 
intensity  representing  FITC  [Fig.  27.10(B)].  For  comparison,  two 
images  obtained  by  filtering  the  emission  in  the  conventional 
way  are  also  shown.  Figure  27.10(C)  shows  the  chlorophyll  signal 


FIGURE  27.9.  Confocal  fluorescence  intensity  (A)  and  fluorescence  life¬ 
time  images  (B)  from  the  alga  Gymnodinium  nagasakience  stained  with 
antibody-FITC  conjugate  against  its  outer  membrane  (T/=  1.1  ns).  In  addition, 
autofluorescence  of  chlorophyll  is  visible  (if  —  0.7  ns). 


recorded  through  a  long-pass  580  filter  and  Figure  27.10(D)  shows 
the  FITC  signal  detected  through  a  bandpass  filter.  There  is  a 
high  degree  of  correlation  between  the  FLIM  and  spectral  images. 
However,  a  subtle  detail  coming  from  a  dimple  in  the  center  of  the 
alga  shell  shows  up  in  both  members  of  the  spectrally  resolved 
image  pair  showing  that  this  filter  combination  is  not  good  enough 
to  discriminate  between  chlorophyll  and  FITC.  This  detail  is  not 
visible  in  the  lifetime  image. 

Ion-Concentration  Determination 

Ion  concentrations  in  biological  systems,  such  as  pH,  Ca2+,  and 
Na+,  are  widely  studied  with  fluorescent  probes.  The  probes  have 
a  high  selectivity  for  specific  ions  and  exhibit  marked  changes  in 
their  photophysical  properties  upon  binding.  Often  the  probes 
exist  in  two  states,  bound  to  the  ion  and  free,  that  have  distinct 
fluorescence  lifetimes.  The  average  fluorescence  lifetime  is  a 
good  measure  for  the  ion  concentration  and  can  be  used  for  the 
quantitative  imaging.  Some  of  the  examples  obtained  in  our  lab 
are  discussed  below.  Others  can  be  seen  in  Chapter  42,  this 
volume. 


FIGURE  27.10.  The  chlorophyll  (A,C)  and  FITC  (B.D)  signals  after  fluorescence  lifetime-based  segmentation  (A.B)  and  segmentation  based  on  differences  in 
emission  spectra  (C,D). 
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Calcium  Imaging 

Calcium  plays  a  central  role  as  a  second  messenger  in  plant  and 
animal  cells  and  is  involved  in  controlling  numerous  biological 
processes.  Fluorescent  probes  are  widely  used  in  macroscopic  and 
microscopic  studies  of  [Ca2+].  These  probes  have  a  high  selectivity 
for  free  Ca2+  over  other  ions  and  the  probes  undergo  a  significant 
change  in  fluorescence  properties  upon  binding  to  Ca2+.  Current 
fluorescent  calcium  probes  can  be  divided  in  two  groups:  ratio  or 
dual-wavelength  probes  and  non-ratio  or  single-wavelength 
probes.  Due  to  calibration  problems,  single-wavelength  probes 
produce  very  ambiguous  results.  The  fluorescence  intensity 
increases  as  a  result  of  an  increase  in  [Ca2+],  However,  the  fluores¬ 
cence  intensity  not  only  depends  on  the  [Ca2+]  but  also  on  the  dye 
concentration,  which  can  vary  as  a  result  of  unequal  cytosol  thick¬ 
ness,  inhomogeneous  dye  distribution  within  or  between  cells,  and 
leakage  and/or  photobleaching  during  an  experiment. 

Quantification  of  [Ca2+]  can  be  more  accurate  when  ratio 
probes  are  used.  These  probes  show  a  marked  shift  in  either  exci¬ 
tation  or  emission  spectra  upon  binding  to  Ca2+  and  are  referred  to 
as  dual-excitation  or  dual-emission  probes.  This  feature  allows 
these  probes  to  be  used  for  ratio  analysis  (Rink  et  al.,  1982; 
Grynkiewicz  et  al.,  1985;  Tsien  and  Poenie,  1986).  The  ratioing 
method  is  independent  of  dye  distribution  inhomogeneities  within 
or  between  cells,  and  leakage  and/or  photobleaching  during  an 
experiment.  Unfortunately,  a  laborious  in  situ  calibration  proce¬ 
dure  is  required  when  ratio  probes  are  being  used  in  fluorescence 
microscopy. 

Most  of  the  commercially  available  ratio  probes  such  as 
Fura-2  and  Indo-1  have  to  be  excited  in  the  ultraviolet  (UV).  UV 
wavelengths  can  potentially  injure  cells  and  tend  to  excite  auto¬ 
fluorescence;  therefore,  it  is  preferable  to  use  probes  that  have  their 
excitation  wavelength  in  the  visible. 

FLIM  is  an  interesting  alternative  for  the  imaging  of  free  Ca2+. 
The  binding  of  Ca2+  to  the  single-wavelength  probes  results  in  a 
large  increase  of  the  quantum  efficiency  of  the  probe.  This  is  in 
general  accompanied  by  a  substantial  increase  of  the  fluorescence 
lifetime  of  the  probe.  Depending  on  the  free  calcium  concentra¬ 
tion  and  the  Kd  of  the  probe,  different  amounts  of  bound  and 
free  probe  will  be  present  in  the  specimen,  Kd  =  ([free  probe]- 
[  Ca2+])/[bound  probe].  In  general  the  fluorescence  decay  will  be 
(at  least)  bi-exponential.  The  amplitudes  of  the  different  lifetime 
components  depend  on  the  calcium  concentration  and  it  turns  out 
that  the  average  lifetime  is  a  good  measure  of  the  calcium  con¬ 
centration  (Sanders  et  al.,  1994). 


Interestingly,  in  principle  FLIM  allows  quantitative  imaging  of 
Ca2+  using  single  wavelength  probes  that  are  excited  with  visible 
light.  There  are  several  indications  that  this  approach  is  more  reli¬ 
able  than  the  approaches  based  on  ratio  imaging  (Sanders  et  al., 
1995;  Herman  et  al.,  1997). 

In  general,  the  free  probe  has  both  a  short  lifetime  and  low 
quantum  efficiency.  The  ion-bound  probe,  on  the  other  hand,  has 
both  a  long  lifetime  and  high  quantum  efficiency.  Therefore,  the 
ion-bound  form  dominates  the  fluorescence  decay  and  the  usable 
ion  concentration  range  of  the  probe  is  shifted  to  lower  ion 
concentrations. 

Early  work  on  the  quantitative  FLIM  of  calcium  required 
comparatively  long  acquisition  times.  Recently,  we  carried  out 
fast-time-gating-based  FLIM  measurements  of  Indo-1  stained  rat 
myocytes  (see  Fig.  27.11)  (Gerritsen  et  al.,  2002).  The  rat 
myocytes  spontaneously  beat  at  rates  of  0.5  to  2  Hz  at  37°C  and 
the  beating  is  accompanied  by  large  Ca2+  fluxes  (Berlin  and 
Konishi,  1993).  The  images  were  recorded  in  a  homemade  two- 
photon  excitation  microscope  (Sytsma  et  al.,  1998)  equipped  with 
an  8-channel  time-gated  detector  and  a  femtosecond  titanium: sap¬ 
phire  laser  operating  at  82  MHz  and  an  excitation  wavelength 
of  700  nm.  Indo-1  exhibits  a  1.4  ns  lifetime  for  the  free  probe 
and  a  1.66  ns  lifetime  for  the  ion-bound  probe  (Lakowicz  and 
Szmacinski,  1993).  Despite  the  small  lifetime  difference  between 
the  free  and  bound  forms  of  the  probe,  lifetime  images  (256  X  256 
pixels)  could  be  recorded  at  a  rate  of  1  frame  per  second.  Here,  a 
Hamamatsu  R1894  PMT  was  used  at  an  average  count  rate  close 
to  10MHz.  Both  the  intensity  [Fig.  27.11(A)]  and  the  lifetime 
image  [Fig.  27.11(B)]  were  median-filtered.  The  edges  of  the  life¬ 
time  image  are  somewhat  sharpened  due  to  the  thresholding,  life¬ 
times  were  only  calculated  for  pixels  with  50  or  more  counts. 
Pixels  with  fewer  counts  were  set  to  zero.  The  calibration  bar 
below  the  lifetime  image  gives  an  indication  of  the  lifetimes.  No 
attempt  was  made  to  convert  the  lifetimes  into  calcium  concentra¬ 
tions.  The  beating  of  the  myocyte  during  image  acquisition  shows 
up  as  a  band  in  both  the  FLIM  and  the  intensity  image.  Here,  the 
myocyte  beat  about  twice  during  the  image  acquisition. 

pH  Imaging 

Dental  biofilm  can  be  several  hundred  micrometers  thick  and 
exhibits  an  intricate  three-dimensional  (3D)  structure  (see  also 
Chapter  51,  this  volume).  This  type  of  specimen  is  strongly  scat¬ 
tering  and  two-photon  excitation  (TPE)  microscopy  is  therefore 
the  preferred  imaging  method  to  study  it.  pH  is  one  of  the  key 


FIGURE  27.11.  Intensity  image  (A)  and  fluorescence 
lifetime  image  (B)  of  an  Indo-1  stained  beating  rat 
myocyte.  The  beating,  approximately  2  Hz,  results  in  the 
bands  in  the  image.  The  256  x  256  pixel  image  was 
recorded  in  1  s  using  a  two-photon  excitation  micro¬ 
scope  equipped  with  a  time-gated  detection  system. 
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FIGURE  27.12.  The  pH  calibration  (carboxy-fluorescein)  of  the  two-photon 
FLIM  system. 


parameters  in  biofilm  research.  The  pH  in  the  biofilm  is  in  the 
acidic  range  and  quantitative  imaging  of  the  pH  in  this  range  is 
usually  accomplished  by  means  of  excitation-ratio  imaging.  Imple¬ 
mentation  of  excitation  ratio-imaging  in  TPE  imaging  is  hampered 
by  the  requirement  of  two  measurements  at  different  excitation 
wavelengths.  Fluorescence  lifetime  imaging  is  an  excellent  alter¬ 
native  for  the  quantitative  imaging  of  pH  in  the  acidic  range. 

Here,  we  employed  TPE  in  combination  with  FLIM  to  study 
the  pH  in  a  biofilm  (Bradshaw  et  al,  1998;  Vroom  et  al.,  1999). 
Carboxy-fluorescein  was  used  as  a  fluorescent  pH  indicator.  The 
carboxy  form  of  the  probe  is  cell  impermeable,  so  it  senses  only 
the  extracellular  pH,  the  pH  of  interest  in  the  biofilms.  TCSPC 
measurements  on  a  series  of  carboxy-fluorescein  buffers  of  dif¬ 
ferent  pH  revealed  a  clear  pH-dependent  average  fluorescence  life¬ 
time  of  4.0ns  at  high  pH  (pH  >  7)  and  3.0ns  at  low  pH  (pH  <  3). 
In  addition  to  the  fluorescence  lifetime  effect,  the  quantum  yield 
of  carboxy-fluorescein  went  down  to  5%  at  pH  3,  relative  to  that 
at  pH  7. 

At  an  excitation  wavelength  of  800  nm,  the  biofilms  show 
some  autofluorescence  with  an  average  fluorescence  lifetime  of 


FIGURE  27.13.  Intensity  image  of  biofilm  (A)  with  individual  bacteria  visible. 
Fluorescence  lifetime  image  (B)  shows  a  homogeneous  pH  of  6.2  ±  0.3. 


less  than  1  ns.  The  difference  in  fluorescence  lifetime  between  the 
autofluorescence  and  the  carboxy-fluorescein  was  exploited  to 
suppress  the  autofluorescence  in  the  pH  measurements.  This  was 
achieved  by  opening  the  first  gate  1  ns  after  the  excitation  pulse. 
This  suppresses  80%  of  the  autofluorescence  and  only  20%  of  the 
carboxy-fluorescein  signal.  In  addition,  a  comparatively  high 
probe  concentration  of  50  to  100pM  was  used  in  the  pH  imaging 
experiments.  The  autofluorescence  was  less  than  1%  of  the  total 
fluorescence  intensity  in  all  the  measurements,  even  at  low  pH. 

In  Figure  27.12,  the  pH  calibration  of  the  microscope  is 
shown.  Intensity  and  pH  xy-images  (30  X  30  pm2,  z  =  60  pm)  of 
carboxy-fluorescein  stained  biofilm  are  shown  in  Figure  27.13. 
The  intensity  image  [Fig.  27.13(A)]  shows  individual  bacteria, 
while  the  fluorescence  lifetime  image  [Fig.  27.13(B)]  shows  an 
almost  homogeneous  pH  of  pH  6.2  ±  0.3.  At  a  constant  pH  the 
quantum  efficiency  of  the  carboxy-fluorescein  is  constant.  There¬ 
fore,  the  heterogeneous  fluorescence  intensity  distribution  can  be 
attributed  to  a  non-homogeneous  probe  distribution.  Binding  of 
some  of  the  probe  to  the  bacteria  may  cause  this. 

A  pH  gradient  was  induced  by  overlaying  the  specimen  with 
a  14  m M  sucrose  solution.  The  fermentation  of  sucrose  lowers 
the  pH  outside  the  bacteria.  In  Figure  27.14,  the  intensity  [Fig. 
27.14(A)]  and  fluorescence  lifetime  [Fig.  27.14(B)]  images  94 min 
after  the  addition  of  the  sucrose  are  shown.  These  images  were 
recorded  at  the  same  position  as  that  of  Figure  27.13. 

The  fluorescence  intensity  image  shows  some  brightly  fluo¬ 
rescing  areas.  At  the  same  position  in  the  fluorescence  lifetime 
image,  a  pH  is  found  of  less  than  3.  This  observation  is  somewhat 
unexpected  because  the  quantum  efficiency  of  the  probe  goes 
down  with  pH.  It  suggests  that  a  high  local  probe  concentration  is 
present  at  the  bright  spots.  This  effect  may  be  caused  by  probe  pre¬ 
cipitation  and  makes  the  measurements  at  these  locations  less 
reliable.  Therefore,  the  bright  spots  were  excluded  from  the  pH 
analysis.  The  average  pH  of  this  image  is  5.2  ±  0.4,  one  pH  unit 
lower  than  the  average  pH  of  the  reference  image. 

Probes  for  Fluorescence  Lifetime  Microscopy 

A  literature  study  and  some  measurements  on  probes  in  our  own 
laboratory  revealed  a  large  list  of  probes  which  can  potentially  be 
used  for  fluorescence  lifetime  imaging,  this  list  is  summarized  in 
Table  27.3. 


A 

B 

f  . 

,  • 
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FIGURE  27.14.  Intensity  (A)  and  lifetime  (B)  images  after  overlaying  the 
biofilm  specimen  with  a  14mM  sucrose  solution.  The  average  pH  is  5.2  ±  0.4. 
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TABLE  27.3.  Ion  Sensitive  Probes 


PH  Probes 

Probe 

KxJKm  (nm) 

^acid  (ns) 

W  (ns) 

Reference 

SNAFL-l 

2957500'’ 

3.58 

1.14 

Whitaker  et  al.  (1991) 

Carboxy-SNAFL-1 

2957500'’ 

3.44 

0.95 

Whitaker  et  al.  (1991) 

Carboxy  SNAFL-2 

2957500'’ 

4.19 

0.87 

Whitaker  et  al.  (1991) 

Carboxy  SNAFL-3 

2957500'’ 

3.63 

1.16 

Whitaker  et  al.  (1991) 

Carboxy  SNARF-5 

2957500'’ 

4.21 

0.73 

Whitaker  et  al.  (1991) 

SNARF-6 

Whitaker  et  al.  (1991) 

Carboxy  SNARF-6 

2957500'’ 

4.50 

1.04 

Whitaker  et  al.  (1991) 

SNARF-1 

2957500'’ 

0.46 

1.54 

Whitaker  et  al.  (1991) 

SNARF-2 

2957500'’ 

0.30 

1.75 

Whitaker  et  al.  (1991) 

Carboxy  SNARF-1 

2957500'’ 

0.60 

1.32 

Whitaker  et  al.  (1991) 

Carboxy  SNARF-1 

2957500'’ 

0.60 

1.32 

Whitaker  et  al.  (1991) 

Carboxy  SNARF-2 

2957500'’ 

0.27 

1.67 

Whitaker  et  al.  (1991) 

2-Naphtol 

313/360 

0.80  (0.35) 

7.30  (1.91) 

4.82 

Laws  and  Brand  (1979) 

Acridine 

348/560 

9.6 

31.1 

Gafni  and  Brand  (1978) 

Virginiamycin  S' 

330/420 

0.48 

1.9 

Clays  et  al.  (1991) 

BCECF 

490/520 

3.0 

3.8 

This  laboratory 

DM-NERF 

490/520 

2.3 

4.0 

This  laboratory 

Carboxy-fluorescein'' 

490/520 

3.0 

4.0 

This  laboratory 

Cl-NERF 

490/520 

1.3 

4.0 

This  laboratory 

Acridine  Orange 

490/520 

1.8 

5.3 

This  laboratory 

Ca2+  Probes 

Probe 

(nm) 

T„„  Ca++“  (ns) 

Tear/  (ns) 

Reference 

Fura-2 

340/420 

0.77(0.49) 

1.77 

Keating  et  al.  (1989) 

380/420 

1.5(0.51) 

Quin-2 

Lakowicz  et  al.  (1994) 

Calcium  green 

Bapta  1 

488/520 

1.6 

3.5 

This  laboratory 

Calcium  green 

Bapta  2 

488/520 

2.1 

3.6 

This  laboratory 

Calcium  green 

Bapta  5N 

488/520 

0.5 

3.4 

This  laboratory 

Oregon  Green 

Bapta  1,2 

488/520 

1.5 

3.7 

This  laboratory 

Oregon  Green 

Bapta  5N 

488/520 

0.5 

2.8 

This  laboratory 

BTC 

0.7 

1.2 

This  laboratory 

Indo  1 

340/400-475 

1.4 

1.7 

Lakowicz  and 
Szmacinski  (1993) 

Indo  5F 

340/400^175 

1.4 

1.4 

This  laboratory 

Cl  Probes 

Probe 

KxJKm  (nm) 

W  (ns) 

Xci°  (ns) 

Reference 

SPQ/Cl- 

— /— 

26.0 

2.0 

Dix  and  Verkman 

(1990) 

DNA  or  RNA  Selective  Probes 

Lifetime  Differences  with  Free  Probe  and  Probe  Bound  to  DNA 

Probe 

KxJKm  (nm) 

KJ  (ns) 

Abound 

Reference 

Proflavine,  acriflavine. 

— /— 

4.5 

4  (0.7) 

Duportail  et  al.  (1977) 

acridine  yellow  (208C) 

7  (0.3) 

9-Aminoacridine 

— /— 

15 

10.5  (0.7) 

31  (0.3) 

Duportail  et  al.  (1977) 

Rivanol 

— /— 

6.5 

5.5  (0.7) 

13  (0.3) 

Duportail  et  al.  (1977) 

Quinacrine 

— /— 

4 

3.5  (0.7) 

19  (0.3) 

Duportail  et  al.  (1977) 

Ethidium  bromide 

532/ — 

1.7 

24.2 

Atherton  and  Beaumont  (1984) 

Acridine  orange,  DNA 

488/520 

— 

1.7— 1.9 

Marriott  et  al.  (1991b) 

RNA 

488/520 

— 

1.7-1 .9 

Marriott  et  al.  (1991b) 

RNA 

488/>630 

— 

5-20 

Marriott  et  al.  (1991b) 

continued 
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TABLE  27.3. 

Ion  Sensitive  Probes  ( Continued ) 

Lifetime  Differences  with  Different  Amounts  of  A  +  T 

Probe 

^extAem  (nm) 

^much  AT  (ns) 

little  AT"  (ns) 

Reference 

Rivanol 

—1— 

5.5  (0.8)' 

13  (0.2) 

11 

Duportail  et  al.  (1977) 

Quinacrine 

- / - 

2.2  (0.9) 

15  (0.1) 

16 

Duportail  et  al.  (1977) 

Quinacrine 

435/516 

2.6  (0.35/ 

1.1  (0.06) 

Amdt-Jovin  et  al.  (1979) 

11  (0.39) 

7.90  (0.33) 

Amdt-Jovin  et  al.  (1979) 

27  (0.26) 

26  (0.60) 

Amdt-Jovin  et  al.  (1979) 

Probes  Which  Are  Sensitive  to  Other  Factors 

as  Ions  or  DNA  or  RNA 

Probe 

Xexc/Xem  (nm) 

W  (ns) 

Abound  (ns) 

Reference 

SR33557 

FFO/EtOH 

336/400 

4 

Chatelain  et  al.  (1992) 

[50/50] 

Calcium — 

EtOH 

336/403 

8.5 

Chatelain  et  al.  (1992) 

entry  blocker 

Erythrocytes 

— /— 

13.3  (0.91) 

2.9  (0.09) 

Chatelain  et  al.  (1992) 

Egg  PC  vesicle 

— /— 

13.6  (0.81) 

5.0  (0.19) 

Chatelain  et  al.  (1992) 

Fluoranthene 

In  N2  gas 

355/— 

51 

Ni  et  al.  (1991) 

In  air 

32 

51 

In  02  gas 

14 

51 

FITC 

pH  7.2 

490/520 

3.9 

This  laboratory 

FITC- 

pH  7.4 

490/520 

1.5  (0.38) 

This  laboratory 

transferin 

3.6  (0.62) 

FITC-KE2 

pH  7.4 

485/— 

1.32  (0.28) 

Matko  et  al.  (1992) 

Fab  complex 

4.11  (0.72) 

A,exc  (nm),  excitation  wavelength;  7,en 

a  (nm),  emission  wavelength;  if. 

fluorescence  lifetime,  a  is 

the  amplitude  of  the  respective  components 

in  case  of  multi-exponential 

decay. 

"Absorption  maxima  between  480  and  580 nm. 
ft500nm  long-pass  filter. 

c  Virginiamycin  S  has  three  different  pKs  and  thus  four  different  protolytic  forms. 
d  Affected  by  other  effects  such  as  binding. 
e28%  A  +  T. 

•^55%  A  +  T. 


SUMMARY 

Fluorescence  lifetime  imaging  is  a  powerful  tool  that  can  be  used 
for  a  large  number  of  different  applications.  Since  its  introduction 
around  1990,  it  has  significantly  matured  and  at  present  several 
companies  supply  complete  FLIM  detection  systems.  The  advan¬ 
tages  and  disadvantages  of  FLIM  and  the  different  implementa¬ 
tions  are  summarized  below. 

•  Fluorescence  lifetime  imaging  is  insensitive  to  intensity  effects 
such  as  shading,  photobleaching,  absorption,  and  light  source 
noise.  This  can  be  an  important  advantage,  especially  in  confo- 
cal  and  multi-photon  studies  of  thick  specimens.  Here,  absorp¬ 
tion  effects  and  photobleaching  are  important  limitations. 

•  Fluorescence  lifetime  imaging  is  a  powerful  tool  in  the  field 
of  ion-concentration  imaging.  The  comparison  between  emis¬ 
sion-ratio  imaging  and  lifetime-imaging  suggests  that  lifetime¬ 
imaging  can  yield  quantitative  results  without  a  cumbersome 
in  vivo  calibration  procedure. 

•  In  theory,  lifetime  imaging  is  able  to  separate  the  signals  from 
a  large  number  of  different  probes  in  parallel  using  a  single 
detector.  Image-processing  can  be  simplified  by  using  fluores¬ 
cence  lifetime  data  as  the  basis  of  segmentation. 

•  In  recent  years  it  has  been  demonstrated  that  FLIM  is  an  excel¬ 
lent  and  straightforward  method  to  carry  out  FRET  experi¬ 


ments.  The  simple  calibration  of  FLIM/FRET  experiments  is 
particularly  appealing  (Fig.  27.15). 

•  The  image  acquisition  times  of  lifetime-imaging  microscopes 
strongly  differ.  Potentially,  the  phase-fluorometry-based  tech¬ 
nique  combined  with  pulsed  excitation  has  the  shortest  acqui¬ 
sition  times  due  to  its  ability  to  cope  with  large  signal 
intensities.  The  phase  technique  employing  sinusoidally 
modulated  excitation,  however,  will  require  very  long  acquisi¬ 
tion  times  when  combined  with  scanning  microscopy.  The 
acquisition  time  of  the  time-gated  or  phase  techniques  in  com¬ 
bination  with  pulsed  lasers  will  be  at  least  one  order  of  mag¬ 
nitude  shorter  than  in  TCSPC. 

•  In  principle  the  photon  economy  of  all  methods  employing 
pulsed  excitation  is  excellent.  The  photon  economy  of 
phase  of  fluorometry  with  sinuoidal  excitation,  however,  is 
poor.  Furthermore,  the  photon  economy  may  vary  per  imple¬ 
mentation.  For  example,  systems  with  a  long  electronics  dead 
time  (TCSPC)  or  slow  detectors  will  show  a  reduction  of  the 
photon  economy  (detection  efficiency)  above  a  certain  count 
rate. 

•  At  present  the  systems  equipped  with  TCSPC  are  best  suited 
for  the  imaging  of  very  short  lifetimes,  provided  that  they  are 
equipped  with  a  fast  detector  (MCP-PMT).  This  type  of  detec¬ 
tor  should,  however,  be  operated  at  very  low  count  rates 
(50-200  kHz). 
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Figure  27.15.  A  FRET-FLIM  experiment  involving  living  HER  14  cells  was  designed  to  determine  if  the  EGF  receptor  (labeled  with  EGF-Alexa-488)  co-local- 
ized  with  the  lipid  raft  marker  CTB  (labeled  with  Alexa-594).  The  top  row  shows  donor  fluorescence  intensity  images  and  the  bottom  row  donor  lifetime  images 
(256  x  256  pixel).  The  graph  on  the  right  plots  a  histogram  of  the  Tf  values  derived  from  pixels  in  the  regions  of  interest  (gray  boxes)  under  the  conditions  noted. 
(A)  The  donor-only  lifetime  image  (lower)  yields  an  average  lifetime  of  4.0  ns  (yellow  curve).  (B)  In  the  presence  of  the  acceptor,  FRET  reduces  the  donor  life¬ 
time  to  about  3.5  ns  (lower  image  and  blue  curve).  (C)  After  photobleaching  the  acceptor,  the  donor  lifetime  (lower  image  and  red  curve)  returns  almost  com¬ 
pletely  to  the  donor-only  lifetime  (experiment  on  a  different  cell). 
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Multi-Photon  Molecular  Excitation  in 
Laser-Scanning  Microscopy 

Winfried  Denk,  David  W.  Piston,  and  Watt  W.  Webb 


INTRODUCTION 

Multi-photon  microscopy  (MPM),  which  is  based  on  molecular 
excitation  by  multi-photon  absorption  (MPA)  and  is  usually  com¬ 
bined  with  laser-scanning  microscopy  (LSM),  has  fulfilled  its  early 
promise  (Denk  et  al.,  1990),  as  evidenced  by  continued  growth  of 
its  application  to  vital  imaging  of  biological  systems  (for  a  recent 
collection  of  reprints,  see  Masters,  2003).  Conventional  fluores¬ 
cence  microscopy  can  provide  submicron  spatial  resolution  of 
chemical  dynamics  within  living  cells,  but  is  frequently  limited 
in  sensitivity  and  spatial  resolution  by  background  due  to  out-of¬ 
focus  and  scattered  fluorescence.  The  superlinear  character  of 
multi-photon  excitation  (MPE)  avoids  background  because  the 
excitation  is  almost  entirely  confined  to  the  high-intensity  region 
near  the  focal  point.  As  excitation  of  the  out-of-focus  background 
fluorescence  is  avoided,  no  confocal  spatial  filter  is  required;  we 
retain  all  of  the  advantages  of  a  (single-photon)  confocal  micro¬ 
scope  and  gain  the  absence  of  out-of-focus  photobleaching  and 
photodamage. 

Multi-photon  molecular  excitation  during  a  single  quantum 
event  was  first  predicted  more  than  75  years  ago  (Goeppert-Mayer, 
1931)  and  consists  of  the  simultaneous  absorption  of  multiple 
photons  that  combine  their  energies  to  cause  the  transition  to  the 
excited  state  of  the  chromophore.  For  example,  simultaneous 
absorption  of  two  photons  of  red  or  infrared  light  can  excite  a  flu- 
orophore  that  normally  absorbs  ultraviolet  (UV)  or  blue/green 
light.  The  fluorophore  then  emits  fluorescence  with  a  wavelength 
that  usually  is  shorter  than  the  exciting  laser  wavelength.  Because 
multi-photon  absorption  requires  at  least  two  photons  for  each 
excitation,  its  rate  depends  on  a  higher  algebraic  power  of  the 
instantaneous  intensity,  just  as  the  rate  of  a  chemical  reaction, 
nA  +  B  — >  C,  varies  with  the  nth  power  of  the  concentration  of  A. 
Because  of  the  large  intensities  required,  the  first  experimental 
observation  of  two-photon  excitation  (2PE)  (Kaiser  and  Garrett, 
1961)  and  three-photon  excitation  (3PE)  (Singh  and  Bradley, 
1964)  had  to  wait  for  the  invention  of  the  laser  (Maiman,  1960), 
more  than  30  years  after  Maria  Goeppert-Mayer 's  prediction. 
In  the  decades  following  Kaiser  and  Garrett’s  work,  a  fair  number 
of  spectroscopic  studies  using  2PE  were  performed  (reviewed, 
e.g.,  by  Friedrich  and  McClain,  1980;  Birge  1986),  mainly  to 
exploit  the  different  quantum-mechanical  selection  rules  that 
govern  2PE. 

Nonlinear  optical  effects  were  first  used  in  microscopy 
to  produce  images  of  second  harmonic  generation  in  crystals 
(Hellwarth  and  Christiansen  1974;  Sheppard  and  Kompfner  1978). 
The  first  MPM  images  (using  2PE)  were  reported  in  1990  (Denk 


et  al.,  1990),  with  the  expectation  from  its  inception  to  develop 
nonlinear  laser  microscopy  as  a  new  tool  for  biophysical  research. 

Technological  advances  in  two  different  areas  have  made 
nonlinear  laser  microscopy,  in  general,  and  MPM,  in  particular, 
practical:  first,  the  development  of  LSM  (Davidovits  and  Egger, 
1969;  Wilson  and  Sheppard,  1984)  and,  second,  the  development 
of  mode-locked  lasers  that  are  capable  of  generating  ultrashort 
pulses  (=100fs)  of  red  or  infrared  light  at  high  repetition  rates 
(=  1 00  MHz).  (For  a  selection  of  reprints  on  this  subject,  see 
Gosnell  and  Taylor,  1991.) 

Early  applications  of  two-photon  microscopy  (2PM)  to  the 
study  of  dynamic  biochemical  processes  in  living  cells  demon¬ 
strated  some  of  the  advantages  for  quantitative  three-dimensional 
(3D)  and  four-dimensional  (4D)  (space  and  time)  resolved  fluo¬ 
rescence  microscopy  (see  below).  Much  progress  has  since  been 
made  in  recognizing  the  important  fundamental  parameters.  Solu¬ 
tions  have  been  found  for  technological  problems  such  as  efficient 
detection  in  scattering  samples  and  effective  commercial  instru¬ 
mentation  has  been  created.  This  has  resulted  in  many  important 
biomedical  research  problems  being  successfully  attacked  using 
MPM  (see  below),  in  particular  those  requiring  visualization  of 
dynamic  cellular  processes.  One  of  the  main  areas  of  application 
has  been  high-resolution  imaging  inside  highly  scattering  brain 
tissue  in  vitro  and  in  vivo. 

The  goal  of  this  chapter  is  to  elaborate  on  the  physical  princi¬ 
ples  of  MPM,  and  to  point  out  their  relevance  to  actual  instrument 
design,  including  the  selection  of  the  appropriate  laser  light  source. 
We  also  discuss  the  challenges  related  to  chromophore  selection 
and  characterization  and  then  list  some  of  the  applications  where 
MPM  has  made  a  difference. 


PHYSICAL  PRINCIPLES  OF  MULTI-PHOTON 
EXCITATION  AND  THEIR  IMPLICATIONS  FOR 
IMAGE  FORMATION 

Physics  of  Multi-Photon  Excitation 

How  and  why  is  MPE  different  from  1PE  and  how  does  this  lead 
to  the  unique  properties  of  MPM?  Because  most  aspects  become 
clear  when  considering  2PE,  we  will  discuss  mostly  2PM  and  point 
to  differences  with  higher  orders  where  necessary.  We  will  espe¬ 
cially  explore  the  complications  involved  in  determining  reliable 
numbers  for  multi-photon  absorption  cross-sections  and  why  and 
how  the  temporal  structure  of  the  excitation  light  can  affect 
imaging  performance. 
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2PE  as  used  here  refers  to  the  simultaneous  absorption  of  two 
photons  of  longer,  not  necessarily  identical,  wavelengths,  and 
X2,  that  combine  their  energies  to  cause  a  molecular  excitation  that 
would  otherwise  require  a  single  photon  with  a  shorter  wavelength 
(Xi1  +  A.21)"1.  This  situation  is  distinct  from  sequential  two-photon 
absorption  (2PA),  not  considered  here,  where  the  molecule  is 
excited  into  an  intermediate  (metastable)  state  by  the  first  photon, 
and  from  there  into  the  final  state  by  the  second  photon. 

The  transition  probability  for  simultaneous  2PA  depends  (as 
mentioned  above)  on  the  square  of  the  instantaneous  light  inten¬ 
sity.  The  use  of  brief  but  intense  pulses,  therefore,  increases  the 
average  two-photon  absorption  probability  for  a  given  average 
incident  power.  It  is  desirable  to  minimize  the  average  excitation 
power  to  minimize  undesirable  1PA,  which  can  occur  all  along  the 
excitation  beam  and  is  usually  responsible  for  most  heating  (see 
below)  and  may  also  cause  photodamage  directly.  The  multi¬ 
photon  “advantage”  (defined  below)  for  n-photon  excitation  is  pro¬ 
portional  to  the  inverse  excitation  duty  cycle  to  the  trl  power.  For 
example,  using  lOOfs  (lfs  =  10~15s)  duration  pulses  at  a  100  MHz 
repetition  rate  leads  to  100,000-fold  and  1010-fold  improvements 
over  CW  illumination  for  2PA  and  3PA,  respectively.  The  use 
of  such  short  pulses  and  small  duty  cycles  is,  in  fact,  essential 
to  permit  image  acquisition  within  a  reasonable  time  while  using 
biologically  tolerable  power  levels.  What  constitutes  a  tolerable 
power  level  is,  however,  hard  to  define  and  depends  on  sample 
properties,  as  well  as  imaging  parameters  such  as  magnification 
and  scan  speed.  With  high  numerical-aperture  (NA)  diffraction- 
limited  illumination,  tolerable  average  power  levels  are  generally 
around  a  few  milliwatts  at  the  focal  spot.  Due  to  losses  in  instru¬ 
ment  optics  and  sample,  source  laser  powers  of  over  1 W  may  still 
be  needed  for  deep  imaging  in  scattering  tissue  (Denk,  1996). 

The  probability  p„  that  a  fluorophore  at  the  center  of  the  focus 
absorbs  a  photon  pair  during  a  single  pulse  is,  using  the  paraxial 
approximation,  given  by  Denk  and  colleagues  (1990): 


Pa 


which  depends  linearly  on  the  two-photon  cross-section  5,  qua- 
dratically  on  the  average  power  (P),  on  the  fourth  power  of  the 
ANA,  and  inversely  on  the  repetition  frequency  F,,;  X,  c,  h  are  the 
wavelength,  the  speed  of  light  in  vacuum,  and  the  Planck  quantum 
of  action,  respectively;  the  two-photon  “advantage”  factor  is 
calculated  as  follows: 


V«,  J 


with  (t,  -  t2 )  =  Fpl.  For  a  pulse  that  is  Gaussian  in  time  (see  below) 
with  a  width  xp  (time  between  the  half-power  points)  one  finds 
4  =  {FplpY' 0.664,  and  for  a  pulse  with  a  hyperbolic-secant  shape 
the  quite  similar  value  of  \  ~  (P,tp)_10.558. 

A  curious  property  of  2PE  (but  not  of  >2  PE)  is  that,  in  spite 
of  the  strong  NA  dependence  of  the  peak  excitation  rate,  the  total 
amount  of  2PE  arising  from  a  focused  laser  beam  in  a  homoge¬ 
neous  distribution  of  fluorophores  is  independent  of  NA.  This  can 
be  understood  intuitively  by  realizing  that  the  decline  of  the  peak 
2PA  probability  by  reducing  NA  is  exactly  compensated  by  an 
increase  in  the  focal  volume,  and  thus  an  increase  in  the  number 


of  fluorophores  in  the  excitation  region.  The  total  absorbed  power 
can  be  calculated  using  a  slightly  modified  form  of  Eq.  4  of  Birge 
(1986): 


2ntic 


(3) 


where  C  is  the  chromophore  concentration  and  T|  is  the  refractive 
index. 

The  quantum-mechanical  selection  rules  for  2PA  differ  from 
those  for  1PA  (Birge,  1979,  1986;  Friedrich  and  McClain,  1980; 
Loudon,  1983).  In  fact,  for  isolated  atoms  a  transition  allowed  for 
1PA  would  be  strictly  forbidden  for  2PA  and  vice  versa.  However, 
due  to  their  reduced  symmetry  and  the  effect  of  molecular  vibra¬ 
tions,  strict  parity  selection  rules  do  not  usually  hold  for  complex 
dye  molecules  (McClain,  1971). 

A  number  of  heuristic  rules  for  the  expected  two-photon 
spectra  can  nevertheless  be  formulated  when  the  single-photon 
spectrum  is  known:  (1)  Some  2PE  usually  occurs  at  a  particular  X 
whenever  1PE  occurs  at  X/2.  (2)  Additional  features  appear,  if  at 
all,  on  the  short  wavelength  side  of  the  spectrum.  (3)  Two-photon 
spectra  are  generally  broader  than  single-photon  spectra.  (4) 
Good  (strongly  absorbing)  single-photon  fluorophores  are  often 
very  good  two-photon  fluorophores,  whereas  bad  single-photon 
absorbers  tend  to  be  very  bad  two-photon  absorbers.  The  absence 
of  additional  long-A,  features  is  simply  due  to  the  fact  that,  toward 
longer  wavelengths,  the  combined  energy  of  the  photons  is  no 
longer  sufficient  to  reach  the  excited  state.  Both  rules  2  and  3  arise 
because  single-photon  inaccessible  states  with  higher  energy  that 
have  no  direct  wave-function  overlap  with  the  ground  state  can 
often  be  reached  with  two-photon  excitation  through  intermediate 
(virtual)  states  that  do  overlap  with  both  the  initial  and  the  final 
state  (Mortensen  and  Svendsen,  1981;  Loudon,  1983;  Birge,  1986). 
Rule  4  arises  because,  as  the  two-photon  excitation  process  uses 
the  typical  transition  matrix  element  twice,  its  size  affects  the  two- 
photon  cross-section  quadratically. 

Equation  1  is  only  correct  as  long  as  the  probability  Pa  for  each 
fluorophore  to  be  excited  during  a  single  pulse  is  much  smaller 
than  one.  The  reason  for  this  is  that  during  the  pulse  (given  a  pulse 
length  of  about  lOOfs  and  a  typical  excited-state  lifetime  x,  in  the 
nanosecond  range),  the  molecule  has  insufficient  time  to  relax  to 
the  ground  state,  which  is  a  prerequisite  of  being  able  to  absorb 
another  photon  pair.  Therefore,  whenever  Pa  approaches  unity,  sat¬ 
uration  effects  begin  to  occur.  In  a  strongly  focused  beam  with 
pulse  lengths  and  repetition  rates  as  mentioned  above,  average 
power  levels  of  several  tens  of  milliwatts  were  estimated  to  cause 
saturation  (Denk  et  al.,  1990).  However,  the  use  of  recently 
developed  fluorophores  and,  particularly,  of  quantum  nanoparticles 
with  large  two-photon  cross-sections  can  lead  to  saturation  at  much 
lower  power  levels.  Because  saturation  depends  on  the  location 
within  the  focal  spot,  the  point  spread  function  is  altered  in  a  way 
that  reduces  the  resolution.  For  comparison,  the  power  levels 
leading  to  ground-state  depletion  in  single-photon  excitation  are 
on  the  order  of  1  mW  (see  Chapter  2,  this  volume).  Often  (but  not 
always)  the  desirable  time  between  pulses  is  around  x,  because 
slower  repetition  rates  leave  the  fluorophore  idle  between  pulses, 
thus  lowering  the  saturation  limit  on  fluorescence  output,  and 
faster  repetition  rates  erode  the  two-photon  advantage  raising 
the  required  average  power  to  achieve  a  particular  fluorescence 
level.  Repetition  rates  of  around  100 MHz  (one  pulse  every  10ns), 
which  are  common  in  commercially  available  mode-locked  lasers, 
are  thus  in  the  desirable  range  even  though  somewhat  higher  rep¬ 
etition  rates  can  reduce  saturation  and  nonlinear  bleaching  effects 
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when  high  fluorescence  rates  are  needed  and  average  power  is  not 
limiting. 

Saturation  due  to  ground  state  depletion  limits  the  amount  of 
two-photon  excitation  power  that  can  be  usefully  directed  into  a 
single,  diffraction-limited  spot  and  thus  limits  the  maximally  avail¬ 
able  fluorescence  power  and  hence  the  signal  acquisition  rate. 
Given  sufficient  laser  power,  simultaneous  illumination  of  multi¬ 
ple  focal  volumes  (e.g.,  with  line  or  microlens-array  illumination 
(Chapters  10  and  29,  this  volume )  (Brakenhoff  et  al.,  1996; 
Straub  and  Hell,  1998;  Egner  and  Hell,  2000;  Andresen  et  al., 
2001)  can  evade  this  image-rate  limitation  by  a  factor  given  by 
the  number  of  simultaneously  illuminated  focal  volumes  («.,). 
It  is,  however,  necessary  to  use  either  n,  descanned  detectors 
(see  below),  or  an  imaging  detector.  This,  in  turn,  precludes  the 
use  of  multi-spot  excitation  in  strongly  scattering  specimens  (see 
below). 

Optical  Pulse  Length 

It  might  appear  that  in  order  to  increase  the  two-photon  advantage 
the  excitation  pulses  should  be  as  short  as  possible.  This  is  not  so 
mainly  for  two  reasons:  first,  and  of  greater  practical  importance, 
is  the  fact  that  during  propagation  through  optical  materials  and 
reflection  off  multi-layer  dielectric  coatings,  pulses  are  spread  in 
time  due  to  group  velocity  dispersion  (GVD).  This  effect,  illus¬ 
trated  in  Figure  28.1,  is  due  to  the  fact  that  the  light  in  ultrashort 
pulses  consists  of  quite  a  range  of  optical  frequencies,  and  thus 
wavelengths.  A  70fs  pulse  centered  at  800nm,  for  example,  is 
spread  over  13nm  in  wavelength.  For  a  Gaussian  pulse  (intensity 
as  a  function  of  time  t:  n(t)  °=  exp  L—  41n(2)(f  -  t0)2X~2 J  with  a  pulse 
width  xs  (between  the  points  of  half  maximum  intensity)  the  so- 
called  “transform-limited”  bandwidth  (AX),  where  the  phases  of  all 
wavelength  components  are  arranged  to  yield  the  shortest  pulse 
possible,  is  related  to  the  spread  in  optical  frequencies  (A/)  by:  1P 
=  21n(2)jr1A/M  =  0.441271/A/,  where  A/=  cAX/(X2),  with  Xc  the 
center  wavelength.  GVD  arises  in  optical  materials  as  wave 
packets  of  different  wavelength  travel  with  different  speeds,  deter¬ 
mined  by  their  group  velocities  cs  =  d(i>/dk  =  d(ck/r\)/k  =  c/r|  - 
ck(dr\/dk)/r\2  (not  to  be  confused  with  the  phase  velocity  co/£  =/X 
=  c/r 1)  where  CO  is  the  angular  frequency,  k  and  X,  are,  respectively, 
the  wave  number  and  the  wavelength  inside  the  material,  and  c  is 
the  speed  of  light  in  vacuum,  and  T|  is  the  refractive  index.  For  a 
given  optical  path,  the  accumulated  GVD  then  gives  rise  to  a 
certain  amount  of  group  delay  dispersion  (GDD),  i.e.,  light  from 
the  red  end  of  the  spectrum  arrives  at  a  different,  usually  earlier, 
time  than  light  from  the  blue  end.  This  leads  to  a  chirped  (fre¬ 
quency  swept)  pulse  that  is  longer  than  the  original  pulse  but  still 
contains  (at  unchanged  spectral  density)  the  same  optical  frequen¬ 
cies  and  hence  wavelengths.  Because  the  pulse’s  total  energy 
content  is  unchanged  by  GDD,  chirping  always  reduces  the  peak 
intensity  and  hence  the  average  squared  intensity,  which,  in  turn, 
determines  the  two-photon  excitation  probability.  The  difference 
in  the  arrival  times  increases  with  increasing  AX,  and  because 
shorter  pulses  have  a  broader  spectrum  (see  above),  they  are,  for 
a  given  amount  of  GDD,  stretched  more  than  longer  pulses.  This 
effect  is  compounded  by  the  fact  that  the  same  amount  of  stretch¬ 
ing  lengthens  a  shorter  pulse  by  a  larger  fraction  of  its  original 
length  x“.  Therefore,  the  two-photon  advantage  which  depends 
on  T,„  is  degraded  by  a  pulse-broadening  factor  depending  on  the 
inverse  square  of  x“.  For  a  Gaussian  pulse  we  find  for  the  spread 
pulse 

f  ,  =  i°P  Vl  +  [41n(2  )lq/x2pf,  (4) 
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FIGURE  28.1.  Simulating  the  effect  of  group  velocity  dispersion  (GVD)  on 
the  pulse  shape  of  an  ultrashort  pulse.  The  pulse  has  initially  a  FWHM  width 
of  40  fs  and  is  then  dispersed  by  about  1250  fs2  of  GDD  (corresponding  to  about 
35  mm  of  fused  silica  or  less  than  5  mm  of  SF59  glass).  For  comparison  the 
electric  field  (top  row  of  panels),  the  intensity  (middle  row),  and  the  squared 
intensity  (bottom  row;  corresponding  to  the  two-photon  excitation  efficiency) 
are  shown  both  without  (left  column)  and  with  dispersion  (right  column).  An 
unrealistically  long  center  X  (4000  nm)  was  chosen  in  order  to  emphasize  the 
chirping  effect.  In  reality  (for  900  nm  light),  a  30  fs  pulse  would  be  10  full 
cycles  long  rather  than  about  2  cycles  as  shown  here. 


where  /  is  the  length  of  the  light  path  inside  the  material  and  £  = 
c~'  52(r|CO)/dco2  the  GVD  parameter.  Therefore,  for  a  given  amount 
of  GDD  (/Q  there  is  an  optimal  x°  that  leads  to  the  shortest  xp  after 
passing  the  group-velocity-dispersing  elements.  For  example,  for 
1  cm  of  fused  silica,  with  £  =  362fs2/cm  at  X  =  800  nm.  the  short¬ 
est  xp  (=45  fs)  is  obtained  for  x“  =  30  fs.  Highly  corrected  lenses 
often  use  optical  glasses  that  have  considerably  larger  GVDs,  and 
in  most  microscopes,  light  passes  through  considerably  more  than 
1  cm  of  glass.  For  example,  GVD  values  at  800 nm  are  338,  453, 
447,  870,  1187,  1193,  1896,  2236,  and  2936fs2/cm  for  the  Schott 
glasses  FK51,  BK7,  BKI,  LFS,  SF2,  TiF6,  SF11,  SF57,  and  SF59, 
respectively  (calculated  from  refractive  index  data  in  the  Schott 
glass  catalog;  Schott  Glass  Technologies,  Duryea,  PA).  The  GDDs 
of  microscope  objectives  and  whole  laser-scanning  microscopes 
have  been  explored  experimentally  and  theoretically  (Guild  et  al., 
1997;  Muller  et  al. ,  1998;  Wolleschensky  et  al.,  1998).  The  effects 
of  GDD  in  MPM  are  discussed  again  below. 
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In  theory,  broadening  that  is  due  to  the  GDD  can  be  compen¬ 
sated  by  prechirping  the  pulse  (giving  the  blue  wavelengths  a  head 
start),  using  a  prism  or  grating  arrangement  (Fork  et  al.,  1984)  in 
such  a  way  that  different  wavelengths  arrive  at  the  sample  almost 
simultaneously  after  passing  the  microscope  optics.  However,  in 
view  of  the  added  alignment  complexity  and  possible  power  losses 
in  the  compensation  optics  as  well  as  the  need  for  readjustment  to 
a  different  GDD  value  for  each  objective  lens  and  excitation  wave¬ 
length,  it  has  to  be  carefully  weighed  whether  prechirping  is  worth 
the  effort,  as  it  might  well  be  if  single-photon  absorption  or  lack 
of  laser  power  are  an  issue.  It  is  worth  noting  (Eq.  4)  that  for  large 
amount  of  GDD  the  pulse  length  roughly  increases  linearly  with 
the  amount  of  GDD,  but  small  amounts  affect  the  pulse  length  dis¬ 
proportionately  less. 

Even  if  pulse  broadening  by  GDD  is  completely  compensated, 
there  is  a  second  factor  putting  a  lower  bound  on  the  optimal  1% 
As  the  X  spectrum  broadens  with  the  shortening  of  the  pulses,  it 
will  eventually  become  wider  than  the  absorption  spectrum  of  the 
chromophore.  This  limit  is,  however,  not  pressing  because  most 
chromophores  used  in  fluorescence  microscopy  have  spectra 
between  20  and  50 nm  wide  (full-width  half-maximum,  FWHM), 
for  which  pulses  with  a  length  of  23  and  9fs,  respectively,  have  a 
matched  (doubled)  spectral  width  at  700  nm.  A  very  interesting 
development  in  this  context  is  the  use  of  coherent  control  tech¬ 
niques  that  apply  complex  phase  relationships  between  the  differ¬ 
ent  X  components  to  select  particular  excitation  pathways  (see, 
e.g.,  Walowicz  et  al.,  2002;  Lozovoy  et  ah,  2003). 

Excitation  Localization 

Most  of  the  properties  that  make  MPM  so  useful  for  fluorescence 
microscopy  derive  from  the  quadratic  or  stronger  dependence  of 
the  excitation  probability  on  the  excitation  light  intensity.  In  a 
strongly  focused  excitation  beam,  the  excitation  probability 
outside  the  focal  region  falls  off  with  z-2",  where  z  is  the  distance 
from  the  focal  plane  and  n  is  the  number  of  photons  absorbed  per 
quantum  event.  In  a  thick  sample  with  a  spatially  homogeneous 
distribution  of  chromophores  and  for  a  Gaussian  beam,  about  80% 
of  the  two-photon  absorption,  and  therefore  80%  of  the  total  fluo¬ 
rescence,  occurs  in  a  volume  bounded  by  the  e~ 1  iso-intensity 
surface,  which  for  an  objective  lens  with  an  NA  =  1 .4  is  contained 
within  an  ellipsoid  (0.3  pm  in  diameter  and  1  pm  long  for  X  =  700 
nm)  or  approximately  0.1  femtoliter  (pm3)  in  volume  (Sandison 
and  Webb,  1994).  This  means  that  MP  (unlike  IP)  excitation  is 
truly  localized  and  as  a  consequence  provides  excellent  depth  dis¬ 
crimination,  which  is  similar  (for  the  2P  case  nearly  identical)  to 
that  of  an  ideal  IP  confocal  microscope.  Because,  in  contrast  to 
the  IP  case,  3D  resolution  is  due  to  the  confinement  of  excitation 
to  the  focal  volume,  out-of-focus  photobleaching  and  photodam¬ 
age  and  the  attenuation  of  the  excitation  beam  by  out-of-focus 
absorption  do  not  occur,  and  because  no  spatial  filter  (detection 
pinhole)  is  required,  none  needs  to  be  aligned.  Figure  28.2  shows 
a  comparison  between  an  .rz-section  through  a  bleaching  pattern 
that  was  generated  by  repeated  2P  scanning  of  a  rectangular  area 
in  a  single  xy-plane  in  a  thick,  rhodamine-stained  Formvar  layer 
and  an  xz-section  through  a  bleaching  pattern  caused  by  IP  scan¬ 
ning.  In  the  IP  case  bleaching  occurs  throughout  the  depth  of  the 
sample. 

Detection 

The  fact  that  resolution  and  discrimination  are  defined  by  the  exci¬ 
tation  process  alone  leaves  substantially  more  freedom  when 


FIGURE  28.2.  Confinement  of  photodynamic  effects,  such  as  bleaching,  to 
the  focal  slice,  xz-profiles  of  the  bleach  patterns  formed  by  repeatedly  scanning 
the  laser  focus  over  a  single  xy  optical  section  in  a  thick  film  of  rhodamine- 
doped  Formvar  until  the  fluorescence  from  the  focal  plane  was  largely 
bleached.  The  scanned  area  extends  through  about  half  the  image  width  shown. 
Single-photon  excitation  was  used  on  the  left  and  two-photon  excitation  on  the 
right.  IP  bleaching  extends  throughout  the  sample  thickness  while  2P  bleach¬ 
ing  is  confined  to  a  thin  region  around  the  focal  plane.  The  widening  of  the 
bleached  region  seen  above  and  below  the  focal  plan  for  IP  bleaching  is  due 
to  the  high  NA  illumination  cone. 

choosing  the  detection  strategy:  (1)  The  emitted  light  does  not  have 
to  pass  through  the  microscope  objective  at  all,  allowing  the  use 
of  emission  wavelengths  that  are  not  transmitted  by  the  objective 
lens  but  could  instead  be  detected  by  a  photodetector  placed,  for 
example,  on  the  far  side  of  the  sample.  (2)  The  emitted  light  does 
not  have  to  be  focused.  Therefore,  scattering  of  emitted  light  can 
be  tolerated  without  any  loss  of  detection  efficiency  or  resolution 
(Denk  et  al.,  1994;  Denk  and  Svoboda,  1997).  This  is  especially 
useful  in  strongly  scattering  samples,  such  as  brain  tissue  (Denk 
et  al.,  1994;  Yuste  and  Denk,  1995;  Svoboda  et  al.,  1997).  Fur¬ 
thermore,  due  to  reduced  scattering  at  longer  wavelength,  the  2P 
excitation  wavelength  ( X2ex )  can  be  focused  to  an  adequately  de¬ 
fined  focus  (which  might  be  impossible  for  the  corresponding  XUx). 
Only  a  vanishing  fraction  of  the  short  wavelength  ( Xem )  and  hence 
strongly  scattered  emission  light  emerges  unscattered  (ballistic) 
and  could  be  used  for  confocal  detection,  virtually  precluding  the 
use  of  IP  confocal  microscopy  in  such  samples. 

(3)  Non-optical  signals  such  as  photo-chemically  induced 
current  signals  in  biological  cells  (Denk,  1994;  Furuta  et  al.,  1999; 
Matsuzaki  et  al.,  2001)  or  photo-induced  currents  from  semicon¬ 
ductor  circuits  (Xu  and  Denk,  1997,  1999)  can  be  used  to  gener¬ 
ate  optically  sectioned  images. 

Wavelengths 

The  range  of  350  nm  <  XUx  <  500  nm  (700  nm  <  XUx  <  1000  nm) 
is  most  widely  used  to  excite  fluorescence  indicators  and  photo- 
activatable  compounds.  The  ability  of  MPM  to  reach  short  UV 
excitation  energies  beyond  those  reachable  with  1PE  (XUx  <  300 
nm)  has  so  far  only  rarely  been  used  for  imaging  applications 
(Wokosin  et  al,  1996b;  Xu  et  al.,  1996;  Maiti  etal.,  1997;  Williams 
etal,  1999).  One  reason  is  that  photodamage  can  occur  due  to  MPE 
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by  intrinsic  chromophores  (Rehms  and  Callis,  1993)  in  proteins  and 
DNA  (Williams  and  Callis,  1990).  While  photodamage  has  been 
insufficiently  studied  (there  are  many  anecdotal  reports  on  tissue 
and  cell  photodamage,  but  few  non-controversial  facts)  it  appears 
that  longer  excitation  wavelengths  are  better  tolerated  by  living 
cells  and  tissues,  especially  at  high  excitation  intensities. 


Resolution 

Another  important  question  is  the  resolution  of  1PCM  versus 
MPM.  This  was  first  quantitatively  discussed  by  Sheppard  and  Gu 
(1990)  and  Nakamura  (1993).  The  answer  depends  strongly  on 
whether  the  fluorophore  or  the  excitation  energy  is  held  fixed. 
Using  the  same  excitation  wavelengths,  the  2PM,  even  without  a 
detector  spatial  filter  (pinhole),  has  a  slightly  improved  resolution 
due  to  the  lack  of  a  Stokes-shift  effect  and  a  very  small  equivalent 
pinhole  size  (Sheppard  and  Gu,  1994).  When  using,  more  appro¬ 
priately,  the  same  fluorophore,  X2„  =  2  X  the  resolution  of  the 
2PM  is  degraded  by  a  factor  of  almost  2  (somewhat  less  if  the  flu¬ 
orophore  has  a  large  Stokes  shift)  compared  to  the  ideal  (zero- 
pinhole  size)  confocal  microscope.  However,  for  a  realistic  pinhole 
size  (Gauderon  and  Sheppard.  1999),  the  performance  of  the 
1PCM  deteriorates,  so  that  in  practice  the  resolution  in  1PCM  and 
MPM  is  about  the  same.  A  significant  resolution  enhancement  in 
multi-photon  microscopy,  albeit  at  the  expense  of  collection  effi¬ 
ciency  (see  below),  can  be  achieved  by  using  a  detection  spatial 
filter  in  conjunction  with  a  relatively  short  excitation  wavelength 
(Stelzer  et  al,,  1994).  The  resolution  can  be  substantially  improved 
in  both  1PM  and  MPM  along  the  axial  direction  by  using  illumi¬ 
nation  from  almost  all  directions  as  with  the  4-Pi  microscope  (Hell 
and  Stelzer,  1992)  (see  also  Chapter  30,  this  volume). 

In  conclusion,  2PE  does  not  normally  lead  to  resolution 
improvements  over  confocal  microscopy.  In  fact,  if  resolution  is 
of  paramount  importance  and  scattering  is  moderate,  1PE  confo¬ 
cal  microcopy  is  usually  better.  Recently  it  has  been  shown  that 
stimulated  emission  depletion  (STED)  microscopy,  a  very  differ¬ 
ent  type  of  nonlinear  optical  microscopy,  can  overcome  the  far- 
field  diffraction  limit  (Dyba  and  Hell.  2002)  (see  also  Chapter  31, 
this  volume). 


Photodamage:  Heating  and  Bleaching 

Photodamage  to  cells  and  tissue  can  result  from  1PA  or  MPA, 
depending  on  illumination  wavelength,  on  type  and  concentration 
of  chromophores  present,  and  on  the  power  level.  In  particular, 
when  infrared  ?is  are  used  (>900  nm),  we  have  to  consider  heating 
due  to  increased  absorption  by  liquid  water,  which  is  not  a  problem 
at  visible  and  near-UV  Xs  where  water  is  very  transparent 
(e.g.,  see  Fig.  3  in  Svoboda  and  Block,  1994  or  Fig.  23.3,  this 
volume). 

For  1PE,  an  upper-bound  estimate  of  the  temperature  rise  can 
be  made  using  a  2D  approximation  because  absorption  occurs  all 
along  the  beam  path.  The  calculation  sketched  here  is  the  same  as 
was  used  to  analyze  thermal  lens  effects  (Whinnery,  1974;  Kliger, 
1983,  and  references  therein).  For  the  temperature  rise  T  at  the 
center  of  the  beam  (r  =  0)  as  a  function  of  the  time  t  after  switch¬ 
ing  on  the  beam  one  obtains 

<5) 

where  a  is  the  absorption  coefficient,  P  the  laser  power,  kT  the 
thermal  conductivity,  and  xr  =  C0s/(4k)  the  thermal  time  constant, 
which  is  a  measure  of  how  fast  steady-state  conditions  are  ap¬ 


proached  and  which  depends  on  co0,  the  Gaussian  beam  parameter 
and  is  equivalent  to  the  beam  radius  (l/ez  intensity)  in  the  focal 
plane,  and  on  the  thermal  diffusivity  K  =  k,/p  where  p  is  the  volume 
heat  capacity.  For  a  diffraction-limited  beam  at  high-NA  (w„  =  200 
nm),  xr  =  70  ns  in  water  (using  kw  =  0.6  WK  'nr ',  Kw  =  1.44  X  10-7 
mV)  and  for  absorption  by  pure  water,  the  pre-factor  in  Eq.  4 
is  0.013,  0.21,  and  0.66K  at  A,s  of  700,  1000,  and  1300nm, 
using  the  absorption  coefficients  for  water  of  0.02,  0.32,  and 
1.0cm-1,  respectively;  the  laser  power  was  assumed  to  be  50mW 
(approximately  the  saturating  intensity;  Denk  et  al. ,  1990).  Slightly 
lower  values,  still  logarithmically  diverging  with  time,  are  found 
if  axial  heat  transport  is  taken  into  account  (Schonle  and  Hell, 
1998).  Due  to  the  small  beam  diameter,  xr  is  rather  short  (=70ns) 
for  high-NA  objectives.  For  video-rate  scanning  microscopes 
(Goldstein  et  al.,  1992;  Fan  et  al.,  1999;  see  Chapter  29,  this 
volume)  this  results  in  a  temperature  rise  of  only  1.55  times  the 
pre-factor  but  at  10|is  dwell-time  (typical  for  non-resonant  mirror- 
scanned  instruments),  the  temperature  rise  is  5  times  the  prefactor. 
For  an  infinite  sample  no  steady-state  value  for  the  temperature 
would  ever  be  reached.  In  practice,  the  temperature  rise  will  even¬ 
tually  be  limited,  by  the  finite  sample  size  and  by  convection  or 
bath  perfusion,  which  remove  heat  at  a  rate  much  faster  than  heat 
conduction  alone.  For  stationary  applications  or  when  continu¬ 
ously  scanning  a  small  area,  rather  large  logarithmic  factors  can 
occur,  however.  Therefore,  water  absorption  may  have  to  be  taken 
seriously,  particularly  at  high  illumination  powers  and  long  wave¬ 
lengths  or  when  attempting  multi-photon  excitation  with  CW 
lasers  (Hanninen  et  al.,  1994,  1996;  Booth  and  Hell,  1998;  Hell  et 
al,  1998).  Fast  scan  rates,  rapid  bath  perfusion,  thin  sample  cells, 
and,  of  course,  maximizing  the  two-photon  advantage  using  the 
shortest  pulses  possible  are  remedies  to  reduce  high  peak  temper¬ 
atures.  To  assess  the  IP  effects  of  infrared  (IR)  beams  on  biolog¬ 
ical  specimens,  we  can  also  exploit  the  experience  gained  with 
optical  tweezers,  which  are  routinely  used  on  living  cells 
at  comparable  or  higher  power  densities  (Ashkin  et  al,  1987; 
Svoboda  and  Block,  1994)  and  for  which  damage  has  been 
assessed  for  most  of  the  wavelength  regime  used  in  2PM  (Neuman 
et  al.,  1999). 

Heating  due  to  2PA  is  restricted  to  the  focal  region.  A  3D  model 
is,  therefore,  appropriate.  Because  we  are  interested  in  the  case  of 
high-NA,  we  can  use  the  approximation  that  the  release  of  heat 
occurs  uniformly  within  a  sphere  with  radius  co0  centered  at  the 
focus.  The  relationship  one  gets  for  the  temperature  rise  is: 
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where  Pahs  is  the  total  absorbed  power  (see  below).  For  large  t, 
when  the  square  root  goes  to  zero,  T3D,  unlike  T2D,  approaches 
an  asymptotic  value,  given  by  the  factor  in  front  of  the  square 
brackets. 

For  high  energy  (mj)  pulses  at  low  repetition  rate,  the  local 
temperature  rise  during  a  single  pulse  can  easily  be  large  enough 
to  cause  damage,  but  we  know  little  about  how  damage  might  be 
exacerbated  for  the  case  of  pulsed  light  at  high  repetition  rates 
(fR  =100  MHz)  compared  to  the  CW  case  with  the  same  average 
power.  Because  xc  is  longer  than  the  interpulse  interval  (1  lfR),  the 
incremental  temperature  rise  during  a  single  pulse  is  smaller  than 
the  steady-state  temperature  rise  TlD  ( t  =  °°)  roughly  by  a  factor  of 
xTv 

We  conclude  that  heating  during  high-repetition-rate  pulsed 
illumination  can  largely  be  treated  like  CW  illumination  and  is 
often  negligible  at  practical  2PM  parameters.  Attention  has  to  be 
paid  to  situations  where  high  local  concentrations  of  chromophore 
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occur,  as,  for  example,  for  DNA  stains,  which  can  bind  at  a  con¬ 
centration  of  one  per  base  pair  or  where  equilibration  of  the  mo¬ 
lecular  temperature  with  its  environment  cannot  be  automatically 
assumed  (Akaaboune  et  al.,  2002). 

Because  of  the  localization  of  excitation  to  the  focal  volume, 
total  photobleaching  in  MPM  is  generally  much  reduced  compared 
to  1PE  microscopy.  However,  it  has  been  shown  that  an  increased 
photobleaching  rate  from  within  the  focal  volume  can  occur  by 
a  mechanism  where  the  fluorophore  is  initially  excited  by  simul¬ 
taneous  2PA,  and  then  one  or  more  photons  interact  with  the 
excited  molecule,  possibly  via  higher-order  resonance  absorption 
(Patterson  and  Piston,  2000).  This  effect  can  be  quite  pronounced 
for  readily  photobleachable  dyes,  such  as  fluorescein,  where  the 
difference  between  one-  and  two-photon  photobleaching  rates  can 
be  a  factor  of  10  at  the  power  levels  that  are  typically  used  in 
biological  imaging  (100|TW  CW  for  single-photon  excitation,  and 
3mW  150fs  pulses  for  MPM).  However,  for  more  stable  dyes, 
such  as  the  green  fluorescent  proteins  (GFPs),  carbocyanines,  and 
AlexaFluors,  the  photobleaching  rate  is  in  our  experience  often  too 
small  for  the  difference  to  be  measurable  at  the  usual  imaging 
intensities.  While  in  some  cases  direct  higher-order  absorption 
(three  or  more  photons)  may  be  relevant,  several  studies  (Koester 
et  al.,  1999;  Konig  et  al.,  1999)  have  found  that  longer  pulses 
(which  reduce  higher-order  absorption)  do  not  reduce  the  damage 
done  per  excitation  event. 

INSTRUMENTATION 

Setup  (Fig.  28.3)  and  operation  of  a  MPM  system  are  very  similar 
to  those  of  a  IP  laser-scanning  microscope.  The  main  differences 
lie  in  the  type  of  excitation  lasers  and  in  the  increased  number  of 


options  for  detection.  In  this  section  we  will  discuss  laser  sources 
suitable  for  MPE,  the  advantages  and  drawbacks  of  the  various 
methods  of  detecting  fluorescence  and  other  contrast  signals,  and 
specific  problems  that  occur  with  non-mechanical  (e.g.,  acousto- 
optical)  beam  power  control  and  deflection.  We  assume  that  the 
reader  of  this  chapter  is  familiar  with  the  principles  of  1PCM  (other 
chapters,  this  volume).  Short  shrift  will,  therefore,  be  given  to  those 
aspects  such  as  mechanical  beam  scanning,  data  collection, 
storage,  and  display  that  are  largely  identical  for  1PCM  and  MPM 
instruments.  The  potential  user  should  also  be  aware  that  MPMs 
are  relatively  easy  to  set  up  and  are  now  available  as  integrated 
systems  from  several  manufacturers.  MPM  systems  are  still  expen¬ 
sive  with  the  price  of  the  laser  system  (>$150,000)  being  between 
one  third  to  one  half  of  the  total  system  cost.  With  a  mode-locked 
laser,  one  has,  however,  also  acquired  the  light  source  necessary 
to  do  time-resolved  fluorescence  measurements  (Piston  et  al., 
1992;  Zhang  et  al.,  2002;  and  Chapter  27,  this  volume). 

Lasers  and  the  Choice  of  Excitation  Wavelengths 
CPM  Laser 

The  first  2P  images  (Denk  et  al.,  1990)  and  2P  photochemical 
microcopy  images  (Denk,  1994)  were  recorded  using  colliding- 
pulse  mode-locked  (CPM)  lasers  (Valdmanis  and  Fork,  1986)  at 
615  nm  excitation  wavelength.  Today  this  laser  type  is  of  only  his¬ 
torical  interest. 

Hybrid  Mode-Locked  Dye  Laser 

Another  early  type  of  ultrashort  pulse  dye  laser  system  is  the 
hybrid  mode-locked  dye  laser.  These  systems  use  an  actively 
mode-locked  argon-ion  or  a  frequency-doubled  neodymium:  YAG 


FIGURE  28.3.  Schematic  diagram  of  a  two-photon  laser-scanning  microscope  illustrating  various  detection  possibilities.  The  stream  of  incoming  laser  light 
pulses  is  raster  scanned  (xy  scanner,  only  one  axis  is  shown  here)  in  a  way  that  is  identical  to  the  single-photon  LSM.  For  fluorescence  microscopy,  several  detec¬ 
tion  possibilities  are  indicated:  (1)  external,  fluorescence  light  bypasses  objective  lens;  (2)  whole-area :  fluorescence  light  passes  objective  lens  and  is  then 
deflected  by  a  dichroic  mirror  to  be  focused  onto  the  detector  by  a  transfer  lens;  (3)  descanned:  as  in  the  1PLSM,  the  fluorescence  light  is  reflected  off  the  scan¬ 
ning  mirrors,  allowing  confocal  detection  (see  text).  Not  shown,  but  possible  and  occasionally  used,  is  focal-array  detection,  where,  after  deflection  by  a  dichroic 
mirror,  fluorescence  light  is  detected  by  an  array  detector  located  in  an  image  plane.  Yet  another  possibility  is  non-optical  detection  using,  for  example,  an  elec¬ 
trically  recorded  signal  from  the  sample.  Time  scales  are  indicated  in  the  left  inset. 


Multi-Photon  Molecular  Excitation  in  Laser-Scanning  Microscopy  •  Chapter  28  541 


laser  to  pump  a  dye  laser  that  also  contains  an  intracavity  saturable 
absorber  jet.  Such  systems  are  rather  expensive  and  difficult  to 
operate  and  are  therefore  rarely  used  for  multi-photon  imaging. 
The  remaining  advantage  over  the  titanium :  sapphire  laser  (see 
below)  is  the  access  to  the  range  550  nm  <  A,  <  700  nm,  which  is 
desirable  for  some  uncaging  experiments  but  has  been  virtually 
abandoned  for  imaging  due  to  photodamage  problems  (Kiskin 
et  al,  2002). 

Titanium :  Sapphire  Laser 

For  most  applications,  the  light  source  of  choice  for  MPM  currently 
is  the  self-mode-locked  titanium :  sapphire  (Ti :  Sa)  laser  (Spence 
et  al,  1991),  nowadays  pumped  by  a  frequency-doubled  diode- 
pumped  Nd :  Vanadate  laser  rather  than  a  power-  and  cooling-water 
hungry  argon-ion  laser.  The  Ti :  Sa  laser  provides  a  large  tuning 
range  (from  slightly  below  700  nm  to  slightly  above  1050nm)  with 
pulse  lengths  shorter  than  lOOfs  and  sufficient  power  (2  W  average 
at  the  peak  of  the  tuning  curve,  down  to  a  few  hundreds  of  milli¬ 
watts  at  the  edges  when  pumped  with  10W)  to  permit  saturating 
excitation  (see  Physical  Principles)  of  most  fluorophores  with  a 
high-NA  objective  over  much  of  the  laser’s  tuning  range.  The 
tuning  range  of  Ti :  Sa  is  now  covered  by  a  single  set  of  cavity 
mirrors,  with  optics  changes  only  required  to  reach  wavelength 
above  lOOOnm  or  below  700  nm.  Currently,  several  manufacturers 
offer  turnkey  laser  systems  that  contain  the  pump  source  and  Ti : 
Sa  laser  inside  a  single  housing,  are  computer  controlled,  and  no 
longer  require  any  mechanical  adjustments  by  the  operator. 

Other  Light  Sources 

If  losses  in  the  excitation  path  are  too  large,  it  is  sometimes  not 
possible  to  achieve  the  desired  excitation  rates  with  the  multi¬ 
photon  advantage  factors  available  for  a  laser  oscillator  alone.  A 
reduction  in  repetition  rate  while  maintaining  average  power  can 
then  increase  the  excitation  efficiency  substantially  (Beaurepaire 
et  al.,  2001).  This  can  be  achieved  by  increasing  the  cavity  length, 
cavity  dumping,  or  regenerative  amplification.  The  last  approach 
has  recently  been  shown  to  allow  imaging  down  to  the  surface¬ 
generated-background  limit  (Theer  et  al,  2003).  Direct  use  or  fre¬ 
quency  doubling  of  femtosecond  pulses  from  optical  parametric 
oscillators  (OPO)  (Cheung  and  Liu,  1991;  Fu  et  al.,  1992;  Powers 
et  al.,  1994;  Keller,  1996)  may  provide  an  almost  universal,  if 
expensive,  solution  to  cover  almost  all  of  the  desired  wavelength 
range. 

One  factor  limiting  multi-photon  microscopy  is  the  cost  of  the 
laser  source,  which,  in  spite  of  early  hopes,  has  not  come  down 
significantly  with  the  introduction  of  diode  pumping  (for  a  review, 
see  Keller,  1994).  One  reason  is  that  gain  materials  that  can  be 
directly  diode-pumped  (Keller,  1996)  have  insufficient  tuning 
ranges  and/or  unfavorable  thermal  characteristics. 

In  niche  applications,  other  sources  (Wokosin  et  al.,  1996a) 
have  been  used,  partly  within,  for  example,  the  Cr:LiSaF  laser 
(Svoboda  et  ah,  1996a)  or  outside,  for  example  the  Cr : Forsterite 
laser  (Liu  et  al.,  2001),  the  Ti:Sa  tuning  range. 

Excitation  Wavelengths 

One  reason  for  the  success  of  the  Ti :  Sa  laser  for  MPM  is  that  the 
range  700  nm  <  <  1050nm  (corresponding  to  350  nm  <  ~Kia  < 

525  nm)  covers  the  range  of  excited  state  energies  for  many  com¬ 
monly  used  fluorophores  (see  below).  Much  shorter  wavelengths, 
in  particular  <  640  nm,  are  likely  to  cause  photodamage  due  to 
intrinsic  absorption,  for  example,  by  tryptophan  rich  proteins 
(Rehms  and  Callis,  1993).  To  minimize  scattering  one  might 


lengthen  the  excitation  wavelengths  and  take  advantage  of  a  dip 
in  the  absorption  spectrum  of  water  around  1040nm,  which  is  well 
known  from  optical  trapping  experiments  (Svoboda  and  Block, 
1994).  Fortunately,  a  large  selection  of  microscope  lenses  has 
become  available  with  excellent  transmission  and  optical  correc¬ 
tion  in  the  near  IR  (Chapter  7,  this  volume).  The  use  of  older  lenses 
that  were  not  designed  for  the  infrared  can  be  problematic,  partic¬ 
ularly  in  the  case  of  highly  corrected  lenses  (Neuman  et  al.,  1999), 
where  non-optimal  performance  of  the  antireflective  multi-layer 
coatings  on  each  of  the  numerous  internal  surfaces  can  reduce 
overall  transmission  catastrophically. 

Beam  Delivery  and  Power  Requirements 

In  general,  the  laser  is  mounted  on  the  same  vibration  isolation 
platform  as  the  microscope  because  delivery  of  ultrashort  pulses 
through  standard,  single-mode  fibers,  which  is  possible  in  princi¬ 
ple  (Wolleschensky  et  al. ,  1998)  (see  also  Chapter  26,  this  volume), 
requires  substantial  technical  efforts  to  prevent  unacceptable 
pulse  broadening  at  the  laser  powers  routinely  required.  Develop¬ 
ment  of  special  optical  fibers,  such  as  photonic  band  gap  fibers  or 
large  cross-section  single-mode  fibers  (Helmchen  et  al.,  2002; 
Ouzounov  et  al.,  2002),  may  facilitate  MPM  applications  where 
fiber  delivery  is  essential  (Helmchen  et  al.,  2001).  In  non-absorb¬ 
ing,  non-scattering  samples  saturating  pulse  energies  (correspond¬ 
ing  to  several  tens  of  milliwatts  of  average  power)  can  easily  be 
reached  over  most  of  the  Ti :  Sa  tuning  range  even  with  5  W  of 
green  pump  power.  However,  one  rarely  has  more  than  the  desir¬ 
able  power  in  scattering  samples  such  as  in  brain  slices  or  the  intact 
brain.  Power  availability  may  also  be  limiting  when  attempting  to 
optimize  resolution  by  overfilling  the  objective  back  aperture. 

Detection 

As  discussed  above,  excellent  3D  localization  is  accomplished 
in  MPM  by  excitation  alone.  This  allows  more  flexibility  in  the 
optical  design  and,  as  a  consequence,  considerable  improvements 
of  fluorescence  collection  efficiency  are  possible  compared  to  the 
1PCM.  Figure  28.3  depicts  the  various  options.  The  positions  of 
non-imaging  detectors  are  designated  PMT  because  photomulti¬ 
plier  tubes  are  usually  the  detectors  of  choice  for  MPM.  In  general, 
considerations  as  to  which  detector  type  to  use  in  MPM  are  quite 
similar  to  those  for  1PCM,  and  the  reader  is  referred  to  Chapter 
12  of  this  book.  Among  non-imaging  schemes  (one  or  a  few  detec¬ 
tor  elements),  the  main  distinction  is  whether  the  emitted  light 
passes  back  through  the  scanning  mirrors  (, descanned  detection)  or 
whether  the  detector  is  sensitive  to  emitted  light  from  the  whole 
image  area  at  all  times  ( whole-area  detection).  A  variant  of  the 
latter  is  external  detection,  where  detected  light  does  not  pass 
through  the  objective  lens. 

Whole-Area  and  External  Detection 

Whole-area  detection  (WAD)  (Piston  et  al.,  1992,  1994)  is  now  the 
detection  mode  of  choice  in  the  majority  of  MPM  applications. 
The  WAD  pathway  uses  a  dichroic  mirror  somewhere  between  the 
scanner  and  the  objective  to  separate  excitation  and  fluorescence 
(alternatively  the  excitation  light  can  be  coupled  in  by  reflection 
from  a  dichroic),  preferably  after  a  minimum  number  of  optical 
surfaces  to  maximize  detection  efficiency.  The  signal  is  then 
passed  through  the  collection  optics,  which  needs  to  avoid 
vignetting.  If  the  back  aperture  of  the  objective  is  conjugate  to  the 
photocathode  of  the  PMT  the  effect  of  spatial  heterogeneities  in 
the  photocathode  sensitivity  is  reduced.  One  of  the  main  advan¬ 
tages  of  WAD  is  the  ability  to  efficiently  collect  fluorescence  from 
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specimens  that  scatter  light  at  X,,„,  so  strongly  that  only  a  very  small 
fraction  can  be  refocused  for  confocal  detection  (Denk  et  al.,  1994; 
Denk  and  Svoboda,  1997;  Beaurepaire  and  Mertz,  2002). 

WAD  is  as  vulnerable  to  contamination  from  ambient  room 
light  as  is  widebeld  imaging  with  highly  sensitive  cameras.  One 
thus  loses  a  convenient  but  rarely  essential  advantage  of  confocal 
imaging.  While  WAD  through  the  excitation  lens  is  usually  the 
most  convenient  and  efficient  mode,  external  detection,  where  the 
detected  light  bypasses  the  objective  lens,  can  be  necessary  when 
light  needs  to  be  detected  that  cannot  (e.g.,  because  it  is  of  too 
short  a  wavelength)  or  did  not  (e.g.,  because  it  went  off  in  the 
wrong  direction)  pass  the  objective.  Combining  through-the-lens 
collecting  with  collecting  the  light  passing  through  the  condenser 
has  been  used  successfully  to  increase  the  signal-to-noise  ratio  in 
embryo  (Denk  et  al.,  1997)  and  in  brain  slice  imaging  (Koester 
and  Sakmann,  1998;  Mainen  et  al.,  1999a,  1999b). 

Another  disadvantage  of  WAD  is  that  detectors  with  a  large 
“phase-space  volume’’  (given  by  the  product  of  detector  area  and 
acceptance  angle)  are  needed  thus  ruling  out  the  use  of  small-area 
photon-counting  avalanche  photodiodes  (Tan  et  al.,  1999)  or  of 
spectrometers  (Lansford  et  al.,  2001). 

Descanned  Detection 

When  converting  a  confocal  microscope  to  multi-photon  operation 
(Denk  et  al.,  1990)  descanned  detection  naturally  results.  While 
this  mode  is  less  efficient  than  WAD,  even  for  clear  specimens, 
descanned  detection  does  allow  the  use  of  detectors  with  small 
apertures  such  as  avalanche  photodiodes  or  spectrometers.  A 
pinhole  that  is  several  times  larger  than  the  optimal  confocal  size 
can  be  useful  for  excluding  room  light  contamination  from  the 
detected  signal  while  still  being  near  optimal  for  signal  collection. 

The  use  of  a  confocal  pinhole  as  a  tight  spatial  filter  in  addi¬ 
tion  to  multi-photon  excitation  (Stelzer  et  al.,  1994)  is  rarely  used 
because  it  is  fraught  with  several  drawbacks:  (1)  A  pinhole  small 
enough  to  produce  any  substantial  increase  in  resolution  causes 
a  large  drop  in  detection  efficiency  due  to  the  fact  that  the 
diffraction-limited  volume  at  the  Xem  is  smaller  than  the  excitation 
volume  determined  by  the  X2ex  because  A,em  <  A,2ex.  Such  a  loss  of 
detection  efficiency  is  particularly  serious  because  fluorescence 
imaging  of  living  specimens  is  often  limited  by  photobleaching 
and  photodamage.  A  technically  complex  yet  feasible  solution  to 
this  problem  might  be  to  use  a  small  array  of  detectors  together 
with  the  appropriate  deconvolution  algorithms  (Sheppard  and 
Cogswell,  1990). 

Chromatic  aberration,  already  a  problem  in  1PCM,  is  exacer¬ 
bated  in  the  confocal  operation  of  MPM  because  the  typical  shift 
between  X^,,  and  X„,  is  much  larger  (50  nm  <  X,,„,  -  XUx  <  200  for 
1PE,  200  nm  <  Xlex  -  X„  <  500  in  2PE,  and  further  increasing 
with  >2  PE). 

Non-Optical  Detection 

A  number  of  non-optical  detection  schemes  have  become  very 
promising  owing  to  the  high  degree  of  spatial  localization  achieved 
during  excitation  alone.  Two-photon  scanning  photochemical 
microscopy  (Denk,  1994;  Furuta  et  al.,  1999;  Matsuzaki  et  al., 
2001)  generates  images  of  receptor  distributions  by  locally  releas¬ 
ing  agonists  such  as  neurotransmitters  from  “caged”  precursors 
and  detecting  the  agonist-induced  ionic  current  in  voltage-clamped 
cells.  In  fact  this  concept  was  one  of  the  motivating  factors  for  the 
initial  development  of  MPM.  Opto-acoustic  detection,  which  has 
been  used  to  measure  two-photon  absorption  coefficients  (Patel 
and  Tam,  1981;  Bindhu  et  al,  1998)  could  be  used  to  measure 


spatially  resolved  absorption  that  is  not  accompanied  by  fluores¬ 
cence  or  induced  current,  but  has  to  date  not  been  tried  as  a  con¬ 
trast  mechanism  in  MPM. 

Focal-Plane  Array  Detection 

A  rather  different  strategy,  which  does  not  rely  on  scan- 
synchronized  detection  to  build  up  the  image,  is  the  use  of  an 
imaging  detector.  As  in  conventional  fluorescence  microscopy,  the 
fluorescence  is  refocused  to  an  image  plane,  and  the  image  is 
generated  by  spatially  sorting  the  fluorescence  photons  into  the 
pixels  of  an  array  detector  such  as  a  charge-coupled  device  (CCD). 
The  lateral  resolution  is  then  determined  solely  by  Xem,  which  is 
considerably  shorter  than  X^.  The  optical  sectioning  effect  due  to 
two-photon  excitation  is,  however,  retained  and  provides  discrim¬ 
ination  and  resolution  in  z-direction.  This  method  is  the  equivalent 
of  widefield  fluorescence  microscopy  with  only  a  thin  focal  slice 
rather  than  the  whole  thickness  of  the  sample  excited.  Focal  array 
detection  is  particularly  useful  in  connection  with  multi-point  illu¬ 
mination,  where  it  allows  the  acceleration  of  image  acquisition 
(Straub  and  Hell,  1998;  Egner  and  Hell.  2000;  Andresen  et  al., 
2001;  Fittinghoff  et  al,,  2001;  Hell  and  Andresen,  2001;  Nielsen 
et  al,,  2001;  Egner  et  al,  2002).  The  main  disadvantage  of  focal 
array  detection  is  that,  different  from  the  case  of  single-point 
scanning,  with  whole-area  detection,  scattering  of  fluorescence 
light  leads  to  an  immediate  degradation  of  image  contrast  and 
resolution. 

Optical  Aberrations 

Aberrations  inherent  in  the  microscope  and  spherical  aberration 
introduced  by  focusing  through  refractile  layers  such  as  the  cov- 
erslip,  immersion  oil,  and  sample  (Sheppard  and  Cogswell.  1991; 
Hell  et  al,  1993)  broaden  the  focus,  shift  the  apparent  focal  point 
(Visser  et  al,  1991),  and  reduce  the  peak  excitation  intensity.  Due 
to  the  mathematical  equivalence  between  the  optical  transfer  func¬ 
tion  of  the  non-confocal  2PM  and  that  of  the  confocal  1PM 
(Sheppard  and  Gu,  1990;  for  a  minor  modification,  see  Visser  et 
al,  1991),  the  effects  of  monochromatic  aberrations,  such  as 
spherical  aberration  and  astigmatism,  on  the  amplitude  and  reso¬ 
lution  of  the  detected  signal  are  the  same  in  both  cases.  In  the 
two-photon  case,  however,  the  number  of  molecular  excitations  is 
actually  reduced  due  to  the  smeared-out  focus  spot  and  the  inten¬ 
sity-squared  dependence  of  the  excitation  probability  (see  above). 
When  photobleaching  or  photodamage  are  the  limiting  factors,  this 
can  provide  a  significant  advantage  of  MPM  over  the  1PCM  case, 
where  the  same  number  of  excitations  occur,  but  the  fraction  of 
the  emitted  light  that  reaches  the  detector  is  reduced.  Nonetheless, 
one  must  take  the  same  precautions  with  MPM  as  with  1PCM 
when  interpreting  absolute  light  levels  as  a  function  of  focusing 
depth.  Note  also,  that  most  aberrations  become  rapidly  less  severe 
as  NA  is  reduced.  The  best  way  to  avoid  spherical  aberration  in 
aqueous  specimens,  even  at  high-NA,  is  the  use  of  water-immer¬ 
sion  objective  lenses,  which  are  now  widely  available  corrected 
even  for  the  IR  range  (Chapter  7,  this  volume). 

A  significant  motivation  for  the  development  of  2PM  was  the 
circumvention  of  the  poor  chromatic  correction  then  found  for 
most  microscope  lenses  in  the  UV.  Chromatic  aberration  problems 
play  a  role  in  (non-confocal)  2PM  only  in  connection  with  the 
broad  X  spectrum  of  ultrashort  pulses  (see  above).  However,  this 
spread  is  generally  smaller  than  a  typical  Stokes  shift  and  chro¬ 
matic  correction  is  easier  in  the  IR  where  glass  dispersion  flattens 
out. 
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Pulse  Spreading  Due  to  Group  Delay  Dispersion 

As  discussed  above,  the  optical  materials  comprising  the  micro¬ 
scope  optics  cause  the  excitation  pulses  to  spread  in  time  and  thus 
become  less  efficient  in  exciting  multi-photon  transitions.  The 
group  delay  dispersion  (GDD)  has  been  measured  for  some 
objectives  (Guild  et  al.,  1997;  Squier  et  al,  1998).  As  mentioned 
above,  the  optical  effort  needed  to  generate  the  prechirping  nec¬ 
essary  to  compensate  for  the  GDD  has  to  be  weighed  against  the 
improvements  expected.  As  a  general  rule,  GDD  compensation 
will  be  helpful  or  even  essential  when  laser  power  is  limiting,  such 
as  for  deep  tissue  imaging,  or  when  single-photon  absorption  con¬ 
tributes  to  damage.  If  coherent  control  techniques  are  used,  com¬ 
plete  dispersion  control  is,  of  course,  essential  but  then  the  optics 
used  to  tailor  phases  can  be  employed  for  dispersion  control  as 
well. 

Control  of  Laser  Power 

For  slow  control  of  the  laser  power,  mechanically  actuated  devices 
such  as  filter  wheels,  graded  neutral  density  filters,  or  rotating  half- 
wave-plate/polarizer  combinations  (Denk,  2001)  can  be  used. 
Faster  shuttering  (e.g.,  in  order  to  blank  the  beam  during  retrace) 
or  modulation  requires  non-mechanical  devices  such  as  acousto- 
optical  (AO)  or  electro-optical  (EO)  modulators  (i.e.,  Pockels 
cells),  which  can  respond  on  the  microsecond  and  even  nanosec¬ 
ond  timescale  (Chapter  3,  this  volume).  EOMs  achieve  high 
throughput  but  often  incomplete  extinction,  while  AOMs  are 
lossy,  and  due  to  limited  diffraction  efficiency,  their  extinction  is 
excellent. 

A  few  problems  arise  specifically  when  ultrashort  pulses  are 
used  together  with  such  devices:  (1)  In  AOM,  AOD,  or  acousto¬ 
optic  tuning  filter  (AOTF)  devices,  an  acoustic  wave  diffracts  the 
incoming  beam  by  an  angle  that  is  dependent  on  A,„.  For  ultrashort 
pulses,  which  are  spectrally  broad,  the  focus,  therefore,  becomes 
distorted  and  diffraction  efficiency  is  reduced.  (2)  Both  EOMs  and 
AOMs  use  dispersive  materials,  which  spread  the  laser  pulse 
temporally  (see  above). 

While  the  temporal  spread  can  be  easily  compensated  (in  a  few 
cases  multi-photon  microscopy  setups  already  contain  GDD  com¬ 
pensators),  it  is  much  more  difficult  but  not  impossible  to  com¬ 
pensate  for  the  angular  spread  in  an  AOM  (Lechleiter  et  al,  2002). 
Limited  extinction  from  the  EOMs  is  often  not  a  serious  problem 
because  the  quadratic  intensity  dependence  of  two-photon  excita¬ 
tion  allows  even  a  moderate  power  reduction  ratio  to  translate  into 
almost  complete  elimination  of  unwanted  excitation. 

Resonance  and  Non-Mechanical  Scanning 

The  time  resolution  of  closed  loop  galvanometer  scanners  is  suffi¬ 
cient  for  most  applications,  in  particular  if  a  limited  number  of  mea¬ 
surement  points  can  be  selected.  However,  because  the  time  per 
line  cannot  be  reduced  significantly  below  the  about  1  ms  with 
closed-loop  scanners,  scan  times  for  large  areas  can  become  too 
long  for  the  time  resolution  desired.  One  solution  to  this  problem 
is  the  use  of  resonant  galvanometer  scanner  (Fan  et  al.,  1999;  see 
also  Chapters  3  and  29,  this  volume)  which  provide  a  fixed  line  rate 
about  10  times  faster,  albeit  at  some  loss  of  flexibility.  Acousto- 
optical  scanning  (Art  and  Goodman,  1993)  requires  correction  of 
the  diffractive  spread  of  the  wavelengths  comprising  short-pulse 
light  (Lechleiter  et  al,  2002),  but  has  the  advantage  of  more  rapid 
access  (still  limited  by  the  acoustic  transit  time  across  the  diffrac¬ 
tion  medium)  and  allows  both  scanning  and  intensity  control. 


CHROMOPHORES  (FLUOROPHORES  AND 
CAGED  COMPOUNDS) 

The  criteria  for  choosing,  or  designing,  fluorophores  for  MPM  are 
essentially  the  same  as  for  any  other  fluorescence  microscopy  tech¬ 
nique:  large  absorption  cross-section  at  convenient  A,«s,  high 
quantum  yield,  low  rate  of  photobleaching,  and  minimal  chemical 
or  photochemical  toxicity  to  living  cells.  In  the  early  days  of  MPM, 
a  heuristic  approach  prevailed  and  fluorophores  were  selected  that 
had  proven  useful  in  widefield  fluorescence  microscopy  or  1PCM. 
In  most  cases,  two-photon  excitation  was  found  whenever  there  is 
single-photon  absorption  at  a  A,  corresponding  to  twice  the  energy 
of  the  excitation  photons.  Most  MPM  imaging  still  uses  conven¬ 
tional  fluorophores,  and  we  now  have  two-photon  spectra  for  many 
of  these  (Xu  and  Webb,  1996;  Xu  et  al.,  1996;  Zipfel  et  al.,  2003). 
On  the  other  hand,  there  is  a  considerable  effort  to  generate  chro- 
mophores  tailored  to  MP  excitation  using  a  donor-acceptor-donor 
or  acceptor-donor-acceptor  strategy.  These  molecules  maximize 
the  electrical  dipole  transition  by  electron  transfer  over  relatively 
long  distances  from  donor  to  acceptor.  By  this  approach,  molecules 
can  be  created  with  two-photon  excitation  cross-sections  about  10- 
fold  greater  than  conventional  fluorophores  (Albota  et  al.,  1998b; 
Ventelon  et  al.,  1999).  Nanoparticles,  also  called  quantum  dots 
(Bruchez  et  al.,  1998;  Han  et  al.,  2001),  which  offer  broad  excita¬ 
tion  spectra,  but  very  narrow  emission  spectra,  have  the  largest 
measured  two-photon  cross-sections  seen  to  date.  This  allows  their 
detection  at  very  low  concentrations,  even  in  vivo  (Larson  et  al. , 
2003). 

Another  notable  development  is  the  movement  to  longer  wave¬ 
lengths.  While  in  the  early  days  of  MPM  the  emphasis  was  on  UV- 
excited  dyes  that  were  2P-excited  by  red  lasers,  the  emphasis  now 
is  on  fluorophores  normally  excited  by  visible  light  and  2P-excited 
by  IR  light.  This  trend  is  mainly  driven  by  the  desire  for  lower 
background  fluorescence  and  deeper  penetration  into  scattering 
tissue. 


Two-Photon  Absorption  Cross-Sections 

Differences  between  one-  and  the  two-photon  excitation  spectra 
have  been  exploited  in  molecular  spectroscopy  because  they 
provide  additional  information  about  the  structure  of  excited  states. 
These  differences  can  be  quite  significant,  see,  for  example,  the 
case  of  Bis-MSB  (Kennedy  and  Lytle,  1986)  or  the  aromatic  amino 
acids  tyrosine  and  phenylalanine  (Rehms  and  Callis,  1993),  but 
note  the  spectral  similarities  for  tryptophan.  As  a  rule  of  thumb,  in 
symmetrical  molecules  one  expects  A,2„  <  2A,,. 

Calculations  of  two-photon  cross-sections  are  difficult  to 
perform  for  complex  molecules.  Direct  experimental  measure¬ 
ments  of  multi-photon  absorption  are  equally  difficult  because 
even  under  optimal  conditions,  the  fraction  of  the  incident  power 
that  is  absorbed  is  rather  small  (using  Eq.  4  we  find,  e.g.,  p,Jp  = 
3  X  10  5  for  a  chromophore  with  a  cross-section  of  10-50m4s  pho¬ 
ton-1,  at  a  concentration  of  10  m M  and  a  laser  power  of  lOOmW 
with  a  two-photon  advantage  of  105).  While  thermal  lensing  or 
acousto-optical  techniques  have  been  used  to  measure  two-photon 
absorption  (Kliger,  1983),  these  techniques  are  much  more  com¬ 
plicated  than  single-photon  spectrophotometry. 

For  fluorescent  molecules,  the  shape  of  the  two-photon  exci¬ 
tation  spectrum  can  be  determined  by  detecting  the  intensity  of  flu¬ 
orescence  emission  as  a  function  of  excitation  wavelength.  In  order 
to  determine  the  action  spectrum,  the  incident  average  laser  power 
(P,),  the  probability  of  detecting  fluorescence  photons,  and  the  two- 
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photon  advantage  t,  (Eq.  2)  need  to  be  known  (Xu  et  al.,  1995). 
The  absolute  value  of  the  two-photon  absorption  cross-section  can 
then  be  calculated  using  the  fluorescence  quantum  yield.  Quite  a 
number  of  measured  spectra  are  now  available  in  the  literature 
(Xu  et  al.,  1996;  Albota  et  al.,  1998b  also  includes  URL.) 

While  precise  calculations  of  two-photon  absorption  cross- 
sections  are  difficult,  several  new  fluorophores  with  particularly 
large  two-photon  absorption  cross-sections  have  been  designed 
using  theoretical  considerations  (He  et  al,  1995,  1997;  Marder  et 
al,  1997;  Albota  et  al.,  1998a;  Ventelon  et  al.,  1999,  2002; 
Adronov  et  al.,  2000;  Kim  et  al.,  2000;  Zojer  et  al.,  2002).  Before 
such  fluorophores  can  come  into  common  use,  however,  problems 
with  water  solubility,  derivatization,  etc.,  will  have  to  be  solved. 

For  the  fluorophores  studied  so  far,  the  spectra  of  the  emitted 
fluorescence  were  found  to  be  essentially  independent  of  whether 
excitation  occurs  via  single-  or  two-photon  excitation  (Curley 
et  al.,  1992).  This  is  not  surprising  because  the  molecular  relax¬ 
ation  process  (on  the  picosecond  scale)  almost  always  occurs  to 
the  same  state  (the  lowest  excited  singlet  state)  prior  to  the  emis¬ 
sion  (on  the  nanosecond  scale)  and  therefore  erases  the  memory 
of  the  excitation  pathway  and  energy. 

Caged  Compounds 

Two-photon  absorption  spectra  for  caged  compounds  are  more  dif¬ 
ficult  to  measure  than  those  for  fluorophores  because  the  amount  of 
uncaged  material  generated  is  too  small  to  be  easily  measured  with 
most  analytical  techniques.  In  some  cases,  uncaging  can  be  detected 
when  fluorescence  assays  for  the  released  agonist  exist,  such  as  for 
caged  ATP  (Denk  et  al.,  1990),  when  the  product  itself  is  fluores¬ 
cent,  as  it  is  with  caged  fluorescein  (Svoboda  et  al. ,  1996b),  or  when 
biological  effects  can  be  detected,  such  as  the  opening  of  ion  chan¬ 
nels  by  the  two-photon-induced  release  of  caged  neurotransmitters 
(Denk,  1994;  Matsuzaki  et  al.,  2001;  Kasai  et  al.,  2002).  Photo¬ 
chemical  reactions  are  often  much  slower  than  fluorescence 
emission  and  their  speed  can  strongly  depend  on  the  chemical  envi¬ 
ronment  such  as  pH  and  ionic  strength  (Milburn  et  al.,  1989;  Corrie 
and  Trentham,  1993;  Kao  and  Adams,  1993).  The  speed  of  release 
is  important  for  at  least  two  reasons:  (1 )  The  pixel  dwell-time  must 
be  at  least  as  long  as  the  duration  of  the  signal  used  to  generate  image 
contrast,  which  at  best  is  as  fast  as  the  photochemical  reaction  rate; 
(2)  diffusion  of  the  released  agonist  tends  to  blur  the  image  and  thus 
prevents  high-resolution  mapping.  A  delay  of  10  ms,  for  example, 
allows  the  released  agonist,  typically  a  small  organic  molecule  with 
a  diffusion  constant  of  5  x  10~9m2s_1,  to  diffuse  a  distance  of  about 
3 pm  (Kiskin  et  al.,  2002). 

CELL  VIABILITY  DURING  IMAGING 

The  survival  of  the  biological  sample  while  it  is  being  imaged  is 
one  of  the  most  important  constraints  on  the  usefulness  of  any  vital 
microscopy  technique.  While  one  of  the  reasons  for  pursuing  MPM 
as  a  new  technique  was  the  expectation  of  greatly  reduced  photo¬ 
damage  (Denk  et  al,  1990),  it  has  to  be  kept  in  mind  that  in  the 
focal  plane,  for  a  given  excitation  rate  the  damage  is  expected  to 
be  at  least  as  large  for  2P  as  it  is  for  IP  excitation.  This  is  because 
any  effect  due  to  reactions  initiated  from  the  excited  state  of  the 
chromophore  are  independent  of  the  mode  of  excitation.  Further¬ 
more,  it  cannot  be  ruled  out  that  some  endogenous  biological  mol¬ 
ecules  have  unusually  large  two-photon  cross-sections  (such  as 
bacteriorhodopsin;  Birge  and  Zhang,  1990)  and  are,  therefore,  par¬ 
ticularly  susceptible  to  damage.  Another  concern  is  the  possibility 


of  excited  state  absorption,  particularly  at  excitation  rates  near 
saturation. 

Considerable  work  has  been  performed  in  this  area  since  the 
first  edition  of  this  book.  Two-photon  excitation,  particularly  when 
using  wavelengths  below  800 nm  (Konig  et  al.,  1996;  Oehring 
et  al.,  2000)  (see  Chapter  38,  this  volume )  can,  not  surprisingly, 
generate  reactive  oxygen  species,  which  are  implicated  frequently 
in  photodamage  (Tirlapur  et  al.,  2001).  On  the  other  hand,  when 
using  longer  wavelengths  (1064nm),  generation  of  reactive  oxygen 
species  by  flavin-containing  proteins  seems  to  be  greatly  reduced 
compared  to  single-photon  excitation  (Hockberger  et  al.,  1999). 

At  higher  excitation  levels,  a  steeper  than  quadratic  power 
dependence  is  often  found  both  for  cellular  photodamage  (Koester 
et  al.,  1999;  Oehring  et  al.,  2000;  Hopt  and  Neher,  2001)  and  for 
photobleaching  (Eggeling  et  al.,  1998;  Patterson  and  Piston,  2000). 
It  appears,  however,  that  the  damage  nonlinearity  is  not  instanta¬ 
neous  (i.e.,  three-  or  four-photon  excitation)  because  for  the  same 
mean  two-photon  excitation  rate  no  change  in  the  damage  is  seen 
with  pulse  width  (Koester  et  al.,  1999;  Konig  et  al.,  1999). 

There  is  virtually  no  experimental  indication  that  heating  by 
water  absorption  (discussed  in  Physical  Principles)  is  a  limiting 
factor  in  multi-photon  microscopy.  Heating  may  yet  become  an 
issue  as  substantially  longer  wavelengths  are  beginning  to  be  used 
for  the  excitation  of  long  wavelength  fluorophores.  A  number  of 
explicit  examples  show  an  actual  and  significant  reduction  of  pho¬ 
todamage  when  using  two-photon  rather  than  single-photon 
imaging  in  biological  specimens  such  as  cultured  cells 
(Hockberger  et  al.,  1999),  cardiac  myocytes  (Niggli  et  al,  1994b; 
Piston  et  al.,  1994),  and  mammalian  (Squirrell  et  al.,  1999)  and 
invertebrate  embryos  (Summers  et  al,  1993). 

The  experience  of  many  a  microscopist  is  that  live-cell 
imaging  can  often  be  performed  by  reducing  the  excitation  light 
intensity  to  the  lowest  possible  level,  using  efficient  optics  and  sen¬ 
sitive  detectors  (Chapters  17,  19,  and  29,  this  volume).  The  expe¬ 
rience  in  2PM  is  similar,  but  the  range  of  imageable  specimens  is 
larger.  For  example,  in  both  the  sea  urchin  (Piston  et  al.,  1993)  and 
hamster  embryos  (Squirrell  et  al,  1999),  two-photon  excitation 
allows  extended  observation  of  embryonic  development,  under 
conditions  where  single-photon  excitation  is  unsuccessful.  In 
another  case,  as  part  of  a  direct  comparison  of  scanned  laser  UV 
and  two-photon  excitation  (Niggli  et  al,  1994a;  Piston  et  al., 
1994),  it  was  found  that  two-photon  excitation  allowed  imaging  of 
the  calcium  indicator  dye  Indo-1  continuously  for  5  min  without 
compromising  cell  viability.  Equivalent  single-photon  scanning 
with  UV  light  resulted  in  considerable  photobleaching,  and  over 
80%  cell  death  (Piston  et  al.,  1994). 

Those  studies  indicate  that,  even  though  damage  is  less  than 
with  conventional  UV  illumination,  cultured-animal-cell  viability 
can  be  compromised  by  two-photon  excitation.  Particularly  wor¬ 
rying,  and  as  yet  unresolved,  is  the  observation  that  at  high  illu¬ 
mination  levels  the  two-photon  photobleaching  rate  can  increase 
much  faster  than  the  excitation  rate  (Patterson  and  Piston,  2000), 
even  though  it  is  not  known  whether  there  is  a  corresponding 
increase  in  phototoxicity  and  whether  these  highly  nonlinear 
bleaching  phenomena  are  limited  to  certain  narrow  classes  of  dyes, 
such  as  the  xanthene  dyes. 

A  question  that  often  arises  is  how  to  determine  the  mechanism 
of  damage.  Important  information  is  provided  by  its  power  depen¬ 
dence  (Neuman  et  al.,  1999;  Hopt  and  Neher,  2001).  For  example, 
two-photon  photochemical  damage  should  be  proportional  to  the 
square  of  the  incident  power.  While  a  linear  power  dependence  all 
but  rules  out  two-photon  effects,  a  superlinear  dependence  on  the 
average  excitation  power  could  result  from  single-photon  absorp- 
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tion  coupled  with  a  nonlinear  mediator  for  damage.  Thermally 
induced  damage  can  have  a  rather  sharp  temperature  threshold  due 
to  cooperative  phenomena  such  as  protein  denaturation.  A  defini¬ 
tive  distinction  between  single-  and  multi-photon  absorption  is  their 
dependence  on  pulse  length;  if  the  pulse  length  is  varied  by  intro¬ 
ducing  a  variable  degree  of  GDD  (see  above),  the  spectrum,  and 
hence  the  amount  of  linear  (single-photon)  absorption,  remains 
completely  unchanged  while  2PA  drops. 

Knowing  the  mechanism  of  damage  is,  of  course,  crucial  for 
choosing  the  optimal  excitation  strategy.  For  example,  to  reduce 
single-photon,  dose-rate-independent  damage,  a  reduction  of  Fp 
might  seem  appropriate  in  order  to  increase  the  two-photon  advan¬ 
tage  but  the  peak  temperature  during  each  pulse  increases  as  F~', 
and  can  become  larger  than  the  thermal  time  constant.  Unpleasant 
surprises  could  also  arise  from  additional  absorption  by  molecules 
already  in  the  excited  state  (something  that  is  more  likely  to  occur 
when  operating  closer  to  saturation)  or  from  proximity  effects 
mediated  by  free  radicals  (Konig  et  al. ,  1996;  Hockberger  et  al., 
1999;  Koester  etal.,  1999;  Konig  etal.,  1999;Oehring  etal.,  2000; 
Hopt  and  Neher,  2001;  Tirlapur  et  al.,  2001). 

APPLICATIONS 

MPM  has  been  used  to  address  questions  in  quite  a  few  areas  of 
biology.  Particularly  the  imaging  of  intact  tissue  has  benefited  from 
the  properties  of  the  multi-photon  (predominantly  two-photon) 
microscope. 

Calcium  Imaging 

Intracellular  messenger  dynamics,  such  as  calcium  ion  concentra¬ 
tion  has  been  measured  in  single  cells  (Piston  et  al.,  1994),  but  the 
particular  advantages  of  MPM  over  single-photon  techniques 
come  to  bear  most  in  scattering  tissue  such  as  brain  slices  (Denk 
et  al,  1995,  1996;  Yuste  and  Denk,  1995;  Mainen  et  al.,  1999b; 
Sabatini  and  Svoboda,  2000;  Wang  et  al.,  2000;  Oertner  et  al., 
2002),  the  stomatogastric  ganglion  (Kloppenburg  et  al.,  2000),  and 
in  vivo  (Svoboda  et  al.,  1997,  1999;  Debarbieux  et  al.,  2003).  In 
isolated  retina  2PM  allowed  the  recording  of  dendritic  calcium 
signals  during  visual  stimulation  (Denk  and  Detwiler,  1999;  Euler 
et  al.,  2002). 

Uncaging  and  Photobleaching 

Multi-photon  photochemistry  has  been  used  to  map  receptor 
sensitivities  in  single  cells  (Denk,  1994)  and  inside  neural  tissue 
(Matsuzaki  et  al.,  2001;  Kasai  et  al.,  2002). 

Autofluorescence 

Because  MPM  easily  reaches  into  UV  transition  energies,  it  has 
increasingly  been  used  to  study  biological  autofluorescence  such 
as  from  NADH  (Piston  et  al.,  1995;  Piston  and  Knobel,  1999), 
serotonin  in  living  cells  (Maiti  et  al.,  1997),  skin  (Masters  et  al., 
1997),  muscle  cells  (Schilders  and  Gu,  1999),  glutathione  in  ara- 
bidopsis  (Meyer  and  Fricker,  2000),  mast  cell  secretion  using  3P 
excitation  of  serotonin  (Williams  et  al,  1999),  arctic  fungus 
(Arcangeli  et  al,  2000),  collagen  (Agarwal  et  al.,  2001),  biofilm 
(Neu  et  al.,  2002),  tryptophan  in  proteins  (Lippitz  et  al.,  2002), 
and  flavoproteins  (Huang  et  al,  2002).  Recently,  the  sources  of 
autofluorescence  from  living  tissue  have  been  analyzed  in  more 
detail  (Zipfel  et  al.,  2003)  (see  also  Chapter  21,  this  volume). 


Developmental  Biology 

Because  of  the  superior  depth  penetration  and  the  localized  exci¬ 
tation  associated  with  MPM,  this  approach  has  proven  useful  in 
many  developmental  biological  applications.  Lineage  tracing  has 
been  performed  using  two-photon  photorelease  of  caged  fluo- 
rophores  in  sea  urchin  embryos  (Summers  et  al.,  1996;  Piston 
et  al,  1998).  Cellular  and  subcellular  dynamics  have  been  imaged 
and  measured  using  MPM  during  development  of  sea  urchin 
embryos  (Summers  et  al.,  1993,  1996),  cell  fusion  in  C.  elegans 
(Mohler  et  al,  1998;  Periasamy  et  al.,  1999),  mammalian  embryos 
(Squirrell  et  al.,  1999),  zebrafish  (Huang  et  al,  2001),  and  birds 
(Dickinson  et  al.,  2002). 

In  Vivo  (Intact  Animal)  Imaging 

In  intact  animals,  the  need  for  tissue  penetration  is  maximal.  High 
resolution  optical  imaging  inside  living  whole  animals  has  there¬ 
fore  become  the  almost  exclusive  domain  of  two-photon 
microscopy  not  only  for  functional  calcium  imaging  (Svoboda  et 
al,  1997,  1999),  but  also  to  image  blood  flow  in  the  fine  capillar¬ 
ies  (Kleinfeld  et  al.,  1998;  Chaigneau  et  al.,  2003),  gene  expres¬ 
sion  and  angiogenesis  (Brown  et  al.,  2001),  and  even  the  dynamics 
of  Alzheimer’s  disease  pathologies  (Christie  et  al.,  1998,  1999, 
2001;  Backskai  et  al,  2001)  and,  having  previously  been  applied 
to  observe  changes  in  dendrite  structure  in  brain  slices  (Engert 
and  Bonhoeffer,  1999;  Maletic-Savatic  et  al.,  1999),  two-photon 
microscopy  has  most  recently  been  used  to  study  the  long-term 
dynamics  of  neuronal  fine  structure  (Grutzendler  et  al.,  2002; 
Trachtenberg  et  al,  2002)  in  living  animals. 

OUTLOOK 

Multi-photon  excitation  microscopy  has  extended  the  range  of 
laser  scanning  fluorescence  microscopy  especially  where  dynamic 
imaging  in  living  specimens  is  needed.  Much  progress  has  been 
made  in  solving  many  of  the  technical  impediments  that  existed  in 
the  early  days  of  MPM.  Still,  only  a  few  of  the  many  potential  con¬ 
trast  mechanisms  established  for  nonlinear  optical  spectroscopy 
have  been  used  for  imaging  purposes.  This  is  mainly  due  to  the 
fact  that  often  only  a  small  number  of  photons  can  be  collected 
from  each  volume  element  in  the  small  amount  of  time  that  the 
beam  dwells  on  each  location.  Increasing  use  is  being  made, 
however,  of  second  harmonic  generation  (Moreaux  et  al,  2001) 
and  Raman  scattering  (Zumbusch  et  al,  1999;  Potma  et  al.,  2002; 
Volkmer  et  al.,  2002)  (see  also  Chapter  33,  this  volume.) 

Phototoxicity  in  cells  is  still  not  well  understood  in  general  and 
for  ultrashort  pulse  illumination  in  particular.  But  the  main  limita¬ 
tion  to  even  more  widespread  use  of  multi-photon  excitation  is  not 
due  to  fundamental  physical,  chemical,  or  biological  problems,  but 
to  the  price  and  complexity  of  the  instrumentation. 
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INTRODUCTION 

Multi-photon  processes  relying  on  the  cooperative  action  of  two 
or  more  photons  can  broadly  be  divided  into  two  families  that  are 
distinguished  by  the  fact  that  the  photons  are  either  absorbed  or 
scattered  (Shen,  1984).  Whereas  the  scattering  events  relevant  to 
microscopy  are  second  and  third  harmonic  generation  (SHG, 
THG),  as  well  as  coherent  anti-Stokes  Raman  scattering  (CARS), 
the  useful  multi-photon  absorption  events  are  two-  and  three- 
photon  excitation  (2PE,  3PE).  The  first  multi-photon  phenomenon 
that  entered  microscopy  was  SHG  (Hellwarth  and  Christensen, 
1974;  Gannaway,  1978),  followed  by  CARS  (Duncan  et  al,  1982), 
2PE  (Denk  et  al,  1990;  Curley  et  al. ,  1992),  3PE  (Hell  et  al.,  1996; 
Maiti  et  al.,  1997),  and  THG  (Barad  et  al.,  1997;  Muller  et  al., 
1998).  Timed  with  the  advent  of  more  accessible  pulsed  lasers,  the 
seminal  work  by  Denk  and  colleagues  (1990)  on  2PE  microscopy 
opened  a  new  epoch  of  research  and  application  with  multi-photon 
processes  in  microscopy  (Guo  et  al,  1997;  Gauderon  et  al.,  1998; 
Zumbusch  et  al.,  1999;  Campagnola  et  al.,  2002;  Cheng  et  al., 
2002;  Muller  and  Schins,  2002;  Yelin  et  al.,  2002;  Theer  et  al, 
2003;  Zipfel  et  al,  2003). 

The  use  of  multi-photon  phenomena  provides  several  advan¬ 
tages  over  their  single-photon  counterparts.  The  most  prominent 
is  the  confinement  of  signal  generation  to  the  focal  region  where 
the  simultaneous  occurrence  of  multiple  photons  is  highest. 
Another  important  advantage  is  the  capability  to  penetrate  deeper 
into  strongly  scattering  specimens  (Denk  and  Svoboda,  1997;  Cen- 
tonze  and  White,  1998).  Moreover,  SHG,  THG,  and  CARS  (Zoumi 
et  al.,  2002;  Cox  et  al.,  2003)  generate  signals  that  are  not  acces¬ 
sible  through  single-photon  interactions,  thus  complementing  flu¬ 
orescence  imaging  in  a  unique  way. 

Unfortunately,  multi-photon  events  have  a  low  probability  of 
occurrence,  that  is,  they  have  a  small  cross-section.  Small  cross- 
sections  can  be  compensated  by  large  excitation  intensities.  In 
microscopy,  the  strong  focusing  provided  by  the  objective  lens 
readily  yields  large  intensities,  in  particular  in  conjunction  with 
pulsed  illumination.  The  only  issue  is  that  the  applicable  intensity 
is  limited  by  photodamage,  which  also  has  a  major  nonlinear 
component  (see  Chapter  38,  this  volume).  In  some  cases,  multi¬ 
photon  absorption  may  also  reach  (singlet-state)  saturation.  With 
the  exception  of  the  important  application  of  imaging  into  strongly 
scattering  tissue,  the  power  of  presently  available  lasers  usually 
greatly  exceeds  the  power  required  at  a  given  point.  Therefore,  the 
use  of  several  parallel  foci  may  be  regarded  as  an  obvious  solution 
to  this  problem.  In  multi-photon  microscopy,  this  solution  is  par¬ 
ticularly  attractive  because  the  optical  sectioning  is  provided  by  the 
multi-photon  interaction  alone.  No  back-imaging  onto  an  array  of 


pinholes  is  needed,  which  otherwise  would  require  delicate  align¬ 
ment  and  the  compensation  of  chromatic  aberrations.  In  this 
chapter,  we  give  an  overview  of  parallelized  multi-photon  imaging 
methods,  which  are  commonly  referred  to  as  multi-focal  multi¬ 
photon  microscopy  (MMM). 

Background 

Owing  to  their  wavelength  tunability,  short  pulse  length,  and  high 
repetition  rate,  mode-locked  titanium :  sapphire  (Ti :  Sa)  lasers  have 
become  the  light  sources  of  choice  for  multi-photon  microscopes. 
Mode-locked  Ti :  Sa  and  similar  laser  systems  typically  provide  1 
to  2W  of  average  power  at  a  repetition  rate  of  ~80MHz  at  pulse 
lengths  of  -200 fs  or  1  to  2ps.  This  is  ample  light  for  a  single  scan¬ 
ning  beam  since  nonlinear  damaging  effects  normally  limit  the 
usable  intensity  to  about  200GW/cm2  at  200  fs  and  70GW/cm2  at 
1  to  2ps  in  the  focus  (Hanninen  et  al.,  1995;  Konig  et  al,  1996, 
1999;  Hopt  and  Neher,  2001).  At  typical  repetition  rates  and  focal 
spot  sizes,  this  maximum  focal  intensity  amounts  to  3  to  10  mW 
at  200  fs  and  10  to  30  mW  in  the  picosecond  range  average  power. 
An  important  exception  is  the  imaging  of  layers  inside  strongly 
scattering  specimens,  such  as  skin  and  brain  at  >250  pm  depth, 
where  most  of  the  laser  light  is  needed  and  femtosecond  opera¬ 
tion  is  preferable.  Therefore,  in  regular,  single-spot  multi-photon 
microscopy,  more  than  90%  of  the  laser  power  is  discarded 
because  applying  more  power  would  be  detrimental.  This  holds 
both  for  the  multi-photon  absorption  and  the  multi-photon  scatter¬ 
ing  microscopy  modes.  By  splitting  up  the  beam  of  a  mode-locked 
Ti :  Sa  laser  into  several  beamlets  and  applying  multiple,  well-sep¬ 
arated  foci  simultaneously,  MMM  exploits  a  much  larger  fraction 
of  the  available  laser  power,  and  at  the  same  time  it  parallelizes 
the  imaging  process  without  significant  trade-offs  in  the  resolution 
(Bewersdorf  et  al,  1998;  Buist  et  al.,  1998). 

Determination  of  the  Optimum  Degree 
of  Parallelization 

If  photodamage,  photobleaching,  or  saturation  of  an  excited  state 
of  the  chromophore  can  be  neglected,  the  signal  S  from  a  single 
focus  n-photon  excitation  microscope,  per  time  unit,  is  propor¬ 
tional  to  CsPZvJ x"-1/"-1,  with  Pmg  being  the  average  laser  power  in 
the  focus,  x  and /being  the  pulse  length  in  the  sample  and  the  rep¬ 
etition  rate,  respectively.  osis  the  multi-photon  cross-section.  In 
MMM,  the  laser  beam  is  split  up  into  N  beamlets  with  an  average 
power  of  P„g,MMM/N  each.  The  signal  of  the  A  independent  foci  adds 
up  to  an  overall  signal  S  °=  OsP'llsMMMK'tfN)"^-  Within  the  framework 
of  sheer  signal  generation,  the  parameters  x/  and  N  are  of  equal 
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importance  and  therefore  the  change  of  one  parameter  can  be  com¬ 
pensated  by  adjusting  one  of  the  others.  This  can  be  illustrated  by 
looking  at  the  laser  pulse  train  at  a  certain  spot  in  the  sample.  The 
number  of  pulses  arriving  per  second  is  proportional  to /times  N. 
Whether  the  repetition  rate  /  is  halved  and  N  is  doubled  or  vice 
versa  is  of  no  importance.  A  doubled  pulse  length  x  can  similarly 
be  interpreted  as  two  subsequent  pulses.  While  x  and  /  are  given 
by  the  laser  system,  the  degree  of  parallelization  N  introduces  a 
new  degree  of  freedom  to  optimize  the  performance  of  a  multi¬ 
photon  microscope.  It  has  to  be  noted  though  that  N  strongly  influ¬ 
ences  the  microscope  design  and  thus  can  be  changed  only  in  a 
certain  range  without  major  technical  modifications. 

The  choice  of  the  parameters  for  MMM  depends  on  the  limit¬ 
ing  factors:  saturation,  photodamage,  and  the  available  laser 
power.  Saturation  obviously  does  not  play  a  role  for  the  scattering 
modes  because  in  this  case  no  long-lived  state  of  the  sample  is 
involved.  The  overall  damage  per  time  unit  can  be  written  as  a 

polynomial  series  D  X  X  5iPLg,MMM/ CtM'1, with  5,  expressing 

(=i 

the  relative  weight  (including  the  damaging  cross-sections)  of  the 
damaging  mechanisms  of  the  different  orders  of  non-linearity.  For 
a  certain  range  of  laser  and  imaging  parameters  Pmg,  x,  /  and  N,  D 
can  be  approximated  by  D  °c  <3DPda,gMmKsP^)d~'^  where  d  is  the 
effective  order  of  non-linearity  that  typically  is  not  an  integer,  d  is 
close  to  the  order  of  non-linearity  of  the  dominating  damaging 
mechanism  which  can  change,  for  example,  with  the  applied  laser 
power  Pm>g.  Similarly,  oD  is  the  effective  damaging  cross-section  in 
this  parameter  range. 

As  a  result,  the  performance  ratio  (3  of  the  signal  S  to  the 
damage  D  is  proportional  to  as/o„(f,„y,,MMM/x/V)”_''  =  os/oBP//4.  The 
goal  obviously  is  to  maximize  the  performance  (3.  For  this  purpose, 
one  has  to  distinguish  between  two  different  situations: 

•  n  >  d  (the  excitation  process  is  of  higher  order  of  non¬ 
linearity  than  the  dominating  damaging  process):  Maxi¬ 
mizing  the  peak  power  Ppeak  yields  the  highest  value  for  (3.  Short 
pulses  and  low  repetition  rates  are  therefore  favorable.  Paral¬ 
lelization  only  decreases  (3.  However,  an  increase  of  Ppeat  is 
only  reasonable  up  to  a  value  where  damaging  processes  of 
higher  order  become  significant. 

•  n  <  d  (the  excitation  process  is  of  lower  order  of  non¬ 
linearity  than  the  dominating  damaging  process):  Ppeak  must 
be  minimized  to  optimize  (3.  Apart  from  applying  long  pulses 
and  high  repetition  rates,  parallelization  is  the  best  alternative. 
Moreover,  by  increasing  the  overall  average  power  P„gMMM  and 
N  simultaneously,  [3  can  be  kept  constant  while  at  the  same 
time  the  recorded  signal  per  unit  time  S  increases  by  a  factor 
of  N.  This  allows  an  acceleration  of  the  imaging  speed  by 
this  factor  without  increasing  the  damage.  The  maximum  N 
is  limited  by  the  available  laser  power  only  as  long  as  no 
low  order  damaging  processes  (such  as  heating)  become 
dominant. 

In  the  case  of  n  =  d,\ 3  does  not  depend  on  the  peak  power. 
Therefore,  parallelization  or  a  change  in  x  or  /  has  no  real  influ¬ 
ence.  We  note  that  the  distance  between  the  focal  spots  and  the 
size  of  the  scanning  field  additionally  influence  the  relative  weights 
5,  of  the  damaging  processes.  Heating  may  be  a  problem  if  all  of 
the  average  power  is  concentrated  on  a  rather  small  scanning  area 
of  a  few  micrometers. 

With  regard  to  the  damage,  parallelization  is  only  reasonable 
in  the  case  of  a  higher  degree  of  non-linearity  d  of  the  dominant 
damage  process  as  compared  to  that  of  the  excitation  process 


(n  >  d).  Another  reason  for  the  parallelization  is  enhanced 
scanning  speed  where  parallelization  is  important  even  if  n  >  d. 

For  the  multi-photon  excitation  processes,  the  (rather  rare  case 
of)  saturation  is  in  the  same  way  a  highly  nonlinear  phenomenon. 
In  this  situation,  a  maximum  acceptable  saturation  level  can  be 
defined  with  a  corresponding  focal  average  power  P„,.  The  ratio 
between  the  totally  available  average  power  PmgMMM  and  P„,  gives 
the  optimum  degree  of  parallelization  Nopt. 

Investigations  of  photodamage  with  pulsed  NIR  illumination 
in  living  cells  showed  prevalent  cell-damage  mechanisms  of  the 
order  of  2  <  d  <  2.5  (Konig  et  al.,  1999;  Hopt  and  Neher,  2001). 
In  vitro  photobleaching  measurements  have  shown  a  power- 
dependence  of  d  >  3  (Patterson  and  Piston,  2000).  These  findings 
make  parallelization  advisable  especially  for  two-photon  processes 
(n  =  2)  such  as  SHG,  CARS,  and  2PE.  Whether  parallelization  is 
beneficial  for  higher-order  multi-photon  processes,  such  as  THG 
and  3PE,  depends  on  the  dominating  damage  mechanism. 

In  the  case  of  2PE  and  a  given  laser  system,  the  optimum 
degree  of  parallelization  Nopt  is  estimated  by  considering  the 
optimal  focal  laser  power  Po;„(x/)  for  given  laser  parameters  and 
samples.  Nopt  is  just  given  by  Nop,  =  Pmg/Pop, (x/).  This  number  is 
also  the  optimum  increase  in  the  recording  speed  of  MMM  com¬ 
pared  to  standard  single-beam  multi-photon  microscopy.  For  a 
mode-locked  femtosecond  laser,  the  repeatedly  confirmed  power 
limit  in  the  focus  is  reached  at  Pop,(x  ~  200fsec,/=  80  MHz)  =  1 
to  lOmW  (Hanninen  et  al.,  1995;  Konig  et  al.,  1996,  1999;  Hopt 
and  Neher,  2001).  If  we  assume  that  because  of  over-filling  and 
reflection  losses  on  average  only  10%  of  the  laser  power  can  be 
transferred  into  the  sample  in  a  nearly  diffraction-limited  manner, 
this  results  in  an  optimum  beamlet  number  Nop,  =  20  to  100  with 
P„vg,MMM  —  100  to  200  nrW  in  the  sample.  The  fraction  of  power 
transferred  can  of  course  be  increased  by  slightly  compromising 
the  axial  resolution  of  the  system,  in  which  case  Nop,  can  be  further 
enlarged  significantly. 

EXPERIMENTAL  REALIZATION 

A  Multi-Focal  Multi-Photon  Microscopy  Setup 
Using  a  Nipkow-Type  Microlens  Array 

We  now  discuss  a  typical  implementation  of  a  MMM  setup  that 
was  originally  designed  for  2PE  fluorescence  (Bewersdorf  et  al., 
1998)  using  a  Nipkow-type  arranged  focal  pattern  for  scanning  the 
object.  Adapting  this  type  of  microscope  to  other  types  of  multi¬ 
photon  microscopes  is  straightforward  as  a  recently  published 
setup  for  SHG  multi-focal  microscopy  (Kobayashi  et  al,  2002) 
shows.  In  the  setup  displayed  in  Figure  29.1(A),  the  expanded  and 
collimated  laser  beam  of  a  mode-locked  Ti :  Sa  laser  illuminates  an 
array  of  microlenses  (460pm  diameter,  6-12mm  focal  length) 
etched  on  a  fused-silica  disk.  The  lenses  are  arranged  in  a  hexag¬ 
onal  pattern  such  that  the  illuminating  beam  is  split  into  small 
beams,  referred  to  as  beamlets,  and  focused  into  an  array  of 
approximately  5x5  foci  of  -6 pm  beam  waist  at  the  prefocusing 
plane  (PFP).  After  the  intermediate  optics,  the  beamlets  are 
directed  into  a  conventional  fluorescence  microscope.  The  role  of 
the  intermediate  optics  is  to  ensure  that  the  array  of  foci  is  imaged 
into  the  focal  plane  of  the  lens  and  that  each  beamlet  is  parallel  at, 
and  over-illuminates,  the  objective  entrance  pupil.  The  objective 
lens  then  produces  a  pattern  of  high-resolution  foci  at  the  sample. 
Figure  29.1(B)  shows  a  recording  of  the  2PE  fluorescence  created 
by  focusing  into  a  dye  solution.  The  typical  number  of  25  foci 
can  be  easily  varied  by  changing  the  intermediate  optics.  The 
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FIGURE  29.1 .  (A)  Schematic  of  the  first  implementation  of  multi-focal  multi¬ 
photon  microscopy  (MMM),  including  time-multiplexing  (TMX).  The  laser 
beam  is  expanded  by  the  lenses  L]  and  L2  and  illuminates  the  microlens  disk 
(ML;  also  shown  in  the  inset).  The  microlenses  focus  the  laser  beam  in  the  pre- 
focal  plane  (PFP),  which  is  imaged  into  the  sample  by  the  lenses  L3  to  L6  and 
the  objective  lens;  L6  is  the  tube  lens.  The  two-photon-excited  fluorescence 
passes  the  dichroic  mirror  (DM)  and  is  focused  onto  the  CCD  camera.  A  short- 
pass  filter  F  eliminates  the  remaining  laser  light.  Alternatively,  by  placing  a 
mirror  M  into  the  optical  detection  path,  the  fluorescence  from  the  focal  plane 
in  the  sample  can  be  viewed  by  eye.  The  dashed  bordered  box  marks  the  parts 
of  the  conventional  inverted  microscope  used  (Leica  DM-IRB).  (B)  The  exci¬ 
tation  pattern  of  the  foci,  recorded  with  the  microlens  disk  stopped  and  a 
fluorescent  solution.  The  experimental  axial  resolution  of  the  microscope  with 
a  1.2NA  water-immersion  lens  (Leica  63  X  1.2NA  water)  is  shown  by  the 
z-response  in  (C)  featuring  a  FWHM  of  890  ±  20  nm. 

microlenses  are  arranged  with  a  constant  helical  pitch  [see  the  inset 
in  Fig.  29.1(A)],  forming  spirals  with  typically  10  rows  in  a 
Nipkow  fashion.  Moreover,  the  layout  and,  in  particular,  the  spiral 
pitch  of  the  lenses  are  designed  in  such  a  way  that  the  disk  con¬ 
tains  several  (typically  5-12)  equivalent  segments.  The  layout  of 
the  microlenses  is  chosen  such  that  upon  rotation,  each  segment 
produces  a  complete  scan  of  the  focal  area.  Upon  rotation  of  the 
disk,  each  microlens  scans  one  line  in  the  sample.  The  distance 
between  the  lines  of  two  subsequent  microlens  foci  in  the  sample 
remains  well  below  the  lateral  resolution  assuring  homogeneous 
scanning  of  the  sample.  In  the  setup  described  the  distance  between 
the  lines  is  60 nm  for  a  lOOx  lens.  With  ~500  lines  per  segment 
this  results  in  a  field  of  view  of  35  pm  diameter.1 

In  our  example,  the  center  diameter  of  the  helix  is  80  mm.  The 
round  microlenses  are  hexagonally  closed-packed.  A  perfect 
hexagonally  closed-packed  plane  possesses  a  fill  factor  of  90.6%. 
However,  because  of  the  spiral  arrangement  of  the  microlenses,  the 


1  With  a  center  diameter  of  the  helical  structure  on  the  disk  of  -80  mm  and  a 
microlens  diameter  of  460  |jm,  the  simultaneous  usage  of  5  X  5  foci  results 
in  -2500  lines  scanning  across  the  sample  per  revolution  of  the  disk.  For  five 
equivalent  segments,  this  results  in  -500  lines  per  complete  scan  of  the  field. 


pattern  is  locally  skewed,  leading  to  a  smaller  fill  factor  of  slightly 
more  than  80%.  The  precise  value  slightly  varies  with  the  position 
on  the  disk.  To  minimize  the  NIR  light  that  may  pass  straight  to 
the  sample  causing  residual  out-of-focus  excitation,  the  area 
between  the  lenses  is  masked.  Tube  lenses  with  different  magnifi¬ 
cations  are  mounted  on  a  revolver  to  vary  the  distances  between 
the  foci  in  the  sample  and  the  over-illumination  of  the  objective 
entrance  pupil.  In  this  way,  the  imaging  parameters  can  be  adapted 
to  the  specimen. 

Rotating  the  disk  by  360°  renders  as  many  complete  lateral 
scans  as  segments  on  the  disk,  typically  5  to  12.  The  disk  can  be 
rotated  at  more  than  100  Hz  resulting  in  scanning  rates  of  more 
than  1000  frames/s.  Unlike  galvanometer-based  scanners,  this 
scanning  mechanism  does  not  involve  any  dead  time.  The  image 
rate  is  ultimately  determined  by  the  camera  frame  rate,  depending 
on  the  readout  rate  and  the  number  of  pixels.  It  may  range  well 
above  30  images/s.  The  signal  is  readily  separated  from  the  NIR 
excitation  light  by  a  dichroic  mirror  and  imaged  directly  onto  a 
charge-coupled  device  (CCD)  camera  mounted  at  the  microscope. 
With  faster  and  more  sensitive  cameras,  the  actual  limit  is  solely 
determined  by  the  number  of  multi-photon-induced  signal  photons 
(fluorescence  or  alternatively  SHG,  THG,  CARS,  etc.)  that  are 
produced  in  the  focal  plane.  Blocking  the  near-infrared  (NIR) 
excitation  light  in  the  eyepieces  with  an  absorption  filter  allows 
real-time  observation  of  multi-photon  generated  images  by  eye. 
Because  the  excitation  is  restricted  to  the  focal  plane,  features 
inside  bulky  objects  are  easily  revealed. 

Resolution 

Because  the  image  acquisition  time  is  usually  much  slower  than 
the  scan  speed,  the  focal  plane  is  scanned  several  times  during 
image  acquisition.  Mathematically,  the  nonlinear  excitation  point- 
spread  function  (PSF)  HexcMMM{x,y,z)  of  the  focal  pattern  is  inte¬ 
grated  over  the  focal  plane,  smearing  out  the  PSF  laterally.  Thus, 
the  excitation  efficiency  is  proportional  to  the  z-response  (Egner 
and  Hell,  2000): 

Jz.eJz)  =  J  J  Hexc,MMM(X’’  />  z)dX’ d/ .  (1) 

The  focal  plane  is  then  imaged  onto  the  CCD  camera.  This  process 
is  described  by  the  detection  PSF  Hia(x,y,z),  resulting  in  the  effec¬ 
tive  PSF 

Heff,MMM^y^)  “  4<*c(z)#det(*’>^)-  (2) 

This  equation  holds  only  for  multi-photon  absorption  phenom¬ 
ena  such  as  2PE  or  3PE.  For  scattering  events,  because  of  their 
coherent  nature  and  the  concomitant  conservation  of  the  phase, 
complex  amplitude  rather  than  intensity  PSFs  have  to  be  combined 
in  the  derivation  of  the  effective  PSF.  This  complicates  the  calcu¬ 
lation  massively. 

According  to  Eq.  2,  the  lateral  resolution  of  the  MMM  is 
determined  by  the  detection  PSF  in  the  visible  wavelength  range, 
thus,  in  a  non-scattering  specimen,  it  is  superior  to  the  resolution 
in  standard  non-descanned  2PE  microscopy.  The  axial  resolution, 
characterized  by  the  z-response,  is  given  by  /-e(f(z)  only,  because 
the  integral  of  Hia(x,y,z)  across  the  lateral  plane  is  of  constant 
value.  Hence  we  have  7,(z)  =  /iCTC(z). 

The  response  to  a  homogeneously  excitable  half  space,  the 
so-called  sea  response,  is  a  good  measure  for  the  axial  imaging  of 
an  axially  extended  object: 
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7«.(z)=  J  J  J  J  HexcMMM(x',  y',  z  )dx'dy'dz.  (3) 

~Z  -z -00-00 

Averaging  a  three  dimensional  (3D)  image  stack  of  a  2PE  flu¬ 
orescent  solution  behind  a  coverslip  recorded  with  a  lOOx  /1.4 
numerical  aperture  (NA)  oil-immersion  lens,  across  an  area  several 
micrometers  in  diameter  yields  the  sea  response  I„a(z).  Figure 
29.1(C)  shows  the  experimental  z-response  Iz(z)  obtained  from  the 
derivative  of  the  experimental  7„(z). 

Time  Multiplexing  as  a  Solution  to 
Interfocal  Crosstalk 

As  with  all  parallelized  3D  microscopes,  standard  MMM  needs 
to  compromise  between  the  degree  of  parallelization  and  the 
crosstalk  between  the  multiple  beams.  For  absorption  processes  of 
the  order  of  n,  the  excitation  PSF  of  the  MMM  describing  the  dis¬ 
tribution  of  the  excitation  efficiency  in  the  sample  at  a  particular 
instant  (i.e.,  scanning  movement  neglected)  is  given  by 

Hexc.miM  (x,  y,  z)  °c  [hjx,  y,  z)  ®  g(x,  y,  z)]2"  (4) 

hm  describes  the  (single-focus)  amplitude  PSF  of  the  illumination 
and  the  grating  function  g  is  the  sum  of  several  5  functions  in  the 
focal  plane,  one  for  each  focus  (Egner  and  Hell,  2000). 

Reducing  the  distance  between  the  lens  foci  increases  the  inter¬ 
ference  between  the  focal  fields  especially  in  the  planes  away  from 
the  focal  plane.  Because  of  the  periodic  arrangement  of  the  lenses, 
the  focal  fields  add  up  constructively  in  the  so-called  Talbot  and 
fractional  Talbot  planes.  In  these  out-of-focus  planes,  the  field  of 
different  foci  reinforce  each  other  by  constructive  interference, 
yielding  periodic  patterns  of  excitation  light,  which  results  in 
increased  out-of-focus  excitation  (Egner  and  Hell,  2000).  Con¬ 
ventional  detection  with  a  CCD  camera  through  a  high  aperture 
lens,  images  these  out-of-focus-planes  onto  the  camera  where  they 
appear  mostly  as  a  featureless  background.  Hence,  the  z-response 
of  an  MMM  may  differ  from  that  of  a  single-beam  system  by  an 
axially  expanded  shoulder  of  low  amplitude  [see  Fig.  29.1(C)  or 
Fig.  29.2(C)].  Problems  arise  if  small  or  dim  structures  need  to  be 
imaged  that  are  located  in  front  of  or  behind  bright  voluminous 
objects.  In  this  case,  the  shoulder  may  lead  to  a  significant  back¬ 
ground  signal.  For  a  thorough  discussion  of  this  subject,  including 
the  influence  of  the  number  of  foci  and  the  distance  between  neigh¬ 


boring  foci  onto  the  z-response  (Eq.  1 )  and  the  sea  response  (Eq. 
3),  we  refer  to  Egner  and  Hell  (2000). 

Introducing  a  temporal  delay  between  the  beamlets  solves  this 
problem  by  ensuring  that  light  pulses  of  neighboring  foci  pass  the 
focal  region  at  different  time  points  (Buist  et  al.,  1998;  Egner  and 
Hell,  2000).  Interference  will  not  occur  because  the  pulses  simply 
do  not  meet  each  other.  Hence,  the  excitation  PSF  is 

H{™$MM{x,y,z)  °=  hm(x,y,z )  ®  g(x,y,z),  (5) 

preventing  the  formation  of  Talbot  planes.  Apart  from  a  constant 
factor  denoting  the  number  of  foci,  the  z-response  of  this  function 
does  not  differ  from  that  of  a  single-focus  multi-photon  excitation 
PSF.  Extensive  calculations  of  different  possible  realizations  of 
this  time-multiplexed  MMM  (TMX-MMM)  as  well  as  its  practi¬ 
cal  implementation  have  been  reported  (Egner  and  Hell,  2000; 
Andresen  et  al,  2001).  Using  typical  operation  parameters  (dis¬ 
tance  between  foci  -4  |j,m,  37  foci),  the  use  of  three  delay  sub¬ 
classes  decreases  the  background  excitation  by  a  factor  of  ~3.5, 
whereas  four  subclasses  decreases  background  excitation  by  a 
factor  of  ~4.5.  The  subclasses  are  chosen  such  that  each  microlens 
is  surrounded  by  microlenses  of  the  other  two  delay  subclasses,  in 
alternating  order.  For  a  random  distribution  of  three  subclasses,  the 
creation  of  distinct  Talbot  planes  is  precluded,  but  the  average 
background  remains  unchanged  (Egner  and  Hell,  2000). 

A  realization  of  TMX  in  MMM  is  sketched  in  the  inset  of 
Figure  29.1(A).  Two  300pm  thick  disks  with  holes  at  the  position 
of  every  third  microlens  (first  disk)  or  at  two  of  three  microlenses 
(second  disk)  were  glued  onto  the  original  microlens  disk.  The 
holes  have  the  same  diameter  as  the  microlenses  (460  pm)  so  that 
the  microlens  beamlets  are  divided  into  three  subclasses:  beamlets 
unaffected  by  the  additional  disks,  beamlets  delayed  by  -500  fs 
due  to  one  additional  layer  of  300  pm  glass,  and  beamlets  delayed 
by  — 1  ps.  Because  of  the  low  microlens  NA  (typically  -0.02),  the 
axial  focal  shift  induced  by  the  additional  glass  is  negligible. 
Figure  29.2  shows  the  comparison  between  the  TMX-MMM  and 
a  typical  Nipkow  disk  microscope  with  single-photon  excitation 
when  imaging  a  challenging  but  realistic  specimen.  In  Figure 
29.2(C),  the  z-profile  through  a  3D-image  series  of  a  fluorescent 
solution  between  two  coverslips  demonstrates  the  superior  sup¬ 
pression  of  the  crosstalk  between  the  excitation  foci  with  the 
TMX-MMM.  This  results  in  a  much  clearer  representation  of  small 
details  above  or  below  bulky  objects;  see  Figures  29.2(A,B). 
Although  TMX  has  been  demonstrated  for  2PE  only,  the  idea  can 


FIGURE  29.2.  (A)  and  (B)  show  surfaces  3D-rendered  using  AMIRA  Voltex  of  2  grains  of  similar  diameter  (—25  pm)  recorded  with  a  lOOx  1.4NA  oil- 
immersion  lens  with  the  TMX-MMM  and  a  typical  Nipkow  confocal  scanner  (NCS),  respectively.  For  a  fair  comparison,  the  parameters  were  selected  to  take 
the  higher  background  of  the  NCS  into  account.  Due  to  specimen-induced  aberrations,  the  background  behind  the  pollen  grain  in  the  NCS  data  is  increased.  For 
details  see  Egner  and  colleagues  (2002).  (C)  The  experimental  sea  response  of  a  Rhodamine  6G  solution  of  -30  pm  thickness.  Note  the  higher  background  of 
the  NCS.  For  example,  at  a  distance  of  5  pm  away  from  the  focal  point,  the  background  of  the  TMX-MMM  is  about  seven  times  lower  than  that  of  the  NCS. 
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be  exploited  in  all  multi-focal  microscopy  modes  using  pulsed  or 
short-coherence-length  illumination.  It  should  be  noted  however, 
that  for  linear  and  nrulti-photon  scattering  events  the  conservation 
of  the  phase  upon  scattering  plays  a  crucial  role  in  the  formation 
of  the  effective  PSF. 

Alternative  Realizations 

Besides  the  Nipkow-type  scanning  scheme,  other  scanning  mech¬ 
anisms  can  also  be  implemented.  For  example  Buist  et  al,,  (1998), 
use  a  rectangular  microlens  arrangement.  Scanning  is  accom¬ 
plished  by  rapidly  moving  the  foci  in  a  Lissajous  pattern  with  an 
xy  galvanometric  mirror.  Proper  adjustment  of  the  frequency  and 
the  amplitude  of  the  scanning  movement  allows  nearly  uniform 
illumination  conditions  at  video  rate,  with  ~25%  efficiency  of  light 
usage.  MMM  was  also  implemented  in  4Pi-microscopy  (Egner  et 
al.,  2002b)  in  which  case  a  square  array  of  microlenses  was 
selected.  This  realization  was  chosen  because  of  its  greater  flexi¬ 
bility  in  combination  with  an  array  of  detection  pinholes  in  the 
focal  plane  of  the  microlenses,  allowing  an  easy  change  of  the 
pinhole  size,  of  the  focal  lengths  of  the  nricrolenses,  as  well  as  of 
the  distance  between  the  foci. 

All  of  these  setups  use  microlenses  illuminated  by  an  expanded 
and  collimated  laser  beam  to  create  the  beamlets.  To  avoid  wasting 
of  the  laser  light,  a  large  area  of  the  Gaussian  beam  profile  is  used 
for  illumination.  The  microlenses  are  much  smaller  than  the 
expanded  beam  profile  so  that  each  individual  lens  is  illuminated 
virtually  uniformly.  However,  the  microlenses  in  the  periphery  of 
the  Gaussian  profile  are  illuminated  by  less  intense  light  than  those 
in  the  center  of  the  laser  beam.  This  difference  is  carried  forward 
to  the  field  of  view,  resulting  in  a  decrease  of  the  intensity  of  the 
foci  towards  the  rim.  Unfortunately,  the  nonlinear  dependence  of 
the  multi-photon  process  on  the  illumination  intensity  enhances 
this  effect.  If  no  measures  are  taken  to  homogenize  this  beam 
profile,  the  efficient  use  of  the  available  laser  power  and  the  in¬ 
homogeneity  across  the  field  of  view  has  to  be  balanced. 

Masking  the  expanded  laser  beam  with  a  rectangular  aperture 
that  blocks  70%  of  the  laser  power  of  a  Gaussian  beam,  results  in 
a  ~50%  decrease  of  the  excitation  efficiency  at  the  edge  of  the 
field  of  view  in  a  2PE  setup.  A  more  constant  light  distribution 
with  a  better  exploitation  of  the  available  laser  power  can  be 
achieved  by  homogenizing  the  illumination  of  the  microlens  array. 
To  avoid  wasting  large  amounts  of  the  laser  light,  sophisticated 
Gaussian-to-flat-top  converters  may  prove  useful.  These  use  either 
a  combination  of  aspheric  lenses  or  an  afocal  spherical  lens 
arrangement  to  redistribute  the  intensity  to  produce  a  homoge¬ 
neous  beam  that  also  has  a  flat  wavefront  (Shafer,  1982). 

Instead  of  dividing  the  expanded  laser  beam  across  its  profile, 
the  beam  intensity  can  also  be  portioned  by  an  etalon  (Fittinghoff 
and  Squier,  2000)  or  a  cascade  of  beam-splitters  (Fittinghoff  et  al., 
2000;  Nielsen  et  al.,  2001).  In  this  case,  the  beam  profile  is  main¬ 
tained  for  each  beamlet  and  not  relayed  to  the  focal  plane.  In  order 
to  achieve  the  best  possible  resolution,  the  objective  entrance  pupil 
still  needs  to  be  uniformly  illuminated.  Dividing  the  beam  into  vir¬ 
tually  equally  intense  beamlets  is  imperative  because  each  beamlet 
illuminates  a  different  subfield.  If  the  ratio  between  the  reflected 
and  the  transmitted  light  is  not  carefully  chosen,  discontinuous 
steps  are  obtained  at  the  edges  of  the  subfields  in  the  image  (e.g., 
a  chessboard  pattern).  As  the  non-linearity  n  of  the  excitation 
process  increases,  the  borders  between  two  subfields  become  more 
noticeable.  By  using  a  high  reflectivity  etalon,  this  problem  can  be 
avoided,  but  a  large  fraction  of  the  laser  light  is  wasted  to  attain 
equal  beamlet  intensities  (Fittinghoff  and  Squier,  2000).  In  addi¬ 


tion,  the  tilting  of  the  beam-splitters  must  be  handled  with  partic¬ 
ular  care  in  order  to  obtain  equidistant  foci  with  each  beamlet  cen¬ 
tered  on  the  objective  entrance  pupil  (Nielsen  et  al.,  2001). 

These  intensity-dividing  approaches  have  several  advantages; 
They  are  flexible,  they  provide  a  homogeneously  illuminated  field 
of  view,  and  inherent  to  the  beam  splitting  is  the  time  multiplex¬ 
ing  which  minimizes  crosstalk.  Unfortunately,  none  of  them  is 
compatible  with  the  spinning  disk  scanner.  They  are  restricted  to 
galvanometer  or  stage  scanning  and  thus  are  limited  in  scanning 
speed.  In  addition,  they  involve  scanning  dead  times. 

Combining  multi-focal  excitation  with  a  sample  scanning 
setup  ensures  that  the  fluorescent  light  emerging  from  the  foci  is 
always  imaged  onto  the  same  pixels  of  the  CCD  (Nielsen  et  al., 
2001).  In  this  case,  one  can  use  the  spatially  resolved  detection  on 
the  CCD  image  to  remove  crosstalk  in  the  detection  path  between 
the  signal  emitted  from  adjacent  foci;  this  may  be  regarded  as  a 
confocal  pinhole  mask  by  software.  The  approach  works  as  long 
as  the  lateral  crosstalk  is  limited  to  a  region  smaller  than  the  dis¬ 
tance  between  two  neighboring  foci. 

Significant  crosstalk  reduction  is  achieved  by  using  a  pinhole 
array  in  combination  with  nricrolenses  in  a  beam-scanning  setup 
(Fujita  et  al.,  2000).  Re-scanning  of  the  signal  across  the  CCD 
camera  behind  the  pinholes  (Egner  et  al.,  2002b)  solves  the 
problems  with  image  acquisition  encountered  by  Nielsen  and  col¬ 
leagues.  Detecting  through  pinholes  slightly  increases  the  resolu¬ 
tion  at  the  expense  of  some  signal  loss.  For  sectioning,  the  use  of 
a  confocal  pinhole  array  (Fujita  et  al.,  2000)  is  of  course  not 
needed.  However,  an  undeniable  benefit  is  that  the  undesired  scat¬ 
tered  (fluorescence)  light  in  multi-focal  microscopy  is  even  more 
suppressed.2 

Confocalization  certainly  increases  the  contrast  in  the  image. 
Unfortunately,  if  chromatic  aberrations  are  not  compensated,  the 
use  of  pinholes  always  results  in  a  significant  signal  loss. 

The  combination  of  MMM  with  a  regenerative  amplifier  fea¬ 
turing  a  repetition  rate  of  1kHz  (800 nrn  wavelength,  llOfs  pulse 
length)  has  also  been  demonstrated  (Fujita  et  al.,  1999).  To  avoid 
severe  photodamage  and  saturation  due  to  the  high  peak  powers, 
the  setup  had  to  be  parallelized  approximately  1000-fold  as  in  a 
typical  single-photon  Nipkow  confocal  system  (Petran  et  al,,  1968, 
1985).  The  interfocal  distance  is  about  the  same  as  in  the  systems 
described  above.  Thus,  the  observation  of  structures  smaller  than 
~20|inr  does  not  profit  from  this  high  degree  of  parallelization. 
Rather,  the  low  duty  cycle  for  each  focus  makes  the  rapid  record¬ 
ing  of  small  structures  nearly  impossible. 

A  commercially  available  version  of  MMM  (La Vision  BioTec, 
Bielefeld,  Germany)  based  on  an  improved  version  of  the  system 
of  Nielsen  and  colleagues  (Nielsen  et  al.,  2001)  produces  a  line  of 
up  to  64  foci,  scanned  in  two  dimensions  by  a  set  of  galvanome¬ 
ter  mirrors.  The  beam-splitter  operates  over  the  whole  spectral 
range  of  the  Ti :  Sa  laser  generating  foci  of  nearly  the  same  inten¬ 
sity  (difference  in  fluorescence  <5%).  The  distance  between  the 
foci  (typically  600  nm  for  a  60x  lens)  can  be  changed  by  the  inter¬ 
mediate  optics  implemented.  Setting  the  focal  separation  to 
approximately  the  lateral  full  width  at  half  maximum  (FWHM)  of 
the  PSF  produces  a  homogeneous  line  of  18  pm  length  with  high 
resolution.  Fast  scanning  perpendicular  to  this  line  (up  to  3.5  kHz) 
allows  the  recording  of  18  pm  wide  rectangles  of  variable  length, 
in  principle,  in  about  0.3  ms.  The  recording  time  is  currently 


2  Because  in  single-beam  multi-photon  microscopes  there  is  no  crosstalk,  pin¬ 
holes  are  reasonable  only  in  MMM. 
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FIGURE  29.3.  Different  realizations  of  MMM.  The  images  show  sketches  drawn  after  the  original  publications  named,  but  are  here  simplified  and  rotated  to 
ease  comparison:  (A)  Bewersdorf  and  colleagues  (1998),  (B)  Buist  and  colleagues  (1998),  (C)  Fittinghoff  and  Squier  (2000),  (d)  Nielsen  and  colleagues  (2001). 
(E)  Fujita  and  colleagues  (2000),  (F)  Egner  and  colleagues  (2002b).  BS  denotes  the  beam-splitting  units  of  Fittinghoff,  as  well  as  of  Nielsen  and  co-workers. 
Details  are  shown  in  the  insets.  TL,  tube  lens;  ML,  microlenses;  PFI,  pinhole  disk;  Obj,  objective  lens. 


limited  by  the  readout  speed  of  the  camera  used  and  the  available 
signal.  By  moving  the  beam-splitter,  the  degree  of  parallelization 
can  be  decreased  by  a  factor  of  2.  In  this  setup,  neighboring  foci 
are  polarized  perpendicular  to  each  other  allowing  polarization- 
sensitive  measurements.  Figure  29.3  compares  the  different 
schemes  of  realizing  MMM. 

ADVANCED  VARIANTS  OF  MULTI-FOCAL 
MULTI-PHOTON  MICROSCOPY 

Space  Multiplexing 

An  option  which  permits  improved  exploitation  of  the  total  laser 
power  and  facilitates  the  changing  of  intensity  levels  in  selected 
regions  without  attenuating  the  total  laser  power  is  space  multi¬ 
plexing  (SMX)  MMM  (Hell  and  Andresen,  2001).  The  basic  idea 
of  SMX  MMM  is  to  modulate  the  intensity  across  the  sample  by 
the  spatially  modulated  interference  resulting  from  overlapping 
arrays  of  slightly  offset  focal  fields.  For  a  given  degree  of  paral¬ 
lelization  and  power,  SMX  increases  the  two-  and  three-photon 
excited  signal  of  parallelized  multi-photon  microscopy  by  a  factor 
of  up  to  1.5  and  2.5,  respectively.  To  some  extent,  sensitive  regions 
may  be  spared,  whereas  in  regions  with  weaker  nonlinear  suscep¬ 
tibilities  the  excitation  intensity  can  be  increased. 

SMX  was  implemented  in  the  MMM  by  splitting  the  colli¬ 
mated  laser  beam,  combining  it  again  with  a  wedge  mirror  and 
then  illuminating  the  microlens  disk.  The  optical  path  difference 
between  the  two  recombined  beams  was  changed  with  a  piezo- 
driven  mirror.  By  carefully  adjusting  the  angle  between  the  two 
beams,  the  interference  pattern  can  be  modified.  Because  the  inter¬ 
ference  pattern  is  generated  in  front  of  the  microlenses,  it  modu¬ 


lates  mainly  the  illumination  of  the  individual  microlenses,  and  the 
influence  of  specimen-induced  aberrations  does  not  differ  from 
standard  MMM.  As  with  TMX,  the  SMX  is  relevant  to  all  modes 
of  multi-photon  microscopy,  including  parallelized  SHG  and  THG 
imaging,  CARS,  and  widefield  multi-photon  excitation. 


Fluorescence  Lifetime  Imaging 

Lifetime  imaging  of  the  fluorescent  state  is  an  important  develop¬ 
ment  in  fluorescence  microscopy.  Fluorescence  lifetimes  are 
sensitive  to  the  fluorophore  environment  and  can  be  used  to  dis¬ 
tinguish  fluorophores  with  overlapping  emission  spectra  where 
spectral  separation  is  difficult.  The  advent  of  gated,  intensified 
CCD  cameras  with  a  gate  width  of  200  ps  and  gating  repetition  rates 
of  up  to  100  MHz  provides  the  opportunity  of  combining  multifo- 
cality  with  lifetime  imaging.  Using  gated  cameras,  lifetime  images 
of  fluorescent  samples  can  be  easily  recorded  with  MMM  (Straub 
and  Hell,  1998a).  The  necessary  modification  includes  the 
exchange  of  the  standard  MMM  camera  with  a  gated,  intensified- 
CCD  camera  (Picostar  HR,  La  Vision,  Gottingen,  Germany)  and 
additional  electronics  to  trigger  the  intensifier  gate  a  fixed  time  after 
the  excitation  laser  pulses.  A  programmable  delay  unit  enables  the 
recording  of  the  fluorescence  decay  curves  in  lOOps  steps.  The  gate 
width  of  200  ps  and  a  time  gap  of  12.5  ns  between  the  laser  pulses, 
allow  for  the  measurement  of  fluorophore  decay  times  in  the  0.5  to 
10ns  range.  Figure  29.4  shows  an  example  of  a  lifetime  measure¬ 
ment  for  two  types  of  beads  taken  with  the  MMM.  By  using  a  series 
of  time  gates  after  each  photomultiplier  in  a  PMT  array  to  detect 
all  time  windows  instead  of  using  a  gated  camera  that  collects 
signal  in  only  a  single  window,  the  acquisition  speed  could  be 
enhanced  by  a  factor  greater  than  2  (see  Chapter  27,  this  volume). 
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FIGURE  29.4.  Fluorescence  lifetime  imaging  with  MMM.  Panels  (A)  and  (B) 
show  images  of  randomly  dispersed  fluorescent  polystyrene  beads  (Poly¬ 
sciences  Inc.,  beads  I)  and  larger  latex  beads  (Molecular  Probes  Inc.,  beads  II) 
recorded  directly  after  the  onset  of  fluorescence  and  after  8.0ns  with  a  gate 
width  of  200  ps.  The  analysis  of  the  decay  of  the  fluorescence  intensity  I  dis¬ 
played  in  (C)  and  on  a  logarithmic  scale  in  (D)  yields  the  lifetimes  of  the  flu- 
orophores  (2.2  and  3.7  ns  for  beads  I  and  II,  respectively). 


Second  Harmonic  Generation  Multi-Focal 
Multi-Photon  Microscopy 

Because  SHG  light  is  mainly  forwardly  scattered,  the  microscope 
is  preferably  used  in  a  transmission  arrangement  (Kobayashi  et  al., 
2002).  By  modifying  the  MMM  design,  Kobayashi  and  co-workers 
imaged  the  focal  plane  with  an  intensified  CCD  camera  on  the  far 
side  of  the  sample.  The  sample  was  illuminated  with  a  1.2NA 
water-immersion  lens  by  applying  ~  100  foci  with  an  average  inten¬ 
sity  of  less  than  2mW  each.  By  placing  appropriate  filters  in  the 
detection  beam,  the  laser  light  (k  =  896  nm,  82  MHz  repetition  rate, 
80  fs  pulse  length  at  the  laser  output)  as  well  as  light  above  ~k  = 
500  nm  from  2PE  fluorescence  was  filtered  out.  Thereby  it  was 
possible  to  record  multi-focal  SHG  images  at  X  =  448  nm  with  a 
typical  overall  exposure  time  between  33  ms  and  -500 ms.  By 
exchanging  the  detection  filters,  2PE  fluorescence  images  of  the 
same  sample  can  also  be  recorded.  Kobayashi  and  colleagues 
(2002)  recorded  the  contraction  of  rat  cardiac  myocytes  with  this 
multi-focal  SHG  microscope  at  video  rate. 


Multi-Focal  Multi-Photon 
Microscopy-4Pi  Microscopy 

The  application  of  MMM  to  4Pi-confocal  microscopy  demon¬ 
strates  that  multiple  beamlets  of  high  optical  quality  can  be  pro¬ 
duced.  The  recording  times  of  living  specimens  with  4Pi-confocal 
microscopy  can  be  decreased  to  a  fraction  of  the  rate  of  former 
setups  (Egner  et  al.,  2002b,  2004). 


IMAGING  APPLICATIONS 

MMM  can  be  advantageously  applied  to  the  3D  imaging  of  bio¬ 
logical  specimens,  including  living  cells,  whenever  rapid  multi¬ 
photon  imaging  of  not  too-strongly-scattering  specimens  is 
needed.  Pollen  grains  and  the  stem  of  Prionium  stained  with 
safranin-fast  green  are  relatively  thick,  scattering  test  objects  that 
have  been  imaged  using  oil-immersion  lenses.  Living  PC12  cells 


stained  with  acridine  orange,  and  neurons  in  the  temporal  cortex, 
that  have  been  ionophoretically  injected  with  Lucifer  Yellow 
(Straub  and  Hell,  1998b)  were  recorded  using  water-immersion 
lenses.  Nielsen  and  co-workers  recorded  3D  stacks  of  a  CHO  cell, 
doubly  stained  with  ethidium  bromide  and  fluorescein  (Nielsen  et 
al,  2001).  Fujita  and  colleagues  imaged  rat  heart  cells  stained  with 
eosin  (Fujita  et  al.,  1999)  as  well  as  sections  of  a  root  of  conval- 
laria  (Fujita  et  al.,  2000).  In  another  study,  bovine  chromafln  cells 
and  NGF-differentiated  PC12  cells,  stained  with  the  dyes  acridin 
orange,  FM1-43  and  DiA  as  well  as  by  transfection  of  the  cells 
with  green  fluorescent  protein  (GFP),  have  been  examined  (Straub 
et  al.,  2000).  Moreover,  the  sectioning  capability  of  MMM  has 
been  demonstrated  in  combination  with  4Pi-confocal  microscopy 
(for  details  see  Chapter  30,  this  volume',  Egner  et  al.,  2002b,  2004). 

Examples  of  fast  imaging  of  dynamic  processes  include  the 
production  of  time  sequences  of  sections  through  living  boar  sperm 
cells  (Bewersdorf  et  al.,  1998)  (Fig.  29.5),  the  imaging  of  Ca2+ 
dynamics  (Fujita  et  al.,  2000),  as  well  as  a  video-rate  movie  of  the 
contraction  of  the  motile  microorganism  Euglena,  showing  chloro¬ 
phyll  autofluorescence  (Fittinghoff  et  al.,  2000).  Using  SHG- 
MMM,  Kobayashi  and  colleagues  recorded  the  contraction  of  rat 
cardiac  myocytes  at  video  rate  (Kobayashi  et  al.,  2002).  A  study 
of  particular  biological  relevance  mapped  Forster  resonance 
energy  transfer  (FRET)  using  MMM  to  reveal  important  aspects 
of  protein  interaction  in  the  Golgi  apparatus  in  living  cells  (Majoul 
et  al.,  2001,  2002)  (see  also  Chapter  45,  this  volume). 

An  interesting  application  of  MMM  in  neurobiology  involved 
the  measurement  of  rapid  changes  in  free  Ca2+  concentration  over 
a  large  field  of  view  (Cossart  et  al,  2003).  By  measuring  the  time 
traces  of  the  2PE  fluorescence  of  the  Ca2+-sensitive  dye  FURA2- 
AM  simultaneously  in  a  few  hundred  cell  bodies  in  hippocampal 
brain  slices  of  mice,  Cossart  and  colleagues  localized  microcircuits 
in  the  neuronal  network.  In  the  example  shown  in  Figure  29.6,  they 
recorded  time  series  of  a  single  xy-section  (443  pm  X  335  pm, 
-100pm  deep  inside  the  tissue)  at  a  rate  of  150ms/frame  over 
5  min  with  the  LaVision  BioTec  version  of  the  MMM  and  identi¬ 
fied  the  cell  bodies  from  the  averaged  frames  [Fig.  29.6(A,B)] 
(Cossart  et  al.,  2005).  With  this  information,  the  fluorescence  time 
traces  [Fig.  29.6(C)]  corresponding  to  the  Ca2+  concentration  in  the 
individual  cells  can  be  analyzed.  Every  dip  in  a  time  trace  is  inter¬ 
preted  as  the  result  of  an  action  potential.  By  searching  for  corre¬ 
lations  in  the  event  patterns  of  all  the  recorded  cell  bodies  [Fig. 
29.6(D)],  neural  microcircuits  were  identified  [marked  red  in 
Fig.  29.6(B)].  These  studies  allow  simultaneous  measurements  of 
the  dynamic  and  global  characterization  of  neuronal  network  activ¬ 
ity  for  the  first  time  and  permit  one  to  determine  the  single-cell 
properties  of  the  unitary  microcircuits  involved  in  this  activity. 


LIMITATIONS 

While  it  does  reduce  higher  order  photodamage  effects  through 
parallelization,  MMM  has  its  own  constraints.  Though  normally 
negligible  in  single-beam  systems  (Schonle  and  Hell.  1998), 
heating  of  the  sample  may  become  a  problem  if  the  overall  average 
laser  power  supplied  to  the  sample  is  relatively  large.  This  holds 
in  particular  for  small  scanning  areas  or  if  the  sample  contains 
strong  single-photon  absorbers. 

Multifocality  also  leads  to  crosstalk  arising  from  the  reinforced 
overlap  of  the  focal  fields  of  the  multiple  foci.  Fortunately,  with 
the  mandatory  pulsed  illumination,  the  crosstalk  of  the  illumina¬ 
tion  light  can  be  almost  entirely  eliminated  by  time-multiplexing 
every  beam.  The  only  relevant  crosstalk  remaining  then  is  that 


FIGURE  29.5.  xy  images  of  living  boar-sperm 
cells  taken  with  the  MMM  within  33  ms  (30 
images/s).  Four  typical  images  from  a  movie  of 
191  images  are  displayed.  Note  the  movement 
of  the  sperm  tail  within  the  first  33  ms. 


FIGURE  29.6.  Identification  of  microcircuits  in  hippocampal  brain  slices  with  MMM.  (A)  Average  of  a  time  series  of  500  consecutive  frames  showing  the  flu¬ 
orescence  of  the  Ca2+-sensitive  dye  FURA2-AM  (Olympus  20  x  0.95  NA  water  immersion,  150ms/frame  recording  time).  (B)  Contours  of  the  cell  bodies  auto¬ 
matically  identified  from  (A).  (C)  Typical  time  traces  of  the  fluorescence  for  single  cell  bodies  taken  from  the  time  series.  Each  dip  corresponds  to  an  action 
potential  causing  the  release  of  Ca2+  and  is  marked  as  an  event  in  (D).  (D)  Rasterplot:  Each  horizontal  line  represents  the  time  trace  of  a  single  cell.  Networks 
are  identified  by  correlated  events  {red  box )  in  the  rasterplot  and  can  be  visualized  in  the  slice  [filled  yellow  contours  in  (B)].  (Cossart  et  al.,  2005.) 
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which  occurs  because  of  scattering  during  the  backimaging  of  the 
signal  onto  the  plane  of  detection.  However,  this  can  be  reduced 
by  imaging  the  signal  through  an  array  of  confocal  pinholes.  It  also 
is  less  severe  than  in  a  conventional  or  in  single-photon  paral¬ 
lelized  microscope,  because  the  multi-photon-induced  signal  orig¬ 
inates  only  at  the  focal  plane.  Figure  29.2  illustrates  that  the  larger 
focal  distance,  the  smaller  number  of  foci,  the  implementation  of 
TMX,  as  well  as  the  nonlinear  signal  dependence  in  MMM  leads 
to  a  crosstalk  much  smaller  than  in  parallelized,  single-photon  con- 
focal  (Nipkow  disk)  microscopy  (Egner  et  al.,  2002a).  However, 
TMX  could  also  be  applied  to  the  latter  if  lasers  with  short  coher¬ 
ence  lengths  are  used. 

The  crosstalk  in  the  detection  path  and  the  attenuation  of  the 
intensity  with  deeper  penetration  depth  may  hamper  the  imaging 
of  planes  lying  deep  in  strongly  scattering  specimens.  For  example, 
MMM  is  not  advantageous  for  imaging  hundreds  of  micrometers 
inside  of  brain  tissue.  Nevertheless,  the  success  of  this  concept  very 
much  depends  on  the  adaptation  of  the  system  layout  to  the  detailed 
optical  properties  of  the  object.  For  example,  one  could  still 
imagine  an  array  of  a  comparatively  small  number  of  distinct  foci, 
say  2x2,  coupled  to  a  matched  array  of  detectors.  Even  without 
this,  imaging  approximately  50  pm  inside  of  a  scattering  pollen 
grain  or  100  pm  inside  of  brain  tissue  is  possible  without  major 
drawbacks  in  resolution  or  signal  (see  Figs.  29.2  and  29.6). 

CURRENT  DEVELOPMENTS 

Of  the  known  multi-photon  imaging  modes  in  microscopy,  2PE 
and  SHG  have  already  been  implemented  in  the  MMM  fashion. 
The  extension  to  3PE  (or  even  4PE)  is  technically  straightforward. 
Furthermore,  the  application  of  other  multi-photon  techniques  to 
multi-focal  microscopy  should  be  uncomplicated  if  the  microscope 
parameters  are  adjusted  to  the  physical  and  biological  limits  of 
interaction  cross-section  and  photodamage. 

With  the  current  progress  in  laser  technology,  the  efficiency  of 
MMM  is  expected  to  improve.  The  parallelization  in  MMM  is  not 
fundamentally  limited  by  photophysical  processes  such  as  highly 
nonlinear  damaging  or  bleaching  or  by  the  scanning  speed.  The 
enhancement  of  the  efficiency  in  signal  generation  by  paralleliza¬ 
tion  scales  directly  with  the  increase  in  laser  power.  Thus,  MMM 
profits  directly  from  the  new  developments  in  laser  technology. 
Even  with  the  presently  available  lasers,  a  doubling  of  the  effi¬ 
ciency  can  be  achieved  by  combining  the  beams  from  two  syn¬ 
chronized  lasers  with  a  polarizing  beam-splitter  to  a  single  beam. 
Apart  from  a  pure  increase  in  laser  power,  the  parameters  of  pulse 
length  and  repetition  rate  can  be  adapted  to  MMM  for  a  more  effi¬ 
cient  operation.  An  increase  in  the  repetition  rate  creates  a  better 
match  than  that  between  the  typical  12.5  ns  pulse  interval  of  a 


80  MHz  laser  and  the  2  ns  characteristic  of  most  fluorophore  decay 
constants,  allowing  a  more  time-efficient  excitation  (Bewersdorf 
and  Hell,  1998).  For  samples  particularly  susceptible  to  highly 
nonlinear  photodamage,  the  pulse  length  can  be  increased  to 
reduce  the  peak  intensity  (Bewersdorf  and  Hell,  1998).  In  the 
opposite  case,  or  to  exploit  excitation  mechanisms  having  a  higher 
degree  of  non-linearity,  shorter  pulse  lengths  can  be  advantageous. 

A  new  generation  of  CCD  cameras  with  on-chip  amplification 
records  with  virtually  no  readout  noise.  Even  though  the  amplifi¬ 
cation  process  increases  the  effective  shot  noise  typically  by  a 
factor  of  V2  (Basden  et  al.,  2003),  this  allows  faster  imaging  with 
frame  rates  at  which  the  signal  had  been  dominated  by  readout 
noise  in  the  past.  Alternatively,  by  using  stage-scanning  (Nielsen 
et  al.,  2001),  or  even  better,  by  descanning  in  the  detection  path, 
the  CCD  can  also  be  replaced  by  an  array  of  photomultipliers  or 
avalanche  photodiodes.  A  setup  can,  for  example,  be  realized  by 
exchanging  the  CCD  camera  in  Figure  29.3(F)  with  a  detector 
array  and  modifying  the  detection  path  so  that  the  fluorescence 
light  is  not  scanned.  This  alternative  makes  sense  when  imaging 
samples  that  exhibit  significant  scattering.  By  separating  the  foci 
and  the  concomitant  detectors  by  a  distance  that  is  larger  than  the 
average  radius  of  the  “scattering  bulb”  induced  by  the  sample,  one 
can  reduce  crosstalk  in  the  detection,  while  still  detecting  most  of 
the  fluorescence. 

The  design  of  the  spinning  microlens  disk  could  easily  be 
changed  to  a  system  that  scans  a  line  rather  than  a  two-dimensional 
field.  Scan  rates  of  several  thousand  hertz  are  also  in  the  range  of 
resonant  galvanometer  scanners.  However,  with  the  spinning  disk, 
dead  times  can  be  avoided  (as  long  as  the  CCD  can  be  read  out 
fast  enough).  These  features  may  be  advantageous  in  physiologi¬ 
cal  applications.  The  modification  of  the  MMM  concept  to  accom¬ 
modate  several  lines  at  a  defined  distance,  to  a  single  broader  line, 
or  to  other  custom-defined  illumination  areas  is  also  possible. 
Implementing  microlenses  with  different  focal  lengths  on  the  same 
spinning  disk  may  be  used  to  vary  the  scan  plane  within  small 
dimensions  as  long  as  these  are  compatible  with  the  aplanatic  cor¬ 
rection  of  the  lens.  A  very  interesting  and  promising  realization  of 
MMM  implies  a  reduction  of  the  lateral  distance  between  the  foci 
down  to  the  size  of  the  spot.  With  such  an  implementation  scan¬ 
ning  becomes  obsolete.  The  result  is  a  scanning  microscope 
without  any  moving  parts  that,  apart  from  the  sample's  linear- 
damage  susceptibility,  is  limited  in  speed  only  by  the  pulse- 
repetition  rate,  the  power  of  the  laser,  the  stability  of  the  specimen, 
and  the  readout  speed  of  the  detector  (Egner  and  Hell.  2000; 
Andresen  et  al.,  2001).  The  realization  of  such  a  system  has  been 
demonstrated  recently  (Fricke  and  Nielsen,  2005). 

For  physiological  applications,  in  which  the  signal  of  a  con¬ 
fined  illuminated  region  must  be  recorded  with  high  temporal  res¬ 
olution,  the  MMM  design  can  be  modified  so  that  the  detection 


TABLE  29.1.  Advantages  and  Disadvantages  of  the  Two  Different  Approaches  of  Subdividing  the  Beam 


Wavefront  Division  (Microlenses) 

[e.g.,  Fig.  29.3(A,B,E,F)] 

Amplitude  Division  (Beam-Splitter,  Etalon) 

[e.g.,  Fig.  29.3(C,D)] 

Beam  profile 

+  Beamlet  profiles  close  to  uniform 

—  Each  beamlet  profile  is  a  replica  of  the  entire  original  laser 
beam  profile 

Homogeneity  of  the  field 

—  Laser  beam  profile  is  transferred  to  field  of  view, 

-I-  Same  amplitude  for  all  beamlets  is  achievable,  but  differences 

of  view 

resulting  in  lower  amplitudes  for  outer  beamlets 

result  in  chessboard  pattern  effects 

Scan  speed 

+  Possible  use  of  spinning  disk  design  allows  extremely 
fast  scan  speeds  >1000  frames/s 

—  Limited  by  galvanometer  scanners 

Handling 

+  Spinning  disk  easy  to  adjust 

—  Adjustment  more  difficult 

+  Robust 

+  Flexible 
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provides  no  spatial  resolution,  for  example,  by  using  a  photomul¬ 
tiplier  tube  that  collects  light  from  the  illuminated  area  of  the 
sample.  While  such  a  device  would  not  provide  spatial  informa¬ 
tion,  it  would  allow  very  fast  readout  of  sequential  data.  The  intrin¬ 
sic  3D  spatial  confinement  of  the  multi-photon-induced  signal 
alone  defines  the  observed  volume.  The  illuminated  region  could 
be  chosen  so  that  signal  generation  is  confined  to  a  functional  com¬ 
partment  allowing  the  accumulation  of  the  signal  of  a  larger  area, 
increasing  the  sensitivity  of  the  system.  In  contrast  to  single-beam 
microscopes,  the  shape  of  the  illuminated  region  can  be  chosen 
more  freely. 


SUMMARY 

Multi-photon  microscopy  complements  conventional  microscopy 
in  numerous  ways.  However,  the  underlying  low  interaction  cross- 
sections  severely  limit  the  resulting  imaging  speed  and  sensitivity. 
The  use  of  multiple  foci  as  practiced  in  MMM  is  an  attractive  solu¬ 
tion  to  this  problem  that  increases  the  data  rate  by  the  factor  of  the 
parallelization  and  avoids  limitations  arising  from  highly  nonlin¬ 
ear  damage  effects  or  saturation.  Several  MMM  setups  have  been 
realized,  all  demonstrating  the  various  technical  possibilities  to 
parallelize  multi-photon  microscopy.  They  can  broadly  be  divided 
into  two  groups  that  differ  by  the  way  they  subdivide  the  laser 
beam  into  beamlets.  Either  it  is  the  wavefront  across  the  beam  that 
is  split  into  several  adjacent  fragments,  or  it  is  the  amplitude  of 
the  laser  beam  that  is  divided  into  several  consecutive  beamlets 
that  are  eventually  spread  out  in  angle  and  then  in  space.  Table 
29.1  compares  the  advantages  and  disadvantages  of  the  two 
approaches. 

For  regular  imaging  applications,  MMM  features  an  axial 
resolution  that  is  largely  uncompromised  with  respect  to  that  of  a 
single-beam  scanning  system.  The  interbeam  crosstalk  results  in 
an  elevated  background  but,  in  the  illumination  path,  this  can  be 
largely  avoided  by  time  multiplexing.  Due  to  the  conservation  of 
the  phase  in  (nonlinear)  scattering,  time  multiplexing  is  even 
more  important  for  CARS,  SHG,  THG,  etc.,  than  for  fluorescence 
imaging  as  it  simply  precludes  non-linearly  scattered  light  from 
interfering  in  the  detection  plane. 

The  spatial  blur  resulting  from  the  transverse  propagation  of 
the  non-linearly  generated  photons  needs  to  be  considered,  but 
owing  to  the  confinement  of  signal  generation  to  the  focal  plane, 
this  effect  is  significantly  less  pronounced  than  in  widefield 
microscopy.  Scattering  can  degrade  the  image  quality  and  thus 
limit  the  maximum  imaging  depth.  Most  of  the  other  advantages 
of  multi-photon  imaging,  such  as  making  ultraviolet  (UV)  illumi¬ 
nation  unnecessary  for  UV  dyes,  as  well  as  the  additional  infor¬ 
mation  gained  from  imaging  modes  that  use  nonlinear  scattering, 
are  retained. 

Progress  in  the  development  of  detectors  and  light  sources  will 
contribute  to  the  enhancement  and  enlargement  of  the  MMM 
family  of  microscopes.  Additionally,  new  scanner  and  paralleliza¬ 
tion  designs  will  further  improve  the  performance  of  MMM, 
leading,  among  other  things,  to  a  fast  scanning  multi-photon 
microscopy  without  any  moving  parts. 
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INTRODUCTION 

The  axial  ( z -)  resolution  of  any  fluorescence  microscope  using  a 
single  lens  is  limited  by  diffraction  to  >500  nm.  While  a  modest 
improvement  by  up  to  a  factor  of  2  may  be  achieved  by  mathe¬ 
matical  deconvolution,  a  substantial  improvement  of  the  axial 
resolution  requires  a  radical  change  of  the  physics  of  imaging. 
Since  the  1990s,  two  families  of  methods  have  evolved  that 
accomplished  substantially  improved  axial  resolution  in  three- 
dimensional  (3D)  imaging.  The  first  family,  comprising  4Pi 
microscopy  and  I5M,  coherently  combines  the  aperture  of  two 
opposing  lenses  (Hell  and  Stelzer,  1992a,  1992b;  Gustafsson  et  al., 
1995,  1999;  Egner  and  Hell,  2005).  The  second  family,  of  which 
stimulated  emission  depletion  (STED)  microscopy  (Hell  and 
Wichmann,  1994;  see  also  Chapter  31,  this  volume)  is  the  most 
established  member,  exploits  photophysical  or  photochemical 
properties  of  the  dye  to  break  the  diffraction  barrier.  Using  two 
lenses  for  STED,  the  axial  resolution  was  improved  down  to  30  to 
50 nm  (Dyba  and  Hell,  2002;  Dyba  et  al.,  2003).  However,  as 
STED  is  still  in  its  infancy  and,  unlike  PM  and  4Pi  microscopy, 
relies  on  the  specific  properties  of  the  dye,  it  is  not  surprising  that 
the  majority  of  recent  3D-imaging  applications  showing  axial 
super-resolution  in  (live)  cells  have  been  carried  out  with  the  4Pi 
microscope  (Egner  et  al.,  2002b,  2004;  Gugel  et  al.,  2004). 

4Pi  microscopy  coherently  illuminates  the  sample  through  two 
opposing  lenses.  Constructive  interference  of  the  counterpropa- 
gating  spherical  wavefronts  narrows  the  main  focal  maximum  of 
the  excitation  light  in  the  z-direction,  and  this  forms  the  basis  for 
a  3-  to  7-fold  improved  axial  resolution.  Unfortunately,  the  inter¬ 
ference  of  spherical  wavefronts  also  produces  so-called  sidelobes1 
that  are  located  at  about  half  a  wavelength  above  and  below  the 
focal  plane.  Moreover,  these  primary  lobes  are  accompanied  by 
weaker  periodic  lobes  that  increase  in  height  with  decreasing  semi¬ 
aperture  angle  a.  The  lobes  produce  ghost  images  that  have  to  be 
removed  by  mathematically  postprocessing  the  3D  data.  The  accu¬ 
racy  with  which  these  ghost  images  can  be  removed  depends  on 
the  relative  height  of  the  sidelobes  compared  to  the  main  peak.  As 
a  rule  of  thumb,  the  primary  lobe  height  should  never  exceed  ~50% 
of  the  main  focal  spot.  Unfortunately,  available  immersion  lenses 
all  feature  a  <  68°  so  that  the  primary  lobe  height  ranges  from 
60%  to  70%  of  the  main  peak.  Additional  optical  techniques  for 
lobe  suppression  thus  have  to  be  implemented,  of  which  the  most 
effective  are: 


1  Actually  they  are  located  above  and  below,  rather  than  to  either  side. 


•  Confocal  detection:  Fluorescence  emitted  from  out-of-focus 
planes  is  detected  less  efficiently,  thus  suppressing  fluores¬ 
cence  detection  from  the  sidelobes. 

•  Two-photon  excitation  (2PE):  The  quadratic  dependence  of 
the  excitation  efficiency  on  the  illumination  intensity  lowers 
the  excitation  sidelobe  height  relative  to  the  central  peak.  Addi¬ 
tionally,  due  to  the  longer  wavelength  of  the  near-infrared  light 
used  (Denk  et  al.,  1990),  the  sidelobes  are  shifted  away  from 
the  focal  plane  and  are  better  suppressed  by  the  pinhole.2 

•  Combined  interference  of  the  excitation  and  the  detection 
wavefront  pairs:  The  wavelength  disparity  leads  to  different 
axial  positions  of  the  excitation  and  the  detection  sidelobes.  As 
a  result,  fluorescence  originating  from  the  excitation  sidelobes 
is  weakened  by  incomplete  detection.  2PE  entails  a  pro¬ 
nounced  and  very  useful  wavelength  disparity  (Hell  and 
Stelzer,  1992a,  1992b;  Gugel  et  al.,  2004). 

The  potential  of  these  three  lobe-suppression  techniques  was 
already  recognized  in  the  early  stages  of  4Pi  microscopy  (Hell  and 
Stelzer,  1992a,  1992b).  In  fact,  the  use  of  2PE  together  with  con- 
focal  detection  has  been  the  method  of  choice  ever  since.  The  third 
technique  is  more  difficult  to  realize.  The  optical  setup  must  be 
designed  such  that  not  only  the  excitation  light  interferes  at  the 
sample  (Type  A  4Pi),  but  also  the  detection  light  interferes  at  the 
point  of  detection  (Type  C  4Pi).  Initially,  Type  C  4Pi-microscopy 
could  be  demonstrated  only  for  scattered  light,  where  excitation 
and  detection  wavelengths  are  the  same  and  the  implementation  is 
much  simpler  (Hell  et  al.,  1994).  A  feasible  realization  for  fluo¬ 
rescence  that  exploits  the  suppression  potential  of  this  technique 
is  shown  in  a  recent  publication  (Gugel  et  al. ,  2004).  The  2PE  Type 
C  mode  combines  all  three  mechanisms  and  thus  features  the 
lowest  achievable  sidelobes.  This  attribute  greatly  simplifies  the 
image  restoration  process.  Additionally,  Type  C  4Pi  microscopy 
obtains  a  -30%  better  resolution  than  its  Type  A  counterpart. 

In  contrast  to  4Pi  microscopy,  I5M  uses  planar  standing  waves 
for  fluorescence  excitation;  in  fluorescence  detection,  the  coherent 
addition  of  spherical  waves  is  performed  just  as  in  a  4Pi  Type  C 
microscope.  In  I5M,  the  joint  action  of  illumination  and  detection 
interference  is  mandatory  because  other  mechanisms  of  lobe  sup¬ 
pression  are  absent  here.  This  results  in  relatively  high  sidelobes 
making  image  restoration  more  difficult  (Bewersdorf  et  al.,  2005; 
see  Appendix,  this  chapter). 


2  On  the  other  hand,  2-photon  4Pi  can  only  be  implemented  optimally  if  the 
total  optical  lengths  of  each  side  of  the  interferometer  are  equal,  ±10|im. 
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Apart  from  the  sidelobe  reduction,  4Pi  microscopy  has  also 
undergone  significant  progress  in  terms  of  increasing  the  record¬ 
ing  speed.  Due  to  the  smaller  focal  volume  of  excitation,  less  total 
fluorescence  is  excited  at  a  given  time  point.  Additionally,  the 
axially  narrower  focal  spot  requires  finer  sampling.  The  deconvo¬ 
lution  required  for  the  lobe  removal  also  sets  a  limit  to  the  accept¬ 
able  minimum  signal-to-noise  ratio  (S/N).  Thus,  to  achieve  an 
adequate  S/N,  the  recording  time  may  easily  take  longer  by  the 
factor  by  which  the  volume  resolution  is  reduced  typically  by  a 
factor  of  two.  Lengthened  recording  times  may  be  counteracted  by 
investigating  smaller  regions.  Another  powerful  approach  is  to  par¬ 
allelize  the  recording  process  by  using  many  4Pi  foci  in  parallel, 
as  in  a  multifocal  multi-photon  microscope  (MMM,  see  Chapter 
29,  this  volume)  (Bewersdorf  et  al,  1998).  In  fact,  the  combina¬ 
tion  of  MMM  with  4Pi  microscopy  has  resulted  in  a  fast  axially 
super-resolving  microscope  (Egner  et  al,  2002b,  2004).  In  this 
chapter  we  give  an  overview  of  modern  beam-scanning  4Pi 
microscopy  based  on  the  two  most  recent  realizations:  the  MMM- 
4Pi  as  described  by  Egner  and  colleagues  and  the  Leica  TCS  4PI, 
which  is  a  compact  Type  C  4Pi  microscope  that  has  recently 
become  commercially  available. 


THEORETICAL  BACKGROUND 
The  Point  Spread  Function 

The  imaging  capabilities  of  a  microscope,  such  as  the  resolution, 
are  quantified  by  the  effective  PSF  h(x),  that  is,  the  response  of 
the  microscope  to  a  fluorescent  point.  The  effective  PSF  can  be 
calculated  as  the  product  of  the  excitation  PSF  and  the  detection 


PSF,  h,.JX )  hj„(x).  The  excitation  PSF  h„Jx  )  is  proportional  to  the 
illumination  intensity  squared  in  the  focal  region  and,  apart  from 
a  constant  factor,  can  be  written  as 

KM)  =  I EmQc)  +  M ■  Em  (M  -x)\4 

with 


M 


fl  0  CD 
0  1  0 
,0  0  -1, 


where  E,„(X)  denotes  the  focal  light  field  of  the  illumination  by  a 
single  lens  and  M  is  a  transformation  matrix  accounting  for  the 
counterpropagation  of  the  fields  in  the  4Pi  microscope.  2PE  is 
incorporated  by  the  forth  power  of  the  illumination  light  field 
(instead  of  the  quadratic  power  for  single-photon  excitation). 

For  the  detection  PSF  hd,:(X)  that  expresses  the  spatial  distri¬ 
bution  of  the  detection  efficiency,  the  finite  pinhole  size  has  to  be 
taken  into  account.  hdeff)  also  depends  on  the  4Pi  mode: 


hdrlAPi-.A(x)  =  \Ed„(x)\2  ®  p(x)  for  Type  A 


hJe,APi-c(X)  =  I Ed„(Z)  +  M  ■  Ede,(M -X)\2  ®  p(X)  for  Type  C 


In  analogy  to  the  illumination  light  field  and  due  to  reasons  of 
symmetry,  Edet(X)  represents  the  normalized  amplitude  of  the  detec¬ 
tion  of  the  fluorescence  light  field  emanating  from  the  focal  region. 
To  consider  the  role  of  the  pinhole,  ndel(x)  is  convolved  with  p(x), 
denoting  the  pinhole  function  is  effective  in  the  focal  region.  Using 
a  vectorial  focusing  theory  (Richards  and  Wolf,  1959)  Em(X)  and 
E,,„(x )  can  be  calculated  thoroughly.  Theoretical  PSFs  for  Type 
A  and  C  4Pi  microscopy  are  displayed  in  Figure  30. 1 .  Comparing 
the  extent  of  the  central  maxima  of  the  4Pi  PSFs  with  that  of  the 
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FIGURE  30.1.  The  PSFs  and  OTFs  of  two-photon  excitation  4Pi  microscopy.  In  Type  A  4Pi  microscopy,  the  sample  placed  between  two  coverslips  is  coher¬ 
ently  illuminated  by  two  opposing  objective  lenses  focusing  a  pulsed  infrared  laser  beam  into  the  same  spot.  The  2PE  fluorescence  is  collected  from  a  single 
side  and  focused  onto  a  (variable)  confocal  pinhole.  While  the  excitation  mode  is  the  same,  in  Type  C  4Pi  microscopy,  the  fluorescence  is  collected  by  both 
lenses  and  coherently  combined  at  the  detector.  As  can  be  seen  from  the  calculated  PSFs  and  the  z-responses  shown  as  insets,  the  4Pi  microscope  (water-immer¬ 
sion  lens  1.20  NA,  Ao texc  =  870  nm,  Xotde,  =  510nm,  pinhole  diameter  of  1  Airy  disk)  features  a  four-  to  sevenfold  enhanced  axial  FWHM  compared  to  confocal 
microscopy  (Ao,CJCC  =  488  nm,  Ao ,det  =  510nm,  pinhole  diameter  of  1  Airy  disk).  Due  to  the  combination  of  the  interference  in  excitation  and  detection,  the  Type 
C  mode  exhibits  lower  sidelobes  and  by  a  ~30%  increased  axial  resolution  compared  to  Type  A.  The  superior  resolution  of  4Pi  microscopy  also  becomes  evident 
in  the  much  larger  support  of  the  4Pi-OTFs  as  compared  to  the  confocal  OTF.  Due  to  the  low  sidelobes  in  the  PSF,  the  4Pi-OTF  is  not  only  free  of  missing 
frequencies  but  also  of  weak  amplitudes  within  the  region  of  support. 
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standard  confocal  PSF  makes  the  resolution  increase  evident.  The 
sidelobes  located  ~Ao,„/2  away  are  also  visible. 

In  the  case  of  the  MMM-4Pi,  the  PSF  has  to  be  modified  in 
order  to  take  into  account  the  multiple  foci  and  the  charge-coupled 
device  (CCD)  camera  detection.  However,  the  differences  between 
the  PSFs  of  a  single  focus  Type  A  4Pi  microscope  and  the  actual 
MMM-4Pi  instrument  are  almost  negligible  (Egner  et  ah,  2002b). 


The  z-Response  and  the  Axial  Resolution 

4Pi  microscopy  improves  the  resolution  only  along  the  optic  axis. 
The  lateral  resolution  remains,  strictly  speaking,  the  same. 
However,  one  has  to  realize  that  the  fundamental  reduction  in  the 
thickness  of  the  optical  sections  also  yields  a  significantly 
improved  perception  of  details  in  the  focal  plane,  almost  as  a  ben¬ 
eficial  side  effect.  This  is  because  details  that  are  only  slightly 
(35-75  nm)  away  from  the  focal  plane  are  suppressed  significantly 
better  than  in  a  confocal  microscope,  in  which  a  similar  reduction 
requires  a  displacement  of  >250  nm. 

The  z-response  of  an  infinitely  thin  fluorescent  layer,  I,(z)  = 
f Jli(x)dxdy,  is  a  good  benchmark  for  the  axial  resolution.  The 
insets  in  Figure  30. 1  show  the  ^-responses  for  Type  A  and  Type  C 
4Pi  microscopes  compared  with  that  of  the  standard  confocal 
system.  For  typical  imaging  conditions,  the  height  of  the  sidelobes 
in  2PE  Type  A  4Pi  microscopy  remains  well  below  50%;  for 
the  Type  C  it  is  lowest.  Importantly,  the  sidelobes  increase  dra¬ 
matically  in  height  for  decreasing  aperture  angles.  Typically, 
one  uses  immersion  lenses  of  a  numerical  aperture  (NA)  of  1.4 
(oil),  1.35  (glycerol),  and  1.2  (water),  featuring  semi-aperture 
angles  of  a  =  67°,  68°  and  64°,  respectively.  In  4Pi  microscopy, 
the  aperture  angle  has  only  a  marginal  impact  on  the  axial  width 
of  the  central  spot.  Because  it  is  an  interference  phenomenon  of 
the  two  counterpropagating  waves,  the  z-width  of  the  central 
spot  scales  predominantly  with  the  wavelength  in  the  embedding 
medium.  For  =  870  nm,  Xa^,  =  510nm,  n  =  1.334 

(water  immersion)  and  an  angle  a  of  -65°,  axial  resolutions  of 
~150nm  (Type  A  4Pi)  and  -105 nm  (Type  C  4Pi)  are  reliably 
obtained. 


The  Optical  Transfer  Function 

Representing  the  Fourier  transform  of  the  PSF,  the  OTF  indicates 
which  spatial  frequencies  of  the  sample  are  transmitted  by  the 
microscope  to  form  the  image.  A  poorly  transmitted  spatial  fre¬ 
quency  means  that  two  objects  separated  by  the  corresponding  dis¬ 
tance  (the  reciprocal  value  of  the  frequency)  are  blurred  into  each 
other  in  the  image.  The  periodic  pattern  of  the  sidelobes  of  the  4Pi 
microscope  represents  such  a  correlation  in  space.  The  more  pro¬ 
nounced  the  sidelobes  in  the  PSF  are,  the  lower  the  OTF  ampli¬ 
tude  at  the  matching  spatial  frequency  is.  The  removal  of  the  ghost 
images  in  the  image  data  by  deconvolution  methods  is  in  fact  an 
amplification  of  the  weakly  transmitted  frequencies  so  that  the  final 
OTF  becomes  smooth  and  contiguous.  The  more  these  amplitudes 
are  masked  by  noise  the  more  difficult  is  their  reconstruction. 
Therefore,  by  analyzing  the  frequency  content  of  the  OTF  and  the 
concomitant  noise  level  presented  by  an  object,  one  learns  whether 
a  microscope  and  the  imaging  of  a  particular  object  are  vulnera¬ 
ble  to  artifacts.  The  theoretical  OTFs  for  Type  A  and  Type  C  4Pi 
together  with  their  confocal  counterpart  are  shown  in  Figure  30.1. 
The  4Pi-OTFs  are  contiguous;  there  are  no  missing  gaps  or 
strongly  suppressed  regions.  Compared  to  the  standard  confocal 


OTF,  the  4Pi-OTF  features  a  4-  to  7-fold  expanded  region  of 
support. 

MULTI-FOCAL  MULTI-PHOTON 
MICROSCOPY-4Pi  MICROSCOPY 

The  MMM-4Pi  microscope  (Egner  el  al.,  2002a)  has  been  designed 
to  increase  the  recording  speed  compared  to  a  single-beam  4Pi 
microscope.  It  consists  of  the  two  subunits  sketched  in  Figure  30.2. 
The  head  of  the  MMM-4Pi  includes  the  beam-splitter  and  the 
objective  lenses  that  are  illuminated  through  balanced  interfero¬ 
metric  arms.  Axial  scanning  is  performed  by  moving  the  sample 
along  the  optic  axis  (z)  as  in  standard  confocal  microscopes.  The 
multi-focal  beam  scanning  arrangement  is  shown  in  the  lower  part 
of  Figure  30.2.  The  beam  of  a  mode-locked  titanium:  sapphire 
(Ti:Sa)  laser  is  split  by  an  array  of  microlenses,  producing  an  n  X 
m  array  of  beamlets  that,  after  passing  through  a  dichroic  mirror, 
are  focused  onto  an  array  of  pinholes.  The  pinholes  act  as  point 
sources  for  creating  the  4Pi  illumination  spots  in  the  sample.  After 
passing  the  pinholes,  each  beamlet  is  collimated  by  an  achromatic 
lens  and  directed  towards  a  rapidly  oscillating  galvanometric 
mirror.  The  following  lenses  ensure  that  the  galvanometric  mirror 
is  imaged  into  the  entrance  pupils  of  the  objective  lenses.  The  rapid 
tilt  of  the  mirror  results  in  lateral  scanning  of  the  illumination  4Pi 
foci  in  the  .r-direction;  this  is  the  standard  scanning  procedure  found 
in  state-of-the-art  confocal  beam  scanners.  Scanning  of  the  beam- 
lets  in  the  y-direction  is  accomplished  by  translating  the  combined 


FIGURE  30.2.  Scheme  of  the  MMM-4Pi.  An  array  of  microlenses  (ML) 
divides  the  laser  beam  of  a  Ti:Sa  laser  into  an  array  of  beamlets.  Focused  onto 
a  pinhole  array  (PH),  the  emerging  cleaned  beamlets  act  as  point  sources  for 
the  4Pi  illumination.  The  beamlets  are  deflected  by  the  scan  mirror  (Galvo)  and 
directed  towards  the  4Pi  unit.  The  beam-splitter  (BS)  splits  up  the  beams  and 
the  beamlets  are  focused  into  the  sample  by  both  objective  lenses.  The  small 
CCD  detector  in  the  4Pi  unit  that  detects  laser  light  transmitted  through  both 
objective  lenses  serves  as  an  alignment  tool.  The  fluorescence  light  is  collected 
by  the  left  objective  lens  and  imaged  back  onto  the  pinholes.  A  dichroic  mirror 
(DM)  separates  the  fluorescence  light  from  the  laser  light  and  directs  the  fluo¬ 
rescence  towards  the  CCD  camera.  Axial  scanning  is  performed  by  moving  the 
sample  stage  along  the  optic  axis.  The  lateral  scan  is  performed  by  a  rapid  tilt 
of  the  galvanometric  mirror  in  conjunction  with  a  slower,  coordinated  move¬ 
ment  of  the  microlens-pinhole  array  unit.  By  reflecting  the  detection  light  at 
the  backside  of  the  galvanometric  mirror,  the  fluorescence  foci  are  scanned 
across  the  CCD  camera. 
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microlens  and  pinhole  array.  The  fluorescence  is  collected  by  the 
objective  lens  on  the  left-hand  side  and  deflected  by  the  galvano- 
metric  mirror  in  order  to  be  back-imaged  onto  the  pinhole  array. 
The  diameter  of  the  pinholes  corresponds  to  1.25  times  the  diam¬ 
eter  of  the  back-projected  Airy  disk  (i.e.,  the  diameter  of  the  first 
dark  ring).  This  slight  confocalization  allows  the  suppression  of  the 
axial  sidelobes  and  also  reduces  the  residual  aberrations  stemming 
from  the  microlenses.  After  the  filtering  by  the  pinholes,  the  fluo¬ 
rescence  light  is  reflected  by  the  long-pass  dichroic  mirror  towards 
the  reflective  backside  of  the  galvanometric  mirror.  The  tilt  move¬ 
ment  of  the  latter  scans  the  array  of  fluorescence  beamlets  across  a 
CCD  camera.  Scanning  the  fluorescence  beamlets  with  the  back¬ 
side  of  the  galvanometric  mirrors  locks  the  scan  of  the  excitation 
foci  with  that  of  fluorescence  (Brakenhoff  and  Visscher,  1992). 
n  and  m  range  between  4  and  8,  so  that  a  total  of  N  =  n  X  m  =  16 
to  64  parallel  4Pi  foci  is  typical.  Each  of  them  usually  per¬ 
forms  ~4  X  4pm2  scans  resulting  in  a  field  of  view  of  up  to 
~32  x  32  pm2. 

The  speed  of  the  MMM-4Pi  depends  on  the  degree  of  paral¬ 
lelization  N.  The  parallelization  in  turn  depends  on  the  available 
laser  power.  The  average  power  of  1  to  2  mW  in  each  4Pi  spot  leads 
to  a  focal  peak  intensity  of  ~80  GW/cm2,  which  is  lower  than  the 
~200  GW/cm2  considered  as  an  upper  limit  with  -200  fs  pulses 
(Konig  et  al,  1999).  As  a  mode-locked  Ti:Sa  laser  emits  1  to  2  W 
of  average  power,  a  parallelization  by  1000  foci  rather  than  by 
N  —  16  to  64  appears  to  be  possible.  At  present,  such  a  degree  of 
parallelization  is  precluded  by  the  losses  connected  with  the 
“cleaning”  of  the  laser  output  and  clipping  of  the  beamlet  profile 
by  the  pinhole  array. 

A  fundamental  problem  of  any  parallelized  3D  microscope 
is  the  potential  crosstalk  between  neighboring  illumination  and 
detection  foci.  If  present,  crosstalk  deteriorates  the  axial  resolution 
of  the  system  by  re-enforcing  specific  planes  above  and  beneath 
the  focal  plane  (Egner  and  Hell,  2000;  Egner  et  al.,  2002a).  It  was 
found  that  in  an  MMM-4Pi  featuring  a  quadratic  grid  arrangement, 
the  4Pi-foci  can  be  brought  as  close  as  3.5  Xac  ~  2.8pm  without 
inducing  significant  crosstalk.  Therefore,  parallelization  does  not 
compromise  the  resolution  of  a  properly  designed  MMM-4Pi  setup 
(see  also  Chapter  29,  this  volume). 

Space  Invariance  of  the  Point  Spread  Function 

To  assure  accurate  image  deconvolution,  the  PSF  must  be  the  same 
throughout  the  recorded  volume.  In  4Pi  microscopy,  this  issue  is 
particularly  relevant  because  the  phase  difference  between  the  two 
interfering  wavefronts  has  to  be  kept  constant.  A  (random)  change 
in  phase  would  mean  that  the  PSF  varies  greatly  in  the  sample. 
While  the  envelope  of  the  4Pi-PSF  usually  does  not  significantly 
change  for  thin  cell  layers,  a  change  in  phase  difference  radically 
alters  the  interference  of  the  two  beams  and  hence  the  structure  of 
the  4Pi  spot.  The  following  issues  are  relevant; 

•  The  MMM-4Pi  and  the  Leica  TCS  4PI  are  beam-scanning 
systems.  To  ensure  the  invariance  of  the  PSF  over  the  whole 
field  of  view,  the  two  interferometric  arms  of  the  4Pi  micro¬ 
scope  are  constructed  symmetrically.  Proper  adjustment  en¬ 
sures  that  the  phase  is  constant  over  a  field  >30  pm,  as  can  be 
seen  in  Figure  30.3(A-C)  (Egner  et  al,  2002a). 

•  Slight  differences  in  the  refractive  index  of  immersion  and 
mounting  medium  lead  to  a  --dependent  phase.  For  the  refrac¬ 
tive  index  mismatches  typically  encountered  in  the  sample,  it 
has  been  shown  that  this  relationship  is  linear  over  a  large  dis¬ 
tance  (Egner  et  al.,  1998).  If  the  magnitude  of  this  distortion 


is  known,  a  linear  change  in  the  length  of  one  of  the  4Pi  arms 
with  changing  z-position  of  the  sample  can  correct  for  this 
phase  shift  during  scanning. 

•  Irregular  changes  of  the  refractive  index  within  the  specimen 
also  influence  the  phase  difference  between  the  interfering 
wavefronts. 

To  explore  whether  these  conditions  can  be  fulfilled,  3D  stacks 
of  (mammalian)  Vero  cells  were  recorded  with  water  immersion 
lenses  (Egner  et  al,  2004).  The  cells  were  grown  on  a  coverslip 
coated  with  Oregon  Green  and  then  covered  with  a  similar  cover- 
slip  so  that  they  were  essentially  sandwiched  between  two  ultrathin 
fluorescent  layers.  To  correct  for  the  difference  in  the  refractive 
indices  of  the  aqueous  mounting  medium  (1.34)  and  water  (1.33) 
used  as  immersion  medium,  the  phase  had  to  be  adjusted  linearly 
with  the  z-position.  Figure  30.3(D)  shows  an  xz-section  of  a  typical 
3D  data  stack  containing  a  part  of  the  nucleus.  The  xz-section 
through  the  layers  on  the  coverslips  and  the  resulting  z-profile  dis¬ 
close  the  quality  of  the  interference  between  the  two  wavefronts. 
Profiles  are  extracted  from  three  different  sites  in  the  nuclear 
region,  in  which  the  local  variations  in  refractive  index  are  strongly 
pronounced:  completely  outside  the  nucleus,  at  the  nuclear  periph¬ 
ery,  and  straight  through  the  nucleus  [Fig.  30.3(E)].  Although  slight 
changes  become  apparent,  the  phase  does  not  notably  change  with 
the  axial  translation.  This  can  be  inferred  from  the  comparison 
between  the  profiles  in  the  same  line,  pertinent  to  the  different  cov¬ 
erslips.  Comparison  of  the  profiles  from  the  same  coverslip  reveals 
that  the  presence  of  the  nucleus  introduces  aberrations  leading  to 
elevated  sidelobes,  but  again  the  changes  in  refractive  index  are  not 
pronounced  enough  to  distort  the  PSF  of  the  4Pi  microscope.  In  the 
perinuclear  region,  the  sidelobe  height  is  lower  than  the  critical 
value  of  50%  of  the  central  peak,  but  if  the  nucleus  is  directly 
involved,  the  lobes  are  slightly  elevated.  The  upper  profile  of  Figure 
30.3(E)  also  shows  that  the  refractive  index  challenges  are  relaxed 
when  largely  omitting  the  nucleus-cytosol  interface. 

Live  Mammalian  Cell  4Pi  Imaging 

In  order  to  further  substantiate  the  applicability  of  4Pi  micro¬ 
scopy  to  living  mammalian  cells,  the  distribution  of  two  Golgi 
marker  enzymes  labeled  by  EGPF  inside  living  Vero  cells  have 
been  imaged  at  32°C  with  the  MMM-4Pi  microscope.  The  marker 
enzymes  were  UDP-galactosyl:glycoprotein,l,4-[3-galactosyl 
transferase  (GalT),  which  is  highly  enriched  in  the  middle  and 
trans  Golgi  membranes  and  3'-phosphoadenylyl-sulfate:uronyl-2- 
O-sulfotransferase  (2-OST)  which  is  a  cis  Golgi  marker. 

The  analysis  of  series  of  subsequent  3D  recordings  of  the 
GalT-EGFP  distribution  in  a  live  Vero  cell  at  32°C  with  the  MMM- 
4Pi  showed  that  the  movement  of  the  living  Golgi  apparatus  does 
not  pose  particular  challenges.  The  slight  morphological  changes 
observed  between  two  subsequent  recordings  occur  on  a  much 
larger  time  scale  than  the  scanning  of  the  4Pi-PSF  across  each  posi¬ 
tion  in  the  cell.  Thus,  they  did  not  compromise  the  space  invari¬ 
ance  of  the  PSF  and  the  slight  changes  did  not  preclude  the  proper 
deconvolution  of  the  data  to  achieve  a  3D  resolution  of  -lOOnrn. 
In  addition,  the  repeated  recording  of  the  Golgi  apparatus  did  not 
show  a  significant  reduction  in  image  brightness.  Therefore,  photo- 
bleaching  of  EGFP  was  not  an  issue  in  these  samples.  The  cells 
remained  viable  even  after  prolonged  mounting  in  the  custom-built 
chamber  for  several  days.  The  comparatively  large  space  of 
175  pm  between  the  two  coverslips  and  the  addition  of  a  suitable 
air/COi  mix  is  adequate  to  maintain  division  and  propagation 
without  apparent  degradation. 
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FIGURE  30.3.  Basics  of  MMM-4Pi  imaging.  (A)  .xy  image  of  a  fluorescent  polydiacetylene  monolayer  taken  from  a  3D  stack.  The  chessboard  appearance  is 
due  to  the  different  intensities  in  the  neighboring  4Pi  foci;  the  tilted  stripes  are  inherent  to  the  polydiacetylene  monolayer.  (B)  Axial  intensity  profiles,  f-(z)- 
through  the  3D  stack  exhibiting  a  sharp  maximum  and  two  lobes,  recorded  at  three  different  coordinates  (x,y).  The  spatial  invariance  of  the  three  profiles  indi¬ 
cates  that  the  4Pi-PSF  is  constant  over  the  whole  field  of  view,  irrespective  of  the  microlens  used.  Fast  linear  one-step  deconvolution  can  be  applied  throughout 
the  field  of  view  to  remove  the  sidelobes  (C).  (D)  xz-section  recorded  at  the  periphery  of  the  nucleus  of  living  Vero  cell  sandwiched  between  two  coverslips  each 
thinly  coated  with  a  fluorescence  layer.  The  nucleus  (N)  is  highlighted  by  Floechst  counterstaining  and  outlined;  the  region  where  the  Golgi  resides  (G)  is  briefly 
sketched.  The  cell  is  significantly  larger  than  the  whole  xz-section.  The  three  vertical  lines  on  either  side  (orange)  are  due  to  the  interference  between  the  coun- 
terpropagating  focused  fields,  representing  the  main  maximum  of  the  4Pi-PSF  and  the  two  axially  offset  sidelobes.  (E)  Enlarged  intensity  profiles  of  the  axial 
responses  of  the  microscope  to  the  thin  fluorescent  layers,  revealing  the  height  of  the  lobes  and  the  relative  phase  in  greater  detail.  Outside  and  at  the  edge  of 
the  nucleus,  the  primary  lobes  are  <50%.  Focusing  through  the  nucleus  slightly  increases  the  lobes  and  induces  a  noticeable  but  comparatively  small  phase  shift. 
The  parallelism  of  the  orange  lines  in  the  displayed  xz-section  indicates  that  the  cell  does  not  gravely  distort  the  interference  between  the  wavefronts. 


The  inset  in  the  upper  left-hand  comer  of  Figure  30.4(A) 
displays  a  conventional  overview  xv-image  of  the  GalT- 
EGFP-labeled  Golgi  apparatus  located  close  to  the  nucleus  coun- 
terstained  with  the  fluorophore  Hoechst.  The  central  panel  of 
Figure  30.4(A)  displays  the  corresponding  3D  surface-rendered 
image  at  100 nm  resolution.  The  3D  image  clearly  reveals  ribbons, 
fractioned  stacks,  as  well  as  smaller  tubular  and  vesicular  sub¬ 
compartments.  Furthermore,  protrusions  and  invaginations  in  the 
200  nm  range  can  be  discerned.  Figure  30.4(B)  shows  a  typical  dis¬ 
tribution  of  2-OST-EGFP  in  interphase  cells.  This  image  also  dis¬ 
plays  a  convoluted  structure  of  the  Golgi  apparatus,  similar  to  that 
observed  for  GalT.  We  think  that  in  the  near  future,  the  discrimi¬ 


nation  of  intra-Golgi  localization  of  2-OST  and  GalT  will  be  pos¬ 
sible  by  applying  two-color  4Pi  microscopy. 

TYPE  C  4Pi  MICROSCOPY  WITH  THE 
LEICA  TCS  4PI 

The  Leica  TCS  4PI  (Gugel  et  al. ,  2004)  consists  of  a  commercially 
available  confocal  microscope  (Leica  Microsystems  Fleidelberg, 
Mannheim,  Germany  )  combined  with  a  4Pi  unit.  The  4Pi  unit  is 
tightly  mounted  to  the  microscope  turret  to  maintain  all  the  capa¬ 
bilities  of  the  scanning  confocal  unit  in  the  microscope  body  (Fig. 
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FIGURE  30.4.  Golgi  apparatus  as  obtained  through 
4Pi  confocal  imaging  of  GalT-EGFP  (A)  and  2-OST- 
EGFP  (B)  in  living  Vero  cells  at  ~100nm  axial  resolu¬ 
tion.  The  insets  show  epi-fluorescence  images  to 
correlate  the  Golgi  with  the  nucleus.  The  Hoechst  stain¬ 
ing  of  the  nucleus  indicates  that  the  cells  were  in  inter¬ 
phase.  This  was  not  the  case  for  the  GalT-EGFP 
transfected  cell  in  (C).  Though  this  cell  is  apoptotic,  the 
data  clearly  pinpoint  the  ability  of  the  4Pi  microscope 
to  resolve  small  structures  like  the  cavities,  which 
become  apparent  when  the  upper  caps  of  the  balloon¬ 
like  structures  are  removed  (D). 
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FIGURE  30.5.  Sketch  of  the  Leica  TCS  4PI.  Excitation  light  originating  from  the 
microscope  stand  is  divided  by  the  beam-splitter  (BS)  and  focused  onto  the  same  spot 
by  the  opposing  objective  lenses  Ol  and  02.  The  lenses  LI,  L2,  L3  and  the  mirrors 
Ml,  M2,  M3  form  the  intermediate  optical  system,  ensuring  that  the  two  scanning 
pivotal  points  coincide  with  the  entrance  pupils  of  the  two  objective  lenses.  Fluores¬ 
cence  is  collected  by  both  lenses,  recombined  at  BS  and  directed  towards  the  micro¬ 
scope  stand.  Dispersion  compensation  over  a  large  wavelength  range  is  ensured  by 
movable  optical  wedges  in  the  lower  interferometric  arm  whose  thickness  is  compen¬ 
sated  by  a  glass  window  in  the  upper  arm.  Scanning  is  performed  by  a  standard  con- 
focal  microscope  unit. 
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30.5).  For  2PE,  the  beam  of  a  mode-locked  Ti:Sa  laser  is  coupled 
into  the  confocal  microscope  and  directed  towards  the  4Pi  unit, 
where  it  is  split  into  two  beams  of  equal  intensity  by  the  beam¬ 
splitter  BS  and  focused  into  the  sample  by  the  objective  lenses  01 
and  02.  To  maintain  the  beam-scanning  capabilities  of  the  confo¬ 
cal  microscope,  the  former  exit  pupil  of  the  confocal  microscope 
is  imaged  into  the  entrance  pupils  of  the  objective  lenses  by  the 
telescope  optics  LI,  L2,  and  L3. 

In  order  to  minimize  the  effects  of  thermal  fluctuations,  the  4Pi 
unit  is  designed  symmetrically.  The  objective  lenses  are  mounted 
on  a  mechanical  translation  stage  and  a  piezo-mechanical  transla¬ 
tion  stage,  respectively.  This  allows  the  alignment  of  the  two  objec¬ 
tive  lenses  along  a  common  optic  axis  and  the  concurrence  of  their 
foci.  The  excitation  beams  can  be  aligned  parallel  to  the  optic  axis 
of  the  objective  lenses,  as  well  as  centric  to  the  entrance  pupil  of 
the  lenses,  by  adjusting  the  mirrors  Ml,  M2,  M3,  and  the  beam¬ 
splitter.  The  mirrors  M2,  mounted  on  a  mechanical  translation 
stage,  and  M3,  on  a  piezo-translation  stage,  also  allow  the  adjust¬ 
ment  and  control  of  the  balance  of  the  interferometric  arms. 

Lateral  scanning  is  performed  by  the  usual  fast  tilting  of  the 
beam  in  the  pupil  of  the  lenses.  Axial  scanning  is  accomplished  by 
translating  the  sample  along  the  optic  axis  with  a  piezo  scanning 
stage.  The  optical  design  of  the  4Pi  unit  ensures  invariance  of  the 
4Pi-PSF  across  a  field  of  50  pm  diameter.  The  sample  stage  is  hor¬ 
izontally  oriented,  as  in  the  case  of  standard  confocal  microscopes. 
In  fact,  the  typical  scanning  conditions  of  a  standard  confocal 
scanner  are  retained. 

The  fluorescence  is  collected  with  both  objective  lenses, 
merged  at  the  beam-splitter  and  partially  reflected  backwards  to 
the  confocal  microscope,  producing  a  Type  C,  4Pi  microscope.  In 
contrast  to  the  Type  A  mode,  the  beam-splitter  divides  the  fluo¬ 
rescence  light  into  a  constructively  interfering  part  that  is  directed 
towards  the  detection  pinhole  (corresponding  to  a  detection  PSF 
consisting  of  a  central  peak  and  symmetrically  arranged  sidelobes) 
and  a  destructive  part  (featuring  a  central  minimum)  emitted  at  the 
fourth  side  of  the  beam-splitter  cube.  The  light  emerging  from  a 
homogeneously  fluorescing  sample  is  in  this  way  divided  into  two 
beams  of  half  the  available  brightness.  This  loss  is  offset  by  the 
doubled  detection  efficiency  achieved  by  the  use  of  two  objective 
lenses.  However,  the  combination  of  constructive  4Pi  excitation 
and  constructive  detection  sharpens  the  central  peak  of  the  effec¬ 
tive  PSF  and  lowers  the  sidelobes. 

Even  though  the  same  optical  components  are  used  in  both 
interferometric  arms  of  the  4Pi  unit,  the  relevant  thickness  of 
the  glass  components  is  not  exactly  the  same.  In  a  Type  A  4Pi 
microscopy,  the  resulting  optical  path  difference  can  be  adjusted 
by  moving  one  of  the  mirrors  along  the  optic  axis,  thereby  chang¬ 
ing  the  length  of  the  air  spacing.  This  is  not  possible  in  Type  C  4Pi 
microscopy.  Due  to  the  different  dispersion  in  the  glasses  and  in 
air,  dispersion  compensation  is  necessary  to  balance  the  arms  for 
multiple  wavelengths.  This  is  implemented  by  a  pair  of  adjustable 
glass  wedges  in  one  of  the  interferometric  arms.  The  mean  thick¬ 
ness  of  the  wedge-pair  is  precompensated  in  the  other  arm  by  a 
glass  window  of  the  average  thickness  of  the  wedges.  The  effec¬ 
tive  dispersion  can  be  tuned  by  displacing  the  wedges  against  each 
other. 


Resolution 

The  experimentally  achievable  resolution  and  sidelobe  heights  of 
4Pi  microscopy  are  shown  in  Figure  30.6  by  measured  "-responses 
of  the  Leica  TCS  4PI  for  water-immersion  lenses  (PL  APO 
63x/1.20W  CORR,  Leica  Microsystems,  Wetzlar,  Germany)  and 
glycerol-immersion  lenses  (HCX  PL  APO  lOOx/1.35  GLYC 
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FIGURE  30.6.  Experimental  .--responses  for  water  and  glycerol-immersion 
lenses  (PL  APO  63x/1.20W  CORR  and  HCX  PL  APO  lOOx/1.35  GLYC 
CORR,  Leica  Microsystems,  Wetzlar,  Germany).  The  measurement  for  the 
water-immersion  lenses  was  carried  out  with  a  fluorescent  layer  consisting  of 
Oregon  green  dye  molecules  excited  with  laser  light  at  a  wavelength  of  880 
nm.  The  z-response  of  the  glycerol-immersion  lenses  was  obtained  with  a  poly¬ 
diacetylene  layer  excited  at  a  wavelength  of  840  nm.  The  central  wavelength 
of  the  emission  light  is  565  nm  in  the  case  of  the  polydiacetylene  layer,  while 
it  is  526  nm  for  the  Oregon  green  layer.  The  pinhole  diameter  was  set  to  the 
size  of  the  backprojected  Airy  disk  at  the  detection  wavelengths  used.  The  z- 
response  of  the  water-immersion  lens  was  recorded  with  an  avalanche  photo¬ 
diode  (APD),  which  results  in  a  higher  signal-to-noise  ratio  compared  to  the 
measured  .--response  of  the  glycerol-immersion  lens  recorded  with  a  photo¬ 
multiplier.  Lor  comparison,  theoretical  calculations  are  shown.  To  account  for 
apodization  at  the  periphery  of  the  aperture,  theoretical  NAs  of  1.15  and  1.30 
were  taken  for  the  water-  and  the  glycerol-immersion  lens,  respectively.  The 
first  minima  in  the  OTP  measured  with  the  glycerol-immersion  lens  exhibits  a 
value  of  0.4.  As  a  result  of  the  higher  sidelobes  of  the  --response,  the  value  of 
the  OTP  measured  with  the  water-immersion  lens  decreases  to  0.22  at  the  first 
minima. 


CORR,  Leica  Microsystems,  Wetzlar,  Germany).  For  both  mea¬ 
surements,  a  detection  pinhole  diameter  corresponding  to  the  size 
of  the  backprojected  Airy  disk  of  the  fluorescent  light  was  chosen. 
A  comparison  of  the  measured  axial  responses  with  the  theoreti¬ 
cal  ones  (Fig.  30.6)  shows  remarkable  agreement.  The  full-width 
at  half-maximum  (FWHM)  of  the  --response  was  measured  as 
105nm  and  112nm  for  the  glycerol-  and  water-immersion  lens, 
respectively.  In  Type  C  4Pi  microscopy,  the  height  and  shape  of 
the  sidelobes  are  determined  by  the  ratio  of  the  excitation  and  the 
detection  wavelength.  The  --response  of  the  glycerol  lens  features 
lower  sidelobes  in  this  experiment  because  of  the  slightly  more 
suitable  wavelength  ratio,  but  also  because  the  glycerol-immersion 
lenses  feature  an  aperture  angle  that  is  larger  by  4°  than  the  water- 
immersion  counterparts. 

The  OTF  of  the  Type  C  4Pi  microscope  along  the  reciprocal 
optic  axis  was  calculated  from  the  theoretical  and  experimental  z- 
responses  by  Fourier  transformation.  It  is  completely  contiguous 
with  low  intraband  modulation.  In  fact,  Type  C  4Pi  shows  the 
best-filled  OTF  support  reported  with  these  types  of  microscopes 
(Nagorni  and  Hell,  2001).  This  well-filled  OTF  makes  it  suitable 
for  routine  operation.  The  largest  transmitted  object  frequencies 
range  between  4514m"1  and  50|4nr‘.  This  corresponds  to  a  7-fold 
enlargement  of  the  OTF  compared  to  the  confocal  microscope 
(714m"1). 
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Type  C  4Pi  Imaging  in  Living  Cells 

The  suitability  of  Type  C  4Pi  microscopy  in  living  cells  was 
demonstrated  by  recording  3D  images  of  the  mitochondrial  com¬ 
partment  in  yeast  cells  (Gugel  et  al.,  2004).  The  mitochondrial 
matrix  of  Saccharomyces  cerevisiae  was  labeled  with  green  fluo¬ 
rescent  protein  (GFP).  The  cells  were  embedded  in  PBS  with  1% 
low  melting  agarose  and  mounted  between  two  coverslips.  Apart 
from  immobilizing  the  yeast  cells,  the  agarose  slightly  increases 
the  refractive  index  of  the  aqueous  medium  from  1.33  to  1.34.  This 
reduces  the  effect  of  refractive  index  changes  in  the  sample.  The 
slight  index  mismatch  between  the  medium  and  the  immersion 
liquid  induces  a  linear  phase  shift,  which  was  actively  compen¬ 
sated.  The  phase  of  the  PSF  in  the  stack  was  monitored  with  a  flu¬ 
orescent  layer  placed  on  one  of  the  coverslips.  The  phase  variation 
of  the  PSF  throughout  the  whole  stack  was  found  to  be  less  than 
jt/6.  Therefore,  aberrations  introduced  by  the  biological  sample 
were  negligible  in  this  case.  The  measurements  were  performed  in 
a  bidirectional  scanning  mode  with  a  line  frequency  of  400  FIz. 
Due  to  the  fast  beam-scanning  system,  imaging  speed  is  only 
determined  by  the  sensitivity  of  the  scanner.  A  typical  mitochon¬ 
drial  stack  of  6.25  x  6.25  x  4.5  pm1  (x  x  y  x  z)  in  size  was  recorded 
in  1.5  min.  During  this  time  span,  the  mitochondria  did  not 
change  significantly.  The  wavelength  of  the  excitation  light  was 
880  nm. 

The  recorded  data  stacks  were  deconvolved  with  theoretical 
PSFs  calculated  according  to  the  parameters  used  in  the  experi¬ 
ment,  that  is,  a  detection  pinhole  diameter  corresponding  to  the 
size  of  the  backprojected  Airy  disk  of  the  light  and  an  emission 
wavelength  of  510nm.  The  sidelobes  obtained  by  recording  the 
yeast  mitochondria  are  of  the  same  height  as  the  sidelobes  mea¬ 
sured  for  the  fluorescent  layer,  that  is,  below  30%  of  the  main  peak 
intensity.  Figure  30.7  shows  a  typical  surface-rendered  recon¬ 
struction  of  a  recorded  stack.  The  data  set  was  restored  with  non¬ 
linear  image  processing  using  the  Richardson-Lucy  algorithm 
[Fig.  30.7(A)]  as  was  applied  for  Type  A  4Pi  images  in  other  pub¬ 
lications  (Nagorni  and  Hell,  2001).  The  good  agreement  of  the 
measured  axial  response  with  the  theory  and  the  low  sidelobes  of 
the  Type  C  4Pi  PSF  enabled  the  recorded  data  stack  to  be  linearly 
deconvolved.  The  linear  filtering  with  a  Wiener  filter  and  an 


FIGURE  30.7.  Mitochondrial  network  of  a  living  yeast  cell  recorded  with  the 
Leica  TCS  4PI.  The  3D  image  displayed  was  obtained  after  nonlinear  (A)  and 
linear  (B)  deconvolution  and  subsequent  surface  rendering  of  the  raw  3D  data. 
Note  the  good  agreement  of  the  two  images  indicating  the  efficiency  of  the 
mathematically  simpler  and  arguably  more  reliable  linear  deconvolution.  The 
mitochondrial  matrix  was  labeled  with  EGFP.  The  diameter  of  the  mitochon¬ 
drial  network  shown  is  -5  pm. 


adapted  theoretical  OTF  not  only  removed  the  sidelobes  but  also 
increased  the  axial  resolution  to  slightly  better  than  80  nm.  Figure 
30.7(B)  illustrates  the  surface-rendered  data  set  of  the  same  mito¬ 
chondrial  compartment,  but  this  time  attained  through  linear  image 
restoration.  The  similarity  of  the  results  obtained  with  linear  fil¬ 
tering  and  nonlinear  deconvolution  demonstrates  the  applicability 
of  linear  filtering  in  Type  C  4Pi  microscopy. 

SUMMARY  AND  OUTLOOK 

With  the  latest  instruments,  MMM-4Pi  and  Leica  TCS  4PI,  4Pi 
microscopy  has  evolved  from  a  laboratory  imaging  setup  to  an 
instrument  for  super-resolved  3D  imaging  of  cells.  Multiple 
images  of  the  Golgi  apparatus  in  living  Vero  cells  (Egner  et  al., 
2004)  as  well  as  extensive  studies  of  the  topology  of  the  mito¬ 
chondrial  network  in  live  yeast  cells  (Egner  et  al.,  2002b)  have 
exhibited  reliable  everyday  operation.  4Pi  microscopy  has  been 
shown  to  work  with  oil-immersion  as  well  as  with  water-  and 
glycerol-immersion  lenses  and  can  thus  be  applied  under  many 
biological  imaging  conditions  for  individual  cells.  However,  at  the 
present  state  of  development,  the  allowable  sample  thickness  is 
limited  by  two  factors.  First,  the  free  working  distance  of  the  high 
NA  lenses  used  sets  an  upper  limit  to  the  maximum  distance 
between  the  two  coverslips  (typically  -80-200  pm).  Second,  thick 
strongly  scattering  specimens  cause  wavefront  aberrations  com¬ 
promising  the  4Pi  interference.  While  for  single  cell  layers  4Pi 
microscopy  has  been  shown  to  work  reliably,  the  applicability  of 
4Pi  microscopy  to  somewhat  thicker  samples  is  left  to  be  solved 
in  the  future.  The  usage  of  adaptive  optics  (Booth  et  al.,  2002)  may 
be  very  helpful  in  that  regard. 

It  is  not  absolutely  necessary  to  mount  the  specimen  between 
two  coverslips:  the  upper  lens  can  be  a  dipping  lens.  Combined 
with  a  standard  water-immersion  lens  of  the  same  NA  and  magni¬ 
fication  in  the  lower  arm,  the  Leica  TCS  4PI  can  be  used  with  only 
one  coverslip.  This  facilitates  chemical,  thermal,  and  also  limited 
mechanical  stimulation  of  the  cells.  Chambers  have  been  devel¬ 
oped  allowing  the  buffered  medium  to  be  exchanged  (Egner  et  al. , 
2004),  thus  enabling  the  investigation  of  time-dependent  changes 
as  well  as  the  effect  of  certain  stimuli  over  many  hours. 

In  conclusion,  with  the  current  4Pi  microscopes  it  is  possible 
to  record  3D  data  stacks  from  living  cells  in  typically  1  to  20  min 
with  an  axial  resolution  of  better  than  100  nm.  Strictly  speaking, 
the  lateral  resolution  does  not  change,  but  the  significantly 
improved  optical  sectioning  leads  to  an  enhanced  clarity  of  the 
details  located  in  the  focal  plane.  The  MMM  version  of  4Pi 
microscopy  has  the  potential  of  imaging  a  whole  cell  in  less  than 
a  minute.  By  combining  coherent  excitation  and  detection,  the 
Type  C  mode  in  the  Leica  TCS  4PI  allows  imaging  with  a  state- 
of-the-art  confocal  scanner  featuring  a  confocal  sectioning  capa¬ 
bility  of  -80  nm. 

A  further  increase  in  resolution  can  be  achieved  by  merging 
4Pi  microscopy  with  other  independent,  resolution-enhancing 
techniques.  Taking  into  account  that  the  axial  resolution  is  now 
slightly  better  than  its  transverse  counterpart,  rotating  the  sample 
(Bradl  et  al,,  1992;  Heintzmann  and  Cremer,  2002)  may  allow  a 
tomographic  version  of  4Pi  microscopy  which  would  further 
increase  the  3D  resolution  optically.  The  combination  of  4Pi  and 
STED  microscopy  has  already  been  demonstrated  (Dyba  and  Hell, 
2002;  Dyba  et  al.,  2003)  to  attain  an  axial  resolution  of  33  nm. 
Being  based  on  a  reversible  saturable  transition  (RESOLFT)  (Hell, 
2004;  Hell  et  al.,  2004),  STED  has  no  inherent  conceptual  resolu¬ 
tion  limit.  The  limits  are  rather  given  by  the  limited  performance 
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of  the  fluorophores  in  terms  of  emission  cycles,  quantum  effi¬ 
ciency,  and  photobleaching.  For  living  cells,  this  demand  will  be 
increased  due  to  rapid  movements  of  proteins  and  organelles 
within  the  sample.  If  these  issues  can  be  satisfactorily  addressed 
in  the  future,  the  synergistic  combination  of  STED  or  another 
RESOLFT  mechanism  is  likely  to  provide  true  optical  resolution 
of  <10  nm  along  the  optic  axis.  At  the  same  time,  an  increase  in 
image  acquisition  speed  through  more  sensitive  and  parallelized 
detection  is  expected. 
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interference  at  the  detector  which  is  a  CCD  camera.  The  OTF  of 
the  I5M  is  much  more  structured  than  that  of  the  2PE  Type  A  and 
Type  C  4Pi  microscopes  (see  Fig.  30.A1)  and  therefore  I5M  trans¬ 
fers  data  at  some  of  these  extended  spatial  frequencies  only 
weakly.  To  visualize  these  components  in  the  PM-OTF,  the  inset 
of  Figure  30. A 1  displays  the  same  OTF  after  normalization  to  10% 
of  its  maximum  value  (Bewersdorf  et  al. ,  2005).  If  the  weakly 
transferred  frequencies  (at  or  even  below  the  noise  level)  fall 
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FIGURE  30.A1.  OTF  of  the  I5M.  The  inset  shows  the  same  OTF  normalized 
to  10%  of  the  maximum  to  highlight  the  weak  regions  of  the  OTF.  The  white 
dashed  line  shows  the  support  of  a  standard  confocal  fluorescence  microscope. 


FIGURE  30. A2.  Measuring  diameters  in  the  subresolution  range  with  4Pi 
microscopy.  (A)  and  (B)  show  the  intensity  profiles  through  a  laterally  oriented 
rod  with  a  diameter  of  40  and  400  nrn,  respectively.  The  depth  of  the  minima 
of  the  profiles  is  correlated  to  the  thickness  of  the  rod.  By  measuring  the  depth 
of  the  minima,  the  thickness  of  rod-like  objects  such  as  the  yeast  mitochondria 
can  be  derived  (C). 


within  the  region  of  support  of  the  OTF,  periodic  artifacts  arise  in 
the  image.  The  white  dashed  line  in  Figure  30. A 1  displays  the 
support  of  the  OTF  of  a  normal  confocal  microscope.  For  the 
FM-OTF  calculations,  a  fluorescence  wavelength  of  510nm,  a 
single-photon  excitation  wavelength  at  480  nm,  and  a  1.20  NA 
water-immersion  lens  were  assumed.  The  height  of  the  singularity 
in  the  center  corresponds  to  an  imaging  size  of  -  1 0  p  m  cubed. 


Additional  Information  from  the  4Pi-PSF 

The  sharper  spot  of  the  4Pi  microscope  can  be  used  to  increase  the 
localization  accuracy  of  point-like  objects  in  tracking  experiments, 
or  to  enhance  the  measurement  of  distances  between  two  known 


objects.  Likewise,  knowledge  about  the  symmetry  of  the  feature 
of  interest  can  be  used  to  determine  its  center  of  mass  and,  there¬ 
fore,  its  position.  The  precision  of  the  position  measurement  is  not 
challenged  by  diffraction  and  may  far  exceed  the  resolution 
(Gelles  et  al,,  1988;  Bornfleth  et  al.,  1998;  Lacoste  et  al.,  2000; 
Thompson  et  al.,  2002).  The  precision  of  this  method  scales, 
however,  with  the  resolution  and  thus  benefits  from  the  narrow  4Pi 
focus.  In  fact,  localization  accuracies  below  1  nm  have  been 
demonstrated  with  4Pi  microscopy  (Schmidt  et  al,  2000).  Addi¬ 
tionally,  the  4Pi-PSF  allows  one  to  measure  object  thicknesses 
down  to  60  nm.  Figure  30. A2  illustrates  the  determination  of  the 
diameter  of  mitochondria  in  a  yeast  cell.  In  this  example,  it  was 
possible  to  measure  the  average  diameter  with  an  accuracy  of 
~5nm  (Egner  et  al,,  2002b). 
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THE  RESOLUTION  ISSUE 

Although  most  specifically  labeled  cellular  constituents  can  readily 
be  detected  in  a  conventional  fluorescence  light  microscope,  their 
submicron-scale  structure  cannot  be  perceived.  For  example, 
despite  the  fact  that  many  proteins  of  the  inner  mitochondrial 
membrane  can  be  labeled  by  tagging  with  the  green  fluorescent 
protein  (GFP),  the  cristae  are  too  small  to  be  represented  in  an 
image  recorded  with  light. 

Resolution  is  the  minimal  distance  at  which  a  microscope  may 
discriminate  two  or  more  features  of  the  same  kind  (Abbe,  1873). 
Therefore,  resolution  must  not  be  confused  with  localization  pre¬ 
cision.  While  it  may  easily  be  possible  to  localize  the  center  of  the 
fluorescence  patch  generated  by  a  membrane-labeled  synaptic 
vesicle  with  nanometer  precision  in  a  microscope,  its  visualization 
as  a  30  to  50  nm  hollow  sphere  fails  because  the  latter  requires 
nanometer  scale  resolution.  In  fact,  conventional  epi-fluorescence 
microscopy  may  not  even  resolve  two  synaptic  vesicles  touching 
each  other.  Similarly,  it  may  be  possible  to  track  the  trajectory  of 
a  single  virus  particle  with  nanometer  precision  but  still  its  inner 
structure  remains  elusive  to  light  microscopy. 

The  resolution  limitation  is  ultimately  rooted  in  the  phenome¬ 
non  of  diffraction  (Abbe,  1873).  Loosely  speaking,  focusing  of 
light  always  results  in  a  blurred  spot  (Born  and  Wolf,  2002)  whose 
size  determines  the  resolution.  According  to  Abbe,  the  transverse 
(i.e.,  focal  plane)  full-width  at  half-maximum  (FWHM)  of  the 
focal  spot  is  given  by 

Ax,  Ay  =  — ^ -  (1) 

2«  sm  a 

with  X,  n,  and  a  denoting  the  wavelength,  the  refractive  index,  and 
the  semi-aperture  angle  of  the  objective  lens,  respectively.  Along 
the  direction  of  light  propagation  (i.e.,  the  optic  axis),  the  spot  size 
is  about  A"  =  2X/(nsm2  a)  (Born  and  Wolf,  2002). 

In  the  visible  optical  range,  the  most  sophisticated  immersion 
lenses  feature  a  maximum  angle  of  a  =  73°;  the  shortest  live  cell 
compatible  wavelength  is  around  400  nm  which  is  almost  in  the 
near  ultraviolet  (UV)  (Konig  et  al.,  1996).  Thus,  it  seemed  obvious 
that  if  visible  light  and  regular  lenses  were  to  be  used,  the  resolu¬ 
tion  in  the  focal  plane  Ax,Ay  would  always  be  poorer  than  150nm 
(Pohl  and  Courjon,  1993).  In  the  axial  direction  the  resolution  is 
even  worse  (Az  >  500  nm).  In  recent  years  significant  improve¬ 
ments  of  axial  resolution  have  been  achieved  through  the  coherent 
use  of  two  opposing  lenses  as  is  realized,  for  example,  in  4Pi 
microscopy  (see  Chapter  30,  this  volume).  However,  these  systems 


are  still  limited  by  diffraction.  In  this  chapter,  we  report  on  con¬ 
cepts  for  radically  overcoming  the  diffraction  barrier  and  attaining 
a  resolution  of  few  nanometers:  a  figure  that  has  been  considered 
impossible  with  focused  light  for  more  than  a  century. 


BREAKING  THE  DIFFRACTION  BARRIER:  THE 
CONCEPT  OF  REVERSIBLE  SATURABLE 
OPTICAL  FLUORESCENCE  TRANSITIONS 

Since  the  inception  of  nonlinear  optics  (Shen,  1984),  it  has  been 
speculated  that  a  nonlinear  relationship  between  the  applied  inten¬ 
sity  and  the  measured  (fluorescence)  signal  could  —  at  least  in 
principle  —  expand  the  resolution  capabilities  of  a  focusing  (far- 
field)  optical  system.  However,  these  notions  remained  vague  and 
without  consequence  because  a  concrete  physical  recipe  could  not 
be  given.  In  fact,  the  multi-photon  processes  that  had  initially  been 
considered  for  significantly  improving  the  spatial  resolution  turned 
out  to  be  unsuitable.  Therefore,  it  was  not  until  the  early  1990s 
that  concrete  physical  concepts  appeared  for  breaking  the  diffrac¬ 
tion  resolution  barrier  with  focused  light  (Hell  and  Wichmann, 
1994;  Hell  and  Kroug,  1995;  Hell,  1997).  In  fact  these  concepts  can 
be  viewed  as  a  family  of  concepts  that  utilize  reversible  saturable 
optica/  (/luorescence)  transitions,  which  we  now  name  reversible 
saturable  optical  fluorescence  transitions  (RESOLFT).  They  can  be 
described  in  a  common  formalism  (Hell,  2003). 

Let  us  assume  a  fluorescent  molecule  with  two  distinct  states 
A  and  B,  whereby  the  transition  from  A  — >  B  can  be  optically 
induced  at  a  rate  kAB=  a  I  [Fig.  31.1(A)].  The  variables  o  and  / 
denote  the  transition  cross-section  and  the  light  intensity,  respec¬ 
tively.  The  rate  for  the  reverse  transition  B  — >  A  is  denoted  with 
kBA.  It  may  be  driven  by  light,  by  a  chemical  reaction,  by  heat,  or 
any  other  means,  or  simply  be  spontaneous.  The  kinetics  of  the 
molecular  states  is  described  by  dNJdt  =  -dNB/dt  =  kBANB-  kABNA, 
with  Nab  denoting  the  normalized  population  probability  of  each 
state.  After  a  time  period  t  »  (kAB  +  kBA)~l,  the  populations  have 
reached  a  dynamical  equilibrium  at  NX =  kBA/(kAB+  kBA).  The  mol¬ 
ecule’s  probability  to  be  in  A  or  B  basically  depends  on  kAB  and 
hence  on  I.  At  the  saturation  intensity  /„«  =  kBA/a,  we  have  equal 
probability  NX  =  1/2.  Increasing  I »  Isa,  renders  kAB  »  kBA,  so  that 
the  molecule  is  virtually  shifted  to  B:  NX  — >  0. 

Figure  31.1(B)  illustrates  how  this  behavior  can  be  exploited 
for  creating  arbitrarily  sharp  regions  of  state  A  molecules.  The 
scheme  in  Figure  31.1(B)  is  one-dimensional  (x),  but  can  be 
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FIGURE  31.1.  Breaking  the  diffraction  barrier  by  reversible  saturable  optical  transitions  (RESOLFT)  requires  (A)  two  states  A  and  B  of  a  label  that  are  dis¬ 
tinct  in  their  optical  properties.  The  optical  transition  A  — >  B  takes  place  at  a  rate  kAB  =  ol  that  is  proportional  to  the  light  intensity  7  applied.  The  reverse  tran¬ 
sition  B  — >  A  of  rate  kBA  brings  the  label  back  to  its  initial  state.  (B)  The  profiles  1  to  4  show  the  spatial  region  in  which  the  label  is  allowed  to  be  in  state  A,  if 
the  region  is  subject  to  a  standing  wave  of  light  with  peak  intensities  70  =  10,  50,  100,  and  500  times  /„,  and  with  a  zero  at  xr  Increasing  70  ensures  that  the  region 
in  which  the  label  may  reside  in  A  is  squeezed  down,  in  principle,  indefinitely.  If  A  is  the  fluorescent  state  of  the  label,  this  ultrasharp  region  functions  as  the 
effective  fluorescent  spot  of  the  microscope  and  Ay  is  its  FWHM.  The  creation  of  a  fluorescence  image  requires  scanning  that  is  moving  the  zero  along  the  x- 
axis  with  subsequent  storage  of  the  recorded  fluorescence.  If  B  is  the  fluorescent  state,  then  the  ultrasharp  regions  of  state  A  are  dark.  In  this  case,  a  sort  of  neg¬ 
ative  image  is  recorded.  Nevertheless,  with  suitable  mathematical  postprocessing,  similar  optical  resolution  can  be  obtained.  In  any  case,  the  resolution  is  no 
longer  limited  by  diffraction,  but  only  determined  by  the  value  of  (C)  The  simplified  energy  diagram  of  a  fluorophore  depicts  possible  schemes  for  imple¬ 
menting  saturable  optical  transitions. 


readily  extended  to  three  dimensions.  For  this  purpose  we  require 
the  spatial  intensity  distribution  7  =  I(x)  to  be  zero  at  a  point  x;  in 
space:  I(x,)  =  0.  Whereas  in  two  dimensions  or  three  dimensions, 
the  light  intensity  distribution  would  be  a  two-dimensional  (2D) 
or  three-dimensional  (3D)  doughnut  mode,  in  one  dimension  the 
zero  is  best  produced  by  a  standing  wave  7(x)  =  I0cos2(2nnx/X).  If 
we  now  apply  7(x)  to  a  spatial  distribution  of  molecules  (in  x )  that 
are  first  in  state  A,  then  for  70  »  Isa„  virtually  all  the  molecules 
will  be  transferred  to  B,  except  for  those  that  are  very  close  to  xt. 
The  larger  the  ratio  ///,,„»  1,  the  sharper  is  the  region  where  state 
A  persists  [note  the  increase  in  curve  steepness  with  increasing  sat¬ 
uration  level  IIIsa,  in  Fig.  31.1(B)],  The  FWHM  of  the  resulting 
spot  of  state  A  is  readily  calculated  as: 


Ax  ~ 


X 


(2) 


In  microscopy,  the  spatial  distribution  7(x)  may  be  produced 
by  the  objective  lens  itself.  If  it  is  produced  through  the  finite  aper¬ 
ture  of  the  objective  lens,  the  smallest  spot  that  can  be  obtained  is 


nnsma^I/Isat 


which  may  be  regarded  as  an  extension  of  Abbe’s  equation  (Hell, 
2003,  2004).  In  fact,  one  can  easily  show  that  if  the  zero  is  pro¬ 
duced  by  focusing  the  light  through  the  lens,  the  equation  becomes: 


Ax  ~ 


_ X 

2nn  sina^/1  +  7/7jo( 


(4) 


For  ///„,,  =  100,  the  theoretical  resolution  improvement  over  Abbe 
is  by  about  10.  Despite  the  dependence  of  Ax  on  A,/sina  and  in 
contrast  to  Eq.  1,  the  new  Eqs.  2,  3  and  4  allow  diffraction- 
unlimited  spatial  resolution. 


For  a  3D  doughnut,  we  obtain  a  confined  spatial  volume  of 
molecules  in  state  A  whose  dimensions  scale  inversely  with 
4TJT~t ,  i  =  x,y,z,  with  7,  denoting  the  peak  intensities  along  the 
respective  axes.  Hence,  the  reduction  in  volume  scales  with 
■sjljyljll,  ■  In  RESOLFT  microscopy  while  the  resolution  still 
scales  with  the  wavelength  X,  its  limit  only  depends  on  the  applic¬ 
able  light  intensity  7  at  a  given  7S„(. 

If  we  now  assume  that  state  A  but  not  state  B  is  a  fluorescent 
state,  the  relevance  to  imaging  becomes  obvious:  our  scheme 
allows  us  to  create  arbitrarily  small  fluorescence  spots  (Hell  and 
Wichmann,  1994;  Hell,  2003,  2004;  Hell  et  al.,  2003).  Moreover, 
by  scanning  the  zero  x,  across  (or  through)  the  specimen,  we  can 
record  the  fluorophore  distribution  point  by  point,  and  thus  assem¬ 
ble  a  fluorescence  3D  image  with  arbitrary  resolution.  Identical  flu¬ 
orescent  objects  can  be  imaged  as  separate  in  space  irrespective  of 
their  proximity  and  size  because  the  fluorescence  spot  (state  A)  can 
be  made  so  small  that  only  one  of  the  objects  fluoresces. 

The  concept  of  RESOLFT  inevitably  requires  scanning  (with 
a  zero),  but  not  necessarily  with  a  single  beam  or  a  point-like  zero. 
Multiple  zeros  or  dark  lines  produced  by  the  interference  of 
counter-propagating  waves  (Cragg  and  So,  2000;  Heintzmann  et 
al.,  2002)  in  conjunction  with  conventional  charge-coupled  device 
(CCD)  camera  detection  can  also  be  used,  provided  the  zeros  or 
the  dark  lines  are  farther  apart  than  about  the  distance  required  by 
the  diffraction  resolution  limit  of  conventional  CCD  camera 
imaging  (Cragg  and  So,  2000;  Hell.  2003).  Dark  lines  increase  the 
resolution  in  a  single  direction  only,  but  stepwise  rotation  of  the 
pattern  plus  interleaved  scanning  of  the  minima  (e.g.,  by  shifting 
the  phase  in  the  interference  pattern)  and  subsequent  computa¬ 
tional  reassignment  (Heintzmann  and  Cremer,  1998;  Heintzmann 
et  al.,  2002)  may  provide,  under  some  conditions,  similar  trans¬ 
verse  resolution  as  with  points  and  do  so  at  higher  recording  speed. 
The  obligation  for  scanning  remains.  The  need  for  scanning  is  also 
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the  reason  why  the  saturable  optical  transition  A  — >  B  has  to  be 
reversible.  The  molecule  in  state  B  must  be  able  to  return  to  the 
state  A  at  the  latest  when  the  zero  comes  across  its  site. 

DIFFERENT  APPROACHES  OF  REVERSIBLE 
SATURABLE  OPTICAL  FLUORESCENCE 
TRANSITIONS  MICROSCOPY 

The  RESOLFT  scheme  and  the  subsequent  breaking  of  the  dif¬ 
fraction  barrier  is  the  actual  idea  behind  stimulated  emission  deple¬ 
tion  (STED)  microscopy  (Hell  and  Wichmann,  1994;  Hell,  1997; 
Klar  et  al.,  2000).  In  STED  microscopy,  many  of  the  molecules 
that  have  just  been  excited  to  the  fluorescent  state  Si  (A)  are  imme¬ 
diately  transferred  by  a  further  light  intensity  1  to  the  molecular 
ground  state  S0  (B),  so  that  fluorescence  emission  is  prevented 
[Fig.  31.1(B,C)  and  Fig.  31.2(A)].  The  physical  effect  responsible 
for  this  transfer  is  stimulated  emission,  a  basic  single-photon 
phenomenon  that  has  about  the  same  cross-section  as  single¬ 
photon  absorption  (o  ~  10-16— 10_18cm2).  Because  STED  competes 
with  the  spontaneous  fluorescence  decay  of  kn  ~  (1  ns)-1  of  the  Si, 
the  saturation  intensity  Im  can  be  approximated  as  £/g  or  about 
1025  to  1027  photons  per  square  centimeter  and  second,  that  is, 
several  tens  to  a  hundred  megawatts  per  square  centimeter  [Fig. 
31.2(B)].  Saturated  depletion  of  the  excited  state  with  a  focal  spot 
containing  a  zero  squeezes  the  extent  of  the  fluorescent  spot  to  a 
subdiffraction  size  that  is  not  any  longer  limited  by  the  wavelength. 


but  only  by  the  applied  intensity.  The  potential  and  details  of  STED 
microscopy  will  be  discussed  later. 

Other  variants  of  RESOLFT  microscopy  employing  different 
physical  realizations  of  states  A  and  B  have  also  been  suggested 
[Fig.  31.1(C)]  (Hell  et  al,  2003).  For  example,  in  ground  state 
depletion  (GSD)  microscopy  (Hell  and  Kroug,  1995;  Hell,  1997), 
the  ground  state  S0  has  the  role  of  state  A,  while  state  B  is  a 
metastable  triplet  state;  more  precisely,  A  and  B  are  the  singlet  and 
triplet  systems  of  the  dye,  respectively.  For  a  number  of  dyes, 
intersystem  crossing  (A— >B)  occurs  as  a  byproduct  of  the  regular 
dye  excitation  because  during  each  excitation  cycle,  the  molecule 
crosses  to  the  triplet  state  with  a  probability  p  ~  0.05  to  0.2. 
Because  of  its  metastability,  the  triplet  state  is  relatively  easily 
filled  up  by  repeated  regular  excitation.  For  saturation,  the  triplet 
buildup  rate  kAB  =  pal  must  be  larger  than  the  decay  to  the  S0,  kBA 
~  (10~®  to  10-2  s)_I,  which  is  comparatively  slow.  Thus,  typical  Im 
is  several  tens  of  kilowatts  per  square  centimeter,  which  is  by  2  to 
4  orders  of  magnitude  lower  than  with  STED.  Low  Iml  is  invalu¬ 
able  with  regard  to  the  attainable  resolution  (see  Eq.  2),  and  with 
regard  to  sample  compatibility.  However,  the  experimental  real¬ 
ization  of  this  member  of  the  RESOLFT  family  will  be  compli¬ 
cated  by  the  fact  that  the  triplet  state  is  involved  in  the 
photobleaching  pathway  (Schafer,  1973). 

Perhaps  the  simplest  way  of  realizing  a  saturated  optical  tran¬ 
sition  is  through  an  intense  excitation  (Heintzmann  et  al.,  2002). 
In  this  case,  the  ground  state  S0  (A)  is  depleted  and  expected  to 
reside  in  the  fluorescent  state  Si  (B).  The  same  RESOLFT  for- 


A  Energy  levels 


B  Saturated  depletion 
of  state  Si 


C  STED  microscope 


Excitation  STED  Saturated 
spot  spot  depletion 


FIGURE  31.2.  STED  microscopy.  (A)  Molecules  in  the  fluorescent  state  Si  (state  A)  return  to  the  ground  state  S0  (state  B )  by  spontaneous  fluorescence  emis¬ 
sion.  Return  to  S0  may  also  be  optically  enforced  through  stimulated  emission.  To  prevail  over  the  spontaneous  emission,  stimulated  emission  depletion  of  the 
Si  requires  relatively  intense  light  pulses  with  durations  of  a  fraction  of  the  Si  lifetime.  (B)  Saturated  depletion  of  the  Si  with  increasing  peak  intensity  of  STED 
pulses  of  ~100ps  duration,  as  measured  by  the  remaining  fluorescence  of  a  monomolecular  layer  of  the  dye  MR-121.  For  the  higher  intensity  levels,  the  Si  is 
optically  depleted.  The  saturation  intensity  is  defined  as  the  intensity  value  at  which  the  Si  is  depleted  by  half.  (C)  Center.  Sketch  of  a  point-scanning  STED 
microscope.  Excitation  and  STED  are  accomplished  with  synchronized  laser  pulses  focused  by  a  lens  into  the  sample,  sketched  as  green  and  red  beams,  respec¬ 
tively.  Fluorescence  is  registered  by  a  detector.  Bottom :  Intensity  distributions  in  the  focus.  The  diffraction- limited  excitation  spot  is  overlapped  with  the 
doughnut-shaped  STED  spot  featuring  a  central  zero.  Saturated  depletion  by  the  STED  beam  confines  the  region  of  excited  molecules  to  the  zero,  leaving  a  flu¬ 
orescent  spot  of  subdiffraction  dimensions.  Outer  left  and  right  insets :  the  confocal  and  the  subdiffraction- sized  spot  left  by  STED,  respectively.  Note  the  doubled 
lateral  and  5 -fold  improved  axial  resolution  over  confocal  microscopy.  The  reduction  in  dimensions  (x,  y,  z )  yields  an  ultrasmall  volume  of  subdiffraction  dimen¬ 
sions,  here  0.67  attoliter,  corresponding  to  an  18-fold  reduction  compared  to  its  confocal  counterpart.  In  spite  of  using  diffraction- limited  beams,  the  concept  of 
STED  fluorescence  microscopy  may,  under  very  favorable  conditions,  reach  spatial  resolution  at  the  molecular  scale. 
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malism  applies,  except  that  it  is  state  B  that  is  now  fluorescent. 
State  A  is  depleted  such  that  ultrasharp  dark  regions  are  created 
that  are  surrounded  by  bright  fluorescent  regions.  In  a  sense,  this 
approach  records  negative  images,  which  are  subsequently  made 
positive  by  mathematical  postprocessing.  The  dark  regions  can  be 
lines  produced  by  interference  patterns,  but  also  3D  doughnuts.  In 
the  latter  case,  one  would  produce  dark  3D  volumes  that  are  con¬ 
fined  by  walls  of  intense  fluorescence.  The  challenges  with  this 
otherwise  very  appealing  approach  are  that  the  mandatory  com¬ 
putations  require  an  excellent  signal-to-noise  ratio.  As  with  STED, 
excitation  saturation  competes  with  fluorescence  emission,  so  that 
Isa,  is  also  given  by  kfl/o  which  is  of  the  order  of  1025  photons  per 
cm2  -  second,  several  tens  of  megawatts  per  square  centimeter 
(Hell,  2003).  Compared  to  STED,  the  intensity  needed  for  satu¬ 
rated  optical  transitions  should  actually  be  up  to  10  times  lower 
because  the  dye  can  be  excited  at  the  maximum  of  the  emission 
spectrum  where  cross-sections  are  largest.  Still,  intense  excitation 
increases  photobleaching.  Relief  could  possibly  be  brought 
by  using  non-blinking  semiconductor  quantum  dots  as  labels 
(Alivisatos,  1996;  Bruchez  et  al.,  1998;  Peng  et  al.,  2000). 

Hence,  while  the  family  of  RESOLFT  concepts  is  not  subject 
to  Abbe’s  diffraction  barrier  anymore,  the  dependence  of  the  res¬ 
olution  gain  on  *JI/ lsat  entails  another  soft  barrier  which  is  the 
maximum  intensity  I  that  the  sample  can  tolerate.  Fortunately,  the 
remedy  is  the  use  of  transitions  with  low  values  of  Isa„  that  is, 
optical  transitions  that  are  easy  to  saturate. 

An  example  is  the  optical  switching  of  bistable  compounds 
from  a  fluorescent  state  (A)  into  a  non-fluorescent  state  (B),  or  vice 
versa.  Optical  bistability  can  be  realized  by  photo-induced  cis- 
trans  isomerization  (Dyba  and  Hell,  2002;  Hell  et  al.,  2003).  If 
both  states  A  and  B  are  (meta)stable,  the  optical  transition  A  — »  B 
or  B  — >  A  can  be  completed  at  very  long,  if  not  arbitrary,  time 
scales.  Thus,  the  light  energy  needed  for  these  transitions  can  be 
spread  in  time  (Hell  et  al.,  2003),  reducing  Imt  to  values  that  are 
lower  by  many  orders  of  magnitude  compared  to  those  of  STED 
or  the  other  RESOLFT  family  members.  These  processes  would 
also  readily  lend  themselves  for  parallelization  through  large  area 
widefield  imaging.  However,  the  principal  advantage  is  the  insight 
that  nanoscale  resolution  does  not  necessarily  require  extreme 
intensities  of  light  (Hell,  2003,  2004;  Hell  et  al,  2003). 

Suitable  candidates  for  this  concept  are  the  optically  switch- 
able  fluorescent  proteins.  For  example,  the  protein  asFP595  can  be 
switched  on  by  green  light  ( B  — >  A)  and  also  switched  off  (A  — » 
B)  by  blue  light  recurrently  (Lukyanov  et  al.,  2000;  Chudakov  et 
al.,  2003).  Although  known  proteins  such  as  asF595  may  have 
major  limitations,  such  as  a  low  quantum  efficiency  and  a  strong 
tendency  to  form  oligomers,  these  problems  could  possibly  be 
solved  by  strategic  mutagenesis.  Alternatively,  new  switchable 
proteins  could  be  found  by  targeted  exploration.  A  RESOLFT 
concept  based  on  genetically  encoded  optically  switchable  tags 
is  extremely  appealing,  because  it  would  allow  highly  specific 
imaging  in  live  cells  with  unprecedented  optical  resolution.  We 
expect  that  further  variants  of  RESOLFT  will  emerge  in  the  future. 

STIMULATED  EMISSION 
DEPLETION  MICROSCOPY 

So  far,  STED  microscopy  is  the  only  member  of  the  RESOLFT 
schemes  that  has  been  realized  (Klar  et  al.,  2000,  2001).  In  its 
initial  demonstration,  STED  microscopy  has  been  realized  as  a 
point-scanning  system,  whereby  excitation  and  STED  is  per¬ 
formed  with  two  synchronized  ultrashort  pulses  [Fig.  31.2(C)]. 


The  first  pulse  excites  the  molecule  into  the  fluorescent  state  Si  at 
a  suitable  wavelength.  The  red-shifted  second  pulse  that  follows  a 
few  picoseconds  later  transfers  the  molecules  away  from  the  zero 
back  to  the  ground  state  S0.  Although  Isat  is  several  tens  of 
megawatts  per  square  centimeter,  it  scales  inversely  with  the  pulse 
duration  of  10  to  300  ps.  Because  the  breaking  of  the  diffraction 
barrier  calls  for  I »  Isal,  focal  intensities  of  100  to  500  MW/cm2 
are  required  (Hell  and  Wichmann,  1994).  For  comparison,  live-cell 
multi-photon  microscopy  typically  uses  103  to  104  times  shorter 
pulses  of  103  to  104  greater  intensity:  200GW/cm2.  As  most  sample 
damage  mechanisms  depend  nonlinearly  on  the  intensity,  the 
typical  values  used  for  STED  so  far  have  been  live-cell  compati¬ 
ble  (Klar  et  al,  2000). 

Figure  31.2(C)  shows  a  typical  experimental  focal  intensity 
distribution  of  the  excitation  spot  (green),  overlapping  with  a 
STED  spot  (red)  featuring  a  central  hole.  Saturated  depletion 
inhibits  fluorescence  everywhere  except  for  the  very  center  of  the 
focal  region  (Klar  et  al.,  2000).  For  the  I/Isal~  100  applied  for  the 
measurement  in  Figure  31.2,  the  net  3D  spot  becomes  almost 
spherical  with  a  diameter  of  -lOOnm,  which  amounts  to  an  almost 
6-fold  and  2.3-fold  increase  in  axial  and  lateral  resolution,  respec¬ 
tively.  Although  theory  permits  much  higher  resolution  (in  princi¬ 
ple,  molecular  scale),  in  this  experiment  the  production  of  a 
smaller  spot  was  challenged  by  experimental  imperfections  (Klar 
et  al.,  2000),  such  as  a  finite  depth  of  the  central  zero,  and 
increased  photobleaching  with  increasing  IIIsal. 

The  fact  that  the  spot  is  squeezed  more  in  the  z-direction  than 
in  the  focal  plane  is  due  to  the  higher  local  intensity  of  this  par¬ 
ticular  quenching  spot  along  the  optic  axis.  Using  a  STED  beam 
of  different  shape,  an  improvement  of  more  than  5-fold  in  the  focal 
plane  resolution,  compared  to  Abbe’s  barrier,  has  recently  been 
demonstrated  with  single  molecules  dispersed  on  a  surface  (see 
Fig.  31.3  and  Westphal  et  al.,  2003).  Furthermore,  Figure  31.3(B) 
proves  that  objects  separated  by  much  less  than  the  diffraction 
limit  can  clearly  be  distinguished.  Recent  experiments  also  indi¬ 
cate  that,  in  accordance  with  Equations  2  and  3,  even  higher  lateral 
resolution  is  possible  with  STED,  provided  that  photobleaching 
can  be  avoided.  In  fact,  it  has  been  shown  that  a  focal  spot  width 
of  16nm,  corresponding  to  little  more  than  2%  of  the  wavelength 
used  can  be  obtained  [see  Fig.  31.7(A)]  (Westphal  and  Hell,  2005). 
Because  the  diffraction  barrier  is  broken,  STED  microscopy  does 
not  have  a  firm  resolution  limit.  The  ultimate  resolution  solely 
depends  on  how  well  the  operational  conditions  can  be  realized. 

In  frequency  space,  the  sample  structure  is  described  in  terms 
of  spatial  frequencies.  Therefore,  microscope  performance  is 
defined  by  the  OTF  (optical  transfer  function),  describing  the 
strength  with  which  these  frequencies  are  transferred  to  the  image. 
Thus,  the  resolution  limit  is  given  by  the  highest  frequency  that 
produces  a  signal  above  the  noise  level.  Point  spread  function 
(PSF)  and  OTF  are  intertwined  by  Fourier  mathematics:  the 
sharper  the  PSF,  the  broader  the  OTF.  The  OTF  extension  of  STED 
is  very  smooth  [see  Fig.  31.3(C)],  without  any  gaps,  simplifying 
the  deconvolution  techniques  needed  to  produce  the  best  final 
results. 

At  present,  the  fact  that  relatively  few  fast,  pulsed,  tunable, 
visible  lasers  are  available  places  some  practical  limits  on  the  dyes 
that  can  be  used  for  STED  microscopy.  The  STED  laser  system 
must  be  able  to  produce  two  short  laser  pulses  that  follow  one 
another  in  the  picosecond  time  domain.  The  first  pulse  must  be  at 
a  wavelength  capable  of  exciting  the  dye,  and  the  second,  more 
powerful  pulse  must  be  at  a  wavelength  capable  of  quenching  it. 
Although  early  STED  studies  were  confined  to  red-emitting  dyes 
by  the  availability  of  appropriate  lasers,  this  is  no  longer  the  case. 
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FIGURE  31.3.  Quantifying  lateral  resolution  in  STED  microscopy  through  imaging  of  point-like  objects.  (A)  The  effective  point  spread  function  (PSF)  of  a 
conventional  microscope  and  a  (laser-diode)  STED  microscope,  determined  on  single  dye  molecules  (JA  26).  (B)  Molecules  spaced  apart  far  below  the  diffrac¬ 
tion  limit  could  be  clearly  separated  in  STED  microscopy  (slightly  augmented  by  deconvolution).  (C)  For  STED  microscopy  the  gain  in  transmitted  bandwidth 
of  the  optical  transfer  function  (OTF)  is  more  than  5-fold,  compared  with  conventional  microscopy.  Objective  lens,  NA  =1.4  (oil);  wavelengths  X,  635  nm  (exci¬ 
tation),  650  to  720nm  (fluorescence  detection),  781  nm  (STED). 


A  list  of  the  dyes  used  so  far,  is  found  in  Table  31.1.  This  list  is 
likely  to  be  expanded  as  the  number  of  pulsed-laser  diodes 
increases  and  lasers  become  available  that  are  capable  of  doing 
STED  on  blue,  green,  and  yellow  fluorophores,  as  well  as  fluores¬ 
cent  proteins.  Studies  aimed  at  identifying  suitable  laser/dye  pairs 
are  ongoing. 

Recently,  a  very  compact  STED  microscope  was  demonstrated 
using  a  laser  diode  for  the  blue  excitation  and  a  second  diode  laser 
for  STED  at  around  780nm  (Westphal  et  al.,  2003).  As  the  set  of 
available  wavelengths  expands  over  the  next  decade,  it  should  in 
the  future  become  possible  to  realize  STED  microscopy  at  lower 
cost  and  on  most  dyes. 

Although  shorter  wavelengths  will  lead  to  higher  spatial  reso¬ 
lution,  a  further  increase  in  intensity  may  be  barred  in  aqueous 
media  by  intolerable  photobleaching.  Saturation  factors  of  >200 
might  not  be  readily  attainable. 

STED  microscopy  is  still  in  its  infancy.  So  far,  most  of  the 
applications  have  been  aimed  at  exploring  its  principles.  Tackling 
cell  biology  questions  will  be  a  task  for  the  years  to  come.  Strong 
fluorescence  suppression  (reduction  by  90%)  is  conceptually  not 
mandatory,  but  practically  important  to  attain  subdiffraction  reso¬ 
lution.  Among  the  first  biological  stains  described  that  allows  this 
level  of  suppression  to  be  reached  were  lipophilic  dyes  such  as 
Styryl  6,  7,  and  8,  9M,  LDS  751.  Pyridine  1,  2,  4  and  RH  414,  or 
Oxazine  170  and  Nile  Red.  Because  STED  was  first  realized  with 
a  titanium :  sapphire  (Ti:Sa)  laser  emitting  in  the  far-red  (750- 
800  nm),  the  emission  maximum  of  these  dyes  is  located  around 
650  to  700 nm  (Klar  et  al.,  2000). 

Pyridine  4  was  used  to  label  the  membranes  of  live  Escherichia 
coli.  A  simultaneous  doubling  of  both  the  axial  and  lateral  resolu¬ 
tion  was  observed  using  a  3D  doughnut  and  STED  pulses  of 


~30ps  duration.  This  initial  improvement  is  likely  to  be  augmented 
by  further  optimization  of  the  wavelengths,  of  the  doughnut, 
and  of  the  pulse  duration.  Indeed,  recent  studies  revealed  that 
STED-related  photobleaching  dramatically  decreases  with  the 
duration  of  the  STED  pulse,  which  indicates  a  strongly  nonlinear 
dependence  of  bleaching  on  the  STED-pulse  intensity  (Dyba  and 
Hell,  2003).  Although  bleaching  is  substantially  reduced  with 
pulses  >120ps  duration,  more  studies  are  required  to  address  this 
critical  issue. 

In  budding  yeast,  the  dye  RH  414  is  taken  up  by  bulk  mem¬ 
brane  internalization  and  subsequently  transported  to  the  vacuolar 
membrane.  The  structural  integrity  of  vacuolar  membranes  is  sen¬ 
sitive  to  many  stress  factors.  Therefore,  the  subdiffraction  resolu¬ 
tion  imaging  of  vacuoles  in  living  yeast  by  STED  microscopy 
confirmed  that  STED  microscopy  is  amenable  to  imaging  living 
cells  (Klar  et  al.,  2000). 

Recent  progress  in  laser  technology  has  enabled  STED 
microscopy  employing  dyes  that  fluoresce  in  the  visible  range. 
These  dyes  offer  several  significant  advantages;  The  shorter  wave¬ 
lengths  involved  inherently  improve  the  Abbe  resolution,  while  a 
better  quantum  efficiency  [compared  with  near-infrared  (NIR) 
dyes]  leads  to  brighter  images.  Furthermore,  the  fact  that  they  are 
visible  by  eye  simplifies  sample  inspection  and  image  region  selec¬ 
tion.  As  a  first  example,  a  dramatic  enhancement  of  the  resolution 
in  the  focal  plane  is  demonstrated  on  yellow-green  fluorescent 
microspheres  (FluoSpheres  505/515,  Molecular  Probes,  OR)  in 
Figure  31.4. 

Although  the  principle  of  STED  applies  to  any  fluorophore, 
new  fluorescent  markers  require  prior  investigation.  Sometimes 
the  narrow  spectral  window  for  efficient  STED  depends  on  the 
chemical  environment  and  has  to  be  established  through  meticu- 


TABLE  31.1.  Listing  of  Example  Dyes  That  Have  Been  Used  Successfully  for  STED 


Category 

Name 

Emission  nm 

STED  nm 

Manufacturer 

Notes 

Green  dye* 

Atto532 

540-570 

615 

Atto-tec  GmbH,  Siegen,  DE 

Used  for  single-molecule  studies 

Yellow  dye* 

DY-510XL 

560-630 

625 

Dyomics  GmbH,  Jena,  DE 

Immunofluorescence  label 

Red  dye* 

Atto647N 

650-720 

760 

Atto-tec  GmbH,  Siegen,  DE 

Used  for  single-molecule  studies 

Far  red  dye 

Pyridine  2 

680-750 

750-780 

SigmaAldrich,  St.  Louis,  MO 

Membrane  label 

Infrared  dye 

Pyridine  4 

710-800 

780-800 

SigmaAldrich,  St.  Louis,  MO 

Membrane  label 

*  These  dyes  have  functional  groups  and  can  be  coupled  to  proteins  making  simultaneous,  3-color,  immuno-fluorescence  STED  imaging  possible. 
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FIGURE  31.4.  Beyond  the  diffraction  barrier:  STED  versus  confocal  fluorescence  microscopy  on  40nm  fluorescent  microspheres  (emission  maximum  at 
515nm)  spread  out  on  a  coverslip.  The  comparison  of  the  confocal  (A)  and  the  STED  (B)  image  demonstrates  the  superior  resolution  of  STED  microscopy; 
neighboring  beads  are  clearly  separated.  (C)  STED  intensity  distribution  employed  for  depletion,  featuring  a  prominent  zero  in  the  center.  Drawn  to  scale 
with  the  images.  (D)  Vertical  sum  of  marked  region  in  (B)  illustrates  both  the  clear  separation  capabilities  and  subdiffraction  resolution  of  STED  microscopy. 
Objective  lens,  NA=  1.4  (oil);  wavelengths  A,  469 nm  (excitation),  500  to  550nm  (fluorescence  detection),  585 nm  (STED). 


lous  screening.  Ongoing  investigations  on  fluorescent  proteins  will 
establish  the  operational  conditions  for  STED  and  its  potential  with 
these  important  labels. 

Recently,  the  resolving  power  of  STED  has  been  synergisti- 
cally  combined  with  that  of  4Pi  microscopy  to  achieve  nanoscale 
axial  resolution  (Dyba  and  Hell.  2002).  Destructive  interference  of 
the  counterpropagating  spherical  wavefronts  of  the  STED  pulse  at 


the  focal  point  produces  a  narrow  focal  minimum  for  STED  with 
an  axial  FWHM  of  ~X/(4 n)  ~  100  to  120nm.  Overlap  with  the 
regular  excitation  spot  of  a  single  lens  has  so  far  rendered  focal 
spots  down  to  Az  =  40  to  50  nm.  Linear  deconvolution  of  the  data 
removes  the  effect  of  the  weak  (<30%)  sidelobes  that  accompany 
the  narrow  focal  spot.  Moreover,  it  further  increases  the  axial  res¬ 
olution  up  to  30  to  40 nm.  This  is  exemplified  in  Figure  31.5(C,D), 


FIGURE  31.5.  Axial  resolution  increase  provided  by  STED— 4Pi  (B,  D)  over  confocal  microscopy  (A,C).  vr- images  from  the  immunolabeled  microtubular 
network  of  a  HEK  cell  as  recorded  with  a  (A)  confocal  and  (B)  STED-4Pi  microscope.  Both  images  have  been  recorded  at  the  same  site  in  the  cell.  The  micro¬ 
tubules  were  labeled  using  a  primary  anti  [)-tubulin  antibody  and  a  secondary  antibody  coupled  to  MR-121.  The  STED  414  images  were  linearly  deconvolved 
to  remove  the  effect  of  the  4Pi  sidelobes.  Note  the  straight  horizontal  line  which  stems  from  an  MR-121  layer  on  the  coverslip.  At  this  layer,  the  resolution  of 
the  STED-4Pi  microscope  is  determined  as  53  nm  after  linear  deconvolution.  The  HEK  cell  was  mounted  in  aqueous  buffer  and  recorded  with  water-immersion 
lenses.  In  another  experiment  the  membranes  of  a  live  bacterium  (Bacillus  megaterium )  were  stained  with  the  RH  414.  Next  it  was  simultaneously  imaged  in 
the  confocal  (C)  and  in  the  STED-4Pi  microscopy  mode  (D).  Note  that  the  axial  resolution  of  this  focusing  microscope  is  of  the  order  of  30  to  40  nm. 
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which  shows  .rz-images  of  the  membrane-labeled  Bacillus  mega- 
terium  (Dyba  and  Hell,  2002). 

The  STED^lPi  setup  realized  so  far  improves  the  resolution 
along  the  z-axis  only,  thus  rendering  a  disk-shaped  focal  spot  whose 
effect  is  also  noticeable  in  Figure  31.5(B,D).  The  spot  could  be 
sculpted  down  to  a  spherical  shape  by  applying  a  second,  STED, 
pulse  whose  spatial  form  is  designed  to  squeeze  the  spot  laterally. 

STED-4Pi  microscopy  has  also  been  applied  to  the  imaging 
of  the  microtubular  cytoskeleton  of  human  embryonic  kidney 
(HEK)  cells  (Dyba  et  al.,  2003).  The  HEK  cells  were  decorated 
with  an  anti-(3-tubulin  antibody  and  a  secondary  antibody  coupled 
to  the  red-emitting  dye  MR- 121.  The  latter  displays  high  STED 
efficiency  (>90%)  at  a  STED  wavelength  of  -780  to  795  nm.  Con¬ 
trary  to  the  confocal  Jcz-sections,  in  the  linearly  deconvolved 
STED^lPi  counterpart,  most  of  the  microtubules  appear  as  distinct 
objects  [Fig.  31.5(A,B)]. 

The  axial  resolution  attained  can  be  inferred  from  the  FWHM 
of  a  fluorescent  mono  layer  that  has  been  deposited  on  the  cover- 
slip;  it  is  ~50nm,  corresponding  to  1/16  of  the  irradiation  wave¬ 
length  of  793  nm.  It  is  interesting  to  note  that  in  the  STED-4Pi 
image,  the  brightness  of  the  monomolecular  layer  is  of  the  same 
order  as  that  of  the  microtubules.  By  contrast,  in  the  confocal 
image,  signal  from  the  layer  is  overwhelmed  by  the  total  signal 
from  the  larger  focal  volume  of  the  confocal  microscope. 

An  important  task  for  the  near  future  is  to  define  the  optimal 
parameters  for  specific  imaging  applications.  The  results  obtained 
with  STED  and  STED-4Pi  microscopy  demonstrate  that  the  basic 
physical  Abbe  limit  has  been  broken  and  we  are  now  moving 
towards  attaining  a  3D  resolution  of  the  order  of  a  few  tens  of 
nanometers. 


CHALLENGES  AND  OUTLOOK 

Given  the  limits  on  the  rate  of  excitation  and  fluorescence  emis¬ 
sion,  high  temporal  and  high  spatial  resolution  may  be  mutually 
exclusive  in  many  cases,  simply  because  of  the  poor  statistics  of 
the  collected  photon  signal.  High  spatial  resolution  also  requires 
small  pixels,  which  is  not  favorable  for  fast  imaging  of  large  areas. 
Downsizing  the  region  of  interest  will  be  inevitable.  A  remedy  is 
to  parallelize  the  scanning  system,  either  by  applying  many  foci 
or  by  utilizing  sophisticated  structured  illumination  schemes.  Like¬ 
wise,  the  limited  number  of  emission  cycles  that  characterizes 
many  fluorophores,  caps  the  signal  that  is  available  from  a  sample, 
and,  thus,  the  signal  per  sample  volume.  For  a  number  of  staining 
protocols  and  applications,  the  available  signal  might  not  match 
up  with  the  number  of  photons  that  must  be  detected  in  order  to 
benefit  from  the  increase  in  spatial  resolution.  Therefore,  potential 
improvements  in  fluorescent  labels  as  well  as  strategies  for  avoid¬ 
ing  photobleaching  will  play  a  vital  role  in  firmly  establishing 
nanoscale  resolution  in  light  microscopy. 

Still,  even  under  poor  signal  conditions,  a  RESOLFT  method 
creating  ultrasmall  focal  volumes,  such  as  STED,  may  be 
extremely  helpful  for  techniques  that  exploit  fluorescence  statis¬ 
tics.  For  example,  fluorescence  correlation  spectroscopy  (Magde 
et  al.,  1972)  depends  on  small  focal  volumes  to  detect  rare  molec¬ 
ular  species  or  rare  molecular  interactions  in  concentrated  solu¬ 
tions  (Eigen  and  Rigler,  1994;  Levene  et  al.,  2003).  STED  may  be 


the  key  to  interrogating  nanosized  volumes  in  intact  cells.  In  fact, 
it  is  so  far  the  only  method  reported  to  squeeze  a  fluorescence 
volume  to  the  zeptoliter  scale  without  mechanical  confinement. 
Published  results  imply  the  possibility  of  sampling  spherical  focal 
volumes  of  only  30 nm  diameter  (Kastrup  et  al.,  2005). 

The  past  decade  has  witnessed  the  emergence  of  a  family 
of  physical  concepts  for  attaining  diffraction-unlimited  spatial 
resolution  in  focusing  fluorescence  microscopy.  Relying  on 
reversible  saturable  optical  (fluorescence)  transitions  (RESOLFT), 
the  spatial  resolution  of  these  concepts  is  eventually  determined 
by  the  saturation  level  that  can  be  realized.  Saturation  brings  about 
an  essential  nonlinear  relationship  between  the  signal  and  the 
applied  intensity  that  allows  one  to  overcome  diffraction 
fundamentally. 

The  nonlinearity  brought  about  by  saturation  is  radically  dif¬ 
ferent  from  that  of  the  well-known  multi-photon  events.  In  the 
latter  cases,  the  nonlinearity  stems  from  the  contemporaneous 
action  of  more  than  one  photon,  which  inevitably  demands  high 
focal  intensities.  In  contrast,  the  nonlinearity  brought  about  by  sat¬ 
urated  depletion  stems  from  the  population  kinetics  of  the  states 
involved  (Hell,  2003).  This  opened  the  door  to  attaining  marked 
nonlinearities  even  with  linear  optical  transitions  such  as  single¬ 
photon  excitation  and  stimulated  emission.  Semistable  molecular 
states  enable  saturable  optical  transitions  at  even  lower  light  inten¬ 
sities.  Bistable  fluorophore  constructs  and  switchable  fluorescent 
proteins  should  allow  very  high  levels  of  saturation  at  the  low  light 
intensities  essential  for  live-cell  imaging.  This  insight  may  be  crit¬ 
ical  to  opening  up  the  cellular  nanoscale  with  visible  light  and 
regular  lenses  (Hell,  2003;  Hell  et  al.,  2003). 

In  fact,  it  is  interesting  that  the  demand  for  high  intensities  is 
the  reason  why  the  typical  nonlinear  optical  processes  of 
multi-photon  fluorescence  excitation  and  multi-photon  scattering 
[second  and  third  harmonic  generation  (SHG,  THG),  coherent  anti- 
Stokes  Raman  scattering  (CARS),  etc.]  could  not  substantially 
improve  the  spatial  resolution.  It  is  also  clear  that  as  77-photon 
excitation  of  the  fluorescent  state  (i.e.,  the  Si)  eventually  requires 
the  subdivision  of  the  excitation  energy  into  n  photons,  it  leads 
to  an  n  times  larger  wavelength.  As  an  n  times  larger  wavelength 
leads  to  n  times  larger  focal  spots,  multi-photon  excitation 
processes  are  counterproductive  when  it  comes  to  sharpening 
the  focal  spot  size  in  the  focal  plane.  Exceptions  have  been  given 
(Hiinninen  et  al.,  1996;  Schonle  and  Hell,  1999;  Schonle  et  al., 
1999)  but  they  currently  appear  less  promising  than  the  concept  of 
RESOLFT. 

The  principles  of  the  concept  of  RESOLFT  have  been  vali¬ 
dated  through  STED  microscopy,  whose  ability  to  break  the  dif¬ 
fraction  resolution  barrier  has  been  experimentally  demonstrated. 
It  is  worth  mentioning  that  the  strategy  of  utilizing  reversible  sat¬ 
urable  optical  transitions  also  has  the  potential  to  break  the  dif¬ 
fraction  barrier  in  nanoscale  writing  and  data  storage  (Hell,  2004). 
The  coming  years  will  show  whether  STED  and  its  RESOLFT 
cousins  are  going  to  establish  themselves  as  part  of  the  micro¬ 
scopic  toolbox  to  elucidate  dynamics  and  structure  of  cellular  net¬ 
works.  The  chances  are  better  than  ever. 

A  taste  of  what  is  to  come  can  be  seen  in  Figure  31.6,  which 
shows  small  patches  of  cell  membrane,  immuno-stained  against 
the  SNARE  protein  SNAP25,  and  viewed  both  by  confocal  and 
STED  microscopy  using  the  exact  same  optics.  The  improvement 
is  even  more  obvious  in  Figure  31.7. 
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FIGURE  31 .6.  Resolution  comparison  of  confocal  versus  STED-microscopy;  plasma  membrane  patches  immuno-stained  against  the  SNARE  protein  SNAP25; 
secondary  antibody  labeled  with  Atto  532-NHS;  Emission  540-570 nm,  STED  at  615  nm.  The  confocal  image  was  recorded  by  simply  turning  off  the  STED 
beam  with  no  other  changes. 
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FIGURE  31.7.  STED-microscopy  hits  the  nanoscale.  (A)  Comparison  of  the  PSF  (x-axis)  of  a  conventional  and  a  STED  microscope  probed  with  a  single  dye 
molecule  whose  orientation  is  parallel  to  the  polarization  of  the  STED-beam;  for  details  see  (Westphal  and  Hell,  2005).  The  up-to  15-fold  reduction  in  lateral 
width  underscores  the  potential  of  STED-microscopy  to  attain  true  nanoscale  resolution.  (B)  The  resolution  of  a  STED  microscope  scales  with  the  square  root 
of  the  intensity  used  in  the  STED  beam,  with  no  firm  limit,  as  predicted  by  equation  (4).  The  gain  in  resolution  also  entails  an  increase  of  the  OTF  bandwidth 
over  the  diffraction  barrier.  (C)  Shows  an  example  where  the  usable  bandwidth  (magnitude  >1%)  of  the  OTF  is  broader  by  about  an  order  of  magnitude  than  in 
a  conventional  microscope.  All  measurements  were  performed  with  the  red  emitting  dye  JA26  and  with  a  STED-wavelength  of  ~780nm.  Being  a  far-field  optical 
microscope,  the  resolution  of  STED-microscopy  increases  inversely  with  the  wavelength.  Therefore,  in  (A)  reducing  the  wavelength  for  STED  to  500  nm  would 
decrease  the  FWHM  down  to  ~10nm. 
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INTRODUCTION 

Confocal  microscopes  commonly  generate  their  images  not  as  real 
or  virtual  patterns  of  light,  but  as  pixel  values  in  the  memory  of 
a  computer  (Cox,  1993).  This  gives  the  image  a  measure  of  per¬ 
manence  —  unlike  a  visual  image,  once  acquired  it  will  not  fade 
—  but  it  will  be  lost  if  the  computer  is  turned  off,  or  if  that  area 
of  memory  is  overwritten.  To  store  that  image  with  all  its  infor¬ 
mation  intact  we  must  write  it  in  digital  form  —  a  copy  on  paper 
or  film,  however  good,  cannot  contain  all  the  information  of  the 
original.  However,  a  copy  on  disk  or  tape  is  not  directly  accessi¬ 
ble  to  human  senses.  For  publication  or  presentation  of  the  image, 
or  even  just  to  access  it,  we  must  have  a  display  or  a  hard  copy,  a 
picture  which  can  be  viewed  by  the  human  eye. 

This  chapter  reviews  the  range  of  possible  solutions  to  these 
two  problems.  Because  this  is  a  rapidly  moving  area,  new  alter¬ 
natives  will  doubtless  become  available  almost  as  soon  as  this  is 
printed.  A  measure  of  the  rate  at  which  this  happens  is  that  many 
of  the  technologies  reviewed  in  the  previous  edition  are  now  obso¬ 
lete,  leaving  users  with  the  task  of  copying  images  to  new  media 
if  they  are  to  retain  access  to  their  data.  As  well  as  assessing 
currently  available  technologies,  therefore,  I  will  try  to  provide 
enough  background  information  to  enable  users  to  assess  the  latest 
high  technology  advances  in  a  rational  way.  It  is  always  worth  con¬ 
sidering  the  scale  of  the  adoption  of  a  technique  as  well  as  its  tech¬ 
nical  efficiency  because  most  of  us  will  still  want  to  be  able  to  use 
our  data  in  10  or  even  20  years’  time,  and  only  mass-market  solu¬ 
tions  are  likely  to  survive  on  that  timescale. 

MASS  STORAGE 

The  major  problem  in  storing  confocal  images  is  their  sheer  size. 
The  smallest  image  we  are  likely  to  acquire  would  be  512  X  512 
pixels,  at  one  plane  only  and  containing  only  one  detector  channel. 
Assuming  that  we  store  only  1  byte  per  pixel  (that  is,  each  point 
in  the  image  can  have  one  of  256  possible  gray  levels)  this  will 
require  one  quarter  of  a  megabyte  (MB)  to  store.  We  will  require 
a  little  more  space  to  store  basic  information  about  how  the  picture 
was  acquired,  either  in  a  header  at  the  start  of  the  hie,  or  at  the 
end,  or  even  in  a  separate  hie.  Most  confocal  microscopes  will 
capture  larger  images  than  this,  and  most  will  capture  more  than 
one  channel.  A  three-channel,  2048  x  2048  pixel  image  (routine 
on  any  current  system)  will  require  12  MB  to  store  one  plane.  A 
three-dimensional  (3D)  image  data  set  could  easily  contain  100 
or  more  planes,  thus  requiring  1200  MB  (1.2  gigabytes,  GB)  or 
more  to  store.  At  the  time  of  this  writing,  current  personal  com¬ 
puters  typically  have  80  to  200  GB  hard  disks,  a  200-fold  increase 


on  the  norm  when  the  last  edition  of  this  chapter  was  written,  but 
still  not  enough  to  be  regarded  as  a  permanent  store.  To  provide 
archival  storage,  we  must  have  some  form  of  removable  storage 
media. 

Data  Compression 

Before  considering  the  contending  bulk  storage  devices,  is  there 
any  way  we  can  reduce  the  size  of  the  problem?  Can  we  compress 
the  image  data  to  make  it  smaller?  Lossless  data  compression 
systems,  which  preserve  the  integrity  of  our  original  image 
data,  generally  work  on  some  variation  or  combination  of  three 
well-known  algorithms.  Run-length  encoding  (RLE)  looks  for 
sequences  of  identical  values  and  replaces  them  with  one  copy  of 
the  value  and  a  multiplier.  It  works  very  well  with  binary 
(black/white)  images  or  simple  graphics  using  a  few  bold  colors, 
and  is  used,  for  example,  for  all  the  splash  screens  in  Microsoft 
Windows. 

Lempel-Ziv-Welch  (LZW)  and  Huffman  encoding  look  for 
repeated  sequences,  assign  each  a  token,  then  replace  each  occur¬ 
rence  by  its  token.  Neither  of  these  works  well  with  real  images 
(though  they  do  an  excellent  job  with  computer-generated  graph¬ 
ics).  Thus,  if  you  save  a  confocal  image  as  a  GIF  (graphics  inter¬ 
change  format)  file,  or  as  a  compressed  TIFF  (tagged  image  file 
format)  file,  both  of  which  use  LZW  compression,  you  will  be 
lucky  to  get  even  a  10%  to  20%  decrease  in  size,  and  sometimes 
your  file  size  will  actually  get  larger.  You  will  not  do  much  better 
with  the  popular  archiving  systems  PKzip,  gzip,  or  WinZip,  which 
(to  avoid  patent  problems)  use  LZ77,  an  earlier  Lempel-Ziv  algo¬ 
rithm,  and  Huffman  encoding  (Deutsch,  1996),  though  these 
systems  do  at  least  recognize  if  compression  is  not  working  and 
insert  the  uncompressed  data  instead,  so  your  file  should  not  get 
larger. 

There  are  a  couple  of  exceptions  to  this  generalization.  First, 
some  confocal  microscopes  store  12  bits  of  data  at  each  pixel  (4096 
gray  levels),  but  they  store  this  as  16-bit  numerical  values.  Clearly 
these  images  have  redundant  space  —  a  quarter  of  the  file  contains 
no  information  —  and  they  will  therefore  at  least  compress  to  75% 
of  the  original  size.  The  file  will  nevertheless  become  even  smaller, 
often  with  little  or  no  real  loss,  if  it  is  converted  to  8-bit  data. 
Second,  even  though  it  may  not  immediately  be  obvious,  a  three- 
channel  image  of  moderate  size,  saved  as  a  24-bit  RGB  file,  must 
always  have  redundant  information.  Twenty-four  bits  of  data  can 
specify  16.7  million  colors,  but  a  512  X  512  image  with  only  a 
quarter  of  a  million  pixels  can  contain  at  most  a  quarter  of  a  million 
colors.  Efficient  algorithms  will  automatically  find  this  redundancy 
and  yield  effective  compression  (how  this  is  done  is  explained  in 
the  description  of  the  PNG  format,  below). 
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PNG,  which  stands  for  portable  network  graphic,  but  is  pro¬ 
nounced  “ping,”  is  a  lossless  compression  system  (Roelofs,  2003). 
It  will  usually  offer  the  highest  lossless  compression  currently 
attainable  for  confocal  images.  The  formal  compression  system  is 
identical  to  that  of  the  “zip”  systems;  the  deflate  algorithm 
(Deutsch,  1996)  a  combination  of  Huffman  and  Lempel-Ziv  algo¬ 
rithms  that  in  essence  looks  for  repeated  patterns.  The  secret  of 
PNG’s  improved  performance  lies  in  its  prefiltering  of  the  image 
to  establish  the  best  way  to  represent  the  data.  In  a  real-world 
image  of  any  kind,  the  difference  between  adjacent  pixels  will 
rarely  be  extreme  so  often  the  data  can  be  reduced  substantially  by 
storing  only  the  difference.  The  different  filters  vary  essentially  in 
the  pixels  used  for  comparison  (no  filtering,  pixel  before,  or  before 
and  after,  or  before  and  above,  etc).  Any  implementation  contains 
all  filters  and  so  will  decode  any  image,  but  the  better  implemen¬ 
tations  will  offer  improved  compression  by  careful  choice  of  which 
filter  to  apply.  (The  standard  allows  different  filters  to  be  used  on 
each  line  of  the  image  if  required.  )  So  if  lossless  compression  is 
important  it  may  be  worth  experimenting  with  different  vendors’ 
implementations  of  PNG  (see  below).  It  tends  to  be  much  slower 
than  LZW  to  compress,  partly  because  it  is  a  two-pass  process  but 
mainly  because,  to  get  the  best  results,  the  program  should  test 
which  algorithm  will  give  best  results.  Decompression  is  fast  (see 
Table  32.1). 

The  demands  of  computer  multi-media  have  led  to  the  devel¬ 
opment  of  compression  techniques  specifically  aimed  at  real-world 
images,  both  still  and  moving.  Unlike  the  compression  tech¬ 
niques  mentioned  above,  which  are  completely  reversible,  these 
approaches  discard  information  from  the  image.  The  picture 
created  after  compression  and  decompression  will  not  be  the  same 
as  the  original.  However,  very  large  file  compressions  can  often  be 
achieved  with  losses  which  are  barely  detectable  to  the  eye,  though 
they  may  affect  numerical  properties  of  the  image. 


TABLE  32.1.  Time  to  Compress  and  Read  Back  an  Image 
Using  Different  Techniques 


Compression 

Save  Time  (s) 

Read  Time  (s) 

File  Size  (KB) 

Uncompressed  TIFF 

5 

3 

9220 

LZW  TIFF 

7 

5 

6014 

PNG 

40 

5 

2861 

Wavelet  (lossless) 

9 

12 

1934 

Wavelet  (high-quality )a 

16 

10 

841 

Wavelet  (high-quality )b 

8 

10 

821 

Wavelet  (low-quality)" 

15 

6 

9 

Wavelet  (low-quality)fc 

6 

6 

10 

Lossless  JPEG 

8 

7 

3877 

DCT  JPEG  (high-quality) 

4 

4 

840 

DCT  JPEG  (low-quality) 

4 

4 

164 

a  Specifying  required  quality. 
b  Specifying  required  file  size. 

The  image  used  was  that  seen  in  Figure  32.1,  but  scaled  up  (using  bicubic  interpo¬ 
lation)  6-fold  to  3072  x  3072  pixels  in  order  to  make  the  times  measurable.  All  con¬ 
versions  were  done  using  Paint  Shop  Pro  version  8  (Jasc  Software);  the  results 
should  only  be  taken  as  relative  and  will  vary  greatly  with  processor  speed.  Scaling 
the  image  means  that  it  contains  substantial  redundancy  and  therefore  the  compres¬ 
sion  levels  achieved  are  unrealistic;  the  file  sizes  are  given  mainly  to  illustrate  the 
trade-off  between  processing  time  and  disk  access.  PNG  was  by  far  the  slowest  in 
compressing  the  image,  but  was  rapid  to  read  back.  The  processing  requirements  of 
DCT  JPEG  compression  were  more  than  compensated  for  by  the  reduction  in  disk 
access,  so  that  it  was  very  fast,  but  lossless  JPEG  was  slower  and  its  compression 
did  not  match  lossless  wavelet  or  PNG.  Wavelet  compression  (JPEG  2000)  showed 
the  curious  result  that  selecting  a  “compression  quality”  gave  much  longer  save 
times  than  selecting  the  “desired  output  file  size.”  At  equivalent  final  sizes,  the  result¬ 
ing  images  seemed  similar.  This  is  probably  a  quirk  of  the  implementation  of  what 
is,  at  the  time  of  this  writing,  a  very  new  standard.  Wavelet  images  were  the  slowest 
to  read  back,  particularly  at  high  image  qualities. 


The  most  common  still  image  format  is  the  Joint  Photographic 
Experts’  Group  (JPEG)  compression  protocol  (Redfern,  1989; 
Anson,  1993;  Pennebaker  and  Mitchell,  1993),  which  is  supported 
by  many  paint  and  image  manipulation  programs.  This  breaks  the 
image  into  blocks  of  8  X  8  pixels,  each  of  which  is  then  processed 
through  a  discrete  cosine  transform  (DCT).  This  is  similar  to  a 
Fourier  transform,  but  much  faster  to  implement,  and  gives  an  8  X 
8  array  in  frequency  space.  The  frequency  components  with  the 
lowest  information  content  are  then  eliminated,  after  which  high- 
frequency  information  (fine  detail)  will  be  selectively  discarded  to 
give  the  desired  degree  of  compression.  The  remaining  compo¬ 
nents  are  stored  (using  Huffman  encoding)  in  the  compressed 
image.  The  amount  to  be  discarded  in  frequency  space  can  be  spec¬ 
ified,  which  gives  the  user  control  over  the  trade-off  between 
image  quality  and  degree  of  compression.  Typically,  monochrome 
images  can  be  compressed  down  to  one  fifth  or  less  of  their  orig¬ 
inal  size  with  no  visible  loss  of  quality  (Avinash,  1993).  Com¬ 
pression  and  decompression  are  similar  operations,  and  require 
similar  amounts  of  computer  time.  Ten  years  ago,  when  the  stan¬ 
dard  was  first  published  (Pennebaker  and  Mitchell,  1993),  the  time 
required  was  quite  noticeable  but  with  a  modern  processor  the 
reduced  amount  of  disk  access  will  more  than  compensate  for  the 
processing  time  (Table  32.1). 

Color  images  can  be  compressed  further  than  monochrome 
because  luminance  (brightness)  and  chrominance  (color)  are 
treated  separately.  The  eye  can  tolerate  a  greater  loss  of  informa¬ 
tion  in  the  chrominance  signal,  so  this  is  normally  handled  at  half 
the  resolution.  (The  standard  allows  many  different  options  here 
but  specific  implementations  usually  do  not  make  these  evident  to 
the  user.)  This  has  certain  consequences  in  confocal  microscopy 
because  a  three-channel  confocal  image  is  not  a  real-color,  real- 
world  image  but  three  images  which  are  largely  independent  of 
each  other.  A  three-channel  confocal  image  compressed  as  a  color 
image  will  look  quite  adequate  but  should  not  be  used  reliably  for 
numerical  analysis;  for  example,  the  lower  resolution  of  the  color 
information  would  make  many  pixels  show  colocalization  when  in 
fact  there  is  none. 

The  JPEG  standard  itself  specifies  a  compression  technique, 
not  a  file  format.  As  such  it  is  used  in  many  different  situations 
(including  one  of  the  compression  options  in  the  TIFF  standard 
and  in  programs  such  as  Microsoft  PowerPoint).  However,  it  is 
most  familiar  to  the  end  user  in  the  form  of  files  conforming  to  the 
JFIF  (JPEG  file  interchange  format)  standard,  which  typically  use 
the  suffix  .jpg.  JPEG  compression  is  designed  for  photographic 
images  so  that  it  only  manipulates  gray-scale  or  true  color  (RGB) 
images.  Adding  a  false-color  palette  to  a  gray-scale  image  will 
make  it  less  suitable  for  JPEG  compression  because  the  JPEG 
algorithm  would  convert  it  to  a  full  color  image,  tripling  its  size, 
before  compression.  Lossless  JPEG  compression  also  exists;  there 
have  been  two  distinct  lossless  compression  modes  specified  in  the 
JPEG  standard  over  the  years,  but  these  do  not  use  DCT  to  com¬ 
press  the  image  and  typically  do  not  perform  very  well,  so  they 
have  not  become  popular.  The  current  version,  JPEG-LS,  uses  a 
predictive  algorithm  formerly  called  LoCo,  and  is  designed  to  be 
both  fast  and  easy  to  implement. 

Other  specific  image  compression  techniques  show  consider¬ 
able  potential  but  have  yet  to  achieve  the  popularity  of  JPEG 
(DCT).  Fractal  compression,  a  proprietary  technique  developed  by 
Iterative  Systems  Inc.  (Anson,  1993;  Barnsley  and  Hurd,  1993), 
creates  mathematical  expressions  which,  when  iterated,  recreate 
the  original  image.  It  can  give  spectacular  levels  of  compression. 
Unlike  JPEG  compression,  creating  the  compressed  image  is  a 
very  time-consuming  process  but  decompression  is  very  quick. 
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This  has  made  it  most  useful  for  such  items  as  CD-ROM 
encyclopedias  but  its  initial  promise  has  not  led  to  widespread 
adoption. 

Wavelet  compression  is  currently  the  hot  topic  in  image  com¬ 
pression  and  will  undoubtedly  be  in  common  use  throughout  the 
lifetime  of  the  current  edition  of  this  book,  though  at  the  time  of 
this  writing  it  is  only  just  appearing  in  the  latest  releases  of  main¬ 
stream  implementations.  It  is,  in  a  sense,  mathematically  compa¬ 
rable  to  JPEG  in  that  it  separates  the  frequency  components  in  an 
image,  but  it  works  in  real  space  rather  than  reciprocal  space.  The 
basic  idea  of  separating  an  image  into  components  of  different  res¬ 
olution  and  discarding  the  lowest  information  content  and  highest 
frequencies  first  is  similar,  but  it  is  achieved  by  passing  a  series  of 
filters  over  the  image  at  a  range  of  different  scales.  The  filters  — 
wavelet  filters  —  are  the  key  to  this,  and  are  designed  to  be 
reversible.  The  claim  is  that  wavelets  can  offer  useful  compression 
without  loss,  and  much  greater  compression  with  losses  that  are 
not  obvious  to  the  eye.  Other  advantages  include  the  ability  to 
rapidly  generate  a  low-resolution  image  (using  the  coarsest 
wavelets)  and  fill  in  the  detail  afterwards. 

Wavelet  compression  can  treat  an  image  as  a  whole  or  break 
it  down  into  blocks  which  are  compressed  individually.  The  JPEG 
has  introduced  wavelet  compression  into  a  new  version  of  the 
JPEG  standard  (JPEG  2000),  and  it  is  in  this  format  that  most 
mainstream  applications  will  offer  wavelet  compression.  In  the 
interests  of  speed  and  portability  (  wavelet  compression  is  intrinsi¬ 
cally  slower  than  DCT),  the  JPEG  2000  implementation  uses  only 
two  wavelet  filters,  one  for  lossless  compression  and  one  for  lossy 
compression.  Even  so,  the  time  required  is  quite  noticeable  even 
on  a  fast  computer  (Table  32.1).  While  a  wider  range  could  offer 
better  performance  by  finding  the  best  wavelet  for  each  image,  the 
practical  difficulties  involved  were  deemed  to  make  it  not  worth¬ 
while.  Also,  in  the  JPEG  implementation  the  image  is  broken  into 
blocks  before  compression.  A  major  criticism  of  the  DCT  JPEG 
standard  was  that  the  8x8  blocks  could  often  become  visible  at 
high  levels  of  compression  and  JPEG  2000  therefore  offers  vari¬ 
able  sized  blocks  within  a  single  image,  so  that  one  compression 
level  can  be  applied  to  featureless  regions  (such  as  sky,  or  the  back¬ 
ground  in  a  confocal  image)  and  another  to  regions  containing  fine 
detail. 

In  practice,  however,  wavelet  compression  does  not  seem  to 
offer  superior  performance  over  DCT  for  confocal  images,  as 
Figure  32.1  shows.  Figure  32.1(A)  shows  a  cultured  He-La  cell 
labeled  with  fluoroscein  isothiocyanate  (FITC)  tagged  to  an  anti¬ 
body  against  |3-tubulin.  It  is  an  average  projection  from  16  confo¬ 
cal  optical  sections  —  a  512  X  512  pixel  8-bit  image.  Using  an 
average  projection  rather  than  a  maximum  brightness  projection 
improves  the  signal-to-noise  ratio,  but  it  also  reduces  the  total 
intensity  (because  so  much  of  the  image  is  dark)  and  this  therefore 
reduces  the  number  of  gray  values  present  (there  are  only  120 
values  in  this  image).  Both  factors  make  the  image  a  better  candi¬ 
date  for  compression.  To  preserve  the  visual  quality  the  contrast 
has  been  scaled  and  the  gamma  changed  (see  below);  these  oper¬ 
ations  simply  change  the  values  assigned  to  each  of  the  120  tones, 
they  do  not  change  the  number  of  tones  and  should  not  affect  how 
it  will  compress.  Figure  32.1(B)  is  one  of  the  original  slice  images 
with  no  modifications  to  gray  values.  It  shows  more  noise  than  the 
projection,  but  contains  248  gray  levels,  showing  that  the  gain  and 
black  level  controls  had  been  used  optimally  to  make  use  of  the 
full  dynamic  range  without  overflow  or  underflow. 

The  raw  image  size  in  each  case  is  256  KB,  and  tif  and  bmp 
files  are  257KB.  An  LZW-compressed  tif  file  of  Figure  32.1(A) 
offered  a  reasonably  useful  reduction  to  170KB,  while  a  PNG  file 


created  with  the  well-known  program  Paint  Shop  Pro  (JASC  Soft¬ 
ware)  did  rather  better  at  143  KB.  The  PNG  optimizing  program 
Pngcrush  (freeware;  see  Roelofs,  2003)  made  an  insignificant 
improvement  to  142KB.  This  is  55%  of  the  original  file  size  and 
shows  that  with  a  restricted  gray  range  and  dark  noise-free  back¬ 
ground  reasonable  compression  can  be  achieved  without  loss. 
Lossless  wavelet  compression  (JPEG  2000)  was  less  effective, 
giving  a  file  size  of  168  KB,  scarcely  better  than  LZW-compressed 
tiff  but  taking  very  much  longer  to  compress  and  decompress. 
Lossless  JPEG  was  comparable,  at  169KB. 

As  predicted,  the  original  single-slice  image  [Fig.  32.1(B)]  did 
not  compress  nearly  so  well;  the  LZW  version,  at  256  KB,  was 
hardly  changed  from  the  original  size.  PNG  did  better,  at  195  KB 
(204  KB  before  optimization).  But  at  76%  of  the  original  size  it 
hardly  seems  worth  the  effort.  It  does,  though,  reinforce  the  point 
that  PNG  is  the  only  format  worth  considering  for  lossless  com¬ 
pression  of  confocal  images. 

DCT  (JPEG)  compression  of  the  projection  [Fig.  32.1(A)]  to 
two  different  levels  is  seen  in  Figure  32.1(C,D).  Figure  32.1(C) 
shows  the  image  compressed  to  26.4  KB,  around  10%  of  its  orig¬ 
inal  size.  While  some  loss  of  quality  is  evident,  the  image  remains 
perfectly  usable  and  the  compression  is  very  substantial.  In  Figure 
32.1(D),  compression  has  been  increased  to  the  point  where  the 
image  is  visibly  degraded  but  still  recognizable  and  even  infor¬ 
mative,  though  the  file  size  is  only  7.7KB,  a  mere  3%  of  the  orig¬ 
inal!  Figure  32.1(E,F)  shows  the  same  levels  of  compression  but 
using  wavelet  compression  with  JPEG  2000.  Both  are  substantially 
worse  than  equivalent  DCT  images.  A  specialist  wavelet  com¬ 
pression  program  (not  using  JPEG  2000)  was  also  tried,  and  gave 
worse  results  at  equivalent  compression  levels.  It  seems  probable 
that  the  relative  failure  of  wavelets  to  compete  with  DCT  lies  in 
the  rather  limited  range  of  resolution  levels  which  contain  sub¬ 
stantial  information  in  these  confocal  images.  The  interest  lies  pri¬ 
marily  in  the  microtubules,  all  of  which  are  the  same  size.  In 
reciprocal  space,  regions  with  no  information  will  automatically 
compress  to  nothing,  whereas  the  wavelet  function  may  perhaps 
be  chosen  to  treat  all  frequencies  more  or  less  equally  because  this 
may  be  the  best  strategy  for  conventional  photographic  images. 
There  may  therefore  be  scope  for  a  wavelet  implementation  dedi¬ 
cated  particularly  to  confocal  images. 

Figure  32.2  shows  the  histograms  of  the  images  in  Figure  32.1. 
In  Figure  32.2(A)  the  missing  gray  values  are  obvious,  whereas 
the  single  optical  section  [Fig.  32.2(B)]  shows  a  continuous  spec¬ 
trum.  At  10%  compression  the  DCT  image  [Fig.  32.2(C)]  shows  a 
similar  spectrum,  but  smoothed  and  with  the  gaps  in  the  gray  levels 
now  filled.  The  wavelet  version  [Fig.  32.2(E)]  also  preserves  the 
same  shape,  but  is  rather  more  smoothed  at  the  same  compression. 
At  3%  of  the  original  size  the  DCT  histogram  [Fig.  32.2(D)]  is 
very  much  changed,  while  the  wavelet  one  [Fig.  32.2(F)]  shows 
little  change  from  the  10%  compression.  In  each  case,  the  mean 
value  remains  unchanged.  These  figures  show  that  photometric 
parameters  are  surprisingly  well  conserved  even  at  levels  of  com¬ 
pression  that  would  seldom  be  used  in  practice.  While  wavelet 
compression  affects  the  histogram  more  than  DCT  at  10%  com¬ 
pression,  it  is  more  accurate  than  DCT  at  3%  compression  so  that 
even  though  the  image  looks  worse,  its  photometric  parameters 
remain  closer  to  the  original. 

In  practice  these  compression  levels  would  only  be  used  for 
such  purposes  as  Internet  transmission  of  images.  Compression  to 
between  25%  and  50%  of  the  original  size  would  give  images  of 
more  general  usefulness,  with  little  visible  change  from  the  origi¬ 
nal.  Even  essential  photometric  parameters  are  preserved.  In  spite 
of  the  current  interest  in  wavelet  compression,  DCT  still  seems  a 
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FIGURE  32.1 .  Effects  of  image  compression  on  a  confocal  fluorescence  image  of  a  cultured  He-La  cell  immunostained  with  FITC  against  p-tubulin.  (A)  Average 
projection  of  the  original  dataset  of  16  optical  sections,  with  contrast  scaled  and  gamma  subsequently  corrected;  original  uncompressed  image.  (B)  One  optical 
section  from  the  stack,  with  no  subsequent  processing.  (C)  JPEG  compressed  (DCT)  to  ~10%  of  the  original  size.  (D)  JPEG  compressed  (DCT)  to  ~3%  of  the 
original  size.  (E)  Wavelet  compressed  (JPEG  2000)  to  ~10%  of  the  original  size.  (F)  Wavelet  compressed  (JPEG  2000)  to  ~3%  of  the  original  size.  Insets  in  (A, 
C-F)  are  part  of  the  image  at  2x  magnification  to  show  the  losses  in  compression  more  clearly. 
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FIGURE  32.2.  (A-F)  Histograms  of  pixel  intensities  in  Figure  32.1(A-F), 
respectively. 


better  choice  for  confocal  images  in  cell  biology.  Not  only  is  it 
more  effective,  it  is  much  faster  then  wavelet  compression  (Table 
32.1).  Lossless  compression  only  gives  useful  results  on  images 
with  large  amounts  of  uniform  background  and  low  noise  but  in 
these  cases  it  can  be  effective.  The  most  likely  use  would  be  for 
storing  the  output  of  3D  reconstructions,  as  in  Figure  32.1(A). 


Although  generating  a  complex  3D  movie  sequence  can  take  as 
long  as  acquiring  the  original  confocal  data,  and  the  output  files 
can  be  just  as  large,  we  typically  do  not  have  the  same  concerns 
about  preserving  data  integrity.  It  is  therefore  sensible  to  use  JPEG 
compression  for  storing  the  output. 

Some  confocal  datasets  contain  only  very  sparse  information. 
Figure  32.3(A)  provides  an  example,  a  frame  (pre-calcium  wave) 
from  a  time  series  of  calcium  transients  induced  by  testosterone. 
There  were  193  images  in  the  series  and  without  compression  this 
dataset  occupies  close  to  50  MB.  However,  as  only  12%  of  the 
pixels  lie  above  the  background  noise  level,  the  dataset  even  in  its 
original  form  compresses  without  loss  to  below  100  kB  per  frame 
—  40%  of  the  original  —  with  LZW  or  PNG.  If  we  remove  back¬ 
ground  by  setting  pixels  with  a  gray  value  of  14  or  below  to  zero 
[Fig.  32.3(B)],  we  have  a  virtually  unchanged  image  which  is  now 
highly  compressible  without  further  loss.  PNG  compression  gave 
a  hie  size  of  only  49.3  KB,  less  than  20%  of  the  original.  Our  orig¬ 
inal  50MB  dataset  will  now  only  be  10MB.  Lossy  JPEG  com¬ 
pression  makes  no  sense  with  such  a  dataset  —  using  a  typical 
setting  for  reasonable  image  quality  the  resultant  hie  size  was  actu¬ 
ally  larger  (59.3  KB)  than  the  lossless  one.  What  is  more,  the  com¬ 
pression  process  brought  background  back  into  the  dark  areas.  So 
the  message  is  either  use  a  lossy  compression  on  the  original  data 
or  compress  it  by  background  subtraction  and  then  save  it  without 
further  loss  —  do  not  do  both. 

Other  image  manipulations  will  also  affect  the  compressibility 
of  images.  Smoothing,  to  remove  noise,  will  reduce  the  high-fre¬ 
quency  content  and  therefore  make  images  more  compressible. 
Deconvolution,  on  the  other  hand,  aims  to  restore  high-frequency 
content.  This  will  make  images  less  compressible,  or  will  mean 
that  more  is  lost  in  lossy  compression.  Figure  32.4  illustrates  this 


FIGURE  32.3.  Calcium  imaging  (non-ratiometric)  of  transients  induced  by  testosterone  in  cultured  cells.  Pre-stimulation,  time  point  42  from  193  images  taken 
at  1-second  intervals.  (A)  original  image  (B)  background  removed  by  setting  all  pixels  below  a  value  of  15  to  zero,  a  process  that  permits  no-loss  compression 
to  reduce  file  size  of  (B)  by  a  factor  of  two  compared  to  (A).  A  false-color  palette  has  been  added  to  show  how  the  background  has  been  set  to  black  but  none 
of  the  “data”  pixels  have  changed.  Taken  using  a  63x/NA  1.2  water-immersion  lens.  Image  width  as  shown  (only  part  of  the  original  image)  is  154pm.  Image 
courtesy  Dr.  Alison  Death. 
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FIGURE  32.4.  Three  different  angle  projec¬ 
tions  (0°,  45°,  and  90°)  from  a  3D  dataset  of 
the  dinoflagellate  alga  Dinophysis.  (A) 
Maximum  intensity  projection  from  the  orig¬ 
inal  dataset.  (B)  Maximum  intensity  projec¬ 
tion  after  smoothing  (3D  median  filter)  and 
one-dimensional  deconvolution. 


point.  This  is  a  3D  dataset  of  the  dinoflagellate  alga  Dinophysis, 
which  was  collected  at  3  pixels  per  resel  and  therefore  is  slightly 
oversampled.  This  provides  the  opportunity  to  smooth  the  data 
down  to  the  Nyquist  limit  and  thereby  reduce  noise  without 
adversely  affecting  resolution.  Figure  32.4(A)  shows  maximum 
intensity  projections,  at  three  angles,  from  the  original  dataset.  This 
type  of  projection  leaves  noise  unchanged  so  the  view  shows  an 
accurate  impression  of  the  noise  content  of  the  original  set.  Com¬ 
pressed  with  LZ77  the  original  4.5  MB  dataset  reduces  to  1.9MB, 
a  useful  saving,  reflecting  the  large  proportion  of  background  in 
the  set.  When  the  entire  dataset  is  smoothed  with  a  median  filter, 
acting  in  three  dimensions  (Cox  and  Sheppard,  1999),  it  becomes 
more  compressible,  now  reducing  to  1.38MB.  If  we  deconvolve 
this  dataset  we  can  restore  some  of  the  resolution  lost  in  the  z- 
(depth)  direction  by  the  transfer  function  of  the  microscope  (Cox 
and  Sheppard,  1993,  1999).  As  expected,  it  is  now  rather  less  com¬ 
pressible,  at  1.47  MB,  but  this  is  still  a  useful  saving  on  the  origi¬ 
nal.  The  smoothed,  deconvolved  dataset  is  shown  in  the  same 
projections  in  Figure  32.4(B). 

In  any  image  compression  strategy,  it  is  important  to  bear  in 
mind  that  confocal  images  can  become  virtually  meaningless  if  the 
information  about  the  acquisition  is  lost.  Some  confocal  systems 
(e.g.,  Bio-Rad)  store  this  data  in  a  header  within  the  same,  single 
file  as  a  series  of  optical  sections.  Even  if  the  slice  images  are 
exported  by  the  Bio-Rad  software,  the  acquisition  data  is  not 
exported  and  the  images  cannot  be  re-imported  for  subsequent  pro¬ 
cessing.  Other  systems  (e.g.,  Leica,  Zeiss)  store  a  database  of  infor¬ 
mation  about  the  images  —  exported  images  generated  within  the 
acquisition  software  will  still  retain  some  of  this  information  but 
typically  3D  reconstructions  can  only  be  done  from  the  original 
images.  In  either  case  it  is  important  to  ensure  that  the  all-impor¬ 
tant  image  acquisition  data  are  preserved,  and  if  possible  that  the 
images  can  be  restored  to  their  original  file  name  and  type. 

A  final  point:  the  most  common  waste  of  disk  space  consists 
of  storing  completely  featureless  areas!  If  your  sample  is  rectan¬ 
gular,  select  a  rectangular  window  to  image  it  rather  than  collect¬ 
ing  a  strip  of  nothing  on  each  optical  section.  And  do  not  collect 


three  channels  if  you  have  only  two  labels!  Modern  systems  make 
it  all  too  easy  to  accept  the  default  method,  or  configuration;  it  will 
save  a  lot  of  time  in  the  long  run  if  you  spend  a  minute  or  two 
changing  settings  to  collect  only  what  you  want. 

Removable  Storage  Media 

Storage  media  can  be  divided  into  those  which  are  sequential 
(records  are  written  and  read  from  one  end  only)  and  those  which 
are  random  access  (it  is  possible  to  move  directly  to  any  record, 
whenever  it  was  written). 

Sequential  Devices 

Sequential  devices  are  tapes  of  various  formats  and  sizes  storing 
up  to  200  GB  on  a  single  cassette.  Tape  is  still  the  largest- 
capacity  bulk  storage  medium  available,  but  is  no  longer  compet¬ 
itive  in  cost  with  optical  storage.  As  an  image  storage  system,  it 
also  suffers  from  the  time  taken  to  locate  and  recover  any  one  file. 
A  single  file  cannot  be  erased  and  replaced  by  another;  one  must 
erase  either  the  whole  tape  or  a  large  group  of  files,  depending 
upon  the  recording  system.  Also,  although  it  is  rewritable  it  will 
not  stand  an  infinite  number  of  uses.  The  tape  surface  has  a  much 
harder  life  than  the  surface  of  a  disk  —  it  comes  into  direct  contact 
with  the  recording  heads  and  capstans,  and  is  coiled  and  uncoiled 
each  time.  Even  reading  files  repeatedly  wears  the  tape,  and  its 
long-term  archival  potential  is  dubious.  Once  tape  drives  were  reg¬ 
ularly  used  for  data  storage  and  transfer  but  now  their  use  is  almost 
exclusively  for  backup  purposes  —  making  a  copy  of  a  complete 
file  system  or  subsystem  which  will  typically  be  read  only  once, 
in  the  event  of  a  hard  disk  failure. 

Modern  tape  systems  are  very  specifically  designed  for  this 
task;  their  purchase  cost  is  high  but  cost  per  megabyte  stored  can 
be  low  compared  to  other  rewritable  media.  This  gives  them  some 
attraction  for  long-term  archival  storage  of  images  that  will  not 
need  to  be  accessed  regularly,  and  for  very  large  collections  of 
images.  Dumping  a  40  or  100  GB  hard  disk  full  of  images  on  to  a 
single  tape  will  be  much  quicker  and  simpler  than  writing  to 
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dozens  of  compact  disks  (CDs)  or  digital  video  disks  (DVDs). 
However,  most  tape  systems  now  rely  on  specific  software  to 
handle  them  and  both  this  software  and  suitable  hardware  will  need 
to  be  available  for  the  tape  to  be  read  in  the  future  —  past  expe¬ 
rience  suggests  that  this  will  limit  effective  use  to  5  years  or  so, 
and  this  is  probably  the  realistic  limit  for  tape  life  also. 

Transfer  rates  up  to  24  MB  s-1  are  available  on  expensive  high- 
end  systems,  although  systems  designed  for  small  computer  use 
will  offer  no  better  than  3 MBs-1.  At  24 MBs-1  writing  one  CD 
worth  of  data  will  take  only  30  s,  but  it  will  take  a  quarter  of  an 
hour  to  copy  a  20  GB  hard  disk.  At  3  MB  s-1  that  same  disk  will 
take  almost  2  h  to  copy. 

Manufacturers  typically  quote  compressed  capacities  for  their 
tape  drives,  based  on  a  notional  2-fold  compression  ratio  that  they 
expect  to  achieve  with  their  archiving  software.  This  is  unrealistic 
when  dealing  with  image  files,  and  when  evaluating  competing 
systems,  it  is  important  to  compare  actual,  uncompressed,  storage 
capacities;  this  is  much  closer  to  the  figure  achievable  with  micro¬ 
scope  images. 

Random-Access  Devices 

Random-access  devices  comprise  a  range  of  disk  media,  either 
magnetic  or  optical,  and  solid-state  devices. 

Magnetic  Disks 

The  oldest  and  simplest  of  removable  media,  rewritable,  random- 
access  systems  is  the  humble  floppy  diskette.  These  are  now  vir¬ 
tually  obsolete,  limited  by  their  small  capacity  —  1.4  MB  in  the 
only  (marginally)  surviving  3.5"  version.  As  many  will  have  found 
out,  finding  a  drive  to  read  the  once  ubiquitous  5.25"  disks  is 
already  difficult.  In  any  case,  they  are  too  small  to  be  relevant  for 
confocal  images. 

Various  types  of  super-floppy  have  had  a  vogue  in  the  past,  but 
the  only  current  survivor  seems  to  be  the  Iomega  Zip  disk,  which 
originally  held  100MB  but  now  comes  in  capacities  up  to  750MB. 
These  are  robust  and  durable  but  seem  unlikely  to  be  current  for 
very  much  longer,  driven  out  by  far  cheaper  optical  technology. 
They  are  also  too  limited  in  space  to  meet  most  modern  needs  for 
confocal  image  storage.  Cost  per  gigabyte  is  around  US$20-100. 

Other  removable  platter  magnetic  devices  have  been  current, 
and  suffer  from  the  same  limitation  that  in  the  course  of  time  there 
may  no  longer  be  hardware  available  to  read  them.  One  of  the  most 
successful  at  the  time  of  writing  is  the  Orb  drive,  available  in 
capacities  from  2  to  5  GB.  Like  many  other  portable  devices  they 
connect  to  the  host  computer  by  the  USB  (universal  serial  bus) 
port,  or  the  parallel  printer  port.  Parallel  port  connection  is  rela¬ 
tively  slow  and  USB  is  by  far  the  preferable  option.  Cost  per  giga¬ 
byte  is  of  the  order  of  US$10  to  US$20,  so  it  is  a  reasonably 
affordable  option. 

There  remains  the  option  of  just  using  conventional  hard  disks. 
Mounting  kits  are  available  to  fit  a  conventional  disk  in  a  pull-out 
mount;  disks  are  also  available  in  cases  for  connecting  to  USB, 
Fire  Wire,  or  SCSI  (small  computer  systems  interface)  ports,  and 
there  are  microsized  ones  which  fit  the  PC  card  (PCMCIA)  slot  in 
notebook  computers.  The  recent  fall  in  price  and  increase  in  capac¬ 
ity  of  hard  disks  has  made  this  a  surprisingly  affordable  option 
(below  US$1.00  per  gigabyte  for  IDE  disks,  more  for  SCSI).  Data 
transfer  is  as  fast  as  the  disk  —  certainly  faster  than  most  other 
options  —  and  rewriting  capacity  is  effectively  unlimited.  The 
long-term  potential  is  less  certain  because  the  durability  of  the 
system  depends  not  only  on  the  longevity  of  the  magnetic  medium, 
but  also  on  the  lifespan  of  the  motor  and  heads. 


Optical  Disks 

In  the  previous  edition,  devices  such  as  WORM  (write  once,  read 
many)  and  MO  (magneto-optical)  disks  were  discussed.  These,  like 
so  many  technologies,  are  not  only  dead  but  virtually  forgotten 
except  by  those  laboratories  which  have  a  huge  stock  of  the  disks! 
However,  optical  technology  is  certainly  the  current  preferred 
option  because  there  is  good  reason  to  have  faith  in  the  archival 
durability  of  the  media.  Furthermore,  mass-market  devices  now 
have  sufficient  capacity  to  meet  many  users’  demands  so  that  one 
can  have  some  confidence  in  the  longevity  of  the  technology. 

Compact  Disks 

Compact  disks  (CDs)  have  already  been  with  us  for  over  20  years, 
and  writable  CDs  for  10.  The  cost,  high  when  the  previous  edition 
was  written,  is  now  very  low  both  in  first  cost  and  media  (around 
US$0.70  per  gigabyte).  Speed,  though  it  has  increased  about 
12-fold  since  then,  is  still  the  major  problem.  The  rate  of  data  trans¬ 
fer  for  an  audio  CD  is  a  rather  pedestrian  150  KB  s-1,  and  this  is 
referred  to  as  single  speed.  Read  and  write  speeds  up  to  52x  this 
base  value  are  now  available.  A  complete  700  MB  CD  can  thus  be 
written  in  5  min  or  so,  and  modern  software  will  adjust  the  writing 
speed  on  the  fly  so  that  the  need  to  maintain  a  constant  data  stream 
is  less  of  an  issue.  This  means  that  CDs  can  now  even  be  written 
across  a  network,  though  this  will  inevitably  carry  a  speed  penalty. 
It  may  still  be  preferable  to  carry  the  additional  overhead  of  first 
copying  files  to  the  writing  computer. 

Rewritable  CDs  are  also  widely  available  at  a  cost  only  a  little 
higher  than  conventional  single-use  CDs.  Erasing  data  for  re-use 
is,  however,  a  relatively  slow  process.  They  may  be  useful  when 
images  are  to  be  stored  for  a  short  time  only,  but  for  long-term 
archival  use  it  would  seem  wiser  to  use  single-use  disks.  Many 
manufacturers  have  conducted  accelerated-aging  tests  on  their 
single-use  CDs  and  their  security  as  archival  storage  seems  to 
be  the  best  of  all  mass-market  computer  media.  It  seems  inevitable 
that  rewritable  disks  could  not  offer  equal  security,  and  the  risk  of 
accidental  deletion  is  always  present  with  any  rewritable  medium. 
In  fact,  the  time  spent  trying  to  decide  which  files  can  be  over¬ 
written  is  usually  worth  much  more  than  the  disk  space  saved. 

Various  formatting  options  now  allow  multiple  use,  either  by 
writing  multiple  sessions  (which  does  carry  an  overhead  of  about 
15  MB  per  session)  or  by  using  the  packet  CD  format,  which 
allows  a  CD  to  be  treated  almost  exactly  like  a  conventional 
mounted  drive.  Multi-session  CDs  can  be  read  on  most  systems 
but  the  packet  CD  format  cannot.  Because  it  reduces  compatibil¬ 
ity  with  other  systems  and  has  little  point  when  the  content  of  a 
CD  is  relatively  small  compared  to  a  modern  hard  disk,  packet  CD 
has  not  become  widely  popular. 

One  of  the  limitations  of  the  CD  format  is  its  handling  of  file 
names.  The  standard  laid  down  by  the  International  Standards 
Organization  (ISO)  requires  file  names  to  fit  an  8  +  3  character 
format  similar  (but  not  identical)  to  that  of  MS-DOS.  ISO- 
compatible  CDs  are  readable  on  Apple,  PC,  and  Unix  computers, 
which  is  very  convenient  for  data  exchange.  Unfortunately,  most 
confocal  microscopes  give  files  and  directories  (folders)  much 
longer  names.  Extensions  to  the  standard  allow  for  longer  files 
names  in  both  Macintosh  and  Windows  computers,  but  these  are 
unfortunately  not  cross-platform  compatible.  Because  most  con¬ 
focal  microscopes  use  Windows  it  is  important  to  use  the  loliet 
extension  which  caters  for  these  file  names,  otherwise  the  disk  will 
contain  a  useless  collection  of  truncated  names,  particularly  with 
microscopes  such  as  current  Leica  models,  which  save  each  plane 
and  channel  as  a  separate  file,  and  rely  on  a  database  program  to 
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identify  these  images.  On  an  ISO  disk,  it  will  be  impossible  to 
identify  which  plane  and  channel  are  which,  and  the  data  becomes 
completely  useless. 

Because  CDs  are  likely  to  be  a  major  archival  medium,  in  the 
medium  term  at  least,  the  question  obviously  arises  as  to  how  per¬ 
manent  they  are.  Pressed  CDs  have  a  polycarbonate  blank  into 
which  the  pits  are  pressed  to  carry  the  information.  This  surface  is 
then  coated  with  an  evaporated  metal  layer,  and  then  a  coat  of 
varnish  and  the  printed  label  (Fig.  32.5,  upper).  The  CD  is  read 
through  the  thickness  of  the  blank.  If  the  clear  side  of  the  blank 
gets  scratched,  it  will  hinder  reading  but  it  can  often  be  repolished. 
The  label  side  is  more  vulnerable  because  only  a  layer  of  varnish 
and  the  printed  label  lie  between  the  data  and  the  outside  world. 
Recordable  CDs  have  a  dye  layer  between  the  polycarbonate  blank 
and  the  metal  film  and  it  is  this  which  is  modified  by  the  writing 
laser  beam  (Fig.  32.5,  lower). 

In  terms  of  pressed  CDs,  excluding  physical  damage,  the  key 
issues  are  the  aluminum  reflective  coating  (which  can  get  oxidized, 
particularly  if  there  are  any  flaws  in  the  varnish)  and  the  polycar¬ 
bonate  blank.  So  far,  no  plastic  seems  to  last  forever  and  I  doubt 
if  polycarbonate  will  stay  clear  and  flexible  indefinitely.  However, 
as  polycarbonate  is  vulnerable  to  almost  all  organic  solvents, 
excluding  light  and  solvent  fumes  will  doubtless  help. 

Archival  quality  recordable  CDs  usually  use  something  better 
than  aluminum.  Several  manufacturers  offer  silver,  silver  +  gold, 
or  pure  gold.  Obviously  pure  gold  should  be  highly  stable,  but  it 
is  less  reflective,  which  increases  the  risk  of  read  errors.  Whatever 
metal  is  used,  archival  life  still  depends  on  the  dye  layer  in  front 
of  it  remaining  stable.  The  claims  made  by  the  manufacturer  for 
their  different  dyes  (typically  cyanine,  phthalocyanine,  or  azo)  are 
difficult  to  evaluate.  The  cyanine  dyes  used  in  the  earliest  record¬ 
able  CDs  were  rather  vulnerable  to  bright  ambient  light.  Some 
manufacturers  have  chosen  to  concentrate  on  extending  the  dura¬ 
bility  of  these  dyes,  whereas  others  have  turned  to  alternative  dyes 
such  as  azo  or  phthalocyanine.  All  archival  tests  depend  on  accel- 
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FIGURE  32.5.  Structure  of  a  pressed  (above)  and  recordable  (below)  CDs  (not 
to  scale). 


erated  aging  (typically  at  higher  temperature)  and,  while  this  is 
valid  up  to  a  point,  it  is  unwise  to  trust  it  too  implicitly  (Nugent, 
1989;  Stinson  et  al.,  1995). 

The  speed  at  which  drives  will  write  CDs  has  increased  enor¬ 
mously  over  the  years,  with  52x  now  routine.  This  has  placed  pres¬ 
sure  on  manufacturers  to  increase  the  response  time  of  the  dye 
layer  but  it  would  seem  logical  that  a  dye  which  can  be  bleached 
at  50  times  the  original  speed  is  unlikely  to  be  as  archivally  stable 
as  the  older  disks.  Often  the  layer  is  made  much  thinner  to  enable 
the  high-speed  writing.  Of  course,  dye  technology  is  also  evolv¬ 
ing  but,  if  an  archival-quality  disk  will  not  support  the  latest 
writing  speeds,  there  may  be  a  good  reason. 

Because  one  is  likely  to  write  or  put  a  label  on  the  back,  the 
varnish  is  important.  Most  makers  object  to  labels  even  though 
labeling  kits  are  widely  sold.  The  varnish  is  typically  water  based 
because  the  polycarbonate  of  the  disks  is  very  vulnerable  to  sol¬ 
vents.  This  leaves  one  in  a  cleft  stick  as  to  how  to  label  it  because 
water-based  inks  may  loosen  the  varnish  but  solvent  ones  may 
attack  the  disk!  Different  manufacturers  vary  in  their  recommen¬ 
dations  and  the  safest  approach  is  to  follow  the  recommendation 
for  each  particular  brand.  Quality  disks  will  have  an  extra  writable 
protective  layer  over  the  base  varnish  giving  you  a  bit  of  extra 
security,  and  this  is  well  worth  having. 

Kodak  recommends  that  CDs  not  be  stacked  adjacent  to  each 
other  or  to  any  other  surface.  They  should  therefore  be  stored  in 
“jewel  cases”  or  in  a  custom  storage  box  which  separates  the  disks, 
and  not  kept  in  envelopes  or  stacked  on  a  spindle. 

Blank  CDs  are  now  so  cheap  that  the  cost  of  storage  is  below 
US$1  per  gigabyte,  depending  on  the  brand  and  quality  of  the 
media.  Common  sense  suggests  that,  however  reasonable  it  may 
be  to  choose  cheap  disks  when  just  sending  data  through  the  post 
or  taking  it  from  laboratory  to  laboratory,  saving  a  few  cents  by 
choosing  unknown  brands  is  a  false  economy  if  the  intention  is 
archival  storage.  Writers  are  very  cheap  and  quite  fast  (48  speed 
corresponds  to  7MB/s,  comparable  with  modern  tape  systems). 
Best  of  all,  every  computer  can  read  the  disk  without  extra  hard¬ 
ware.  The  huge  range  of  commercial  CD-ROMs  ensures  that 
readers  will  remain  available  for  many  years,  so  that  archival  mate¬ 
rial  will  be  accessible  as  well  as  secure. 

Digital  Video  Disk  (DVD) 

DVD  (digital  video  disks)  represent  the  next  stage  of  optical  disk 
technology.  Using  similar  technology,  but  shorter  wavelength 
lasers  so  that  resolution  is  better,  4.7  GB  can  be  stored  on  one  side 
of  a  disk  with  the  same  size  as  a  CD.  Because  the  optics  that  read 
the  disk  are  confocal,  a  DVD  can  carry  two  separate  layers  of  infor¬ 
mation,  thus  storing  over  9  GB,  but  recordable  two-layer  disks  are 
only  just  coming  on  to  the  market  at  the  time  of  this  writing. 

DVD-R  write-once  DVD  disks  —  are  now  reasonable  in  price, 
at  around  US$1.00  per  disk  from  the  cheapest  sources,  and  the 
writers  are  now  reasonable  at  US$200  to  US$300  (a  mere  1%  of 
their  price  4  years  ago!).  Two  standards  (DVD+R  and  DVD-R) 
exist  for  these  disks.  This  has  hindered  their  general  acceptance, 
but  newer  players  will  handle  both  (Nathans,  2003).  At  4.7GB,  it 
is  clear  that  DVD  is  now  both  cheaper  and  more  convenient  than 
CD-R  for  storage  of  confocal  images,  though  the  question  of  the 
long-term  stability  and  longevity  of  the  format  is  still  not  so  well 
known  as  that  of  the  CD  format.  Nevertheless,  DVD  players  are 
now  common  domestic  appliances,  so  it  seems  likely  that  the  stan¬ 
dard  will  be  durable. 

Rewritable  DVD  disks  also  suffer  from  incompatible  standards 
and  because  they  are  of  lower  reflectivity  are  sometimes  difficult 
to  read  in  DVD  players.  The  older  standards  (DVD-ROM  and 
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DVD-RAM)  were  also  lower  capacity  than  4.7  GB.  As  with  CD- 
RW,  they  are  probably  not  the  best  alternative  for  archival  storage, 
but  could  have  their  place  for  data  transfer.  Current  drives  mostly 
handle  RW  and  R  disks  in  both  +  and  -  options.  As  with  CDs, 
rewritable  disks  are  always  slower  to  write. 

It  is  only  in  the  past  couple  of  years  that  the  DVD  market  has 
really  showed  signs  of  maturity.  Because  most  computer  drives 
will  read  and  write  the  CD  format  as  well,  it  would  seem  to  be  the 
logical  choice  when  purchasing  a  new  system,  and  DVD  writers 
are  now  routine  on  new  confocal  microscopes. 

Solid  State  Devices 

A  development  which  was  not  foreseen  in  the  last  edition  of  this 
chapter  has  been  the  proliferation  of  ultra-compact  solid  state 
memory  devices  which  retain  data  even  without  a  source  of  power. 
While  small  in  capacity  compared  to  a  hard  disk,  these  range  up 
to  more  than  the  capacity  of  a  CD  in  a  tiny  fraction  of  the  space. 
Much  of  this  development  has  been  driven  by  the  explosive  growth 
of  the  digital  camera  market. 

Compact  flash  cards  (Compact  Flash  Association,  2003)  are 
used  by  many  such  cameras,  making  the  computer  accessories  to 
read  them  an  essential.  Typically  these  use  either  the  PC-card 
(PCMCIA)  slots  in  notebook  computers  or  else  the  USB  or  Fire 
Wire  ports  found  on  both  desktop  and  notebook  systems.  A  key 
feature  is  that  both  are  designed  to  appear  as  hard  disks  to  the  com¬ 
puter  without  the  need  to  install  any  drivers.  As  the  card  is  the  size 
of  a  postage  stamp,  and  about  3  mm  thick,  it  represents  a  highly 
portable  data  store,  and  many  people  use  them  for  convenient 
portable  storage  or  transfer  between  computers  without  reference 
to  digital  cameras.  Flash  drives  currently  can  hold  up  to  2  GB.  Data 
transfer  rates  of  the  compact  flash  (confocal)  chips  are  currently  5 
to  7MB/s,  but  the  latest  revision  of  the  interface  is  designed  to 
cope  with  rates  of  up  to  16MB/s  to  allow  for  advances  in  chip 
technology  in  the  future.  Practical  test  speeds  achieved  in  com¬ 
puters  (Digital  Photography  Review,  2003)  are  around  3  to  4MB/s 
with  writing  being  slower  than  reading;  performance  in  digital 
cameras  will  always  be  much  slower.  The  cost  is  still  around 
US$100  a  gigabyte  so  it  will  not  compete  with  CDs  for  archival 
storage,  but  as  a  fast,  rewritable  method  of  transporting  relatively 
large  files  —  whether  images,  documents,  or  digital  presentations 
—  compact  flash  has  an  important  place. 

Memory  Stick  (Sony)  and  Smart  Media  (Samsung)  are  similar, 
more  proprietary  flash  memory  devices  which  fulfill  similar 
functions,  but  so  far  offer  a  smaller  range  of  useful  options 
than  the  more  open  standard  Compact  Flash.  They  tend  to  be  more 
popular  in  the  portable  music  player  market,  showing  again  how 
several  once-different  technologies  are  converging.  The  Sony 
Micro  Vault  is  a  dedicated  USB-only  version  that  comes  in  capac¬ 
ities  from  32MB  to  256MB,  and  requires  no  further  accessories; 
it  even  has  a  cover  for  the  plug  when  removed  from  the  computer. 
Similar  “keychain”  memory  devices  are  available  from  other 
manufacturers. 

These  flash  memory  devices  have  established  a  quite  different 
market  niche  from  other  removable  storage  devices,  but  as  pho¬ 
tography  becomes  increasingly  a  digital  process,  this  convergence 
seems  likely  to  continue. 

DISPLAY 

Before  looking  in  detail  at  how  the  image  is  displayed  and  printed, 
we  should  consider  the  nature  of  the  confocal  image  (see  Chapter 
4,  this  volume).  The  image  in  a  conventional  optical  microscope 


has  an  infinite  gradation  of  tones  within  it,  whereas  the  confocal 
image  typically  has  just  256,  if  it  is  monochrome  or  false  color. 
Merged  two-  or  three-channel  images  may  have  up  to  2563  colors, 
but  often  have  considerably  fewer.  Confocal  images  have  a  finite 
number  of  pixels,  whereas  photographic  images  have  limited  res¬ 
olution,  but  a  smooth  transition  from  point  to  point.  In  more 
general  terms,  a  confocal  image  is  quantized  in  both  spatial  ( x ,  y, 
z)  and  intensity  dimensions  (see  Chapter  4,  this  volume ). 

What  the  microscopist  actually  sees  is  not  the  image  itself,  but 
a  display  on  a  monitor.  Both  the  monitor  and  the  way  it  is  driven 
will  have  a  major  effect  on  the  appearance  of  the  image.  This  in 
turn  is  interpreted  by  the  human  eye  when  we  see  it  directly,  or  by 
a  camera  if  we  record  the  image  photographically.  As  a  prelimi¬ 
nary,  we  should  therefore  look  at  how  monitors  display  confocal 
images. 

Monitors 

Monochrome  cathode  ray  tube  (CRT)  monitors  simply  have  a  layer 
of  phosphor  coated  on  the  inside  of  the  glass,  so  that  an  illumi¬ 
nated  spot  will  be  produced  wherever  the  electron  beam  hits.  The 
resolution  of  the  monitor,  therefore,  depends  solely  on  how  small 
the  electron  beam  hitting  the  screen  can  be.  Color  monitors,  on  the 
other  hand,  have  red,  green,  and  blue  phosphors  arranged  either  in 
dots  (shadow-mask  tube)  or  stripes  (Trinitron  tube).  The  image  on 
a  color  monitor  will  always  be  made  up  of  a  mosaic  of  the  three 
primary  colors;  the  finer  this  mosaic,  the  better  the  image  will  be. 
This  is  specified  by  the  dot  pitch  of  the  tube  in  millimeters  —  0.28 
mm  would  be  a  typical  value  for  a  good  quality  modern  PC 
monitor,  though  pitches  as  small  as  0.18  mm  are  available,  and 
cheaper  or  older  monitors  will  have  pitches  up  to  0.4  mm.  These 
are  absolute  values,  so  a  larger  monitor  will  have  more  dots  in  the 
total  width  of  the  image. 

The  number  of  pixels  that  may  be  displayed  on  the  monitor  is 
a  function  of  the  speed  at  which  the  electron  beam  can  respond  to 
a  changing  signal,  and  is  not  related  to  the  actual  dot  pitch  of  the 
CRT.  Thus,  a  confocal  image  displayed  on  a  color  monitor  will 
have  each  pixel  subdivided  into  a  pattern  of  red,  green,  and  blue 
dots.  If  the  pixel  spacing  of  the  data  comes  close  to  the  dot  pitch 
on  the  screen,  aliasing  (below)  may  occur,  creating  undesirable 
effects.  Many  color  monitors  can  be  set  up  to  display  more  pixels 
than  there  are  dot  triplets  available,  so  the  full  resolution  of  the 
image  cannot  actually  be  shown.  Thus,  a  large  monitor  is  essen¬ 
tial  on  a  confocal  microscope  if  we  are  to  be  able  to  see  the  detail 
in  a  high-resolution  image. 

A  CRT-based  monitor  is  intrinsically  capable  of  displaying  an 
almost  infinite  number  of  colors.1 *  However,  the  video  board  inside 
the  computer  imposes  its  own  restrictions.  Display  boards  suitable 
for  a  confocal  microscope  will  permit  256  gradations  in  each 
primary  color,  so  that  a  24-bit  (three  8-bit  channels)  confocal 
image  can  be  displayed  without  compromising  the  intensity  range. 
(12-bit  or  16-bit  images  will,  however,  need  to  be  reduced  to  8-bit 
for  display.) 

The  other  factor  determining  the  appearance  of  the  image  is 
the  frequency  with  which  the  display  is  scanned.  The  more  rapidly 
the  screen  is  refreshed,  the  less  the  image  will  flicker,  and  a  suit¬ 
able  monitor  should  redraw  the  entire  image  at  least  70  times  per 
second.  Low  cost  boards  will  often  compromise  one  or  other  of 
these  attributes  at  their  highest  resolution  and  are  therefore  inher- 


1  Limited  only  by  Poisson  noise  as  it  affects  the  charge  deposited  by  the  elec¬ 

tron  beam  during  the  pixel  dwell  time  and  the  number  of  phosphor  grains  in 

each  dot,  etc.). 
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ently  unsuitable  for  confocal  use,  but  because  high-end  display 
boards  are  now  very  cheap  compared  to  confocal  microscopes,  this 
is  not  likely  to  be  a  problem  so  long  as  it  is  understood  that  just 
any  computer  will  not  do. 

Displaying  large  numbers  of  pixels  on  low-priced  monitors  can 
reduce  the  refresh  rate  to  60  per  second,  which  is  about  the  lowest 
tolerable  value.  Interlaced  scanning  is  a  strategy  used  to  reduce 
flicker  when  it  is  not  possible  to  scan  the  entire  image  at  a  suffi¬ 
cient  rate.  First,  the  odd  lines  of  the  image  are  drawn,  then  the  next 
scan  draws  the  even  lines.  This  technique  is  primarily  used  for 
broadcast  video  signals  to  enable  the  signal  to  fit  into  the  available 
bandwidth,  but  it  has  in  the  past  also  been  used  to  obtain  higher 
resolution  on  low-cost  computer  monitors. 

International  television  standards  use  625  scan  lines  per  frame, 
with  each  interlace  drawn  50  times  a  second  (and  thus  a  full-frame 
rate  of  25/s).  The  system  used  in  the  Americas  and  Japan  uses  only 
525  lines,  but  a  faster  refresh  rate  of  60  interlaces  per  second.  (Not 
all  scan  lines  are  available  for  display;  standard  video  can  display 
only  5 12  pixels  vertically,  US  video  only  483.)  Video  displays  once 
played  a  significant  part  in  confocal  microscopy  (they  were  stan¬ 
dard,  e.g.,  on  the  widely  popular  Bio-Rad  MRC  500  and  600)  but 
they  are  not  used  now.  Apart  from  the  low  resolution  and  refresh 
rate,  the  problem  of  displaying  multi-channel  images  adequately 
led  to  their  demise.  In  broadcast  television,  the  color  signal  is 
encoded  as  a  chrominance  signal  at  much  lower  resolution  than  the 
monochrome  luminance  signal,  and  this  does  not  give  adequate 
quality  for  a  multi-channel  confocal  image.  The  alternative  is  to 
generate  a  three-channel  video  signal  that  will  give  a  much  higher 
quality  display  (on  the  same  monitor  so  long  as  the  appropriate 
inputs  are  provided).  However,  this  signal  cannot  now  be  recorded 
on  a  standard  video  cassette  recorder  (VCR)  or  printed  on  a  low- 
cost  video  printer.  The  utility  of  video  in  microscopy  is  primarily 
in  recording  fast-moving  items  and  it  may  still  have  a  part  to  play 
with  direct-vision  confocal  systems  (Nipkow  disk  or  slit  scanners), 
but  it  is  no  longer  relevant  to  point-scan  systems. 

In  a  confocal  image,  pixel  intensity  values  are  linearly  related 
to  the  numbers  of  photons  captured  from  the  specimen.  However, 
this  linearity  may  not  be  preserved  when  the  image  is  displayed. 
In  the  simplest  case,  if  the  value  of  the  pixel  is  converted  directly 
to  the  voltage  at  the  control  grid  of  the  CRT,  the  actual  brightness 
of  the  pixel  on  the  screen  will  be  proportional  to  the  three-halves 
power  of  the  pixel  value.  This  may  not  be  all  bad  because 
the  human  eye  responds  logarithmically,  not  linearly,  to  light 
(Mortimer  and  Sowerby,  1941).  However,  confocal  images  often 
look  excessively  contrasty  on  an  uncorrected  display,  and  fine 
detail  in  the  mid-tones  will  be  lost. 

This  relationship  may  be  modified  by  more  sophisticated 
display  electronics,  and  high-quality  display  cards  typically  come 
with  software  to  allow  the  user  to  set  up  the  display  optimally. 
These  are  all  too  often  ignored  by  researchers  who  “don’t  have 
time”  (and  then  waste  much  more  time  struggling  with  pictures 
which  fail  to  show  details  which  “were  there  when  they  took  the 
picture”).  Failing  such  an  option,  image  manipulation  programs 
such  as  Photoshop,  Paint  Shop  Pro,  and  Corel  PhotoPaint  place  the 
display  gamma  under  user  control;  but  there  again  the  user  must 
make  the  effort  to  use  that  control  (for  more  discussion,  see 
Chapter  4,  this  volume). 

Liquid  Crystal  Displays 

Flat  screen  (liquid  crystal)  monitors  are  inherently  different  from 
CRTs.  They  use  liquid  crystal  devices  between  crossed  polarizers 
to  display  the  image,  and  as  each  pixel  is  addressed  independently, 
the  question  of  display  resolution  not  matching  the  pixel  resolu¬ 


tion  does  not  arise.  Typically  different  resolutions  are  not  available 
on  LCD  monitors;  in  the  rare  cases  that  they  are,  the  pixels  are  re¬ 
mapped  in  software  (see  below  and  Fig.  32.6).  Therefore,  LCD 
monitors  will  often  give  a  crisper  display  than  CRTs,  though  on 
the  other  hand  the  pixels  may  be  more  visible  simply  because  their 
edges  are  more  clearly  defined.  Some  older  displays  (typically 
used  on  lower  cost  notebook  computers)  used  passive  supertwisted 
nematic  (STN)  displays  instead  of  active  thin-film  transistor  (TFT) 
technology.  These  are  both  cheaper  and  far  less  demanding  of 
power.  However,  they  may  not  offer  full  24-bit  color,  and  the 
image  may  be  less  bright  and  have  a  smaller  viewing  angle.  Large 
freestanding  monitors  are  always  TFT. 

Large  LCD  monitors  have  many  advantages  in  the  confocal 
laboratory.  Because  the  display  is  not  continually  redrawn  as  in  a 


FIGURE  32.6.  Halftoning  (A)  versus  dithering  (B).  Highly  enlarged  view  of 
part  of  an  image  of  an  integrated  circuit  chip;  above  printed  by  halftoning  using 
a  4  X  4  matrix  of  laser  dots,  below  printed  by  dithering  using  a  3  X  3  matrix. 
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CRT,  flicker  is  not  an  issue.  Consequently,  neither  is  refresh  rate 
(it  can  become  an  issue  with  video  images).  The  screen  is  flat  and 
compact,  and  as  confocal  systems  often  include  two  monitors  and 
three  lasers  in  a  small  room,  minimizing  heat  is  worthwhile. 
However,  LCDs  are  costly,  and  the  cheaper  models  often  sacrifice 
color  quality,  viewing  angle,  or  both.  These  are  sacrifices  which 
are  not  worth  making.  Good  saturation,  wide  control  over  contrast 
and  gamma,  and  a  wide  viewing  angle  are  all  essential.  If  you  can 
afford  it,  buy  a  pair  of  top-quality  LCD  monitors  (and  don’t  just 
take  price  as  a  criterion  of  excellence,  check  them  out  carefully 
yourself).  If  cost  is  a  major  issue,  buy  high-quality  CRT  monitors 
rather  than  low-quality  LCDs. 

Data  Projectors 

Data  projectors  are  now  a  very  common  display  format  for  con- 
focal  images,  and  naturally  they  often  represent  the  occasion  when 
high-quality  display  is  most  important.  However,  they  typically 
have  lower  resolution  and  often  a  poorer  gray-scale  rendition  than 
computer  monitors.  In  terms  of  resolution,  800  x  600  pixels 
(SVGA)  is  common  on  projectors  intended  for  the  home  market 
and  1024  X  768  (XGA)  on  ones  intended  for  academic  and  teach¬ 
ing  use,  though  higher  resolutions  are  available  at  correspondingly 
higher  prices. 

Two  different  technologies  are  used  in  these  projectors.  A  very 
good  description  is  given  by  Powell  (2004).  LCD  projectors  use 
liquid  crystal  screens,  as  in  flat  panel  monitors,  except  these  do  not 
have  a  color  mosaic.  Instead,  three  panels  are  used,  one  for  each 
of  the  primary  colors.  Digital  light  processor  (DLP)  projectors  use 
a  micromirror  array,  where  the  pixels  are  tiny  mirrors  and  these 
are  tilted  to  send  more  or  less  light  to  the  image.  Very  expensive 
projectors  use  three  DLP  chips,  one  for  each  primary  color,  but 
these  are  rare.  The  projectors  one  is  likely  to  encounter  in  a  lecture 
room  or  conference  have  one  DLP  chip,  with  a  rapidly  spinning 
filter  wheel  in  front  of  it.  The  colors  are  therefore  generated 
sequentially  and  merged  by  persistence  of  vision. 

The  two  types  have  their  own  strong  and  weak  points,  and  typ¬ 
ically  DLP  projectors  are  favored  for  home  theatre  use  and  LCD 
for  data  projection.  As  a  comparison,  for  this  review  the  signal 
from  a  notebook  computer  was  sent  simultaneously  to  two  mod¬ 
erately  high-end  projectors,  projecting  on  to  adjacent  screens.  One 
was  a  DLP  projector,  the  other  an  LCD,  and  price  and  luminous 
output  were  comparable.  Contrast,  brightness,  and  other  display 
parameters  were  set  to  midpoint  values  on  both  projectors. 

The  native  resolution  of  both  projectors  was  1024  x  768,  and 
the  computer  was  set  to  the  same  value.  Both  projectors  were  able 
to  handle  higher  resolutions  and  scale  them  down,  but  the  quality 
suffered  very  markedly  when  this  was  done.  The  first  lesson,  there¬ 
fore,  is  to  set  your  screen  display  to  the  resolution  of  the  projec¬ 
tor,  if  possible.  Even  at  the  native  projector  resolution  there  may 
still  be  some  pixel  re-mapping  taking  place  because  projectors 
correct  for  keystone  distortion  caused  by  a  non-horizontal  projec¬ 
tion  angle.  This  means  that  either  the  top  or  bottom  of  the  image 
cannot  use  all  the  displayable  pixels. 

The  LCD  projector  gave  a  much  sharper  image,  which  was 
obviously  preferable  for  fine  text.  However,  its  color  rendition, 
particularly  on  real-world  photographic  images,  was  inferior, 
having  a  slight  color  cast  and  excessive  saturation.  The  DLP  pro¬ 
jector  gave  images  with  a  very  accurate  color  rendition,  free  of  any 
cast  and  natural  in  appearance.  The  two  projectors  differed 
markedly  in  gamma.  The  DLP  projector  had  a  gamma  of  1  (mea¬ 
sured  with  the  gamma  test  function  of  an  imaging  package),  while 
the  LCD  projector  was  around  1 .6  (slightly  higher  in  blue  and  red 


than  in  the  green).  This  means  that  the  LCD  projector  was  very 
comparable  to  both  the  screen  of  the  laptop  and  to  a  CRT  monitor, 
both  of  which  checked  out  with  similar  values,  but  the  DLP  is  more 
accurate  for  confocal  images  in  which  pixel  value  is  typically 
linear  with  number  of  photons. 

To  test  the  displayable  gray  scales,  a  test  image  with  intensity 
scaling  from  0  (black)  on  one  side  to  255  (white)  on  the  other  was 
used.  All  256  values  were  present,  and  on  both  CRT  and  notebook 
monitors  the  change  seemed  totally  smooth.  On  the  LCD  projec¬ 
tor  it  also  seemed  smooth,  though  with  some  minor  streaks,  which 
may  have  been  aliasing  rather  than  posterizing  (see  Digital 
Printers,  below).  However,  there  were  noticeable  bands  with  the 
DLP  projector.  This  implies  it  was  incapable  of  reproducing  a  full 
8  bits  in  each  color,  and  in  fact  posterizing  was  noticeable  in  large 
pale  areas  of  scanned  pictures.  Also  relevant  in  this  context  is  the 
contrast  range  of  the  projector:  the  difference  between  its  whitest 
white  and  darkest  black.  This  is  an  important  figure  of  merit  for  a 
digital  projector  and  is  always  quoted  by  manufacturers.  The 
number  of  tones  which  can  be  reproduced  has  little  relevance  if 
they  are  squeezed  into  such  a  small  range  that  the  eye  cannot  dis¬ 
tinguish  them.  In  the  past  this  has  been  a  major  concern  when 
projecting  confocal  images,  with  detail  disappearing  in  both  high¬ 
lights  and  shadows.  This  is  an  area  in  which  digital  projectors  (of 
either  technology)  have  made  huge  strides  in  recent  years.  DLP  is 
normally  regarded  as  leading  in  contrast  ratio  but  in  this  test  both 
projectors  seemed  comparable,  with  good  rich  blacks. 

Both  projectors  seemed  evenly  balanced  in  response  time,  with 
rapid  mouse  movements  appearing  equally  (and  acceptably)  jerky 
in  both  (at  60 Hz  refresh  rate).  At  very  close  quarters  some  mis¬ 
alignment  of  the  different  color  images  was  visible  with  the  LCD 
projector.  This  was  invisible  at  normal  viewing  distance  and  may 
be  inevitable  in  a  projector  with  three  different  LCD  arrays  (espe¬ 
cially  one  which  is  regularly  transported).  This  would  not  be 
expected  in  a  DLP  projector  because  there  is  only  one  display 
element,  but  in  fact  some  color  fringing  could  still  be  seen  at  the 
edges  of  the  screen,  though  not  in  the  center. 

The  verdict  on  this  test  was  that  the  LCD  projector  was  way 
ahead  for  text,  diagrams,  and  other  computer-generated  graphics, 
but  the  DLP  had  the  edge  for  micrographs  and  other  real-world 
images.  This  is  in  line  with  the  commonly  accepted  merits  of  the 
two  technologies.  The  question  of  different  gamma  is  likely  to  be 
significant  when  projecting  confocal  images,  and  in  the  rare  case 
where  one  knows  in  advance  which  type  of  projector  will  be  used, 
the  images  in  a  presentation  could  be  adjusted  to  suit.  But  the  most 
useful  point  to  remember  when  giving  a  presentation  at  a  confer¬ 
ence  is  still  to  set  your  screen  resolution  to  the  native  resolution 
of  the  projector. 

HARD  COPY 

When  it  comes  to  recording  images,  the  confocal  microscopist  has 
to  make  a  choice  between  two  fundamentally  different  technolo¬ 
gies.  One  option,  photography,  was  in  the  past  familiar  ground  to 
most  microscopists.  The  other  option,  computer  printers  of  one 
sort  or  another,  are  more  likely  to  be  relevant  in  the  21st  century. 

Photographic  Systems 

In  the  10  years  since  the  previous  edition  of  this  book,  photography 
has  almost  completely  disappeared  from  the  cell  biology  labora¬ 
tory.  So  far  as  confocal  images  are  concerned,  this  is  all  to  the  good 
because  there  is  a  fundamental  mismatch  between  film  and  the 
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digital  image.  A  photograph  can  reproduce  far  more  tones  than  the 
256  that  an  8-bit  image  possesses,  but  the  interposition  of  lenses 
and  film  means  that  pixel  positions  are  not  reproduced  sharply  and 
with  complete  accuracy.  Blurring  below  the  level  of  microscope 
resolution  will  not  be  noticeable  in  a  conventional  micrograph,  but 
if  pixels  are  not  rendered  clearly  in  a  confocal  image  it  will  look 
soft  —  especially  if  there  is  any  text  superimposed  on  the  image. 
At  one  time,  screen-shooting  devices  were  standard  equipment 
with  confocal  microscopes  but  now  they  are  no  more  than  histori¬ 
cal  curiosities. 

Digital  Printers 

Printers  typically  work  by  putting  dots  on  a  page  of  paper.  As 
printer  technology  has  evolved,  the  size  and  resolution  of  these 
dots  has  become  smaller  and  more  precise,  but  in  general  the  dots 
are  still  either  present  or  absent,  which  limits  the  ability  of  a  printer 
to  represent  images  with  a  range  of  tones.  However,  these  pixels 
are  placed  with  extreme  accuracy,  so  providing  the  data  is  handled 
properly  (below),  it  is  possible  for  each  pixel  in  a  confocal  image 
to  be  printed  sharply  and  in  its  correct  place.  There  are  two  ways 
in  which  we  can  break  up  a  gray-scale  image  into  a  pattern  of  black 
dots  for  printing:  halftoning  or  dithering. 

Halftoning  is  the  way  images  are  reproduced  in  printed  books 
and  newspapers.  The  image  is  broken  into  a  series  of  black  dots 
of  varying  size,  darker  grays  being  represented  by  larger  dots. 
Halftoning  is  unarguably  the  method  of  choice  if  the  resolution  of 
the  output  medium  is  sufficient.  However,  it  will  be  clear  that  to 
produce  halftone  dots  of  varying  sizes,  each  dot  must  be  a  multi¬ 
ple  of  the  basic  dot  pattern  of  the  printer.  If  the  halftone  dots  are 
to  be  small,  the  printer’s  basic  dots  must  be  very  small.  The 
halftone  screen  in  a  printed  book  is  typically  133  or  150  dots  per 
inch  (Cox,  1987).  A  1200dpi  printer  can  give  is  8  X  8  dots  —  65 
gray  shades  —  within  that  resolution.  To  get  256  gray  shades  at 
150  dpi,  we  need  a  2400  dpi  printer. 

Dithering  uses  a  probabilistic  method  to  decide  whether  a 
printer  dot  should  be  present  or  not.  If  the  pixel  is  dark,  there  is  a 
high  probability  that  a  black  dot  will  be  printed;  if  it  is  light  that 
probability  is  low.  The  effect  is  of  a  grainy  image  without  any  reg¬ 


ularly  repeating  pattern.  When  using  a  low-resolution  output 
device,  dithering  is  the  only  option;  halftoning  would  result  in  an 
impossibly  coarse  screen.  Figure  32.6  shows  a  magnified  view  of 
a  confocal  image  of  an  integrated  circuit,  (IC)  device,  printed  by 
halftoning  and  dithering.  In  the  dithered  image  each  pixel  of  the 
original  micrograph  is  represented  nine  times,  by  a  3  X  3  matrix 
of  laser  dots,  the  probabilistic  dithering  calculation  being  applied 
independently  each  time.  Thus,  on  average,  a  50%  gray  would 
have  either  four  or  five  dots  black,  the  others  being  white,  but 
which  of  the  nine  dots  were  black  would  vary  each  time. 

The  eye  can  perceive  about  64  shades  of  gray  reflected  from 
a  solid  surface.  If  fewer  shades  than  this  are  used  to  reproduce  an 
image,  areas  that  should  show  a  smooth  transition  in  tone  will 
reproduce  as  a  series  of  bands.  Because  the  human  visual  system 
is  extremely  sensitive  to  edges,  this  can  be  extremely  distracting. 
This  effect  is  termed  posterizing.  In  printing  it  occurs  when  the 
printer  is  unable  to  reproduce  at  least  64  shades  of  gray.  Figure 
32.7  shows  an  example  of  this.  In  Figure  32.7(A),  using  16  gray 
tones  (roughly  what  an  old  300  dpi  laser  printer  can  give)  the 
smoothly-graded  gray  appears  as  a  series  of  discrete  bands.  To 
some  extent,  the  problem  can  be  reduced  by  combining  dithering 
with  halftoning  —  values  which  bridge  the  boundaries  between  the 
levels  the  printer  can  produce  are  randomized  to  decide  which 
value  they  should  have  [Fig.  32.7(B)],  Ultimately,  though,  good 
reproduction  of  a  confocal  image  will  require  at  least  64  gray  levels 
to  be  reproduced  on  paper  [Fig.  32.7(C)]. 

Proper  reproduction  of  a  confocal  image  by  a  printer  will  gen¬ 
erally  require  the  image  to  be  reproduced  either  pixel  for  pixel,  or 
with  an  integer  multiple  (or  fraction)  of  printer  pixels  or  half-tone 
dots  to  each  dot  of  the  confocal  image.  If  this  is  not  the  case,  alias¬ 
ing  (Chapter  4,  this  volume )  will  generate  artifactual  patterning  in 
the  image.  This  is  shown  in  Figure  32.8;  scaling  the  original  465- 
pixel  wide  image  [Fig.  32.8(A)]  to  fit  the  common  512-pixel  size 
has  given  the  diagonal  lines  and  the  circle  a  very  jagged  appear¬ 
ance  [Fig.  32.8(B)].  This  introduces  a  considerable  constraint  on 
printing  confocal  images;  photography  can  reproduce  them  with 
equal  accuracy  at  any  size,  but  printing  works  best  with  integer 
multiples  or  fractions  of  the  image  pixels.  The  only  ways  out  of 
this  are  either  to  use  such  a  high-resolution  print  device  that  it 


FIGURE  32.7.  Posterizing.  The  smooth  ramp  of  shade  in  the  original  shows  banding  when  reproduced  with  only  16  gray  levels  (A).  This  can  be  partly  dis¬ 
guised  by  dithering,  still  only  using  16  levels  (B ),  but  using  80  gray  levels  is  enough  to  give  a  smooth-looking  result  (C). 
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FIGURE  32.8.  Aliasing.  The  circle  and  the  diagonal  lines  appear  as  smooth 
as  the  horizontal  and  vertical  ones  in  the  original  image  (A),  but  when  it  is 
scaled  up  slightly  they  become  jagged  (B).  In  the  original  image  the  curved 
and  slanting  lines  are  actually  made  up  of  black  and  various  shades  of  gray, 
shown  much  enlarged  in  (C). 


exceeds  the  Nyquist  criterion  at  the  output  resolution  or  else  to 
remap  the  image  with  sophisticated  software  (several  algorithms 
are  in  general  use,  and  bilinear  or  bicubic  resampling  are  proba¬ 
bly  the  commonest)  to  the  output  resolution.  The  first  option  is 
becoming  more  common  as  printer  resolution  improves  but  cannot 
yet  be  guaranteed.  Figure  32.9  shows  the  effect  of  different  remap¬ 
ping  algorithms.  The  image  (shown  in  the  inset)  is  a  tiny  part  of 
Figure  32.1(A),  enlarged  by  the  odd  amount  of  467%.  Direct  pixel 
scaling  [Fig.  32.9(A)]  gives,  as  expected,  very  poor  results,  and 
bilinear  resampling  [Fig.  32.9(B)]  is  very  little  better.  Bicubic 
resampling  [Fig.  32.9(C)]  is  hugely  better  and  obviously  the  only 
useful  choice  in  this  case. 

Aliasing  can  also  appear  if  the  printer’s  gray  levels  do  not 
match  the  intensity  levels  of  the  image.  When  a  black  diagonal,  or 
curved,  line  is  reproduced  in  a  pixelated  image  it  will  appear  as  a 
mixture  of  gray  and  black  pixels  [Fig.  32.8(C)].  Pixels  which  lie 
wholly  within  the  line  are  black;  those  which  were  partially  inter¬ 
sected  by  the  line  are  gray.  So  long  as  they  are  reproduced  at  their 
original  intensity  the  line  will  retain  the  illusion  of  smoothness,  but 
as  soon  as  they  are  made  lighter  or  darker  the  line  will  appear 
jagged. 

Color  images  present  all  the  above  problems,  as  well  as  some 
of  their  own.  Most  laser-scanning  confocal  microscopes  (CLSM) 
do  not  produce  color  images  in  the  sense  that  a  conventional 
optical  microscope  does.  Color  images  produced  on  a  CLSM  are 
either  pseudo-color  images  in  which  a  false-color  palette  is  applied 
to  a  gray-scale  image,  or  multi-channel  images  in  which  two  or 
three  different  signals  are  each  assigned  to  a  different  primary 
color.  An  image  with  a  fluorescein  signal  in  the  green  channel  and 
a  rhodamine  image  in  the  red  channel  might  look  very  similar  to 
a  real  color  photomicrograph  of  the  same  slide,  but  the  way  the 
image  is  made  up  is  very  different. 

A  further  problem  arises  because  multi-channel  images,  and 
some  false-color  palettes,  tend  to  use  fully  saturated  colors.  These 
almost  never  occur  in  nature.  On  a  monitor,  which  emits  light, 
these  can  look  very  effective,  not  least  because  confocal  micro¬ 
scopes  commonly  operate  in  dimly  lit  rooms.  They  can  also  make 
good  slides.  Flowever,  when  printed  on  paper,  where  the  image  is 
created  by  light  reflected  from  the  paper  through  the  ink,  the  image 
will  look  very  dark.  This  problem  is  exacerbated  by  the  different 
color  models  used  to  form  the  image. 

A  computer  image  is  usually  stored,  and  always  displayed, 
using  an  RGB  (red/green/blue)  or  additive  color  model.  Adjacent 
points  on  the  monitor  screen  emit  light  of  the  three  primary  colors. 


FIGURE  32.9.  Scaling  techniques.  A  small  part  of  the  image  shown  in  Figure 
32.1(A)  enlarged  by  the  odd  amount  of  467%,  which  cannot  be  achieved  by 
simple  scaling  of  pixel  size.  (A)  Pixel  enlargement,  as  well  as  looking  blocky, 
severe  aliasing  is  obvious.  (Inset)  The  original  image.  (B)  Bilinear  resampling: 
aliasing  is  much  reduced  but  the  image  is  still  blocky.  Little  use  is  made  of  the 
extra  pixels  now  available.  (C)  Bicubic  resampling  gives  a  hugely  better  result; 
it  cannot  produce  more  resolution  than  the  original  data  contains,  but  it  does 
the  optimum  job  of  mapping  it  to  the  output  resolution. 
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and  the  different  colors  are  created  by  adding  these  together.  A 
printed  page  uses  the  CMY  (cyan/magenta/yellow)  or  subtractive 
model.  A  red  point  is  created  by  putting  on  the  paper  both  magenta 
(which  subtracts  green  from  the  reflected  light)  and  yellow  (which 
subtracts  blue)  so  that  only  red  will  be  reflected.  [Commercial 
printing,  and  many  computer  printers,  also  add  a  black  ink  to  com¬ 
pensate  for  the  fact  that  the  three  color  inks  may  not  add  up  to  a 
perfect  black.  This  is  then  a  CMYK  (cyan/magenta/yellow/black) 
color  model.] 

Thus,  reproducing  each  of  the  primary  colors  that  look  so  bril¬ 
liant  when  formed  by  a  single  phosphor  on  a  monitor  requires  the 
light  to  pass  through  two  separate  color  dyes  before  being  reflected 
by  the  paper.  To  put  back  brilliance  into  such  an  image,  it  is  nec¬ 
essary  to  unsaturate  the  colors  —  to  add  some  “white”  into  them. 
Many  computer  image-manipulation  programs  provide  this  facil¬ 
ity.  Often,  some  experimentation  is  required  before  a  good  screen 
image  can  be  turned  into  a  good  print  and  it  can  be  very  useful  to 
have  a  program  that  permits  one  to  print  an  array  of  small  test 
images,  each  made  using  slightly  different  settings. 

Laser  Printers 

Laser  printers  are  the  workhorse  of  the  modern  office.  Their  crisp 
black  type  and  graphics  are  unrivaled  for  most  computer  output. 
For  reproduction  of  confocal  (or  other)  images  their  abilities  are 
more  limited,  though  they  can  produce  quite  reasonable  proof 
prints.  Laser  printers  use  a  low  power  laser  to  write  dots  directly 
onto  the  charged  drum  of  a  photocopier. 

The  limitation  of  a  laser  printer  is  that  it  is  difficult  to  get  the 
pattern  of  dots  fine  enough  to  reproduce  a  full  range  of  tones  by 
halftoning  (above).  However,  laser  printers  with  1200dpi  resolu¬ 
tion  are  now  common,  and  many  also  have  the  capability  of  mod¬ 
ulating  the  size  of  the  spot  to  some  extent.  Output  from  such  a 
printer  is  adequate  for  many  purposes,  and  the  cost  is  far  lower 
than  either  photography  or  higher-class  printing.  Their  weakest 
point  is  that  large  expanses  of  black  are  still  not  rendered  as  uni¬ 
formly  as  in  a  photograph. 

Color  laser  printers  have  only  recently  started  to  make  a  sig¬ 
nificant  impact  on  the  marketplace.  This  is  partly  because  of  a  huge 
decrease  in  price,  and  partly  because  of  improvements  in  resolu¬ 
tion  (600  dpi  is  common)  and  in  the  tricks  used  by  the  built-in 
firmware,  which  at  last  make  near-photographic  quality  routinely 
attainable.  The  quality  does  not  yet  match  that  offered  by  inkjet 
and  dye  sublimation  printers,  but  on  a  cost-versus-quality  basis  it 
hugely  exceeds  it,  so  that  a  color  laser  printer  is  ideal  for  such  pur¬ 
poses  as  printing  preprints  of  journal  articles  in  quantity. 

Ink  Jet  Printers 

These  have  long  ago  taken  over  from  dot  matrix  as  the  everyday 
printer  for  home  and  single-user  office  use.  They  operate  by  squirt¬ 
ing  small  jets  of  ink  on  the  paper  (for  best  results,  a  slightly 
absorbent  paper).  They  typically  offer  three-  or  four-color  printing 
at  much  finer  resolution  than  any  other  printers.  Some  use  seven 
inks  (high  and  low  intensities  of  the  three  subtractive  primaries) 
to  give  more  realistic  results.  Though  the  tendency  of  the  ink  to 
spread  limits  their  ultimate  performance,  it  also  helps  improve  the 
perceived  realism  of  the  image  by  making  individual  pixels  less 
visible.  Printing  to  photographic  quality  requires  special  paper  and 
also  uses  large  amounts  of  the  expensive  ink,  so  it  is  not  cheap, 
but  the  results  bear  comparison  with  those  from  expensive  dye  sub¬ 
limation  printers.  Because  (unlike  other  printer  technologies)  there 
is  no  inherent  limitation  on  the  size  of  the  paper,  A3  and  larger 
printers  are  readily  available  for  such  tasks  as  printing  conference 
posters.  Unlike  dye  sublimation  printers  there  is  always  the  option 


with  an  inkjet  of  printing  at  lower  quality  and  hugely  lower  cost 
for  proofing  and  layout  purposes. 

Dye  Sublimation  Printers 

These  printers  have  changed  little  since  they  the  previous  edition 
of  this  book.  They  are  still  an  excellent  output  medium  for  routine 
production  of  photographic  quality  output,  and  although  still  not 
cheap,  the  price  has  not  increased  in  line  with  inflation  so  they  are 
not  much  more  expensive  per  page  than  the  inkjet  for  photo-quality 
output,  though  the  purchase  price  is  much  greater.  They  use  a  full- 
page  sheet  of  ink  for  each  color  they  print,  but  in  this  case  the  ink 
sublimes  when  heated  and  is  absorbed  by  a  specially  coated  sheet 
of  paper.  The  vaporized  ink  will  tend  to  diffuse  laterally  to  a 
limited  extent,  making  pixelation  less  obvious.  More  or  less  ink 
can  be  transferred,  depending  upon  the  amount  of  heat  applied,  so 
that  true  gray  scales  can  be  produced.  The  output  from  the  best  dye 
sublimation  printers  can  be  comparable  to  photography,  but  the 
cost  is  also  similar  —  up  to  US$5.00  for  an  A4  size  color  print. 
The  cost  per  page  is  fixed,  unlike  a  laser  or  inkjet  printer,  where 
the  cost  per  page  depends  upon  the  degree  of  coverage. 

CONCLUSION 

The  big  change  since  the  previous  edition  of  this  book  10  years 
ago  is  that  now  mass-market  media  are  effectively  equal  to  the 
demands  of  the  confocal  user.  It  is  a  truism  that  for  several  years 
now  developments  in  the  personal  computer  market  have  been 
driven  not  by  business  or  scientific  usage,  but  by  the  domestic 
market.  The  requirements  of  games,  music,  digital  photography, 
and  video  have  been  the  driving  factors  for  processor  speed,  inter¬ 
faces,  display  quality,  print  output,  and  data  storage.  We  no  longer 
need  specialist  image  manipulation  hardware,  custom  video  cards, 
non-standard  monitors,  dedicated  data  buses,  or  expensive  storage 
devices.  Computers  are  fast  enough  for  the  necessary  image 
manipulation,  everyday  video  cards  handle  24-bit  images  at  high 
resolutions  and  fast  refresh  rates,  as  well  as  providing  hard-wired 
image  manipulation  functions.  Monitors  offer  megapixel  displays 
at  high  bit  depths  and  refresh  rates.  Fire  Wire  and  USB-2  will  carry 
data  faster  than  any  point-scanning  confocal  can  scan,  and  domes¬ 
tic  video  disks  are  big  enough  to  handle  huge  data  sets.  Just  about 
every  home  has  a  printer  giving  photographic  quality  output  — 
and  these  are  printing  higher  resolution  images  than  most  micro¬ 
scope  users  generate. 

The  major  computer  magazines  generally  run  annual  surveys 
of  color  printers  and  it  is  always  worth  seeking  out  the  latest  of 
these  before  making  a  purchase.  A  final  word  of  warning:  If  you 
are  evaluating  a  hard  copy  system,  of  whatever  sort,  insist  on 
testing  it  on  real  confocal  images  from  your  own  work.  Every  man¬ 
ufacturer  has  a  gallery  of  images  which  reproduce  superbly  on  his 
own  hardware,  and  if  you  try  to  judge  a  system  on  the  basis  of 
such  pictures  you  will  be  disappointed  once  you  start  using  it 
yourself. 

SUMMARY 
Bulk  Storage 

Image  compression  has  made  huge  strides  but  still,  as  always, 
needs  to  be  used  with  care.  The  new  wavelet  compression  system 
seems  to  offer  little  to  the  confocal  user  but  the  PNG  format  has 
at  last  given  us  a  lossless  technique  that  works.  Recordable  CDs 
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are  currently  the  most  common  and  most  reliable  choice  for  mass 
storage,  though  recordable  DVDs  seem  likely  to  take  over  during 
the  lifetime  of  this  edition.  For  archival  purposes,  there  seems  little 
merit  in  selecting  the  rewritable  version  of  either  of  these  media. 
Tiny  solid-state  FLASH  memory  devices  have  become  a  very 
effective  way  to  carry  presentations  from  place  to  place. 

Display 

Monitors  are  no  longer  a  problem;  24-bit  displays  of  adequate  size 
and  refresh  rate,  as  well  as  the  display  boards  to  drive  them,  are 
now  the  norm  rather  than  expensive  exceptions.  Flat-screen  LCD 
displays  are  still  expensive,  but  worth  the  cost  for  their  extra  sharp¬ 
ness  and  for  the  complete  elimination  of  flicker. 

Hard  Copy 

Inkjet  printers  have  now  essentially  replaced  dye  sublimation 
printers  for  optimal  photographic  output.  They  have  much  lower 
purchase  cost  but  the  cost  per  glossy  print  is  still  high.  Color  laser 
printers  are  now  more  than  adequate  for  proofing  and  preprints. 
Finally,  if  you  are  evaluating  a  hard  copy  system,  insist  on  testing 
it  on  real  confocal  images  from  your  own  work. 
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X.  Sunney  Xie,  Ji-Xin  Cheng,  and  Eric  Potma 


INTRODUCTION 

Advances  in  biological  sciences  are  often  facilitated  by  new  tools 
in  microscopy.  Confocal  and  nonlinear  or  multi-photon  fluores¬ 
cence  microscopy  (Chapters  21  and  28)  have  become  powerful 
techniques  for  three-dimensional  (3D)  imaging  of  living  cells.  This 
coincides  with  developments  of  various  natural  and  artificial  fluo¬ 
rescent  probes  for  cellular  constituents  (Chapters  16  and  17).  For 
biochemical  species  or  cellular  components  that  neither  fluoresce 
nor  tolerate  labeling,  other  contrast  mechanisms  with  molecular 
specificity  are  needed.  Vibrational  microscopy  based  on  infrared 
absorption  and  spontaneous  Raman  scattering  has  been  used  for 
chemically  selective  imaging.  Although  infrared  microscopy  is  a 
powerful  tool,  it  is  limited  to  low  spatial  resolution  because  of  the 
long  wavelength  of  light  used  (Humecki,  1995;  Jamin  et  al.,  1998). 
Furthermore,  the  absorption  of  water  in  the  infrared  region  makes 
it  difficult  to  image  through  living  cells.  In  contrast,  Raman 
microscopy  can  overcome  these  limitations,  as  the  wavelength  of 
the  excitation  light  is  much  shorter  and  there  is  essentially  no 
absorption  of  water  at  these  wavelengths.  Raman  spectroscopy  has 
been  applied  extensively  to  biological  molecules  and  cellular  con¬ 
stituents  (Turrell  and  Corset,  1996;  Puppels,  1999;  Shafer-Peltier 
et  al.,  2002).  Confocal  Raman  microscopy  of  biological  samples 
has  resulted  in  high-resolution  images  of  living  cells.  However,  the 
intrinsically  weak  Raman  signal  necessitates  high  laser  power 
(typically  >100mW)  and  long  integration  times,  and  the  signal  is 
often  overwhelmed  by  the  fluorescence  background  of  the  sample, 
limiting  its  application  in  biology.  In  this  chapter,  we  present 
coherent  anti-Stokes  Raman  scattering  (CARS)  microscopy,  a  non¬ 
linear  vibrational  imaging  technique  that  overcomes  the  limitations 
of  the  linear  techniques  mentioned  above. 

Coherent  anti-Stokes  Raman  scattering  as  a  nonlinear  optical 
process  was  first  reported  in  1965  by  Maker  and  Terhune  at  Ford 
Motor  Company  (Maker  and  Terhune,  1965),  and  later  named 
CARS  by  Begley  and  colleagues  (1974).  Since  then,  CARS  spec¬ 
troscopy  has  been  used  widely  as  a  spectroscopic  tool  for  chemi¬ 
cal  analyses  in  the  condensed  and  gas  phases  and  has  become 
the  most  extensively  used  nonlinear  Raman  technique  (Clark  and 
Hester,  1988;  Tolies  et  al.,  1977).  In  CARS  spectroscopy,  a  pump 
laser  and  a  Stokes  laser,  with  center  frequencies  of  G),,  and  co„ 
respectively,  stimulate  a  sample  through  a  wave-mixing  process. 
The  pump  field  Ep(ap)  and  the  Stokes  field  Es{ cos)  interact  with  the 
sample  and  generate  an  anti-Stokes  field  Eas  at  the  frequency 


Part  of  this  chapter  is  a  condensation  of  a  feature  article  by  J.  X.  Cheng  and 
X.S.  Xie  in  the  Journal  of  Physical  Chemistry. 


CQOJ  =  2 co(,  -  CD,.  The  energy  diagrams  of  CARS  are  shown  in 
Figure  33.1. 

The  induced  nonlinear  polarization  at  the  anti-Stokes  fre¬ 
quency  is  given  by  P(3)  =  Xe)Epf*  Where  /(3)  is  the  third-order 
susceptibility, 


x(3)  = 
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-K-ra,Hr, 


-  +  5C' 
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where  Q  is  the  vibrational  frequency  and  rK  is  the  half  width  at 
half  maximum  of  the  Raman  line.  AR  is  a  constant  representing  the 
Raman  scattering  cross-section.  The  first  term  in  Equation  1  is  a 
vibrationally  resonant  contribution,  which  is  maximized  when  the 
Raman  shift  ap  -  CO,  coincides  with  the  frequency  of  a  molecular 
vibration  of  the  sample.  The  second  term  includes  contributions  to 
the  third-order  susceptibility  that  are  off  the  vibrational  resonance 
and  are  independent  of  the  Raman  shift.  Figure  33.1(B)  shows 
the  energy  diagram  of  one  of  the  non-resonant  contributions.  The 
CARS  signal  intensity  is  proportional  to  the  square  modulus  of  P(3) 
and  has  a  quadratic  dependence  on  the  pump  field  intensity  and  a 
linear  dependence  on  the  Stokes  field  intensity: 
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The  last  factor  in  Equation  2  is  maximized  when  the  wave  vectors 
of  the  pump,  Stokes,  and  CARS  fields,  k,„  k„  and  k,„,  respectively, 
satisfy  the  phase-matching  condition: 

lAkl  •  L  =  lk„,  -  (2k,,  -  k,)l  ■  L  <  n,  (3) 

where  L  is  the  sample  interaction  length  (Shen,  1984;  Clark  and 
Hester,  1988;  Levenson  and  Kano,  1988;  Mukamel,  1995). 

The  physical  picture  is  that  CARS,  unlike  fluorescence,  is  a 
coherent  process  in  which  the  induced  dipoles  fP<3))  within  the 
sample  have  a  well-defined  phase  relationship.  Consequently,  the 
CARS  signal  is  normally  only  generated  in  a  specific  direction, 
shown  in  Figure  33.1(C),  at  which  constructive  interference  of  the 
anti-Stokes  field  occurs. 

Like  other  nonlinear  optical  processes,  high  peak  powers  are 
necessary  for  the  efficient  generation  of  a  CARS  signal.  These 
are  readily  available  using  picosecond  or  femtosecond  light 
pulses.  CARS  detection  exhibits  higher  sensitivity  than  spon¬ 
taneous  Raman.  For  spontaneous  Raman,  the  molecular  oscillators 
within  the  sample  are  random  in  phase.  For  CARS,  however,  the 
molecular  oscillators  are  stimulated  to  vibrate  in  phase  when  the 
beating  frequency  of  the  pump  and  Stokes  laser  beams,  cop  -  co„ 
matches  their  Raman-active  vibrational  frequency.  The  construc¬ 
tive  interference  of  the  coherent  radiation  results  in  a  stronger 
signal  than  attained  in  spontaneous  Raman.  The  CARS  signal 
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FIGURE  33.1.  Energy  diagram  of  CARS.  (A)  Resonant  CARS  characterized 
by  As/[£2  —  (C0p  —  COJ  —  iTJ.  (B)  Non-resonant  CARS  from  an  electronic 
contribution  to  where  the  dotted  lines  indicate  virtual  states.  (C)  The 
phase-matching  condition. 


intensity  is  proportional  to  n 2  (or  l/(3)l2),  with  n  being  the  number 
of  oscillators. 

Unlike  spontaneous  Raman,  CARS  is  not  background  free. 
The  non-resonant  background,  the  second  terms  in  Equation  1, 
limits  the  vibrational  contrast  and  spectral  selectivity.  Fortunately, 
as  shown  below,  methods  for  suppressing  non-resonant  back¬ 
ground  can  overcome  this  difficulty. 

The  Reintjes  group  at  the  Naval  Research  Laboratory  reported 
the  first  CARS  microscope  in  1982.  Using  two  picosecond  dye 
lasers  with  a  non-collinear  beam  geometry,  they  detected  the  signal 
in  the  phase-matching  direction  with  a  two-dimensional  detector. 
However,  the  non-collinear  beam  geometry  used  in  their  work  had 
many  limitations,  and  the  use  of  visible  excitation  created  an  over¬ 
whelming  non-resonant  background  (high  /'„3?,  thus  low  contrast) 
because  of  its  two-photon  electronic  (rather  than  vibrational) 
resonance  at  the  excitation  frequencies. 

Due  to  these  technical  difficulties,  there  were  limited  further 
developments  until  the  Xie  group  at  Pacific  Northwest  National 
Laboratory  revived  CARS  microscopy  in  1999.  In  this  work, 
Zumbusch  and  colleagues  used  near-infrared  laser  pulses  that  led 
to  a  significant  reduction  of  the  two-photon  enhanced  non-resonant 
background  (Zumbusch  et  al.,  1999).  More  importantly,  they  used 
highly-focused,  collinear-propagating  pump  and  Stokes  beams  that 
relaxed  the  phase-matching  condition.  This  implies  that  the  large 
cone  angle  of  the  k  vectors  and  the  small  sample  interaction  length 
(L)  guarantee  that  the  phase-matching  condition  Equation  3  is  still 
satisfied  under  the  tight  focusing  condition.  CARS  images  were 
acquired  by  raster  scanning  the  sample  with  respect  to  the  small 
laser  focal  volume,  resulting  in  high  3D  resolution  and  sectioning 
capability  based  on  the  nonlinear  intensity  dependence. 

To  sum  up,  CARS  microscopy  offers  the  following 
advantages: 

1.  CARS  microscopy  provides  a  contrast  mechanism  based  on 
molecular  vibrations  that  are  intrinsic  to  the  sample.  It  does 
not  require  natural  or  artificial  fluorescent  probes.  This  avoids 
the  toxicity  and  artifacts  associated  with  staining,  and  avoids 
photobleaching  of  fluorophores. 

2.  CARS  microscopy  is  orders  of  magnitude  more  sensitive 
than  spontaneous  Raman  microscopy.  Therefore,  CARS 


microscopy  permits  fast  vibrational  imaging  at  moderate 
average  excitation  powers  (i.e.,  up  to  ~10mW)  that  are  toler¬ 
ated  by  most  biological  samples. 

3.  Because  the  nonlinear  CARS  signal  is  only  generated  at 
the  focus  where  the  laser  intensities  are  the  highest,  CARS 
microscopy  has  a  3D  sectioning  capability  and  is  useful  for 
imaging  thick  tissues  or  cellular  structures. 

4.  The  anti-Stokes  signal  is  blue-shifted  from  the  two  excitation 
frequencies,  and  can  thus  be  easily  detected  in  the  presence  of 
one-photon  fluorescence  background. 

5.  With  the  near-infrared  excitation  wavelengths  used,  CARS 
microscopy  has  a  large  penetration  depth  of  more  than  0.3  mm 
into  a  sample,  allowing  imaging  through  thick  tissues. 

6.  Under  typical  conditions,  there  is  little  linear  or  nonlinear 
absorption  of  the  two  excitation  beams.  This  reduces  photo¬ 
damage  to  biological  specimens. 

In  addition  to  CARS,  other  nonlinear  coherent  processes, 
including  second  harmonic  generation  (SHG)  (Gannaway  and 
Sheppard,  1978;  Campagnola  et  al.,  1999;  Moreaux  et  al.,  2000) 
and  third  harmonic  generation  (THG)  (Barad  et  al.,  1997;  Muller 
et  al.,  1998;  Yelin  and  Silberberg,  1999;  Cheng  and  Xie,  2002; 
Yelin  et  al.,  2002),  have  also  been  incorporated  with  scanning 
microscopy,  and  are  discussed  in  Chapter  40.  Like  CARS,  the 
coherent  radiation  of  these  multi-photon  processes  is  intrinsically 
different  from  the  incoherent  emission  of  multi-photon  fluores¬ 
cence.  SHG  and  THG  arise  from  electronic  contributions  to  the 
second  and  third  order  susceptibility,  respectively,  and  have  essen¬ 
tially  no  spectroscopic  specificity.  CARS  microscopy,  however,  is 
more  informative  than  SHG  and  THG  microscopy  in  that  it  con¬ 
tains  rich  spectroscopic  information  about  specific  molecular 
species. 

The  advances  made  in  1999  immediately  triggered  the  research 
of  several  groups  and  brought  about  a  rapid  growth  of  the  CARS 
microscopy  literature.  CARS  microscopy  has  now  matured  into  a 
powerful  method  for  biological  imaging.  In  this  chapter,  we  sum¬ 
marize  recent  progress  in  this  area. 


UNIQUE  FEATURES  OF  COHERENT  ANTI¬ 
STOKES  RAMAN  SCATTERING  UNDER  THE 
TIGHT-FOCUSING  CONDITION 

CARS  microscopy  is  distinguished  from  CARS  spectroscopy  in 
that  the  laser  beams  are  tightly  focused  with  a  high  numerical- 
aperture  (NA)  objective  lens.  In  CARS  spectroscopy,  non-collinear 
beam  geometries  such  as  the  BOXCARS  geometry  (Shirley  et  al., 
1980)  were  conventionally  used  to  minimize  the  wave  vector  mis¬ 
match  and  thus  maximize  the  interaction  length.  Although  these 
geometries  have  also  been  used  for  CARS  microscopy  (Muller 
et  al.,  2000),  it  became  clear  that  the  non-collinear  geometry  is 
difficult  to  implement  and  results  in  poor  lateral  resolution. 
The  collinear  geometry  (Zumbusch  et  al.,  1999)  has  become  the 
method  of  choice  because  of  its  ease  of  use  and  high  spatial  reso¬ 
lution,  and  has  been  widely  implemented  (Zumbusch  et  al.,  1999; 
Hashimoto  et  al.,  2000;  Potma  et  al.,  2000;  Cheng  et  al.,  2001a, b; 
Volkmer  et  al.,  2001,  2002;  Cheng  et  al.,  2002a). 

It  is  important  to  emphasize  some  of  the  differences  between 
coherent  and  incoherent  imaging.  For  incoherent  imaging  methods 
such  as  fluorescence  and  spontaneous  Raman  microscopy,  the 
image  profile  is  the  convolution  of  the  object  profile  with  the 
diffraction-limited  point  spread  function  of  the  excitation  intensity. 
In  CARS,  however,  the  image  intensity  is  a  squared  modulus  of 
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FIGURE  33.2.  (A)  Intensity  distribution  on  a  log  scale  and  (B)  axial  intensity 
and  phase  shift  in  the  focal  region  of  a  fundamental  Gaussian  beam  focused  by 
an  objective  lens  of  NA  =  1 .4.  p  =  -Jx2  +y2 .  (C)  Forward-detected  CARS  and 
(D)  forward-detected  THG  signals  calculated  as  a  function  of  the  diameter  of 
a  spherical  sample  centered  at  the  focus. 


the  coherently-superimposed  radiation  fields  from  different  parts 
of  the  sample  in  the  focal  volume.  Consequently,  standard  decon¬ 
volution  methods  cannot  be  used.  Cheng  and  colleagues  have 
investigated  signal  generation  in  CARS  and  THG  microscopy  by 
using  the  Green’s  function  method,  in  which  the  CARS  signal  is 
calculated  as  a  coherent  addition  of  the  radiation  fields  from  the 
induced  dipoles  in  the  focal  region  (Cheng  and  Xie,  2002;  Cheng 
et  al.,  2002d).  This  model  makes  it  possible  to  calculate  nonlinear 
optical  signals  from  a  3D  sample  of  arbitrary  size  and  shape  under 
the  assumption  that  the  scatterer  and  the  surrounding  medium  are 
index-matched. 

Tightly-focused  excitation  fields  were  described  by  Richards 
and  Wolf  using  diffraction  theory  (Richards  and  Wolf,  1959).  The 
intensity  distribution  in  the  focal  region  of  a  Gaussian  beam  at 
wavelength  X  focused  by  a  NA  =1.4  objective  lens  is  depicted  in 
Figure  33.2(A).  The  FWHM  (full-width  at  half-maximum)  of  the 
lateral  profile  at  z  =  0  and  of  the  axial  profile  is  0.4  X  and  1.0  A,, 
respectively.  A  negative  phase  shift  (-180°  in  addition  to  kz )  along 
the  axial  direction,  shown  in  Figure  33.2(B),  is  known  as  the  Gouy 
phase  shift  (Siegman,  1986). 

For  tightly-focused  laser  beams,  the  Gouy  phase  shift  gives 
rise  to  a  n  phase  delay  of  the  excitation  fields  within  the  depth  of 
focus,  which  is  on  the  order  of  the  optical  wavelength.  The  total 
Gouy  phase  shift  of  the  excitation  fields  is  the  sum  of  the  phase 
shift  5(| ),  of  each  participating  field  £j.  For  CARS,  the  total  Gouy 
phase  shift  in  the  excitation  fields  is  Scouts  =  25(f),,  -  5(|)x  ~  n.  To 
understand  the  buildup  of  coherent  signal  in  the  focal  volume,  this 
rapid  phase  change  needs  to  be  taken  into  consideration.  Such  a 
Gouy  phase  shift  is  not  only  important  in  CARS  microscopy  but 
also  in  other  forms  of  nonlinear  coherent  microscopy,  such  as  SHG 
and  THG.  The  Gouy  phase  shift  is  a  critical  factor  that  determines 
the  imaging  properties  of  the  microscope.  In  THG,  for  instance, 
the  total  Gouy  phase  shift  is  b$THG  =  35(1),,  ~  37T. 

It  is  interesting  to  compare  the  CARS  and  THG  signals  as  a 
function  of  the  diameter  of  a  spherical  sample  centered  at  the  laser 


focus,  as  displayed  in  Figure  33.2(C,D),  respectively.  Although 
both  CARS  and  THG  are  four-wave  mixing  processes,  the  effect 
of  the  Gouy  phase  shift  on  the  CARS  field,  Ells  =  xa)EpE*  is  par¬ 
tially  cancelled  through  the  interaction  of  the  pump  field  ( Ep )  and 
the  conjugate  Stokes  field  (Ef).  On  the  other  hand,  the  effect  of 
the  Gouy  phase  shift  on  the  third  harmonic  field,  E3m  =  is 

much  larger.  The  large  Gouy  phase  shift  in  the  THG  process  results 
in  a  substantial  phase  mismatch  under  tight-focusing  conditions 
(Boyd,  1992).  Consequently,  THG  signal  from  an  isotropic  bulk 
that  fills  the  focal  volume  is  extremely  low  because  of  destructive 
interference.  On  the  other  hand,  the  phase  mismatch  in  CARS  is 
much  smaller.  This  results  in  constructive  interference  across  the 
entire  focal  volume,  giving  rise  to  a  strong  signal.  Details  about 
the  effect  of  the  Gouy  phase  shift  on  SHG,  CARS,  and  THG 
microscopy  can  be  found  elsewhere  (Moreaux  et  al.,  2000;  Cheng 
and  Xie,  2002;  Cheng  et  al.,  2002d). 

FORWARD  AND  BACKWARD 
DETECTED  COHERENT  ANTI-STOKES 
RAMAN  SCATTERING 

The  conventional  wisdom  is  that  the  CARS  signal  travels  in  the 
forward  direction,  the  phase-matched  direction,  which  is  the  same 
direction  as  the  collinearly-propagating  pump  and  Stokes  beams. 
Contrary  to  this,  Volkmer  and  colleagues  (2001)  also  observed  a 
backward-going  CARS  signal  using  an  epi-detection  microscope. 
The  presence  of  an  epi-detected  CARS  (E-CARS)  signal  can  be 
rationalized  as  follows:  When  the  size  of  the  scatterer  is  much 
smaller  than  the  pump  wavelength,  the  CARS  radiation  goes 
forward  and  backward  symmetrically  [Fig.  33.3(A)],  similar  to 
radiation  from  an  induced  Hertzian  dipole  in  the  plane  normal  to 
the  light  propagation.  Coherent  addition  of  induced  dipoles  in  this 
plane  results  in  CARS  radiation  propagating  in  both  the  forward 
and  the  backward  direction  but  confined  to  a  smaller  cone  angle 
[Fig.  33.3(B)].  In  contrast,  coherent  addition  of  the  induced  dipoles 
along  the  axial  direction  results  in  the  constructive  interference  in 
the  forward  direction  and  destructive  interference  in  the  backward 
direction  [Fig.  33.3(C)],  which  is  the  basis  for  the  phase-matching 
condition.  However,  this  is  only  true  when  the  sample  interaction 
length  is  larger  than  the  wavelength,  as  was  the  case  for  most  of 
the  experiments  prior  to  the  CARS  microscopy  work.  It  is  impor¬ 
tant  to  realize  that  the  phase-matching  condition  is  relaxed  when 
the  size  of  the  object  is  comparable  or  smaller  than  the  wavelengths 
of  the  excitation  beams.  Hence,  when  the  backward-going  wave 
has  a  large  lAkl,  the  phase  mismatch  lA/d  •  L  is  compensated  by  the 
short  interaction  length  L  of  the  object  and  the  phase-mismatch 
condition  of  Equation  3  is  still  satisfied. 

Figure  33.3(D)  displays  the  calculated  F-CARS  and  E-CARS 
signals  of  a  spherical  scatterer  with  centered  in  the  focus  as  a 
function  of  the  object’s  diameter  (Cheng  et  al.,  2002d).  The  epi- 
detected  signal  is  only  significant  when  the  diameter  of  the  object 
is  small.  The  first  maximum  is  reached  when  the  diameter  D  equals 
0.3  Xp.  The  intensity  oscillation  as  a  function  of  the  diameter  is  due 
to  the  interference  effect.  At  D  =  8.0 Xp,  the  epi-detected  signal  is 
105  times  smaller  than  the  forward-detected  signal.  For  a  scatterer 
with  embedded  in  a  solvent  with  ,  the  epi-detected  signal 
exhibits  the  same  behavior  but  with  a  modified  sample  suscepti¬ 
bility  of  -  x?di-  Interestingly,  the  result  is  independent  of  the 
sign  of  x?m  ~  X S  (Cheng  et  al.,  2002d).  The  distribution  of  CARS 
radiation  is  also  dependent  on  the  shape  of  the  object.  When 
shaped  like  a  long  rod  along  the  axial  (z)  direction,  the  signal 
predominantly  goes  forward  [Fig.  33.3(C)].  When  shaped  like  a 
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FIGURE  33.3.  (A)  Sketches  of  the  Hertzian  dipole  radiation  pattern;  (B) 
Hertzian  dipoles  are  coherently  added  up  in  the  sample  plane.  (C)  Addition  of 
multiple  Hertzian  dipoles  leads  to  constructive  interference  in  the  forward 
direction  and  destructive  interference  in  the  backward  direction.  (D)  Forward- 
and  backward-detected  CARS  signals  as  a  function  of  the  diameter  D  of  a 
spherical  sample  centered  at  the  focus. 


thin  disk,  the  signal  goes  in  both  forward  and  backward  directions 
[Fig.  33.3(B)]  (Cheng  et  al.,  2002d). 

The  forward  and  backward  CARS  signals  provide  comple¬ 
mentary  information  about  a  sample.  Forward-detected  CARS 
(F-CARS)  microscopy  is  suitable  for  imaging  objects  of  a  size 
comparable  to  or  larger  than  the  excitation  wavelength.  For  smaller 
objects,  the  F-CARS  contrast  can  be  overwhelmed  by  the  large 
non-resonant  background  from  the  solvent.  The  main  purpose  of 
E-CARS  microscopy  is  to  provide  a  sensitive  means  of  imaging 
objects  with  an  axial  length  smaller  than  the  excitation  wavelength, 
circumventing  the  large  background  from  the  solvent  (Cheng 
et  al.,  2001b,  2002d;  Volkmer  et  al.,  2001). 

As  a  demonstration.  Figure  33.4  shows  the  simultaneous  F- 
CARS  and  E-CARS  images  of  an  epithelial  cell  with  the  Raman 
shift  tuned  to  the  CFl-stretching  vibration  frequency  at  2847  cm-1. 
The  resonant  CARS  signal  arises  from  lipids  that  are  rich  in  C-Fl 
vibration.  Although  the  F-CARS  image  has  large  signal  amplitudes, 
it  has  a  constant  non-resonant  background  from  the  surroundings. 
A  better  contrast  for  small  objects  is  seen  in  the  E-CARS  image 
with  minimal  non-resonant  background  from  the  surroundings. 

Figure  33.5  shows  the  experimental  configuration  of  the  CARS 
microscope.  The  collinearly-overlapped  pump  and  Stokes  laser 
beams  are  tightly  focused  into  a  sample  by  a  high-NA  objective  in 
an  inverted  microscope.  A  condenser  lens  (or  objective  lens)  is 
used  to  collect  the  forward  CARS  signal.  The  forward  propagat¬ 
ing  signal  has  a  relatively  small  cone  angle  and  an  air  condenser 
generally  suffices  for  efficient  signal  collection  (Cheng  et  al., 
2002a).  The  large  working  distance  of  the  air  condenser  increases 
the  accessibility  to  the  sample  and  thus  facilitates  imaging  of  living 
cells. 

The  CARS  signal  is  conveniently  separated  from  the  excita¬ 
tion  beams  using  bandpass  filters.  The  backward  CARS  signal  is 


Distance  (pm)  Distance  (pm) 

FIGURE  33.4.  Forward-detected  (F-CARS)  and  epi-detected  (E-CARS)  images  of  an  epithelial  cell  with  the  Raman  shift  tuned  to  the  C-H  stretching  vibration 
at  2850  cm  '.  The  intensity  profiles  of  the  two  white  lines  are  shown  in  the  lower  panels.  Note  the  offset  in  the  F-CARS  image  due  to  the  non-resonant  back¬ 
ground  of  the  solvent.  The  background-free  E-CARS  image  shows  better  contrast  for  small  features. 
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FIGURE  33.5.  Schematic  of  a  CARS  microscope  with  both  forward  and  back¬ 
ward  detection.  The  synchronized  pump  and  Stokes  picosecond  pulse  trains  are 
collinearly  combined  on  a  dichroic  mirror  and  are  directed  to  a  beam-scanning 
microscope.  A  high-NA  objective  focuses  the  beams  to  a  diffraction-limited 
spot  in  the  sample.  CARS  signal  is  detected  simultaneously  in  the  forward  and 
backward  direction.  Non-descanned  photomultipliers  (PMT)  are  typically  used 
as  detectors. 


collected  with  the  same  objective  used  for  focusing  the  laser 
beams,  passing  through  the  dichroic  beam-splitter  that  is  used  to 
reflect  the  incoming  excitation  beams.  F-  and  E-CARS  images  are 
simultaneously  recorded  by  raster  scanning  the  two  laser  beams 
with  a  pair  of  galvanometer  mirrors.  With  a  laser-scanning  CARS 
microscope  using  two  near-infrared  (IR)  laser  beams  of  high  rep¬ 
etition  rate  (Cheng  et  al.,  2002a),  forward-  and  epi-detected  images 
(512  x  512  pixels)  from  the  same  sample  can  be  taken  simultane¬ 
ously  in  less  than  1  s. 

The  imaging  properties  of  F-CARS  and  E-CARS  have  been 
characterized  by  using  laser-scanning  CARS  images  of  polymer 
beads  embedded  in  agarose  gel  (Cheng  et  al.,  2002a).  The  typical 
FWHMs  of  the  lateral  and  axial  intensity  profiles  for  a  0.2  pm  bead 
are  0.28  pm  and  0.75  pm,  respectively.  Using  a  NA  =1.2  water- 
immersion  objective  lens,  the  lateral  FWHM  for  0. 1  pm  beads 
is  0.23  pm.  These  FWHM  should  be  used  with  caution  as  simple 
deconvolution  methods  cannot  be  applied  to  CARS  images. 

It  should  be  noted  that,  besides  the  mechanism  associated  with 
small  scatterers,  there  are  two  additional  mechanisms  responsible 
for  backward  CARS.  One  occurs  at  the  interface  of  two  homo¬ 
geneous  media  with  different  /(3)  (Cheng  et  al.,  2002d),  while  the 
other  results  from  the  back  reflection  or  backscattering  of  the 
forward  CARS  in  a  heterogeneous  sample.  Such  animal  tissue 
(Evans  et  al.,  2005). 


OPTIMAL  LASER  SOURCES  FOR 
COHERENT  ANTI-STOKES  RAMAN 
SCATTERING  MICROSCOPY 

It  is  known  that  the  non-resonant  electronic  contribution  to  the 
signal  can  be  enhanced  by  two-photon  electronic  resonance 
(Maeda  et  al.,  1988).  This  is  evidenced  by  the  fact  that  CARS 
images  of  liposomes  taken  with  visible  dye  lasers  are  dominated 
by  the  non-resonant  background  (Duncan,  1984).  The  use  of  near- 
IR  laser  beams  reduced  the  two-photon  enhancement  of  and 


accordingly  improved  the  image  contrast  (Zumbusch  et  al.,  1999; 
Cheng  et  al.,  2001b).  On  the  other  hand,  CARS  spectroscopy  with 
visible  laser  beams  can  take  advantage  of  one-photon  electronic 
resonance  (Hudson  et  al.,  1976;  Carreira  et  al.,  1977;  Dutta  and 
Spiro,  1978;  Dutta  et  al.,  1980;  Andrews  et  al.,  1981;  Igarashi 
et  al.,  1981;  Schneider  et  al.,  1988;  Ujj  et  al.,  1994;  Voroshilov 
et  al.,  1995)  for  chemical  species  with  absorption  in  the  visible 
wavelength  range,  such  as  cytochrome  c  (Dutta  et  al.,  1980; 
Andrews  et  al.,  1981),  |3-carotene  (Carreira  et  al.,  1977;  Dutta  et 
al.,  1980),  bateriorhodopsin  (Ujj  et  al.,  1994),  and  hemoglobin 
(Voroshilov  et  al.,  1995).  Electronically-resonant  CARS  provides 
a  way  to  suppress  the  non-resonant  solvent  background  by  enhanc¬ 
ing  the  signal  from  the  scatterer.  The  signal  enhancement  and  the 
photodamage  associated  with  one-photon  electronic  resonance  has 
yet  to  be  explored  in  CARS  microscopy. 

The  1999  work  used  femtosecond  lasers  that  have  a  high 
peak  power  and  are  widely  used  in  multi-photon  fluorescence 
microscopy.  However,  it  turns  out  that  femtosecond  pulses  are  not 
optimal  for  vibrational  imaging.  Whereas  a  few  molecules  such  as 
water  have  broad  Raman  spectra,  the  typical  Raman  line  width  at 
room  temperature  is  around  10  cm-1  in  the  condensed  phase.  The 
spectral  width  of  a  100  femtosecond  pulse  is  333  cm-1,  much  wider 
than  the  Raman  line  width.  Therefore,  most  of  the  pulse  energy 
ends  up  being  used  to  generate  the  non-resonant  background.  On 
the  other  hand,  the  spectral  width  of  a  picosecond  pulse  is  com¬ 
parable  to  the  Raman  line  width,  so  that  the  excitation  energy  is 
fully  utilized  to  maximize  the  vibrationally  resonant  CARS  signal. 
It  has  been  shown  theoretically  that  the  optimal  signal-to- 
background  ratio  occurs  with  spectral  pulse  widths  of  l~2ps  for  a 
typical  Raman  band  at  room  temperature  (Cheng  and  Xie,  2003). 
A  pulse  width  of  a  few  picosecond  provides  a  good  compromise 
between  the  spectral  resolution  and  the  peak  power,  and  improved 
the  signal-to-background  ratio.  In  2001,  Cheng  and  colleagues 
constructed  a  new  CARS  microscope  with  two  synchronized 
picosecond  titanium :  sapphire  (Ti:Sa)  lasers,  and  experimentally 
demonstrated  improved  spectral  resolution  and  high  signal-to- 
background  ratio  (Cheng  et  al.,  2001b). 

In  general,  the  timing  jitter  between  the  pump  and  the  Stokes 
pulses  introduces  a  fluctuation  to  the  CARS  intensity  and  this 
limits  the  image  acquisition  rate.  This  is  particularly  important 
when  two  independent,  passively  mode-locked  lasers  are  used. 
Recently,  tight  synchronization  between  the  pulse  trains  with  jitters 
of  20  to  lOOfs  has  been  realized  (Jones  et  al.,  2002;  Potma  et  al., 
2002).  This  has  allowed  acquisition  of  high-quality  CARS  images 
using  2ps  pulse  trains  (Potma  and  Xie,  2003). 

Picosecond  pulses  from  a  Ti :  Sa  oscillator  typically  have  an 
energy  ~6nJ  at  an  average  power  level  of  about  500  mW  and  a  rep¬ 
etition  rate  of  around  80  MHz.  Such  laser  pulses  have  been  suc¬ 
cessfully  applied  for  high-speed  CARS  imaging  (Cheng  et  al., 
2002a).  The  integrated  CARS  signal  intensity  can  be  enhanced  by 
a  factor  of  m2  (=  m3/m)  if  the  pump  and  Stokes  pulse  energies  are 
increased  by  a  factor  of  m  and  the  repetition  rate  is  lowered  by  the 
same  factor.  On  the  other  hand,  high  peak  power  can  be  hazardous 
to  the  sample  and  a  high  repetition  rate  is  beneficial  for  fast  image 
acquisition.  A  repetition  rate  of  100  kHz  to  1  MHz  is  a  good 
compromise. 

Recently,  Ye  and  co-workers  have  shown  that  enhancement  of 
pulse  energy  can  be  realized  through  coherent  storage  of  radiation 
in  a  high  finesse  cavity  without  a  gain  medium  (Jones  and  Ye, 
2002).  Using  a  high  finesse  cavity  equipped  with  a  cavity  dumper, 
Potma  and  colleagues  demonstrated  amplification  of  mode-locked 
picosecond  pulses  that  is  greater  than  a  factor  of  30  at  a  repetition 
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rate  of  253  kHz  (Potma  et  al.,  2003).  These  novel  sources  provide 
new  possibilities  for  improving  the  speed  and  sensitivity  of  CARS 
imaging. 

The  timing  jitter  between  the  pump  and  probe  pulses  can  be 
avoided  if  one  uses  synchronously  pumped  optical  parametric 
oscillator  (OPO)  systems.  When  a  commercial,  near-IR  picosec¬ 
ond  OPO  based  on  periodically-poled  nonlinear  crystals,  pumped 
by  a  mode-locked  laser  with  semiconductor  saturable  absorber 
mirrors  (SESAMs)  becomes  available,  it  can  serve  as  an  ideal  laser 
source  for  CARS  imaging. 


SUPPRESSION  OF  THE  NON-RESONANT 
BACKGROUND 

The  suppression  of  both  the  non-resonant  background  from  the 
solvent  and  that  from  the  scatterers  is  key  to  improving  the  detec¬ 
tion  sensitivity  and  spectral  selectivity  of  CARS  microscopy. 
Several  schemes  for  background  suppression  have  been  developed 
and  are  summarized  as  follows. 

Use  of  Picosecond  Instead  of 
Femtosecond  Pulses 

A  straightforward  way  to  minimize  the  non-resonant  background 
is  not  to  generate  it  in  the  first  place.  As  outlined  above,  compared 
to  the  broadband  pulses  of  fs  lasers,  the  spectral  bandwidth  of  1  to 
2ps  pulses  matches  the  linewidth  of  the  Raman  bands  and  mini¬ 
mizes  generation  of  non-resonant  spectral  components.  A  pico¬ 
second  laser  system  thus  represents  a  better  light  source  for  CARS 
imaging  in  terms  of  the  signal-to-non-resonant  background  ratio. 

Epi-Detection 

As  already  discussed,  this  method  introduces  a  large  wave  vector 
mismatch,  which  acts  as  a  size-selective  filter  that  rejects  the  back¬ 
ground  signal  (resonant  or  non-resonant)  from  the  bulk  solvent  and 
allows  high-sensitivity  imaging  of  small  objects  (Volkmer  et  al., 
2001;  Cheng  et  al.,  2001b).  Because  E-CARS  rejects  signal  con¬ 
tributions  based  on  size  and  not  on  vibrational  properties,  the  non¬ 
resonant  signal  from  the  small  objects  themselves  still  contributes 
to  the  image.  Hence,  for  weak  resonant  signals  (e.g.,  from  the 
protein  amide  I  band),  the  vibrational  sensitivity  of  E-CARS  can 
be  limited  due  to  the  non-resonant  background  from  the  small 
objects  themselves. 

Polarization-Sensitive  Detection 

This  method  is  based  on  the  different  polarization  properties  of  the 
electronic  (p(3>NE)  and  resonant  (P(3,R)  portions  of  the  third  order 
polarization  (Akhmanov  et  al.,  1977;  Oudar  et  al.,  1979;  Brakel 
and  Schneider,  1988).  The  polarization  direction  of  the  emitted 
CARS  field  is  determined  by  the  polarization  of  the  incident  pump 
and  Stokes  fields  and  the  Raman  depolarization  ratio  p  of  the  mate¬ 
rial.  The  depolarization  ratio  of  the  resonant  signal  pK  depends  on 
the  molecular  vibrational  mode  and  usually  differs  from  the  con¬ 
stant  depolarization  ratio  of  the  isotropic  non-resonant  background 
pNR  (Kleinman,  1962).  With  linearly-polarized  pump  and  Stokes 
beams,  a  difference  between  pR  and  p;VR  leads  to  non-parallel, 
linearly-polarized  resonant  and  non-resonant  anti-Stokes  fields.  In 
polarization-sensitive  CARS,  a  polarization  analyzer  in  front  of  the 


detector  blocks  the  non-resonant  signal,  whereas  a  portion  of  the 
differently  polarized  resonant  signal  leaks  through  the  analyzer. 
The  detected  resonant  signal  is  optimized  when  the  polarization 
difference  between  P<3,R  and  P(3)NR  is  maximized.  This  is  achieved 
by  introducing  a  polarization  difference  of  7 1 .6°  between  the  pump 
and  the  Stokes  beams  (Brakel  and  Schneider,  1988). 

The  efficient  background  rejection  of  the  P-CARS  microscope 
permits  vibrational  imaging  of  intracellular  proteins  (Cheng 
et  al.,  2001a).  Figure  33.6(A)  shows  the  P-CARS  spectrum  of  N- 
methylacetamide,  a  model  compound  containing  the  characteristic 
amide  I  vibration  at  1652  cm”1,  which  is  a  signature  band  for  pep¬ 
tides  and  proteins.  As  the  P-CARS  band  positions  coincide  with 
the  corresponding  Raman  band  positions,  one  can  assign  the 
P-CARS  bands  based  on  the  Raman  literature.  The  difference  in 
the  relative  intensity  of  the  bands  in  the  two  spectra  arises  from 
the  quadratic  dependence  on  the  number  of  vibrational  oscillators 
and  the  Raman  depolarization  ratios  of  the  bands.  Figure  33.6(B,C) 
shows  the  background-free  P-CARS  images  of  an  unstained 
epithelial  cell  with  cop  -  cos  tuned  to  the  amide  I  band  [Fig.  33.7(B)]. 
Tuning  (0,,  -  cos  away  from  the  amide  I  band  to  1745  cm”1  resulted 
in  a  faint  contrast  [Fig.  33.7(C)],  proving  that  the  image  contrast 
was  due  to  proteins,  the  distribution  of  which  is  heterogeneous  in 
the  cell  (Cheng  et  al.,  2001a). 

Time-Resolved  Coherent  Anti-Stokes  Raman 
Scattering  Detection 

The  vibrationally-resonant  signal  can  be  separated  from  the  non¬ 
resonant  electronic  contribution  by  use  of  pulse-sequenced  detec¬ 
tion  with  femtosecond  pulse  excitation  (Laubereau  and  Kaiser, 
1978;  Kamga  and  Sceats,  1980).  In  time-resolved  CARS  detection, 
a  signal-generating  probe  pulse  is  time  delayed  with  respect  to  a 
temporally-overlapped  pump/Stokes  pulse-pair.  Because  of  the 
instantaneous  dephasing  time  of  the  non-resonant  signal,  the 
non-resonant  CARS  signal  only  exists  when  the  pump/Stokes 
pulse-pair  overlaps  with  the  probe  pulse.  On  the  other  hand,  the 
vibrationally-resonant  CARS  signal  decays  with  the  finite  dephas¬ 
ing  time  of  the  vibrational  mode.  The  dephasing  time  is  related  to 
the  spectral  width  of  the  corresponding  Raman  band  and  is  typi¬ 
cally  several  hundred  femtoseconds  for  a  mode  in  the  condensed 
phase  (Fickenscher  et  al.,  1992).  The  non-resonant  background  in 
the  CARS  images  can  thus  be  eliminated  by  introducing  a  suitable 
delay  between  the  femtosecond  pump/Stokes  and  the  probe  pulses. 
Time-resolved  CARS  imaging  has  been  demonstrated  by  Volkmer 
and  colleagues  with  a  three-color  excitation  scheme  (Volkmer  et 
al.,  2002).  The  disadvantages  of  this  approach  are  the  photodam¬ 
age  induced  by  the  femtosecond  pulses  and  the  necessity  of  adding 
a  third  laser  beam  of  a  different  color. 

Phase  Control  of  Excitation  Pulses 

With  femtosecond  lasers,  another  way  to  reduce  the  electronic 
contribution  is  phase-shaping  of  the  pulses.  Phase-shaping  of 
femtosecond  pump  and  Stokes  pulses  suppresses  the  non-resonant 
signal  by  introducing  a  phase  mismatch  in  the  coherent  addition 
of  non-resonant  spectral  components,  whereas  the  resonant  con¬ 
tribution  still  adds  up  in  phase  (Oron  et  al.,  2002).  This  method 
has  been  applied  to  CARS  spectroscopy  and  microscopy  with 
single,  femtosecond  pulses  (Dudovich  et  al.,  2002).  Recently,  Oron 
and  colleagues  reported  a  combination  of  phase  and  polarization 
shaping  for  background-free  single-pulse  CARS  spectroscopy 
(Oron  et  al.,  2003). 
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FIGURE  33.6.  (A)  P-CARS  and  spontaneous  Raman  spectra 
of  pure  A-methylacetamide  liquid  recorded  at  room  tem¬ 
perature.  (B,  C)  Polarization  CARS  images  of  an  unstained 
epithelial  cell  with  CD,,  —  cp,  tuned  to  1650  cm-1  and  1745  cm-1, 
respectively.  Each  image  was  acquired  by  raster  scanning  the  ^ 

sample  with  an  acquisition  time  of  8  min.  The  pump  and  Stokes  ^ 

power  were  1.8  and  1.0  mW,  respectively,  at  a  repetition  rate  of  c/j 

400  kHz.  § 
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FIGURE  33.7.  (A)  Energy  diagram  for  multiplex  CARS  microspectroscopy  using  a  picosecond  pump  beam  and  a  femtosecond  Stokes  beam.  (B)  Schematic  of 
the  setup  for  multiplex  CARS  microspectroscopy.  (C)  Multiplex  CARS  spectrum  of  a  DSPC  (l,2-distearoyl-sn-glycero-3-phosphocholine)  liposome.  The  pump 
and  Stokes  beam  were  centered  at  713  nm  and  900  nm,  with  average  powers  of  0.6  mW  and  0.3  mW  at  a  repetition  rate  of  400  kHz.  The  integration  time  was 
2  s.  (D)  Polarization-sensitive  multiplex  CARS  spectrum  of  the  same  liposome  sample.  The  average  power  of  the  pump  and  the  Stokes  beams  was  1.2mW 
and  0.6  mW,  respectively.  The  other  parameters  are  the  same  as  in  (C). 
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MULTIPLEX  COHERENT  ANTI-STOKES  RAMAN 
SCATTERING  MICROSPECTROSCOPY 

While  CARS  microscopy  allows  high-sensitivity  vibrational 
imaging  of  particular  molecules,  CARS  microspectroscopy  can 
provide  molecular  structure  information  about  samples  with  3D 
spatial  resolution.  Picosecond  excitation  enables  high-sensitivity 
CARS  imaging  based  on  a  particular  band;  however,  it  is  time  con¬ 
suming  to  record  a  CARS  spectrum  by  tuning  the  Stokes  frequency 
point  by  point  (Hashimoto  et  al.,  2000;  Cheng  et  al.,  2001a, b).  Mul¬ 
tiplex  CARS  (M-CARS)  spectroscopy  first  demonstrated  by 
Akhmanov  and  colleagues  permits  fast  spectral  data  acquisition 
(Akhmanov  et  al.,  1977).  In  previous  work,  a  narrowband  and  a 
broadband  dye  laser  were  used  for  the  pump  and  the  Stokes  beams, 
respectively  (Toleutaev  et  al.,  1994;  Ujj  et  al.,  1994;  Voroshilov 
et  al.,  1996).  Recently,  M-CARS  microspectroscopy  has  been 
developed  for  fast  spectral  characterization  of  microscopic  samples 
(Muller  and  Schins,  2002;  Cheng  et  al.,  2002c;  Kee  and  Cicerone, 
2004;  Otto  et  al.,  2001). 

Figure  33.7(A)  shows  the  energy  diagram  and  the  schematic 
of  an  M-CARS  microscope  using  a  picosecond  and  a  femtosecond 
pulse  train.  In  a  theoretical  investigation  by  Cheng  and  colleagues 
(Cheng  et  al.,  2002c),  it  was  shown  that  the  frequency  chirp  in  the 
femtosecond  Stokes  pulse  induces  little  distortion  to  the  CARS 
spectrum  of  a  sample.  The  chirped  pulses  are  desirable  because  of 
reduced  sample  damage.  Polarization-sensitive  detection  has  been 
incorporated  into  an  M-CARS  microscope  to  suppress  the  non¬ 
resonant  background  (Cheng  et  al.,  2002c).  Figure  33.7(B,C) 
shows  the  M-CARS  spectra  of  the  C-H  stretching  vibrational 
bands  in  a  DSPC  liposome  recorded  with  parallel-polarized  beams 
and  with  polarization-sensitive  detection,  respectively.  The  P- 
CARS  spectrum  is  free  from  the  complication  of  the  non-resonant 
background  and  spectral  distortion.  The  symmetric  CH2  stretching 
vibration  at  2845  cnf 1  exhibits  a  high  signal-to-background  ratio 
and  has  been  used  for  imaging  lipids  as  shown  in  the  images  of 
Figures  33.4,  33.8,  and  33.11. 

M-CARS  microspectroscopy  has  been  applied  to  distinguish 
the  gel  and  the  liquid  crystal  phases  in  liposomes.  Muller  and  col¬ 
leagues  have  shown  that  it  is  possible  to  distinguish  the  two  dif¬ 
ferent  phases  by  using  the  CARS  spectra  of  the  C-C  stretching 
vibration  (Muller  and  Schins,  2002).  Cheng  and  colleagues  have 
shown  that  the  different  lipid  phases  are  also  reflected  in  the  spec¬ 
tral  differences  in  the  C-H  stretching  vibration  region  (Cheng 
et  al.,  2002c). 


COHERENT  ANTI-STOKES  RAMAN  SCATTERING 
CORRELATION  SPECTROSCOPY 

Fast  dynamical  processes  can  be  probed  by  optical  intensity 
correlation  spectroscopy.  Fluorescence  correlation  spectroscopy 
(FCS)  measures  the  concentration  fluctuation  of  specific  fluores¬ 
cent  molecules  (Magde  et  al.,  1972;  Elson  and  Magde,  1974).  FCS 
has  found  wide  application  with  the  development  of  confocal 
detection  (Eigen  and  Rigler,  1994;  Maiti  et  al.,  1997).  For  probing 
diffusion  processes  with  vibrational  selectivity,  CARS  correlation 
spectroscopy  (CARS-CS)  has  been  developed  recently  (Cheng 
et  al.,  2002b;  Hellerer  et  al.,  2002).  This  technique  measures  the 
fluctuation  of  the  CARS  signal  from  a  subfemtoliter  excitation 
volume.  CARS-CS  has  been  carried  out  with  epi-detection  and/or 
polarization-sensitive  detection.  Figure  33.8  shows  the  epi- 
detected  CARS  intensity  fluctuations  and  autocorrelation  curves 
that  characterize  the  diffusion  of  0.175|im  polystyrene  beads  in 
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FIGURE  33.8.  (A)  CARS  and  spontaneous  Raman  spectra  of  a  polystyrene 
film  coated  on  a  coverslip.  The  CARS  spectrum  was  recorded  with  parallel- 
polarized  pump  and  Stokes  beams.  The  pump  frequency  was  fixed  at 
14,047  cm  1  whereas  the  Stokes  frequency  was  scanned.  (B)  Epi-detected 
CARS  signal  traces  of  a  diluted  aqueous  suspension  of  0.1 75  pm  polystyrene 
spheres  ((TV)  =  0.04).  The  average  pump  and  Stokes  power  measured  after  the 
beam  combiner  were  1.3  and  0.6  mW,  respectively.  (C)  Epi-detected  CARS 
intensity  autocorrelation  curves  corresponding  to  the  signal  traces  in  (B). 


water.  The  spectral  selectivity  is  illustrated  by  the  CARS  spectrum 
of  polystyrene,  in  which  the  aromatic  C-H  (3052  cm-1),  the  sym¬ 
metric  aliphatic  C-H  (2852  cm-1),  and  the  asymmetric  aliphatic 
C-H  (2907  cm-1)  stretching  bands  display  a  high  signal-to- 
background  ratio.  With  cop  -  CO,  tuned  to  the  aromatic  C-H  band  at 
3050  cm-1  and  the  aliphatic  C-H  band  at  2843  cm-1,  the  intensity 
autocorrelation  revealed  a  diffusion  time  of  27.3  ±  0.7  ms  and 
28.6  ±  1.0ms,  respectively,  fitted  with  a  CARS-CS  model  (Cheng 
et  al.,  2002b).  Tuning  cop  -  CO,  to  3 146  cm-1  away  from  any  Raman 
resonance,  resulted  in  an  autocorrelation  curve  that  is  smaller  by 
16  times  in  amplitude  than  the  curve  at  3050  cm-1.  The  residual 
intensity  autocorrelation  with  a  diffusion  time  of  160  (±6)  ms  was 
attributed  to  the  water  signal  back-reflected  by  the  diffusing  beads. 
It  has  been  demonstrated  that  CARS-CS  can  provide  quantitative 
information  about  the  diffusion  coefficient,  particle  concentration, 
and  viscosity  of  a  medium  in  a  chemically  selective  and  non- 
invasive  manner  (Cheng  et  al.,  2002b). 

The  intensity  autocorrelation  function  of  CARS-CS  bears 
much  similarity  to  the  FCS  autocorrelation  function  at  low  scat- 
terer  concentration.  However,  a  careful  analysis,  in  which  the 
phase  of  the  CARS  field  is  included,  shows  that  the  E-CARS  cor- 
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relation  function  is  different  from  the  FCS  correlation  function. 
Whereas  the  autocorrelation  amplitude  in  FCS  is  inversely  pro¬ 
portional  to  the  average  number  of  particles  ( N)  in  the  excitation 
volume  and  vanishes  at  high  concentration,  the  E-CARS  correla¬ 
tion  function  contains  a  term  that  is  independent  of  ( N ).  This 
implies  that  the  E-CARS  autocorrelation  amplitude  does  not 
vanish  for  higher  ( N }.  This  concentration-independent  term  has 
been  confirmed  experimentally  and  provides  a  way  of  probing  the 
diffusion  of  objects  at  high  concentrations  (Cheng  et  al.,  2002b). 


COHERENT  ANTI-STOKES  RAMAN 
SCATTERING  IMAGING  OF 
BIOLOGICAL  SAMPLES 

The  capability  of  CARS  microscopy  to  generate  chemically- 
selective  images  with  a  high  3D  spatial  resolution  offers  an 
attractive  method  for  cellular  imaging.  In  Figure  33.9,  a  3D  recon¬ 
struction  made  from  CARS  recordings  of  three  epithelial  cells  is 
depicted.  Similar  to  multi-photon  fluorescence  microscopy,  the 
intrinsic  sectioning  capability  of  CARS  ensures  that  the  cells  are 
resolved  in  three  dimensions.  The  Raman  shift  was  tuned  to  the 
symmetric  CH2  stretch  vibration  of  the  lipid’s  aliphatic  chains  at 
2845  cnf1,  a  vibrational  mode  that  is  abundant  in  lipids.  The  con¬ 
trast  observed  in  the  image  is  therefore  largely  due  to  the  phos¬ 
pholipid  membranes,  such  as  the  plasma  membrane,  and 
intracellular  lipid  droplets  mainly  composed  of  triglyceride.  The 
CARS  signal  from  the  aliphatic  CH2  vibrational  band  has  been 
used  in  imaging  studies  of  various  cell  types  (Zumbusch  et  al., 
1999;  Floltom  et  al.,  2001;  Cheng  et  al.,  2002a). 

The  CARS  sensitivity  to  lipids  is  so  high  that  single 
phospolipid  bilayers  can  be  visualized  (Potma  and  Xie,  2003). 
Figure  33.10(A)  depicts  an  image  of  the  natural  membrane  of  an 
erythrocyte  ghost,  taken  at  the  symmetric  CH2  stretch  vibration  fre¬ 
quency  of  lipids.  This  image  was  recorded  in  the  forward  direc¬ 
tion  with  the  bilayer  parallel  to  the  propagation  direction  of  the 
beams.  The  higher  signal  intensities  at  the  top  and  bottom  of  the 
cell  image  relative  to  the  left  and  right  of  the  cell  image  are  a  result 
of  the  relative  orientation  of  the  acyl  chains  in  the  membrane.  The 
stronger  signals  were  generated  when  the  CH2  stretching  normal 
modes  are  aligned  with  the  polarization  direction  of  the  pump  and 
Stokes  beams.  This  feature  can  be  used  to  determine  molecular  ori¬ 
entation  with  CARS  microscopy  (Wurpel  et  al,  2005;  Cheng  et  al. 
2003).  When  the  membrane  surface  is  perpendicular  to  the  optical 
axis,  as  in  Figure  33.10(B),  the  bilayer  is  conveniently  visualized 
in  the  epi-direction,  demonstrating  the  high  sensitivity  of  CARS 
microscopy. 

The  ability  of  visualizing  single  bilayers  enables  the  study  of 
lipid  phase  segregation  in  membranes.  Phase-segregated  domains, 
also  known  as  lipid  rafts,  form  when  lipids  of  different  thermody¬ 
namic  phase  are  mixed.  Lipid  rafts  are  thought  to  play  crucial  roles 
in  such  processes  as  protein  sorting  and  signaling  on  cell  mem¬ 
branes.  With  CARS,  micrometer-sized  lipid  domains  can  be 
directly  detected  without  the  introduction  of  fluorescent  membrane 
labels  (Potma  and  Xie,  2005).  In  Figure  33.10(C),  a  clear  phase 
separation  is  observed  between  the  phospholipids  DOPC  (1,2- 
dioleoyl-.sn-glycero-3-phosphocholine)  and  deuterated  DSPC  (1,2- 
distearoyl-sn-glycero-3-phosphocholine)  by  tuning  into  the  C-D 
stretching  frequency  of  the  latter.  This  underlines  the  potential  for 
CARS  microscopy  to  detect  lipid  rafts  in  cells  non-invasively. 

Lipid  droplets  are  typically  found  in  adipocytes  and  hepato- 
cytes  as  energy  reservoirs  and  are  thought  of  as  playing  important 
roles  in  cellular  processes  (Murphy,  2001).  Because  lipid  droplets 


FIGURE  33.9.  Three-dimensional  reconstruction  of  three  epithelial  cells.  The 
Raman  shift  is  tuned  to  the  C-H  stretch  vibration  at  2845  cm  The  3D  image 
is  composed  of  71  planes  of  512  X  512  pixels  each.  Pump  beam  power  was 
15mW  and  Stokes  beam  power  was  lOmW  at  a  repetition  rate  of  76  MHz. 
Each  plane  was  recorded  in  2 s.  The  total  dimension  is  92  X  92  X  50prn. 


are  aggregates  of  neutral  lipids,  they  are  rich  in  CH2  bonds  and 
yield  very  strong  CARS  signals.  Traditionally,  lipid  droplets  can 
be  labeled  with  Oil  Red  O  in  fixed  cells  and  imaged  with  fluores¬ 
cence  microscopy.  In  a  recent  study.  Nan  and  colleagues  applied 
CARS  microscopy  to  monitor  the  growth  of  triglyceride  droplets 
during  the  differentiation  process  of  3T3-L1  cells  (Nan  et  al., 
2003).  In  addition  to  the  traditional  picture  of  lipid  accumulation 
in  the  differentiation  process,  the  P-CARS  images  shown  in  Figure 
33.11  indicate  an  intermediate  stage,  that  is,  the  removal  of  cyto¬ 
plasmic  lipid  droplets  after  the  addition  of  the  induction  medium. 
This  reduction  of  lipid  droplets  was  attributed  to  an  increased 
activity  of  hormone-sensitive  lipase,  the  enzyme  responsible  for 
hydrolyzing  intracellular  triglyceride  and  sterol  esters  (Kawamura 
et  al.,  1981). 

So  far,  we  have  discussed  imaging  of  aggregates  of  lipids 
(membranes  and  droplets)  and  proteins  in  cells  using  the  C-H 
stretching  band  and  the  amide  I  band,  respectively.  In  order  to 
follow  the  distribution  and  diffusion  of  specific  molecules  in  living 
cells,  isotopic  substitution  provides  a  good  strategy  to  enhance 
molecular  selectivity.  For  example,  the  Raman  shift  of  the  aliphatic 
and  the  aromatic  C-D  vibration  bands  lies  in  the  region  of  2100  to 
2300  cnf1,  isolated  from  the  Raman  bands  of  endogenous  mole¬ 
cules.  In  earlier  work  using  picosecond  dye  lasers,  Duncan 
and  colleagues  showed  that  CARS  microscopy  could  distinguish 
deuterated  liposomes  from  non-deuterated  ones  (Duncan,  1984). 
The  deuteration  method  has  been  applied  to  living  cells  by  Holtom 
and  colleagues,  who  demonstrated  selective  mapping  of  deuterated 
lipid  vesicles  in  a  macrophage  cell  (Holtom  et  al.,  2001). 

The  high  signal  level  in  CARS  microscopy  permits  high-speed 
imaging  of  intracellular  dynamics.  Potma  and  colleagues  used  the 
resonant  CARS  signal  from  the  O-H  stretching  vibration  of  water 
to  visualize  intracellular  hydrodynamics  (Potma  et  al.,  2001). 
By  line-scanning  the  laser  beams,  they  measured  the  intracellular 
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FIGURE  33.10.  CARS  images  of  erythrocyte  ghosts  recorded  at  C-H  stretching  vibration  of  2845cm  (A)  F-CARS  image  in  the  equatorial  plane  of  the  cell. 

Note  the  orientation-dependent  signal  of  the  C-Fi  bonds  in  the  bilayer.  (B)  E-CARS  image  of  the  bottom  of  the  cells  close  to  the  coverslip.  Pump  beam  power 
was  0.3  mW  and  Stokes  beam  power  was  0. 1  mW  at  a  repetition  rate  of  250kFlz.  Images  are  256  X  256  pixels  and  the  pixel  dwell  time  was  1  ms.  (C)  Part  of  a 
giant  unilamellar  vesicle  composed  of  an  equimolar  ratio  of  the  phospholipids  DOPC  and  DSPC,  showing  a  clear  phase  separation  of  the  lipids.  To  enhance  con¬ 
trast,  deuterated  DSPC  was  used  and  the  image  was  taken  at  the  CD  stretch  vibrational  band  at  2090  cm  '.  giving  rise  to  bright  signals  from  the  DSPC-enriched 
domains. 


water  diffusion  coefficient  and  the  membrane  permeability.  Using 
laser-scanning  CARS  microscopy,  Cheng  and  co-workers  charac¬ 
terized  apoptosis  induced  by  L-asparaginase  in  unstained  NIH3T3 
fibroblasts  (Cheng  et  al.,  2002a).  They  tuned  cop  -  CO,  to  the 
aliphatic  C-H  vibrational  frequency  and  were  able  to  identify  dif¬ 
ferent  stages  in  the  apoptotic  process,  such  as  the  compaction  of 
cytoplasmic  organelles  and  the  rounding  of  the  cells. 


CONCLUSIONS  AND  PERSPECTIVES 

CARS  microscopy  is  a  new  approach  for  chemical  and  biological 
imaging  of  unstained  samples  with  vibrational  selectivity  and 
3D  spatial  resolution.  On  the  method-development  side,  various 
schemes  including  picosecond  excitation,  epi-detection,  and 
polarization-sensitive  detection  have  significantly  improved  the 


FIGURE  33.11.  P-CARS  images  3T3-L1  preadipocytes  differentiating  into  adipocyte  cells  at  different  stages,  recorded  at  the  C-FI  stretching  vibration  of 
2845  cm"1.  Differentiation  is  induced  with  insulin,  isobutylmethylxanthine,  and  dexamethasone  2  days  postconfluence  (A).  The  dots  seen  in  (A)  are  lipid  droplets. 
Note  the  clearance  of  lipid  droplets  from  the  cytoplasm  48 h  after  initiation  (B).  After  that  cells  accumulate  lipid  droplets  again  as  differentiation  went  on.  In 
fully  differentiated  cells  (C),  the  cytoplasm  are  full  of  large  droplets  (Nan  et  al..  2003).  Pump  beam  power  was  15  mW  and  Stokes  beam  power  was  7.5  mW  at 
a  repetition  rate  of  80MFlz.  Image  acquisition  time  was  2.7  s. 
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ratio  of  the  resonant  signal  to  the  non-resonant  background.  High 
imaging  speed  (video  rate)  has  been  achieved  with  a  laser¬ 
scanning  microscope.  New  developments  in  laser  technologies  are 
boosting  the  imaging  sensitivity  to  higher  levels.  Meanwhile,  a  rig¬ 
orous  theoretical  model  based  on  the  Green’s  function  method 
has  led  to  a  better  understanding  of  the  contrast  mechanisms  of 
CARS  microscopy  and  nonlinear  coherent  microscopy  in  general. 
Multiplex  CARS  microspectroscopy  for  localized  structure 
characterization  and  CARS  correlation  spectroscopy  for  probing 
fast  diffusion  dynamics  have  also  been  developed. 

On  the  application  side,  CARS  microscopy  has  been  used  for 
mapping  the  distribution,  orientation,  and  diffusion  of  specific 
compounds,  for  imaging  lipids,  proteins,  and  chromosomes  in 
unstained  living  cells,  and  for  monitoring  cellular  processes  such 
as  apoptosis  and  lipogenesis.  Looking  into  the  future,  CARS 
microscopy  provides  exciting  possibilities  for  tackling  a  broad 
range  of  chemical  and  biological  problems.  It  is  expected  to  have 
potential  applications  in  such  fields  as  membrane  biology,  neuro¬ 
biology,  and  pathology.  In  the  meantime,  there  are  many  oppor¬ 
tunities  to  push  the  fundamental  limits  of  CARS  microscopy  to  the 
next  level. 
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INTRODUCTION 

The  ability  of  confocal  laser-scanning  microscopy  to  collect  stacks 
of  optical  sections  has  made  three-dimensional  (volumetric) 
imaging  a  standard  analytical  tool  in  experimental  cell  and  devel¬ 
opmental  biology.  Parallel  developments  in  deconvolution  tech¬ 
niques,  especially  as  computational  power  increased  and  costs 
decreased,  offered  tools  to  make  three-dimensional  (3D)  imaging 
from  widefield  as  well  as  confocal  microscopes  possible.  Despite 
the  high  spatial  resolution  provided  by  these  3D  methods,  they  all 
suffer  from  a  common  limitation:  light  scattering  in  the  specimen 
limits  them  to  operating  in  the  outer  few  hundred  micrometers  of 
the  specimen.  Even  the  greater  depth  of  penetration  and  reduced 
photobleaching  provided  by  two-photon  laser-scanning  micro¬ 
scopy  falls  short  for  larger  specimens  such  as  embryos.  To  perform 
volumetric  imaging  on  larger  specimens,  alternative  approaches 
are  needed.  This  chapter  examines  four  light-optical  methods: 
surface  imaging  microscopy  (SIM),  optical  coherence  tomography 
(OCT),  optical  projection  tomography  (OPT),  and  the  recently 
developed  selective  plane  illumination  microscope  (SPIM),  as  well 
as  microscopic  magnetic  resonance  imaging  (pMRI)  and  micro¬ 
scopic  computed  tomography  (CT)  as  potential  solutions  to  the 
challenges  of  collecting  high-quality  3D  image  data. 

Each  of  these  volumetric  imaging  methodologies  has  particular 
strengths  and  limitations;  no  single  method  addresses  all  3D 
imaging  requirements.  SIM  (also  referred  to  as  episcopic  fluores¬ 
cence  image  capture  (EFIC)  takes  a  brute  force  approach,  physi¬ 
cally  sectioning  the  specimen  and  obtaining  a  series  of 
two-dimensional  images  that  can  later  be  assembled  into  a  volu¬ 
metric  reconstruction.  Both  OPT  and  OCT  employ  different  tomo¬ 
graphic  approaches  to  obtain  data  from  the  depth  of  the  specimen, 
building  up  a  3D  model  from  a  series  of  2D  data  sets.  OPT  transil- 
luminates  the  specimen,  and  collects  a  large  number  of  2D  images 
from  different  orientations;  OCT  uses  depth  encoding  of  reflected 
light  and  is  conceptually  similar  to  ultrasound  imaging.  SPIM 
represents  a  hybrid  between  these  approaches,  using  transverse, 
sheet-like  illumination  to  generate  series  of  2D  image  stacks  from 
different  orientations  that  can  be  assembled  with  computational 
tools.  Finally,  pMRI  and  micro-computerized  tomography  (micro- 
CT)  are  non-optical.  The  former  derives  image  contrast  from  the 
water  contained  in  the  specimen.  This  permits  the  entire  volume  of 
an  opaque  object  to  be  imaged.  The  latter  is  merely  a  micro  version 
of  the  CAT  scanner  used  in  hospitals.  We  will  describe  each  of  these 
techniques  and  suggest  how  each  might  be  used. 


SURFACE  IMAGING  MICROSCOPY 
AND  EPISCOPIC  FLUORESCENCE 
IMAGE  CAPTURE 

Surface  imaging  microscopy  is  a  form  of  physical  serial  section¬ 
ing  where  one  images  the  cut  face  of  the  embedded  specimen 
instead  of  the  ribbon  of  removed  sections. 

A  SIM/EFIC  device  is  essentially  a  widefield  microscope 
mounted  on  a  precision  microtome.  Dry  objectives  ranging  from 
2x  to  40x  magnification  are  used  to  image  the  face  of  the  embed¬ 
ding  block  to  an  in-plane  resolution  of  8.8  to  0.4pm,  respectively. 
As  each  section  is  cut,  the  face  of  the  remaining  block  is  recorded 
with  a  charge-coupled  device  (CCD)  camera  (Ewald  el  al.,  2002, 
2004;  Weninger  and  Mohun,  2002)  (Fig.  34.1).  The  stack  of 
images  is  then  reconstructed  into  a  high-resolution  image  volume. 

Imaging  the  block  face  rather  than  the  ribbon  has  significant 
advantages.  The  mass  of  embedding  medium  supports  the  speci¬ 
men  and  minimizes  sectioning  artifacts  such  as  compression, 
folding,  and  tearing.  Perfectly-registered  data  volumes  are  easily 
reconstructed  as  the  block  face  is  photographed  at  the  same  posi¬ 
tion  in  the  microtome’s  sectioning  cycle. 

SIM/EFIC  is  a  fluorescence  imaging  modality  that  relies  on 
endogenous  autofluorescence  or  exogenous  dye  labeling  of  the 
specimen.  To  prevent  out-of-focus  fluorescence  from  compromis¬ 
ing  image  quality,  specimens  are  embedded  in  highly  light 
absorbent  black  paraffin  or  dark  embedding  resins;  the  opacity  of 
the  embedding  medium  can  be  adjusted  to  match  the  in-plane  res¬ 
olution  of  the  objective  being  used  and  the  section  thickness,  pro¬ 
ducing  isotropic  image  volumes  (Fig.  34.2).  SIM  can  produce 
stunning  3D  image  volumes  with  isotropic  voxel  resolutions  of 
-0.5  pm. 

At  such  high  resolution,  individual  cell  morphology  can  be 
examined  within  the  context  of  tissues  and  entire  organisms.  SIM 
has  been  used  in  morphogentic  studies  to  reveal  the  relationship 
between  cell  morphology  and  overall  tissue  shape  and  to  follow 
the  changes  that  occur  during  embryonic  development  (Ewald  et 
al.,  2002).  SIM  has  also  been  used  in  the  materials  sciences  to 
understand  how  microscopic  structure  relates  to  the  strength,  feel, 
texture,  and  other  properties  of  textiles  and  papers. 

SIM,  however,  is  a  destructive  imaging  modality  that  allows 
no  other  data  collection  from  a  given  specimen.  In  combination 
with  the  non-destructive  methods  discussed  below,  SIM  can  be 
used  to  add  a  content-rich  description  of  a  single  specimen. 


J.  Michael  Tyszka,  Seth  W.  Ruffins,  and  Scott  E.  Fraser  •  California  Institute  of  Technology,  Pasadena,  California  91 125 

Jamey  P.  Weichert  •  University  of  Wisconsin,  Madison,  Wisconsin  53706 

Michael  J.  Paulus  •  ORNL  and  CTI-Concorde  Microsystems,  Knoxville,  Tennessee  37932 


Handbook  of  Biological  Confocal  Microscopy,  Third  Edition,  edited  by  James  B.  Pawley,  Springer  Science+Business  Media,  LLC,  New  York,  2006. 


607 


FIGURE  34.1.  Surface  imaging  microscopy/episcopic  fluorescence 
image  capture  setup.  SIM/EFIC  is  a  block-face  imaging  modality.  Essen¬ 
tially,  a  microtome  is  outfitted  with  a  microscope  imaging  system.  A 
sample  is  embedded  in  an  opaque  medium  and  sectioned  using  a  diamond 
knife.  A  horizontally-mounted  microscope  captures  an  image  of  the  block 
face  as  the  chuck  returns  to  a  specific  position  after  each  section  is  cut. 
The  resulting  image  series  is  reconstructed  into  a  perfectly-registered 
image  volume  with  high  resolution. 


FIGURE  34.2.  SIM  of  12.5 dpc  mouse  embryo  stained  with  fluorescent  analog  of  H&E  staining.  The  image  volume  was  collected  with  a  2x  objective  and  pro¬ 
vides  approximately  8jxm/pixel  resolution.  This  data  volume  is  1517  x  1016  x  664  pixels.  (A)  Whole  embryo  view  shows  external  structures  with  high  fidelity. 
(B)  A  slab  section  -1700  (im  thick.  (C)  A  single  section  through  the  embryo,  and  (D)  the  high  magnification  view.  SIM  provides  a  global  view  of  the  specimen 
at  near  cellular  resolution. 
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OPTICAL  COHERENCE  TOMOGRAPHY 

Optical  coherence  tomography  (Huang  et  al.,  1991;  Tearney  et  al., 
1997;  Masters,  1999)  is  well  suited  to  examining  tissue  geometry 
and  organization,  even  in  living,  unstained  tissues  (Boppart  et  al., 
1996, 1997, 1998;  Yazdanfar  etal.,  1997;  Rollins  etal.,  1998).  OCT 
is  based  upon  the  reflection/backscattering  of  light  and  is  not  unlike 
sonar,  detecting  the  amount  of  light  reflected  from  different  depths 
(up  to  a  few  millimeters)  in  the  specimen  to  build  up  a  voxel-by¬ 
voxel  image.  The  challenge  of  OCT  is  to  optically  section  the  huge 
amount  of  backscattered  light  to  create  an  image  of  a  single  plane 
at  depths  several-fold  greater  than  the  mean  free  path  of  light  in  the 
specimen.  To  meet  this  challenge,  OCT  uses  interference  between 
the  light  reflected  by  the  specimen  and  the  light  reflected  by  a  ref¬ 
erence  mirror  to  isolate  individual  optical  sections. 

Although  its  ability  to  image  deep  in  a  specimen  far  outper¬ 
forms  confocal  microscopy,  an  OCT  shares  many  features  with  the 
optics  of  a  reflected  light  confocal  microscope.  An  OCT  micro¬ 
scope  images  the  output  of  an  optical  fiber  through  an  objective 
lens,  and  re-images  the  backscattered  signal  from  specimen  into 
the  same  fiber,  creating  a  confocal  aperture;  the  difference  is  that 
an  OCT  uses  a  short  coherence  length  light  source  and  an  inter¬ 
ferometer  to  add  a  “coherence  gate”  that  suppresses  the  signal  from 
out-of-focus  planes  by  6  orders  of  magnitude  (Rollins  and  Izatt, 
1999).  The  Michelson  interferometer  of  a  typical  OCT  is  created 
from  optical  fibers;  the  beam-splitter  is  created  by  fusing  the  two 
optical  fibers  into  a  2  x  2  coupler.  The  reference  arm  fiber  is  ter¬ 
minated  with  a  collimating  lens  and  retro-reflector;  the  specimen 
arm  is  terminated  by  a  collimating  lens,  beam-steering  galvonome- 
ters,  and  an  imaging  lens  such  as  a  microscope  objective.  Both  the 
length  of  the  two  arms  and  the  materials  must  be  closely  matched 
to  avoid  unequal  group  velocity  dispersion  in  the  specimen  and 
reference  arms.  The  creation  of  interference  fringes  involves  mod¬ 
ulating  the  length  of  the  arms  by  means  ranging  from  stretching 
the  optical  fibers  with  piezoelectric  devices  to  a  grating-based 
phase-delay  retro-reflector. 


In  an  OCT,  the  signal  is  derived  from  interference  fringes,  as 
light  back-reflected  from  the  reference  mirror  and  the  specimen 
interferes  coherently.  Fringes  will  only  result  if  the  optical  path 
lengths  of  the  reference  and  specimen  arms  are  equal  to  within  half 
of  the  coherence  length  of  the  light  source  (the  “coherence  gate”). 
Signal  from  multiply-scattered  photons  are  rejected  by  the  con¬ 
certed  action  of  the  confocal  aperture  and  the  coherence  gate.  The 
frequency  of  the  interference  fringes  is  set  by  the  motion  of  the 
reference  reflector  (or  interferometer  arms),  giving  the  detection 
electronics  a  defined  lock-in  frequency  to  detect,  and  thereby 
rejecting  the  signal  from  light  that  is  backscattered  (but  not  inter¬ 
fering)  from  above  and  below  the  focal  plane.  Thus,  resolution  of 
an  OCT  microscope  is  set  by  two  factors:  the  coherence  length  of 
the  light  source  (for  depth)  and  the  size  of  the  illuminated  spot  (for 
lateral  resolution). 

A  typical  OCT  generates  a  scan  perpendicular  to  the  surface, 
much  like  the  .vz-scans  generated  in  confocal  microscopy,  creating 
a  slice  through  the  depth  of  the  specimen  (Fig.  34.3).  A  galvonome- 
ter  mirror  translates  the  beam  in  one  axis;  at  each  spot  along  this 
line,  a  rapid  change  in  the  length  of  the  reference  arm  (changing 
the  length  or  the  retro-reflector  position)  scans  the  coherence 
volume  by  millimeters.  Such  a  rapid  depth  scan  with  a  slower 
lateral  scan  creates  an  image  reminiscent  of  that  created  by  ultra¬ 
sonography.  Volumetric  (3D)  images  can  be  assembled  by  col¬ 
lecting  a  series  of  neighboring  depth  scans.  OCT  scans  can  be 
generated  quickly,  with  depth  discrimination  set  by  the  coherence 
length  of  the  light  source;  however,  the  lateral  resolution  of  the 
images  is  typically  compromised  because  the  coherence  volume 
must  move  very  rapidly  in  depth,  much  more  rapidly  than  it  is  pos¬ 
sible  to  refocus  the  light  in  the  specimen.  As  a  result,  the  lens 
focusing  the  light  into  the  specimen  must  employ  a  relatively 
shallow  convergence  angle  so  that  the  spot  size  does  not  change 
too  dramatically  over  the  ±2  mm  motion  of  the  coherence  volume 
in  depth.  Thus,  B-mode  instruments  often  achieve  resolutions  of 
less  than  10  pm  in  depth,  but  greater  than  10  pm  in  lateral  extent 
(see  Izatt  et  al.,  1996).  Given  the  laminar  organization  of  many 


FIGURE  34.3.  In  vivo  ultrahigh  resolution  OCT  B  mode  ( xz )  section  through  a  human  retina,  viewed  through  the  cornea.  A  depth  resolution  of  less  than  5 
microns  was  achieved  using  a  very  broad  bandwidth  light  source.  Transverse  resolution  was  limited  by  the  numerical  aperture  of  the  lens  in  the  human  eye  and 
by  imperfections  in  its  optical  characteristics.  Abbreviations:  NFL  =  Nerve  Fiber  Layer,  GCL  =  Ganglion  Cell  Layer,  IPL  =  Inner  Plexiform  Layer,  INL  =  Inner 
Nuclear  Layer,  OPL  =  Outer  Plexiform  Layer,  ONL  =  Outer  Nuclear  Layer,  ELM  =  External  Limiting  Membrane,  IS/OS  =  Photoreceptor  Inner/Outer  Segment 
Junction,  RPE  =  Retinal  Pigment  Epithelium,  CC  =  Choriocapillaris.  (Reproduced  with  permission  from  Wojtkowski,  2004  ©  Optical  Society  of  America.) 
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FIGURE  34.4.  Schematic  of  an  en  face  OCT  system.  The 
superluminescent  diode  (SLD),  which  provides  the  broad  band¬ 
width  light  for  OCT  imaging,  is  combined  with  an  alignment 
laser  diode  at  635  nm  using  a  2  X  1  coupler.  The  2x2  coupler 
splits  the  light  between  the  sample  arm  with  the  focusing  lens 
and  the  reference  arm  with  the  motorized  retro-reflector.  The 
reflected  light  from  both  arms  interferes  and  is  measured  by  the 
photodiode  at  the  output  of  the  2x2  coupler.  Electronic  demod¬ 
ulation  of  the  photodiode  signal  produces  an  OCT  signal  for  a 
given  depth  position  in  the  sample.  Transverse  scanning  and 
subsequent  depth  scanning  produce  a  2D  en  face  image  and  a 
3D  image,  respectively.  (Reproduced  with  permission  from 
Haskell  et  al,  2004  ©  IEEE.) 


specimens,  such  as  the  layered  organization  of  the  retina,  striking 
images  can  result. 

En  face  OCT  is  an  alternative  approach  for  OCT  imaging  that 
can  optimize  lateral  resolution  by  using  a  higher  numerical- 
aperture  (NA)  imaging  lens  (Hoeling  et  al.,  2000,  2001).  A  smaller 
spot  size,  more  similar  to  that  used  in  confocal  microscopy  is 
created,  and  galvomometer  mirrors  are  used  to  scan  the  beam 
through  a  complete  optical  section  (Fig.  34.4).  The  focal  depth  and 
coherence  depth  must  be  aligned  to  the  same  optical  section  by  the 


FIGURE  34.5.  OCT  section  of  a  paraformaldehyde-fixed  stage  41  Xenopus 
laevis  embryo  viewed  along  the  spinal  cord  from  the  tail  towards  the  head  with 
dorsal  at  the  top  of  the  figure.  The  beam  is  incident  from  the  left  side  of  the 
embryo  and  the  colorscale  encodes  backscatter  intensity  from  blue  (low )  to  red 
(high).  (Reproduced  with  permission  from  Hoeling  et  al..  2000.  ©  Optical 
Society  of  America.) 


positioning  of  the  retro-reflector.  Interference  fringes  are  generated 
by  changing  the  length  of  the  reference  arm,  typically  by  oscillat¬ 
ing  a  small  mirror  or  by  using  an  acousto-optical  device,  by  an 
amount  far  less  than  the  coherence  length.  Thus,  the  depth  resolu¬ 
tion  is  set  at  roughly  half  the  coherence  length  of  the  light  source. 
Volumetric  images  are  built  up  from  collecting  a  series  of  2D  (en 
face )  scans,  adjusting  the  focus  (and  coherence  depth)  after  each 
scan  to  build  up  a  z-series  of  images  (Fig.  34.5).  Given  the  per¬ 
formance  of  inexpensive  wide-bandwidth  light  sources,  and  the 
index  of  refraction  of  tissue  (>1.3),  en  face  OCT  microscopy  can 
generate  images  with  isotropic  resolution  of  ~5  pm. 


OPTICAL  PROJECTION  TOMOGRAPHY 

Optical  projection  tomography  is  a  projection  tomographic  method 
akin  to  X-ray  computed  axial  tomography  (commonly  called  CT 
or  CAT,  see  micro-CT  below)  and  electron  microscopy  tomogra¬ 
phy  (EMT).  Like  other  projection  tomogram  methods,  OPT  uses 
2D  projections  of  3D  objects  to  reconstruct  the  volume  of  an  object 
using  a  Radon-backplane  reconstruction  algorithm  (for  review,  see 
Herman,  1980).  Each  point  in  the  projected  image  is  the  result  of 
the  cumulative  absorption  or  emission  of  light  rays  propagating 
through  an  object.  OPT  operates  in  both  blight-field  (transmission) 
and  fluorescence  (emission)  modes  and  is  proving  useful  in  cap¬ 
turing  patterns  of  gene  expression  and  antibody  staining  in  situ. 
Image  volumes  reconstructed  from  projection  tomogram  data 
differs  from  the  optical  and  physical  sectioning  methods  described 
above  in  that  the  location  of  features  within  an  object  are  not 
explicitly  sampled.  Instead,  the  location  of  features  is  computed 
from  a  series  of  2D  projections  collected  from  many  axial  posi¬ 
tions  around  the  object. 

An  OPT  instrument  is  a  relatively  simple  device  consisting  of 
a  sample  chamber  with  a  rotating  sample  platform,  imaging  optics 
for  capturing  projections,  and  a  diffuse  light  source  for  bright-field 
mode  (Fig.  34.6).  In  fluorescence  mode,  the  imaging  optics  are 
used  for  widefield  illumination  of  the  specimen.  OPT  requires  that 
the  specimen  be  transparent  and  indexed-matched  with  the  sur¬ 
rounding  media  to  permit  light  to  pass  through  the  object  with 
minimal  refraction.  Currently  this  is  achieved  by  clearing  and 
imaging  specimens  in  a  mixture  of  benzyl  alcohol  and  benzyl  ben¬ 
zoate  (BABB).  Typically,  a  sample  is  suspended  in  agarose  and 
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FIGURE  34.6.  Optical  projection  tomography  setup  (bright-held  mode).  (A)  Cleared  specimen  is  suspended  in  agarose  gel,  mounted  to  a  rotating  platform,  and 
placed  in  chamber  containing  indexed-matched  media.  Imaging  optics  (B)  are  used  to  focus  the  projected  image  and  reject  light  scattered  by  the  specimen.  The 
projected  image  (C)  is  collected  with  a  CCD  camera.  Multiple  projections  are  used  to  reconstruct  an  image  volume. 


serially  dehydrated  in  methanol  and  then  cleared  in  BABB.  The 
sample  in  the  agarose  block  is  placed  onto  the  rotating  stage  in  a 
sample  chamber  containing  BABB  (Sharpe  et  al.,  2002). 

X-ray  computed  axial  tomography  (CAT  or  CT)  and  electron 
microscopy  tomography  (EMT)  assume  that  the  specimen  is  trans¬ 
parent  to  the  probing  rays  and  that  refraction  is  non-existent.  The 
paths  of  the  projected  rays  are  straight  and  produce  a  perfect 
“shadow  projection.”  Because,  in  fact,  the  probing  light  rays  are 
susceptible  to  scatter  and  some  degree  of  diffraction  and  refraction 
as  they  pass  through  the  specimen,  the  resulting  projection  is  “con¬ 
fused”  and  blurred  as  it  is  composed  of  rays  that  have  followed  the 
predicted  path  through  the  subject  as  well  as  stray  rays.  The 
imaging  optics  focus  the  projection  and  collimate  the  rays  that  are 
projected  along  an  approximately  straight  path  through  the  sample 
(Fig.  34.7). 

The  limited  depth  of  focus  inherent  in  all  optical  microscope 
systems  requires  that  modifications  be  made  to  the  collection  and 
projection  reconstruction  (Sharpe,  2004).  Projections  captured  by 
OPT  represent  only  the  volume  residing  within  the  region  of  the 
depth  of  focus.  During  imaging,  the  center  of  focus  is  set  to  a  plane 
approximately  half  the  distance  between  the  axis  of  rotation  and 
the  near  side  of  the  specimen.  Because  each  projection  only  rep¬ 
resents  about  half  the  specimen  volume,  a  complete  projection  set 
requires  the  specimen  be  rotated  through  360°  at  0.9°  intervals. 
Volume  reconstruction  is  performed  using  a  modified  back-plane 
projection  algorithm  to  compensate  for  the  limited  projected 
volume. 

For  small  objects  (on  the  order  of  5  mm  diameter)  light  scatter 
is  not  severe  and  internal  details  can  be  readily  resolved.  The 


-  rays  contributing  to  projection 

-  scattered  rays 

FIGURE  34.7.  Due  to  the  limited  focal  depth  of  the  imaging  optics,  the  focus 
is  set  to  the  region  between  the  axis  of  rotation  and  the  imaging  optics.  Each 
projection  represents  only  half  the  specimen  volume  requiring  that  the  subject 
be  rotated  through  360°  to  collect  a  full  projection  set. 
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FIGURE  34.8.  Autofluorescent  OPT  reconstruction  of  17-day  lamprey  larva.  The  larva  was  imaged  using  a  Texas  Red  filter  set  and  mounted  with  the 
anterior-posterior  axis  (long  axis)  along  the  axis  of  rotation.  The  light  path  is  between  1.5  to  3  mm  (left-right,  dorsal-ventral  axis,  respectively).  The  larva  is 
virtually  opaque  in  visible  light  yet  internal  structures  are  easily  identifiable  using  near  IR  autofluorescence. 


quality  of  the  projections  diminishes  as  the  objects  imaged  become 
larger  because  of  the  increased  light  scattering.  The  result  is  that 
fewer  internal  details  are  resolved,  and  opaque  regions,  such  as 
highly  pigmented  tissues,  similarly  degrade  the  projections  (Fig. 
34.5). 

The  time  required  to  collect  a  complete  projection  set  (400 
images)  ranges  between  3  (for  bright-held  and  strong  fluorescent 
signals)  and  15  min,  depending  on  signal  strength.  Current  recon¬ 
struction  performance  is  about  6  min  for  a  5123  voxel  image 
volume  on  a  standard  4  GHz  Pentium  4  class  workstation. 

OPT  has  only  recently  become  a  practical  device  for  routine 
3D  imaging,  but  it  is  proving  to  be  useful  as  an  analytical  tool  for 
developmental  biology  as  it  is  capable  of  imaging  anatomy,  gene 
expression  patterns  and  the  distribution  of  specific  cell  popula¬ 
tions,  extracellular  matrix  components,  etc.,  as  revealed  by  anti¬ 
body  labeling.  One  of  its  first  uses  was  to  collect  anatomical  data 
of  early  human  and  mouse  development  for  the  creation  of  volu¬ 
metric  atlases  for  the  analysis  of  aberrant  embryogenesis  (Sharpe, 
2003;  Kerwin  et  al.,  2004;  Robson  et  al.,  2004). 

The  anatomy  of  unlabeled  embryos  can  be  collected  in  either 
bright-held  and/or  fluorescent  modes.  Each  modality  reveals  dif¬ 
ferent  features  depending  on  the  wavelength  of  the  probing  light. 
In  the  bright-held  mode,  tissue  transparency  increases  as  the  illu¬ 
mination  moves  into  the  near-infrared  (IR),  allowing  deeper 
imaging  of  larger  embryos.  Structures  that  are  opaque  in  visible 
light,  such  as  the  pigmented  retina,  can  be  imaged  using  transmit¬ 
ted  light  at  these  longer  wavelengths.  In  the  fluorescence  mode, 
the  vasculature  is  revealed  as  blood  fluoresces  brightly  when  Texas 
Red  Alter  sets  are  used  (562  nm  excitation  to  624 nm  emission). 
The  autofluorescence  of  the  lamprey  larva  has  been  used  to  reveal 
internal  structures  without  physical  sectioning  (Fig.  34.8). 
However,  the  information  content  within  an  embryo  is  not  limited 
to  its  intrinsic  optical  properties.  Cell-  and  tissue-specific  chemi¬ 
cal  composition  can  be  made  visible  using  labeled  antibodies  and 
classic  histological  stains  while  RNA  hybridization  and  green  flu¬ 
orescent  protein  (GFP)  reporter  constructs  show  gene  activity. 
Many  of  these  protocols  are  compatible  with  OPT  imaging. 


Bright-held  OPT  has  successfully  been  used  to  image  the  intri¬ 
cate  spatial  arrangement  of  the  developing  skeleton  using  the 
common  histological  stain  Alcian  Blue  (Fig.  34.9).  In  situ  RNA 


FIGURE  34.9.  Bright-field  OPT  reconstruction  of  Alcian  Blue  stained  12.5 
dpc  mouse  embryo.  Non-ossified  skeletal  elements  stained  with  Alcian  Blue 
reveal  the  intricate  and  delicate  structure  of  the  fetal  mouse.  Note  that  dermal 
bone  has  not  yet  formed. 
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hybridization  reveals  the  global  pattern  of  specific  gene  activity 
using  a  color  precipitate  indicator.  Standard  whole-mount  photo¬ 
graphy  reveals  regions  of  gene  activity  but  sections  are  needed  to 
define  the  precise  location  of  expressing  tissues  and  cells.  Once 
sectioned,  the  3D  context  is  lost.  Additionally,  weak  signals  may 
not  be  detected  in  thin  sections.  OPT  is  a  powerful  tool  in  that  a 
single  imaging  experiment  places  local  and  specific  activity  in  the 
context  of  the  entire  embryo.  Potentially,  OPT  may  be  able  to 
capture  weak  signals  as  projections  that  are  cumulative  through  the 
sample  volume  (up  to  2  mm  compared  with  10-15  pm  for  histo¬ 
logical  sections). 

OPT  requires  that  the  light  rays  be  able  to  pass  through  the 
object  with  minimal  scatter.  This  limits  OPT  use  to  transparent 
specimens  with  a  homogeneous  refractive  index  (RI).  Thus  far, 
OPT  has  primarily  been  used  for  fixed  specimens  but  there  is  a 
potential  for  imaging  live  specimens,  such  as  tissue  explants 
and  early  embryogenesis  of  aquatic  organisms.  This  method  could 
be  used  for  time-lapse  studies  with  the  shortest  interval  being 
the  time  required  to  acquire  a  full  set  of  projections.  Currently, 
the  minimum  time  interval  for  collecting  400  projections  is 
about  3  min  but  as  there  is  a  trade-off  between  spatial  resolu¬ 
tion  and  number  of  projections,  one  can  acquire  a  smaller  set 
of  images  more  rapidly  if  one  is  willing  to  settle  for  less 
resolution. 


LIGHT  SHEET  MICROSCOPY 

“Light  sheet”  fluorescence  microscopy,  originally  developed  by 
Voie  (1995,  2002),  is  an  optical-sectioning  technique  whereby  only 
the  plane  being  imaged  is  illuminated.  The  concept  is  not  unlike 
that  of  multi-photon  microscopy  in  that  only  the  fluorophores 
in  the  focus  plane  are  excited,  thus  minimizing  photodamage. 
Rather  than  using  the  imaging  optics  to  illuminate  the  specimen, 
as  in  epi-fluorescence  microscopes,  light-sheet  microscopy  uses 
separate  illumination  optics  to  produce  an  illuminated  layer  per¬ 
pendicular  to  the  imaging  axis.  Optical  sectioning  is  achieved 
by  moving  the  light  sheet  through  the  specimen  coincident  with 
the  focus  plane  of  the  imaging  optics  (also  see  Chapter  37,  this 
volume). 


Optical  Setup 

Selective  plane  illumination  microscopy  (SPIM)  is  the  name  given 
to  an  updated  version  of  light-sheet  microscopy  developed  by  the 
Stelzer  group  at  the  EMBL/Heidelberg.  In  this  system,  the  micro¬ 
scope  optics  are  arranged  horizontally  to  accommodate  a  cylindri¬ 
cal  sample  immersed  in  a  fluid-filled  sample  chamber  and  capable 
of  being  rotated  around  its  vertical  axis.  This  arrangement  provides 
an  ideal  environment  for  imaging  living  specimens  as  it  permits 
the  media  to  be  replaced  and  additional  agents,  such  as  nutrients, 
growth  factors,  gases,  etc.,  to  be  introduced  during  imaging. 
Samples  are  typically  suspended  in  soft  agarose  and  placed  into 
the  sample  chamber  containing  liquid  medium.  The  sample  is 
imaged  using  a  water  dipping  lens  inserted  through  one  wall  of 
the  sample  chamber.  Windows  on  the  lateral  walls  (perpendicular 
to  the  imaging  axis)  allow  the  light  sheet  to  illuminate  the  speci¬ 
men.  The  light  sheet  is  formed  by  passing  collimated  laser  light 
through  a  cylindrical  lens  and  controls  are  provided  to  adjust 
the  thickness  and  convergence  of  this  sheet  (Huisken  et  al.,  2004) 
(Fig.  34.10). 


det.  ill. 


FIGURE  34.10.  Selective  plane  illumination  microscope  (SPIM)  setup.  The 
specimen  is  suspended  in  agarose  in  a  medium-filled  sample  chamber. 
The  SPIM  device  optically  sections  the  sample  as  it  is  illuminated  by  a  “light 
sheet”  perpendicular  to  the  imaging  axis  and  coincident  with  the  focus  plane 
of  the  imaging  optics.  The  sample  can  be  rotated  to  collect  multiple  views  for 
the  reconstruction  of  subjects  that  cannot  be  imaged  through  their  entire 
volume. 


The  lateral  (xy)  resolution  of  SPIM  is  limited  by  the  light  scat¬ 
tering  and  absorption  properties  of  the  specimen  being  imaged 
while  the  axial  (z)  resolution  is  dependent  on  the  thickness  of  the 
light  sheet.  The  lateral  penetration  and  scatter  of  the  light  sheet 
will  affect  the  thickness  and  intensity  of  the  illuminated  layer 
across  the  field  of  view. 

SPIM  is  an  excellent  imaging  modality  for  transparent  speci¬ 
mens  such  as  Medaka  and  zebrafish  embryos.  Developing  larval 
Medaka  fish  have  been  imaged  to  a  depth  of  500  pm  with  an  axial 
resolution  of  6  pm.  The  imaging  depth,  however,  is  not  great 
enough  to  encompass  the  entire  embryo  and  images  of  deeper 
structures  are  of  a  lower  quality  than  those  of  features  near  the 
surface.  A  solution  to  this  problem  has  been  to  collect  multiple 
image  stacks  from  different  views  of  the  specimen.  Each  view 
produces  a  set  of  data  that  includes  information  not  available  in 
other  views.  Combining  the  multiple  views  into  a  single  image 
volume  produces  a  reconstruction  of  the  entire  specimen.  Entire 
volumes  nearly  2  mm  thick  can  be  reconstructed  at  a  voxel  size  of 
~5  pm. 

SPIM  is  also  useful  as  a  tool  for  capturing  dynamic  events  on 
time  scales  ranging  from  fractions  of  a  second  to  days.  The  use  of 
a  narrow  light  sheet  plane  allows  widefield  optics  to  be  used,  max¬ 
imizing  the  emitted  light  collection  without  compromising  axial 
resolution.  Maximizing  light  collection  minimizes  the  excitation 
time  required  for  each  image.  Short  exposure  times  minimize 
photodamage  and  permit  long-term  imaging  experiments.  The  use 
of  high-speed  cameras  allows  SPIM  to  be  used  for  observing 
dynamic  events  such  as  the  beating  heart  of  a  Medaka  embryo 
(Fig.  34.11). 
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FIGURE  34.1 1 .  GFP  expression  in  the  developing  heart  of  Medaka  imaged  by  SPIM.  Optical  sectioning  of  Medaka  heart  tube  can  be  performed  because  flu¬ 
orescent  signal  is  only  emitted  from  the  plane  illuminated  by  of  the  light  sheet.  This  series  of  images  demonstrates  the  axial  resolving  capabilities  of  SPIM.  Each 
image  is  separated  by  30|im  along  the  imaging  axis.  The  three-dimensional  reconstruction  is  very  good  with  no  out-of-focus  data  confounding  the  image. 


MICRO-COMPUTERIZED 
TOMOGRAPHY  IMAGING 

The  recent  introduction  of  commercial  ultra-high-resolution  labo¬ 
ratory  micro-computerized  tomography  (micro-CT)  scanners  has 
revolutionized  our  ability  to  non-invasively  examine  tissues  and 
organs  in  living  experimental  animal  models,  including  mice,  at 
spatial  resolutions  of  less  than  20  microns.  In  a  3D  sense,  this 
makes  it  possible  to  image  a  1  pL  volume  in  vivo.  Several  excel¬ 
lent  reviews  have  recently  appeared  that  discuss  in  detail  the 
physical  differences  as  well  as  the  advantages  and  limitations 
associated  with  each  type  of  small  animal  imaging  system,  includ¬ 
ing  micro-CT  (Paulus  et  al.,  2000,  2001;  Holdsworth  eta/.,  2002). 

Operating  Principle 

The  general  principles  of  micro-CT  are  similar  to  those  used  in 
clinical  CT  scanners.  Most  clinical  systems  consist  of  a  high-power 


X-ray  point  source,  between  1  and  64  rows  of  detector  elements, 
and  a  mechanical  gantry  that  rotates  around  the  patient  over  a 
period  of  minutes  [Fig.  34.12(A)].  X  rays  emitted  by  the  source 
are  collimated  into  a  fan-shaped  beam,  oriented  perpendicular  to 
the  rotation  axis  passing  through  the  patient.  Most  of  the  X  rays 
not  absorbed  by  the  patient  are  detected  by  the  detector  rows. 

Contrast  and  Dose 

The  relationship  between  the  X-ray  flux  emitted  by  the  source,  cp0 
and  the  detected  flux,  (p,  is  given  by  the  expression 

<l>  =  <l>0exp|-Jpx<&j 

where  pr  is  the  spatially-varying  X-ray  attenuation  coefficient  of 
the  tissue  and  l  is  the  path  of  the  X  rays.  This  expression  may  be 
solved  to  determine  the  line  integral  of  the  attenuation  coefficient 
to  yield 


A 


B 


FIGURE  34.1 2.  (A)  Layout  of  a  CT  scanner  showing  orientation  of  body,  X-ray  source,  and  detector  for  a  hospital  scanner  (A)  that  collimates  the  X  rays  into 
a  fan-shaped  beam  with  the  long  axis  parallel  to  the  rotation  axis  and  (B)  a  micro-CT  that  uses  a  cone-shaped  X-ray  beam  and  a  large  2D  X-ray  detector  that  is 
slightly  larger  than  the  mouse  and  has  an  effective  pixel  size  of  25  to  40  pm.  The  detector  counts  about  50%  of  the  X-ray  photons  striking  it  (depending  on  kilo¬ 
volts)  and  X  rays  passing  through  the  body  of  the  mouse  are  usually  digitized  to  10-bit  accuracy  (12-  to  14-bits  overall).  The  distance  from  the  source  to  the 
rotation  axis  is  variable  from  about  5  to  30  cm;  shorter  spacing  gives  shorter  exposure  times  but  lower  MTF  and  more  complex  computations. 
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Tomographic  image  reconstruction  is  the  process  of  using  a 
number  of  these  integral  expressions  to  calculate  values  for  m  in 
a  2D  slice  or  3D  volume.  The  calculated  values  for  the  attenuation 
produced  by  each  voxel  are  typically  normalized  to  the  attenuation 
coefficient  of  water  and  scaled  following  the  expression 

ft 

CT  Number  =  2  x  1000. 

M-  H20 

Although  the  scaled  attenuation  coefficient  is  actually  unitless, 
the  reported  values  are  typically  cited  in  pseudo-units  of  CT 
Numbers  or  Hounsfield  Units.  Of  course,  like  any  other  imaging 
system  that  relies  on  the  detection  of  photons,  micro-CT  is  limited 
by  the  Poisson  noise  associated  with  actually  detecting  the  signal. 
However,  simple  calculations  relating  visibility  to  contrast  and 
detected  signal  levels  (such  as  those  in  Chapters  4  and  8,  this 
volume)  are  complicated  by  the  fact  that  the  imaging  algorithms 
extract  3D  information  from  an  entire  series  of  between  90  and 
360  2D  projection  images. 

Micro-CT  systems  typically  have  poorer  contrast  resolution 
than  clinical  systems.  The  combination  of  small  detector  elements 
and  low  X-ray  flux  leads  to  signal-to-noise  ratios  in  micro-CT 
systems  that  are  typically  an  order  of  magnitude  lower  than 
those  of  their  clinical  counterparts.  Signal-to-noise  ratios  can  be 
improved  by  either  increasing  the  detector  element  size  (at  the 
expense  of  resolution)  or  by  increasing  the  number  of  X  rays  col¬ 
lected  (at  the  expense  of  scan  time  and  the  dose  to  the  animal). 

As  in  other  imaging  modalities,  the  ability  to  discern  structure 
depends  on  the  contrast  of  the  structure.  Generally  speaking,  con¬ 
trast  in  the  raw  data  from  the  2D  X-ray  detector  goes  down  as  the 
overall  thickness  of  the  object  increases,  and  goes  up  as  the  atten¬ 
uation  of  a  feature  varies  from  that  of  its  surroundings  (as  mea¬ 
sured  in  Hounsfield  units).  In  the  micro-CT,  the  kilovolts  applied 
to  the  X-ray  source  changes  both  the  peak  and  the  average  energy 


of  the  X  rays  emitted:  higher  energy  X  rays  are  more  likely  to  pen¬ 
etrate  through  the  mouse  but  usually  show  less  differential  atten¬ 
uation  (i.e.,  raw  contrast).  In  theory,  information  about  features 
with  low  contrast  (soft  tissue),  will  be  maximized  if  about  60%  of 
the  X  rays  are  absorbed  in  the  mouse  and  the  remaining  40% 
detected  by  the  detector.  In  practice,  much  mouse  work  is  done 
with  the  X  ray  source  operating  at  ~80kV  (range,  40-100 kV),  with 
the  beam  filtered  through  about  0.5  mm  of  A1  to  remove  very  low 
energy  photons  that  can’t  penetrate  the  mouse  but  can  cause 
damage. 

In  general,  micro-CT  scanners  only  have  sufficient  contrast 
resolution  to  differentiate  between  different  soft  tissue  types  after 
radio-opaque  contrast  agents  have  been  added.  Figure  34.13  illus¬ 
trates  the  use  of  these  compounds  in  micro-CT  animal  studies. 

Computed  Tomography  Scanning  Systems 

Even  the  largest  multi-detector  row  (multi-slice)  clinical  X-ray  CT 
systems  image  only  a  slice  of  tissue  a  few  centimeters  thick  in  a 
single  rotation.  To  image  a  larger  volume,  the  X-ray  source  and 
detector  rotate  through  multiple  orbits  as  the  patient  bed  moves 
linearly  through  the  gantry.  This  process  of  acquiring  volumetric 
image  data  by  translating  the  patient  through  the  imaging  path  of 
a  continuously  rotating  X-ray  source  and  detector  is  called  spiral 
CT,  and  is  now  the  dominant  clinical  method  for  acquiring  volu¬ 
metric  CT  data. 

In  contrast,  most  preclinical  micro-CT  systems  [Fig.  34.12(B)] 
employ  a  large  area  detector,  sized  to  detect  X-rays  illuminating 
much  of  the  test  subject  in  a  single  exposure.  The  X-ray  source  is 
uncollimated,  generating  a  cone  beam  rather  than  a  fan  beam  as  in 
a  clinical  system.  X-ray  data  is  typically  collected  as  a  series  of  90 
to  720  discrete  exposures  (i.e.,  step-and-shoot  acquisition),  and  the 
entire  image  volume  is  often  acquired  in  a  single  orbit  acquisition 
rather  than  in  a  multi-orbit  spiral  acquisition.  The  bed  does  not 
move  during  the  acquisition. 

The  motivation  for  using  a  cone  beam  configuration  for  high- 
resolution  micro-CT  studies  stems  from  the  requirement  for  a  very 


FIGURE  34.1 3.  Micro-CT  images  of  mice  with  (A)  no  contrast  media,  (B)  intraperitoneally  injected  water-soluble  contrast  medium,  (C)  intraperitoneally  injected 
water-soluble  and  oral  contrast  medium,  and  (D)  blood  pool  and  hepatophillic  contrast  media.  (Images  courtesy  of  CTI/Concorde  Microsystems,  LLC,  Knoxville, 
TN.) 
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small  (10-15  pm  focal  spot)  microfocus  X-ray  source.  The  small 
focal  spot  is  required  to  achieve  the  <50  pm  resolution  required 
for  most  laboratory  animal  imaging  studies.  Microfocus  X-ray 
sources  operate  at  very  low  power,  typically  less  than  100W,  and 
consequently  generate  several  orders  of  magnitude  lower  X-ray 
flux  intensities  than  their  clinical  counterparts.  In  order  to  make 
efficient  use  of  the  available  X  rays,  the  clinical  fan  beam  is 
expanded  into  a  cone  beam,  leading  to  the  scanner  geometry 
described  above. 

The  2D  micro-CT  detector  is  typically  a  large  format  CCD 
camera  which  collects  images  appearing  on  a  high-resolution 
phosphor  screen.  The  phosphor  screen  images  are  transferred  to 
the  CCD  by  either  a  conventional  lens  or  a  fiber-optic  array. 

Micro-CT  scanners  are  intrinsically  high-resolution  devices. 
Available  X-ray  sources  can  have  a  focal  spot  size  of  <10pm  and 
detector  pixel  sizes  range  from  10  to  40  pm.  With  these  compo¬ 
nents,  volumetric  image  resolutions  on  the  order  of  10  pm  are 
readily  achievable  but  only  at  the  cost  of  either  being  restricted 
to  imaging  high-contrast  structures,  such  as  bone,  or  exposing 
the  mouse  to  dangerous  levels  of  X-ray  radiation.  A  typical  high- 
resolution  micro-CT  image  of  a  mouse  femur  is  shown  in  Figure 
34.14. 

When  seeking  to  enhance  micro-CT  contrast/resolution.  X-ray 
dose  is  an  important  consideration.  A  typical  micro-CT  study  with 
~  100  pm  spatial  resolution  and  sufficient  contrast  resolution  to  dif¬ 
ferentiate  between  bone,  soft  tissue,  and  fat  exposes  a  mouse  to 
approximately  5  to  lOcGy.  This  is  below  the  dose  expected  to 
produce  chronic  or  acute  physiological  responses  in  mice 
(~20cGy).  As  the  spatial  and  contrast  resolution  are  increased, 
some  physiological  effects  may  be  expected  and  in  the  limit,  the 
typical  LD  50/30  of  7  Gy  may  be  approached.  Figure  34. 15  shows 
the  approximate  dose/resolution  trade-off  for  studies  designed  to 
image  only  bones  (2500 HU  contrast  resolution);  to  separate  bones, 
soft  tissue,  and  fat  (100 HU  contrast  resolution);  and  for  studies 
designed  to  differentiate  between  soft  tissues  (5  HU  contrast  reso¬ 
lution).  Based  on  this  chart,  if  one  sets  an  exposure  limit  to  lOcGy, 


FIGURE  34.14.  High  resolution  micro-CT  image  of  a  mouse  femur.  Field 
width,  4  mm.  (Image  courtesy  of  CTI/Concorde  Microsystems,  LLC, 
Knoxville,  TN.) 

it  is  possible  to  image  bone  at  a  resolution  of  20  pm;  separate  bone, 
soft  tissue,  and  fat  at  a  resolution  of  100  pm  and  to  differentiate 
between  soft  tissue  organs  at  a  resolution  of  400  pm  without 
causing  harm  to  the  animal. 

In  the  recent  past,  the  resolution,  image  quality,  and  scan  times 
of  micro-CT  have  increased  substantially,  chiefly  because  of 
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FIGURE  34.15.  Approximate  relationship  between  resolution  and  dose  for  micro-CT  studies  as  a  function  of  required  contrast  resolution.  (Chart  courtesy  of 
CTI/Concorde  Microsystems,  LLC.  Knoxville,  TN.) 
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FIGURE  34.16.  Contrast-enhanced  micro-CT  scan  of  a  living 
mouse  with  a  CT26  colon  tumor  (2  mm  diameter,  arrow )  in  the 
liver.  Other  organs  including  kidney  (K)  and  gall  bladder  (GB) 
are  easily  identified  in  the  corresponding  axial  (A),  coronal  (B), 
and  sagittal  views  (C).  Bar  in  (A)  is  2mm. 


improvements  in  2D  CCD  X-ray  detectors  and  the  algorithms  used 
to  reconstruct  the  image  data.  The  main  areas  in  which  additional 
improvements  can  be  expected  include  the  development  of  new 
iterative  reconstruction  algorithms  and  higher  power  microfocus 
X-ray  sources.  Should  both  these  succeed,  whole  animal  scan 
times  of  a  few  seconds  may  be  achieved  and  high-resolution,  high- 
contrast  images  may  be  generated  at  significantly  reduced  doses. 

Although  micro-CT  scanning  of  bones  and  skeletal  features 
produces  outstanding  images  due  to  the  inherently  high  density  of 
these  tissues,  success  in  imaging  native  soft-tissue  structures  has 
thus  far  been  modest  at  best.  As  mentioned  above,  soft  tissue  con¬ 
trast  can  be  enhanced  in  clinical  CT  with  the  use  of  non-targeted, 
water-soluble  radiographic  contrast  agents  such  as  iohexol.  These 
agents  are  non-specific  in  that  they  are  not  designed  to  target  any 
particular  tissue.  The  contrast  agents  used  on  clinical  CT  scanners 
are  not  suitable  for  use  with  research  micro-CT  scanners  because 
they  rapidly  undergo  renal  elimination  and  localize  in  the  urinary 
bladder  within  a  few  minutes  of  injection.  This  pharmacodynamic 
mismatch  between  the  agent  lifetime  and  the  relatively  long  data 
acquisition  times  associated  with  micro-CT  precludes  their  use. 
Recently  however,  tissue-specific  agents  suitable  for  use  in  micro- 
CT  have  significantly  extended  the  usefulness  of  this  non-invasive 
imaging  technique,  allowing  exquisite  vascular  and  soft  tissue 
images  to  be  made  in  living  mice  (Figs.  34.16  and  34.17;  Weichert, 
2004).  Moreover,  it  is  now  possible  to  non-invasively  perform 
virtual  serial  colonoscopy  in  mouse  colon  cancer  models 
(Pickhardt  et  al.,  2005).  This  finding  is  paramount  in  drug  devel¬ 
opment  because  it  is  now  possible  to  non-invasively  monitor  the 
effectiveness  of  anticancer  agents  over  time  in  living  mouse  tumor 
models.  Finally,  although  in  vitro  systems  are  currently  capable  of 
sub-lOpm  spatial  resolution,  these  systems  are  typically  only  used 
for  imaging  non-living  tissue  specimens  due  to  the  long  acquisi¬ 
tion  time  requirements  and  associated  radiation  dose.  While  in  vivo 
scanners  are  certainly  capable  of  obtaining  sub-20  pm  spatial 


FIGURE  34.17.  Three-dimensional  volume-rendered  micro-CT  image  of  a 
Colon-51  tumor-bearing  live  mouse,  showing  tumor  vasculature  contrast 
enhanced  using  a  blood  pool  agent  developed  by  Weichert  and  now  being  mar¬ 
keted  as  Fenestra  VC  (Alerion  Biomedical,  San  Diego,  CA).  Tumor  (T)  feeder 
vessels  are  easily  visible  ( arrows )  with  the  aid  of  this  contrast  agent.  The  heart 
is  visible  within  the  rib  cage  and  exquisite  vascular  details  are  also  visible  in 
the  liver  (L)  and  mesentary  (M). 


618  Chapter  34  •  J.M.  Tyszka  et  al. 


resolution,  absorbed  radiation  dose  becomes  a  concern  due  to 
increased  scan  times.  Using  appropriate  long-acting  contrast 
agents,  exquisite  micro-CT  images  can  be  obtained  from  soft 
tissues  and  tumors  in  live  mice  at  90  to  100  pm  without  harming 
the  animal.  In  our  hands,  we  have  performed  serial  imaging  studies 
at  this  resolution  in  the  same  mice  over  several  weeks  with  no 
apparent  harm  to  the  animals. 

Technological  enhancements  have  truly  revolutionized  small 
animal  imaging  capabilities  within  the  past  several  years  alone.  It 
is  now  possible  to  generate  spectacular  anatomic  images  at  ultra- 
high  resolution  in  living  animals  by  micro-CT.  Although  this 
affords  anatomic  information,  the  recent  introduction  of  hybrid 
microscanners,  including  a  micro-CT/micro-SPECT  hybrid,  will 
significantly  extend  our  capabilities  to  non-invasively  garner  both 
anatomic  and  biochemical  information  simultaneously  with  the  use 
of  molecular  imaging  agents. 


MAGNETIC  RESONANCE  MICROSCOPY 

Magnetic  resonance  imaging  (MRI)  is  very  well  established  as  a 
non-invasive  diagnostic  technique  in  human  medicine  with  partic¬ 
ular  strength  in  visualizing  soft  tissue.  In  comparison  to  a  typical 
clinical  MRI  with  spatial  resolution  on  the  order  of  1  mm,  mag¬ 
netic  resonance  microscopy  (MRM)  can  be  broadly  defined  as 
MRI  with  a  spatial  resolution  less  than  100  microns. 

MRM  applications  are  dominated  by  in  vivo  and  ex  vivo  struc¬ 
tural  biological  imaging,  concentrated  almost  exclusively  on 
mapping  water  and  mobile  lipid  distributions  in  soft  tissues.  MRM 
is  unfettered  by  the  optical  opacity  of  a  material,  and  is  particu¬ 
larly  suited  to  absolute  measurements  of  mass  transport  and  flow 
that  cannot  be  obtained  by  optical  methods.  However,  MRM  is  hin¬ 
dered  by  a  low  intrinsic  sensitivity,  which  ultimately  limits  spatial 
and  temporal  resolution  to  significantly  more  modest  values  than 
can  be  achieved  with  optical  techniques. 


Basic  Principles  of  Nuclear  Magnetic 
Resonance 


MRM  relies  on  nuclear  magnetic  resonance  (NMR)  as  a  source  of 
spatially  resolvable  signal.  NMR  is  a  phenomenon  observed  when 
nuclei  possessing  a  non-zero  nuclear  spin,  and  therefore  a  mag¬ 
netic  moment,  are  placed  in  a  strong  magnetic  field  and  excited 
away  from  equilibrium  with  radiofrequency  energy  [Fig. 
34.18(B)].  The  most  common  biologically-abundant  nuclei  tar¬ 
geted  by  NMR  and  MRI  include  H,  13C,  23Na,  and  31P.  19F  is  also 
used  as  an  NMR/MRI  label  due  to  its  low  natural  biological  abun¬ 
dance.  The  magnetic  field  polarizes  the  population  of  nuclear 
dipoles  according  to  quantum  mechanical  rules,  generating  a  bulk 
nuclear  paramagnetism  within  a  sample  [Fig.  34.8(A)]. 

At  equilibrium,  the  bulk  magnetization  is  aligned  parallel  to 
the  polarizing  magnetic  field  and  is  undetectable.  The  magnetiza¬ 
tion  becomes  detectable  following  excitation  by  a  radiofrequency 
pulse  at  a  specific  frequency,  /,  determined  by  the  Farmor 
equation 


/  =  —B 
2n 


(1) 


where  B  is  magnetic  flux  density  in  Tesla,  and  y  is  the  gyromag- 
netic  ratio  for  a  given  nucleus  in  rad/s/T.  For  example,  y/2%  for  the 
H  nucleus  is  approximately  42MHz/T,  so  for  a  magnetic  field 
strength  of  7.0T,  the  Larmor  or  resonance  frequency  is  close  to 
300 MHz.  The  excited  bulk  magnetization  has  a  component  per- 
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FIGURE  34.18.  Basic  principles  of  magnetic  resonance  microscopy.  (A)  A 
strong  magnetic  field  polarizes  the  population  of  nuclear  dipoles  within  a  mate¬ 
rial.  In  this  case  a  spin- 1/2  nucleus  is  shown,  which  can  adopt  only  one  of  two 
energy  states  (red  and  blue  nuclei)  in  the  magnetic  field.  The  dipoles  precess 
around  the  applied  field  at  the  Larmor  frequency  (see  text).  The  slight  excess 
of  dipoles  aligned  with  the  field  in  the  low  energy  state  (typically  10-100  ppm) 
leads  to  a  bulk  nuclear  paramagnetism.  (B)  A  radiofrequency  pulse  at  the 
Larmor  frequency  excites  the  bulk  nuclear  magnetization  away  from  equilib¬ 
rium.  Once  excited,  the  bulk  magnetization  precesses  at  the  Larmor  frequency 
and  induces  a  small  voltage  in  the  transceiver  coil  which  decays  as  the  nuclear 
spin  system  returns  to  equilibrium.  (C)  In  a  uniform  magnetic  field  (above  left) 
there  is  no  spatial  variation  in  resonance  frequency  as  seen  in  the  spectrum  for 
a  water  sample  (below  left).  Application  of  a  magnetic  field  gradient  (above 
right)  encodes  spatial  position  in  the  precessional  frequency  of  the  spin  system. 
Fourier  analysis  of  the  frequency  and  phase  of  the  detected  signal  allows  the 
spatial  distribution  of  spins  in  the  gradient  direction  to  be  reconstructed  as  a 
ID  projection  of  the  sample  (below  right).  Additional  pulsed  gradients  in 
orthogonal  directions  allow  2D  and  3D  image  formation  (not  shown). 
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FIGURE  34.19.  (A)  Simplified  schematic  of  a  magnetic  reso¬ 
nance  imaging  pulse  sequence  with  the  gradient  pulses  associ¬ 
ated  with  slice  selection  omitted  for  clarity.  Two  excitation 
pulses  are  shown  separated  by  a  repetition  time  (TR)  in  the  range 
10  ms  to  10  s.  The  time  between  the  excitation  pulse  and  the 
signal  echo  (TE)  is  in  the  1ms  to  50  ms  range  for  high-field 
microscopy.  The  1  ms  time  bar  is  a  representative  reference  only. 
(B)  Following  the  RF  excitation  pulse,  all  ‘H  nuclear  spins  are 
at  the  center  of  £-space:  (kx,  ky)  =  (0,  0).  The  negative  x-gradi- 
ent  (green)  and  y-gradient  (blue)  pulses  are  played  out  simulta¬ 
neously  and  move  the  spin  system  in  £-space  as  shown. 
The  positive  x-gradient  pulse  (red)  is  then  played  out  and  the 
spin-system  moves  in  the  kx  direction  at  a  fixed  ky  coordinate 
(lower  red  line).  During  this  gradient  pulse,  the  received  signal 
from  the  sample  is  digitized  and  saved  to  a  2D  data  array.  This 
process  is  then  repeated  as  the  amplitude  of  the  blue  pulse  is 
changed  to  sweep  data  acquisition  across  &-space  (upper  red 
line).  The  final  spatial  resolution  of  the  real-space  image  is  deter¬ 
mined  by  the  maximum  k  coordinate  reached  in  each  dimension. 
Conversely,  the  field  of  view  of  the  final  image  is  determined  by 
the  digitized  sampling  interval  in  £-space.  Following  a  complete 
traverse  of  the  required  region  of  &-space,  the  final  real-space 
image  is  reconstructed  using  a  2D  Fourier  transform.  In  prac¬ 
tice,  phase-sensitive  quadrature  detection  of  the  received  signal 
is  equated  to  the  real  and  imaginary  parts  of  £-space  required  for 
the  Fourier  transform.  The  complex  image  resulting  from 
Fourier  transformation  is  then  converted  to  a  magnitude  image 
for  display  purposes. 
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pendicular  to  the  polarizing  field  (the  transverse  component)  and 
precesses  around  the  field  direction  at  the  Larmor  frequency,  a 
process  that  induces  a  detectable  voltage  in  an  appropriately- 
placed  receiver  coil  [Fig.  34.19(B)]. 

Following  excitation,  the  precessing  transverse  component 
decays  or  relaxes  exponentially  with  a  time  constant,  T2.  Simulta¬ 
neously,  the  magnetization  component  parallel  to  the  main  field 
relaxes  back  to  its  equilibrium  value  with  a  time  constant,  7). 
Typical  relaxation  times  for  liquid  water  in  tissues  range  from  mil¬ 
liseconds  to  seconds  with  7)  always  greater  than  T2.  Consequently, 
the  natural  resonance  line-widths  in  NMR  fall  in  the  1  Hz  to  1  kHz 
range  and  are  therefore  extremely  narrow  when  compared  with  the 
Larmor  resonance  frequency. 


Magnetic  Resonance  Image  Formation 

Unlike  most  other  imaging  modalities,  MRM  does  not  physically 
“scan”  real  space  because  the  wavelength  of  the  radiofrequency 
radiation  involved  is  typically  much  larger  than  the  length-scale  of 
the  sample  (extreme  near  field).  It  turns  out  instead,  that  the 
Larmor  Eq.  (1)  can  be  exploited  to  spatially  resolve  the  NMR 
signal.  If  we  arrange  for  the  magnetic  field  to  vary  linearly  across 


a  sample,  then  the  resonance  frequency  becomes  proportional  to 
distance  along  the  gradient  direction  (frequency  encoding).  In  the 
following  description  of  MR  image  formation,  we  will  assume  that 
the  sample  contains  only  water  (singlet  resonance)  and  that  other 
parameters  such  as  relaxation,  diffusion,  motion,  etc.,  can  be 
ignored.  At  this  point  it  is  worth  looking  at  how  the  phase  of  the 
NMR  signal  varies  with  time  in  a  field  gradient  that  may  also  vary 
with  time.  In  one  dimension,  with  the  gradient  along  the  Jt-axis, 
we  have 

t )  =  |  dt' co(x,  t') 

=  dt'yB(x,t') 

=  \‘dt'yGx(t')x  (2) 

•'0 

where  Gx(t)  is  a  time-varying  magnetic  field  gradient  (in  T/m)  and 
x  is  the  position  of  a  water  molecule  (spin)  in  the  gradient  direc¬ 
tion,  ignoring  diffusion  and  bulk  motion.  We  can  define  a  new  vari¬ 
able,  kx(t),  the  significance  of  which  will  become  apparent,  as 

kJt)  =  j‘dt'jGx(t')  (3) 
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The  time-varying  signal  arising  from  water  at  a  given  position  in 
the  gradient  can  be  expressed,  using  Eqs.  2  and  3,  as 

S(x,t)  =  p(x)e,i>(x’''1 

=  p(x)eik‘i,)x  (4) 

where  p(.r)  is  the  so-called  proton  density  which  is  proportional  to 
H  nucleus  concentration.  Additional  terms  describing  the  effects 
of  relaxation,  motion,  and  RF-pulse-sequence  timing  are  omitted 
here  for  simplicity.  This  expression  in  turn  leads  to  the  total  signal 
detected  from  a  sample  as  a  function  of  kx 

=  j^dxp(x)e‘k'x  (5) 

which  is  the  inverse  Fourier  transform  of  the  proton  density  in  x. 
kx  can  be  thought  of  as  a  spatial  frequency  of  the  proton  signal, 
with  units  of  nF1  (as  opposed  to  s'1  for  temporal  frequency)  and  is 
the  coordinate  for  the  one-dimensional  (ID)  Fourier  inverse-space 
of  x.  This  expression  is  easily  extended  to  three  dimensions 

Stot(kx,ky,  kz)  =  [jxf  jyf  jz  p(x,  y,  zy(V*W) 

with  p(x,y,z)  being  the  sought-for  image.  All  positions  in  real  space 
only  occupy  one  position  in  fc-space  at  a  given  time,  so  an  MR 
image  must  be  acquired  by  scanning  &-space,  then  Fourier  trans¬ 
forming  the  resulting  inverse-space  image  to  reconstruct  the  real- 


space  image  (Fig.  34.19).  From  Eq.  3  we  can  see  that  the  sample 
can  be  moved  in  £-space  by  applying  appropriate  sequences  of 
time-varying  field  gradients  to  traverse  fc-space  in  one  or  more 
dimensions.  An  example  fc-space  trajectory  for  the  gradient  echo 
imaging  sequence  is  detailed  in  Figure  34.19.  Digital  signal  acqui¬ 
sition  can  take  place  continuously  or  discontinuously  during  only 
specific  parts  of  the  k-space  trajectory.  The  most  common  fc-space 
scanning  trajectory  involves  acquiring  one  line  of  data  in  kx  (with 
ky  and  k,  fixed)  per  radiofrequency  (RF)  excitation  pulse.  Rapid 
scanning  can  be  achieved  by  scanning  larger  regions  of  £-space 
per  excitation  pulse,  as  is  the  case  for  the  echo-planar  imaging 
(EPI)  sequence  1.  Finally,  the  real-space  image,  p(.r,y,z),  is 
reconstructed  by  Fourier  transformation  of  the  scanned  Azspace 
data. 

The  deviation  in  the  Larmor  frequency  brought  about  by  the 
field  gradient  creates  a  frequency  difference  across  a  sample  of 
between  10  kHz  and  1  MHz  in  conventional  liquid-state  MRM.  A 
single  line  of  k-space  is  acquired  in  milliseconds,  with  a  whole 
plane  of  &-space  traversed  in  anything  from  tens  of  milliseconds 
to  minutes  depending  on  the  pulse  sequence  employed.  Signal 
averaging  is  achieved  by  simple  repetition  of  a  trajectory  segment. 
The  RF  excitation  pulse  resets  the  fc-space. 

The  complement  of  frequency-encoded  signal  detection  is  fre¬ 
quency  selective  excitation.  Here  a  magnetic  field  gradient  is 
applied  in  conjunction  with  an  amplitude-modulated  (and  in  some 
cases)  phase-modulated  RF  pulse  (Fig.  34.20).  The  modulation  of 
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FIGURE  34.20.  An  amplitude  modulation  wave¬ 
form  for  the  RF  carrier  at  the  Larmor  frequency  (A) 
produces  a  band-limited  excitation  of  the  nuclear 
spins  (B).  If  such  an  RF  waveform  is  used  in  combi¬ 
nation  with  a  linear  magnetic  field  gradient,  then  only 
a  limited  slice  or  slab  will  be  excited  perpendicular  to 
the  gradient  direction.  Three  such  slice-selective 
pulses  could  be  combined  to  limit  excitation  to  a 
cubic  sub-volume  within  a  sample. 
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FIGURE  34.21.  Atypical  MRM  hardware  configuration:  (A)  an  11.7  Tesla  vertical  bore  superconducting  magnet.  This  hardware  is  identical  to  that  used  for 
high-resolution  NMR  spectroscopy.  (B)  Three  millimeter  transverse  RF  solenoid  (copper  winding  at  upper  center)  and  tuning  circuitry  for  excitation  and  detec¬ 
tion  of  the  MR  signal.  (C)  Simplified  schematic  cross-section  of  a  vertical  bore  MRM  setup.  The  gradient  and  shim  coils  are  secured  within  the  magnet  bore 
and  the  RF  coil/sample  assembly  introduced  for  each  experiment.  (D)  Enlargement  of  a  transverse  configuration  for  the  solenoidal  RF  coil  and  sample  tube.  The 
sample  is  often  surrounded  by  a  suitable  fluid,  such  as  buffered  saline  or  culture  medium.  For  fixed  samples  where  background  signal  needs  to  be  eliminated, 
the  sample  can  be  surrounded  by  H-free,  fluorinated  fluids.  Environmental  control  can  be  provided  by  thermostatically  controlled  airflow  through  the  bore  of 
the  coil/sample  assembly  base. 


the  carrier  frequency  of  the  excitation  pulse  limits  the  frequency 
content,  so  that  the  excitation  becomes  spatially  limited  in  the 
presence  of  a  field  gradient.  For  example,  a  1  ms  shaped  pulse 
might  have  a  bandwidth  of  4  kHz  that,  in  the  presence  of  a  lOmT/m 
field  gradient  would  limit  NMR  excitation  to  a  region  approxi¬ 
mately  1  cm  wide  in  the  gradient  direction,  but  would  be  unlim¬ 
ited  perpendicular  to  the  gradient  direction.  This  process  is 
termed  slice  or  slab  selection  and  can  be  extended  to  line  and 
volume  selections  by  using  multiple,  selective  RF  pulse/gradient 
combinations.  Slice  selection  is  often  used  for  in  vivo  imag¬ 
ing  where  interleaved  acquisition  of  multiple  2D  ^-spaces 
separated  in  z  is  more  time-efficient  than  scanning  the  complete 
3D  fc^-space. 

In  a  typical  MRM  experiment,  a  sample,  such  as  an  anes¬ 
thetized  mouse  or  a  fixed  embryo,  is  secured  within  an  appropri¬ 
ate  holder  and  placed  within  the  RF  coil.  Additional  physiological 
monitoring,  anesthesia  lines,  and  environmental  control  are  often 
integrated  into  the  RF  coils/sample  assembly.  The  gradient  and 


shim  coils  are  normally  secured  more  permanently  within  the  main 
magnet  bore  (Fig.  34.21)  requiring  only  the  RF  coil  and  sample  to 
be  introduced  for  each  experiment.  The  RF  coil/sample  assembly 
is  then  placed  at  the  center  of  the  main  field  and  gradient  coils  for 
imaging.  The  shim  coil  currents  are  adjusted  to  minimize  magnetic 
inhomogeneities  in  the  sample,  the  synthesizer  frequency  set  to  the 
Larmor  frequency  determined  from  the  received  signal  and  the 
strength  of  the  RF  excitation  pulses  calibrated.  The  duration  of  an 
image  acquisition  may  vary  from  less  than  a  second  to  more  than 
a  day  and  is  largely  determined  by  the  required  spatial  resolution 
and  signal-to-noise  ratio.  The  RF  power  deposition  in  the  sample 
varies  widely  with  pulse  sequence  design  and  has  the  potential  to 
cause  significant  heating.  Changes  to  the  pulse  sequence  timing 
and  RF  pulse  waveform  specifications  can  minimize  peak  and 
average  power  deposition.  Heating  effects  increase  rapidly  with 
the  Larmor  frequency  (increasing  field  strength)  and  power  depo¬ 
sition  can  be  estimated,  monitored  and  controlled  for  in  vivo 
imaging  studies. 
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Magnetic  Resonance  Microscopy  Hardware 

The  hardware  for  MRM  consists  of  three  major  components  and 
their  associated  electronics:  ( 1 )  a  permanent  or  electromagnet  that 
generates  the  main  polarizing  field,  (2)  one  or  more  radiofrequency 
coils  for  excitation  and  signal  detection,  and  (3)  a  three-axis  set 
of  magnetic-field-gradient  coils.  The  sample  is  placed  within  the 
RF  coil  which  in  turn  is  placed  within  the  gradient  and  the  main 
solenoid.  The  main,  polarizing  field  is  typically  generated  by  a 
cryogenically-cooled,  superconducting  electromagnet,  although 
permanent  magnets  and  even  resistive  electromagnets  are  in  use. 
The  uniformity  or  homogeneity  of  the  polarizing  field  plays  a 
critical  role  in  successful  high-quality  MRM.  Additional  room- 
temperature  field  correction  or  shim  coils  are  routinely  used  to 
improve  the  fundamental  homogeneity  of  the  magnetic  field  within 
the  imaging  volume.  RF  coils  for  MRM  are  most  commonly 
simple  loops,  solenoidal  resonators,  or  a  volume  resonator  design 
called  a  birdcage.  A  single  coil  may  be  used  for  both  transmis¬ 
sion/excitation  and  reception,  or  a  larger  transmitting  volume  coil 
with  good  RF  field  homogeneity  can  be  used  with  a  high-sensi- 
tivity  receiver  loop  coil  (surface  coil). 

The  RF  power  absorbed  by  a  sample  during  imaging  varies 
widely  with  the  type  of  pulse  sequence  employed.  Some  sequences 
involve  rapid  RF  pulsing  at  relatively  high  power  which  ultimately 
will  lead  to  tissue  heating.  Power  deposition  also  increases  rapidly 
with  field  strength  for  a  given  pulse  sequence. 

The  signal  received  from  the  RF  coil  is  amplified  by  a  low 
noise  preamplifier  prior  to  demodulation  from  the  Larmor  fre¬ 
quency  (see  sidebar)  and  digitized  at  rates  from  about  lOOkS/s  to 
2  MS/s.  Raw  k-space  data  is  reconstructed  and  displayed  by  the 
operating-console  computer. 

The  coils  generating  magnetic  field  gradients  (gradient  coils) 
are  usually  wound  on  a  cylindrical  former  within  the  main  magnet 
bore.  The  coil  windings  are  designed  to  produce  independent, 
linear  field  gradients  over  the  imaging  volume  in  each  of  the  three 
cardinal  directions  ( x ,  y,  z).  The  maximum  gradient  strengths 
required  for  MRM  typically  exceed  500mT/m  and  may  rise  to 
more  than  5T/m  in  high-performance  hardware.  Even  at  these 
strengths,  the  maximum  field  perturbation  from  the  gradient  coils 
rarely  exceeds  1%  of  the  main  magnetic  field. 

A  functional  MRM  system  requires  a  high-stability,  high- 
accuracy  frequency  synthesizer  that  provides  the  reference  oscil¬ 
lation  for  both  RF  transmission  and  reception.  Amplifiers  for  the 
RF  and  gradient  coils  are  controlled  by  a  pulse  sequencer  that  in 
turn  is  coordinated  by  the  operating-console  computer. 

Strengths  and  Limitations  of  Magnetic 
Resonance  Microscopy 

Whereas  magnetic  resonance  imaging  of  humans  is  very  well 
established  as  a  medical  diagnostic  tool,  microscopy  has  developed 
at  a  slower  rate.  MRM  cannot  compete  with  optical  microscopy  in 
terms  of  spatial  resolution  and  sensitivity,  but  has  several  unique 
strengths  that  benefit  certain  applications.  Specifically,  MRM  is 
not  limited  by  the  opacity  of  an  object  to  light,  because  it  employs 
radiofrequency  radiation  with  wavelengths  much  larger  than  the 
typical  sample  size. 

Optimizing  MRM  is  a  complex  trade-off  between  acceptable 
signal-to-noise  ratio,  temporal  resolution,  and  spatial  resolution. 
The  NMR  experiment  has  an  intrinsically  low  sensitivity,  so 
requirements  for  minimum  SNR  tend  to  place  lower  bounds  on 
spatial  and  temporal  resolution.  Consequently,  acquiring  high 
spatial  resolution  MRM  with  voxel  sizes  in  the  10  to  100  micron 


range  typically  requires  tens  of  minutes  to  hours  to  achieve  an 
acceptable  SNR. 

Various  estimates  place  the  ultimate  limit  for  MRM  spatial 
resolution  in  the  neighborhood  of  1  pm  for  liquid  water  at  room  or 
physiological  temperatures.1  These  estimates  consider  the  effects 
of  molecular  diffusion,  T2  relaxation,  microscopic  field  inhomo¬ 
geneity,  and  sensitivity  (Callaghan,  1991).  Other  factors,  includ¬ 
ing  hardware  design  and  sample  size,  lead  to  practical  MRM 
isotropic  resolution  limits  greater  than  10  pm.  The  achievable 
spatial  resolution  is  limited  by  sensitivity  factors,  specifically  the 
signal-to-noise  ratio  achievable  in  a  given  acquisition  time. 

As  a  general  rule,  high-resolution  MRM  with  spatial  resolu¬ 
tions  less  than  100  pm  benefits  from  the  use  of  polarizing  magnetic 
fields  greater  than  3  Tesla,  sometimes  as  high  as  17.5  Tesla. 
Although  this  leads  to  a  greater  nuclear  paramagnetic  polarization, 
other  factors  such  as  increasing  7j  relaxation  times,  decreasing  T2 
relaxation  times,  and  increased  susceptibility-based  field  inhomo¬ 
geneities  tend  to  diminish  the  expected  gains  in  sensitivity  and 
signal-to-noise  ratio  efficiency. 

Fligh-performance  gradient  hardware,  with  higher  maximum 
amplitudes  (27m)  and  faster  slew  rates  ( T/m/s )  are  almost  always  an 
advantage  in  MRM.  Microscopy  gradients  capable  of  10  Tim  allow 
high  spatial  and  temporal  resolution  imaging  of  small  samples. 

Another  approach  to  increasing  the  sensitivity  of  MRM  is 
through  RF  coil  design.  Various  groups  have  developed  microcoils 
with  dimensions  smaller  than  1mm  to  boost  sensitivity  in  very 
small  samples.  Some  of  the  highest  resolution  MRM  images 
obtained  have  employed  such  designs  (Grant  et  al.,  2001;  Lee  et 
al.,  2001;  Ciobanu  and  Pennington,  2004). 

Image  Contrast  in  Magnetic 
Resonance  Microscopy 

Signal  differences  between  tissues  in  'FI  MRM  of  biological 
samples  largely  arise  from  differences  in  the  microscopic  envi¬ 
ronment  of  intracellular  and  interstitial  water.  Chemical  differ¬ 
ences  between  molecules  containing  'H  nuclei  also  contribute  to 
image  contrast,  for  example,  between  water  and  lipid-rich  tissues. 
Parameters  influencing  MRM  image  contrast  include  7j  and  77 
relaxation  times,  NMR  nucleus  concentration,  temperature,  diffu¬ 
sion  coefficient,  fluid  velocity,  magnetic  susceptibility,  and  mag¬ 
netization  transfer  coefficients  of  a  material.  The  influence  that 
each  of  these  parameters  has  on  image  contrast  can  be  accentuated 
or  suppressed  by  careful  design  of  the  sequence  of  radiofrequency 
and  field  gradient  pulses  played  out  prior  to  signal  acquisition.  For 
example,  specific  pulse  sequence  designs  allow  the  time  delay 
between  excitation  and  signal  acquisition  to  be  increased,  accen¬ 
tuating  differences  in  the  T2  relaxation  time  of  materials.  Reduc¬ 
ing  the  time  delay  between  excitation  pulses  accentuates  short  7j 
materials  that  appear  brighter  in  such  images. 

Motion  of  water  and  other  fluids  can  be  encoded  in  the  phase 
or  amplitude  of  MRM  images  using  gradient  pulse  combinations. 
Both  coherent  motion  (flow)  and  incoherent  motion  (diffusion)  can 
be  quantified  by  MRM  over  a  2D  plane  or  3D  volume  of  an  opti¬ 
cally  opaque  material.  MRM  measurements  of  water  diffusion 
within  organized  tissues  have  been  particularly  well  developed  and 
are  capable  of  quantifying  subresolution  ensemble  molecular 
interactions  with  restrictive  or  hindering  boundaries  (Tuch,  2003). 
For  example,  diffusion  tensor  imaging  (DTI)  maps  diffusion 


1  About  3  to  4  |Jm  has  been  obtained  on  very  small  specimens  (9  Tesla;  Ciobanu 
and  Pennington,  2004). 
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anisotropy  within  tissues  using  a  simplified  model  of  molecule¬ 
boundary  interactions  (Pierpaoli  et  al.,  1996).  Diffusion  tensor 
MRM  has  been  applied  to  brain  development  in  mice  (Zhang  et 
al.,  2003),  dysmyelination  models  (Song  eta/.,  2002),  and  myocar¬ 
dial  fiber  structure  (Jiang  et  al.,  2004). 

In  addition  to  endogenous  image  contrasts,  such  as  relaxation 
time  and  molecular  diffusion,  it  is  possible  to  introduce  exogenous 
MRM  contrast  agents  that  target  specific  tissues  or  physiology 
within  a  living  system.  Gadolinium  chelates  are  widely  used,  both 
individually  and  bound  to  larger  molecules,  to  reduce  the  T i  relax¬ 
ation  time  of  surrounding  water  (Weinmann  et  a/.,  1984,  2003). 
Ionic  manganese  is  a  calcium  analog  and  7)  contrast  agent  that,  at 
low  concentrations,  has  been  used  as  an  in  vivo  trans-synaptic 
axonal  transport  tracer  (Pautler  and  Koretsky,  2002;  Pautler  et  al., 
2003;  Aoki  et  al.,  2004).  Tracking  of  progenitor,  stem,  or  immune 
cells  using  MRM  is  a  recent  growth  area  (Hinds  et  al.,  2003; 
Shapiro  et  al.,  2004)  typically  employing  variants  of  superpara- 
magnetic  iron  oxide  (SPIO)  particles  as  intracellular  T2*  contrast 
agents  (Foster-Gareau  et  al.,  2003).  The  development  of  molecu¬ 
lar  imaging  for  MRM,  targeting  both  gene  expression  and  metab¬ 
olism  within  a  living  organism  is  likely  to  be  the  next  significant 
field  of  investigation  (Louie,  2000;  Allen,  2004)  and  holds  great 
promise  for  future  non-invasive  biomedical  research. 


Magnetic  Resonance  Microscopy  Applications 
Phenotyping 

MRM  is  widely  used  for  phenotyping  genetically  manipulated 
organisms  such  as  transgenic  mice  (Johnson  et  al.,  2002a,  2002b; 
Lo  et  al.,  2003)  leading  to  the  development  of  high  throughput 
techniques  for  imaging  more  than  one  animal  simultaneously 
(Bock  et  al.,  2003).  MRM  brain  atlases  of  adult  inbred  mouse 
strains  are  being  constructed  by  several  groups  (Kovacevic  et  al., 
2005;  Segars  et  al.,  2004). 

Histology 

Ex  vivo  MR  histology  exploits  the  lack  of  physiological  motion 
and  extended  data  acquisition  times  to  generate  high-resolution 
structural  images  of  various  organisms  (Figs.  34.22  and  34.23) 
(Johnson  et  al.,  1993;  Johnson  et  al.,  2002a).  MRM  lacks  the 
spatial  resolution  and  staining  flexibility  of  optical  histology,  but 
preserves  the  3D  structure  of  tissues  and  eliminates  the  need  for 
dehydration,  embedding,  and  physical  sectioning  and  their  associ¬ 
ated  artifacts.  As  a  non-invasive  technique,  MRM  can  even 
precede  conventional  histology,  providing  a  valuable  structural  ref¬ 
erence  for  subsequent  histological  sections. 


FIGURE  34.22.  Examples  of  MRM  histology  of  biological  samples.  (A)  High-resolution  3D  MRM  of  a  fixed  late  gastrula  stage  Xenopus  laevis  embryo  with 
a  nominal  isotropic  spatial  resolution  of  16  microns.  The  vegetal  cell  mass  (vcm),  blastopore  (bp),  and  archenteron  (arch)  are  clearly  visualized  in  this  section. 
(B)  Volume  texture  rendering  of  a  3D  MRM  image  of  a  fixed  8.5  dpc  mouse  embryo  within  its  yolk  sac.  (C)  Central  section  through  a  composite  MRM  image 
of  a  fixed  mouse  eye.  The  red  channel  encodes  the  isotropic-diffusion-weighted  image,  highlighting  regions  of  restricted  diffusion  within  the  chorioretina  (orange). 
The  green  channel  encodes  the  7V weighted  image  in  which  free-fluid  appears  bright.  The  low  intensity  circular  feature  within  the  globe  of  the  eye  is  the  crys¬ 
talline  lens.  (D)  Montage  of  sections  from  a  3D  MRM  image  of  a  fixed  7  dpc  mouse  embryo  with  nominal  20  micron  isotropic  spatial  resolution. 
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FIGURE  34.23.  Volumetric  rendering  of  an  11-day  postcoital  mouse  embryo.  Because  the  sample  has  not  been  dehydrated,  the  global  geometric  preservation 
of  the  embryo  is  excellent. 


Developmental  Biology 

MRM  has  been  applied  to  structural  imaging  of  mouse  (Jacobs, 
1999;  Chapon,  2002)  and  quail  embryos  and  to  serial  in  vivo 
imaging  of  the  Xenopus  laevis  embryo  (Papan  et  al.,  2001;  Jacobs 
et  al.,  2003).  MRM  leverages  its  ability  to  image  optically  opaque 
specimens  to  generate  3D  data  which  would  be  difficult  or  impos¬ 
sible  to  obtain  by  other  methods.  MRM  is  particularly  suited  to 
studies  of  dynamic  developmental  processes  such  as  morphogen¬ 
esis.  As  for  phenotyping,  MRM  is  proving  valuable  in  the  con¬ 
struction  of  reference  developmental  atlases  for  mouse,  quail,  and 
other  embryos  that  are  too  large  or  opaque  at  later  stages  for  optical 
imaging  (Dhenain  et  al.,  2001;  Matsuda  et  al.,  2003). 

Other  Applications 

MRM  is  in  many  ways  underexploited  outside  the  realm  of  verte¬ 
brate  imaging.  MRM  is  an  excellent  resource  for  botanical 
imaging,  exploiting  minimal  physiological  motion  to  generate  high 
spatial  resolution  images  (Edzes,  1998;  Kockenberger,  2001, 
2004)  and  is  especially  suited  to  imaging  flow  and  water  distribu¬ 
tion  within  plant  samples  (Kuchenbrod  et  al.,  1998).  MRM  has 
also  seen  applications  in  opaque  bioreactors  and  biofilms,  study¬ 
ing  local  flow  and  structure  beyond  the  capabilities  of  optical 
microscopy  (As  and  Lens,  2001;  Paterson-Beedle  et  al.,  2001; 
Manz  et  al.,  2003;  Seymour  et  al.,  2004). 

Future  Development  of  Magnetic 
Resonance  Microscopy 

The  future  of  MRM  is  likely  to  include  incremental  improvements 
in  spatial  resolution  and  sensitivity  as  hardware  design  and  fabri¬ 
cation  methods  are  refined.  Ironically,  clinical  MRI  has  led  the  way 
in  technical  advances,  driven  by  the  needs  and  resources  of  medical 
diagnosis  and  therapy.  Some  of  these  technologies,  including 


phased-array  RF  coils  and  advanced  pulse  sequence  designs,  will 
be  reapplied  to  MRM  applications.  However,  it  is  in  the  develop¬ 
ment  of  new  molecular  imaging  agents  in  model  organisms  that  we 
will  most  likely  see  the  greatest  advances  in  MRM  in  the  near  future. 


CONCLUSION 

The  imaging  modalities  discussed  in  this  chapter  are  emerging 
tools  directed  at  meeting  the  growing  need  for  volumetric 
microscopy  of  challenging  specimens.  They  join  confocal  laser¬ 
scanning  and  two-photon  laser-scanning  microscopy  in  the  suite 
of  microscopic  imaging  techniques  that  enable  entire,  whole- 
mounted  specimens  to  be  studied.  Each  of  these  volumetric 
imaging  tools  plays  an  important  role  in  the  imaging  toolkit,  as 
none  answers  all  needs.  SIM/EFIC  is  a  high-resolution  method  that 
is  capable  of  capturing  the  entire  volume  of  large  specimens; 
however,  it  is  inherently  destructive  as  biological  samples  must  be 
fixed,  dehydrated,  and  embedded,  and  then  must  be  sectioned. 
Although  imaging  the  block  face  avoids  the  distortions  resulting 
from  the  sectioning  process,  the  specimen  is  still  subject  to  any 
fixation  and  dehydration  artifacts.  As  presently  employed,  OPT 
requires  extensive  clearing  of  a  specimen  to  render  it  translucent, 
and  so  is  subject  to  the  same  fixation  and  shrinkage  artifacts  as 
SIM/EFIC.  A  major  strength  of  OPT  on  fixed  specimens  is  its 
ability  to  capture  the  distribution  of  molecular  markers  such  as 
antibody  staining  and  gene  expression  in  situ  using  common  mo¬ 
lecular  biological  and  histological  techniques.  Of  course,  on  living 
specimens  that  are  small  and  transparent,  OPT  can  be  performed 
without  processing,  and  at  the  time  of  writing,  live  OPT  imaging 
experiments  are  being  performed. 

A  subset  of  the  volumetric  imaging  tools  are  designed  to  image 
living  specimens:  OCT,  SPIM,  and  MRM.  OCT,  a  reflected  light 
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modality,  requires  no  extrinsic  labels  and  is  ideal  for  optical 
imaging  deep  within  specimens,  and  for  following  motions  such 
as  blood  flow.  SPIM  offers  an  important  tool  at  the  confluence  of 
imaging  and  molecular  biology  technologies,  as  it  is  a  fluorescent 
optical  sectioning  technique,  ideal  for  capturing  the  expression  of 
GFP  reporter  genes  in  living  specimens.  The  use  of  transgenic  GFP 
reporter  constructs  allows  for  the  direct  in  vivo  visualization  of 
gene  transcription  during  development  within  the  context  of  the 
living  embryo:  the  SPIM  sample  chamber  is  designed  for  main¬ 
taining  physiological  conditions  and  the  use  of  the  light  sheet 
minimizes  sample  bleaching  and  reduces  blur  from  structures 
not  in  the  focus  plane.  Using  SPIM,  dynamic  events  ranging  from 
fractions  of  a  second  to  days  can  be  recorded  at  voxel  sizes  of 
-0.5  pm. 

MRM  is  not  an  optical  modality  and  therefore  is  not  limited 
by  the  optical  properties  of  the  subject.  MRM  is  non-invasive  and 
allows  long-term  observation  of  living  biological  specimens  as  no 
ionizing  radiation  is  employed.  Although  MRM  is  slower,  more 
expensive,  and  offers  lower  resolution  than  most  optical  methods, 
it  offers  direct  observation  of  morphogenic  movements  in  opaque 
embryos.  Previously,  these  movements  could  only  be  inferred  from 
static  sections.  Clinically,  MRI  has  a  proven  diagnostic  history  but 
is  only  in  its  infancy  as  a  microscopic  technique.  Tools  proven  in 
the  clinic  are  being  modified  for  microscopic  use,  promising  a 
growing  set  of  techniques  for  following  tissue  fine  structure  and 
blood  flow. 

As  with  any  review  of  finite  length,  there  are  a  great  many 
modalities  and  modifications  to  existing  techniques  that  have  not 
been  treated  here.  Adapting  non-conventional  elements  to  con¬ 
ventional  devices  pushes  microscopy  development  forward.  The 
field  of  view  of  confocal  microscopy  has  been  greatly  increased 
(20  mm2  at  2  pm  resolution)  with  the  use  of  a  high-NA  f-theta  tele- 
centric  objective  (Dixon,  1995).  Microscopic  versions  of  clinical 
imaging  tools  are  emerging  and  improving  at  an  alarming  rate. 
These  include  positron  emission  tomography  (micro-PET),  and 
ultrasonography  (micro-US).  Typing  in  any  of  these  key  terms  in 
a  Web-  or  library-based  search  will  find  a  rapidly  growing  set  of 
offerings.  The  resolution  of  these  techniques  has  improved  dra¬ 
matically.  The  omission  of  these  powerful  approaches  is  in  no  way 
intended  to  diminish  their  potential  role  for  volumetric  imaging, 
but  instead  a  statement  of  how  rapidly  these  fields  are  advancing. 
The  future  will  see  the  toolkit  for  volumetric  imaging  grow  at  an 
increasing  rate. 
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INTRODUCTION 

The  other  chapters  in  this  book  give  the  reader  an  in-depth  descrip¬ 
tion  of  every  important  aspect  of  biological  confocal  microscopy 
that  we  could  think  of.  This  chapter  is  to  provide  the  novice  user 
of  this  instrument  with  a  basic  understanding  of  the  practical  infor¬ 
mation  needed  to  use  it  effectively.  Because  the  computer  inter¬ 
faces  of  the  various  commercial  instruments  vary  greatly,  this 
chapter  will  stress  the  important  features  of  microscopical  optics 
and  the  basics  of  sampling  that  are  common  to  all  instruments.  The 
underlying  agenda  of  these  suggestions  is  based  on  two  principles: 

•  Don’t  waste  photons. 

•  Get  all  of  the  optical  performance  that  you  have  paid  for. 

Getting  Started 

In  order  to  be  able  to  operate  a  confocal  microscope  properly,  one 
must  first  be  thoroughly  familiar  with  the  basic  principles  of  non- 
confocal  microscopy.  The  user  must  understand  the  concept  of 
conjugate  image  planes,  Kohler  illumination,  and  how  to  set  up  a 
microscope  to  produce  it  [i.e.,  the  field  diaphragm  must  be  in  focus 
in  the  image  plane  and  the  condenser  aperture  set  so  that  the  illu¬ 
mination  almost  fills  the  back-focal  plane  (BFP)  of  the  objective 
lens.  See  the  Appendix  to  this  chapter  or  Chapter  6,  this  volume , 
for  a  refresher].  The  user  must  also  understand  how  the  phenom¬ 
enon  of  diffraction  acts  to  place  the  fundamental  limitation  on  the 
resolution  that  any  optical  system  can  attain  (see  Chapters  1,  8, 
and  11,  this  volume,  or  an  introductory  text  such  as  Bradbury, 
1984,  Inoue  1986,  Pawley  and  Centonze,  1997). 

The  next  step  is  to  get  an  image  of  some  specimen  that  you 
understand.  Without  going  into  the  details  of  the  many  available 
commercial  confocal  systems,  we  note  that  most  confocal  micro¬ 
scopes  permit  the  user  to  set  up  the  microscope  for  normal,  non- 
confocal  use,  and  then  switch  over  to  confocal  operation  by 
changing  the  position  of  a  single  control.  The  user  should  follow 
the  instructions  from  the  manufacturer  to  produce  such  a  confocal 
image  using  a  live  mode  (i.e.,  producing  an  image  that  is  contin¬ 
uous  and  not  frame-averaged  or  Kalman  filtered). 

In  most  cases,  the  fluorescent  specimen  chosen  for  this  first 
attempt  at  using  the  confocal  microscope  should  be  similar  or  iden¬ 
tical  to  one  that  has  just  been  viewed  with  success  on  a  conven¬ 
tional  instrument.  Almost  invariably,  new  users  will  close  the 
detection  pinhole  in  order  to  get  the  highest  resolution  and  increase 
the  zoom  magnification  to  increase  the  size  of  the  image  on 
the  display.  They  will  then  try  to  adjust  the  focus,  but  after  a 


few  seconds,  the  image  will  begin  to  fade  and  become  indistinct. 
Reducing  the  zoom  magnification  will  reveal  the  terrible  truth:  the 
observations  so  far  have  severely  bleached  the  dye  in  the  rectan¬ 
gular  area  that  was  being  scanned  at  the  higher  zoom  magnifica¬ 
tion.  This  is  the  moment  at  which  would-be  users  either  decide 
right  then  and  there  that  the  confocal  microscope  clearly  produces 
far  more  bleaching  than  they  are  used  to  and  leave  the  room,  never 
to  return,  or  they  decide  that  there  may  be  a  bit  more  to  operating 
this  instrument  than  they  had  bargained  for  and  set  out  to  do  better. 
This  chapter  is  written  for  those  who  fall  into  the  latter  category. 

Bleaching  —  The  Only  Thing  That  Really  Matters 

There  are  three  reasons  that  the  bleaching  rate  of  a  confocal  micro¬ 
scope  often  seems  to  be  much  higher  than  that  of  a  conventional 
epi-fluorescent  microscope: 

1 .  The  instrument  can  be  used  in  such  a  way  that  the  suppo¬ 
sition  is  correct!  A  laser  is  a  far  more  intense  light  source  than  a 
Hg  arc.  Although  the  total  power  striking  the  entire  specimen  may 
be  less  with  the  laser,  the  area  of  the  specimen  over  which  it  is 
absorbed  can  be  reduced  arbitrarily  by  increasing  the  setting  of  the 
zoom  magnification  control,  thus  increasing  the  power/unit  area. 
The  crucial  difference  between  the  two  microscopical  methods  is 
that  in  normal  epi-fluorescence,  the  power/area  on  the  specimen  is 
fixed  by  the  type  of  arc  and  the  illumination  optics,  while  in  con¬ 
focal  microscopy,  the  power/area  increases  with  the  square  of  the 
zoom  magnification.  In  normal  epi-fluorescence,  one  does  not 
expect  to  see  an  image  that  is  both  large  enough  to  see  the  finest 
details  and  bright  enough  to  view  by  eye  because  it  is  not  possi¬ 
ble.  In  the  confocal  microscope,  the  extreme  intensity  of  the  laser 
does  make  it  possible,  though  only  for  the  moment  until  bleach¬ 
ing  occurs! 

2.  If  one  is  accustomed  to  viewing  a  widefield  (WF)  image  in 
which  fluorescence  from  a  large  number  of  focal  planes  is  added 
together  into  a  single  image,  a  confocal  image  may  look  somewhat 
anemic  because,  ideally,  it  records  only  the  fluorescence  features 
present  in  a  single  optical  section.  When  presented  with  an  image 
of  the  few  features  that  happen  to  be  in  the  focus  plane,  one  may 
be  tempted  to  use  a  much  longer  exposure  than  is  necessary  in  a 
vain  attempt  to  record  features  that  are  not  really  there. 

3.  A  final  factor  contributing  to  the  perception  that  confocal 
bleaches  faster  is  that,  in  normal  epi-fluorescence,  one  expects  to 
make  one  exposure  that  may  require  30  to  60  s,  while  in  the  con¬ 
focal  microscope  one  often  wants  an  image  that  is  bright  enough 
to  look  at  after  collecting  for  only  1  to  2  s  and  then  sets  out  to 
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record  20  to  30  similar  images  of  the  adjacent  planes  to  form  a 
three-dimensional  (3D)  stack.  Each  of  these  20  to  30  images  will 
cause  additional  bleaching. 

In  actual  fact,  what  one  should  be  interested  in  is  not  so  much 
the  bleaching  rate  but  some  sort  of  efficiency  ratio  of  information 
recorded  to  illuminating  dose.  This  ratio  is  set  by  the  numerical 
aperture  (NA)  and  transmission  (T)  of  the  optics  and  the  quantum 
efficiency  (QE)  of  the  photodetector.  Therefore,  assuming  equal 
optics,  the  confocal  microscope  is  lOx  to  lOOx  more  efficient  than 
photography,  because  at  these  low  light  levels,  the  photomultiplier 
tube  (PMT)  photodetector  that  it  uses  is  lOx  to  lOOx  more  efficient 
than  film  (Chapter  12,  this  volume).  The  only  photodetector  more 
efficient  than  the  PMT  is  the  cooled  charge-coupled  device  (CCD), 
and  this  detector  can  be  used  in  disk-scanning  confocal  micro¬ 
scopes  (Chapters  10,  12,  21,  25,  and  Appendix  3,  this  volume). 

While  on  the  subject  of  bleaching,  there  are  a  few  additional 
items  that  should  be  mentioned.  It  is  generally  assumed  (1)  that 
bleaching  is  proportional  to  the  total  dose  of  light  where  dose  is 
the  illumination  level  (in  photons/pm2s)  x  time,  and  (2)  that,  as 
absorption  is  usually  low,  the  total  number  of  photons  passing 
through  any  horizontal  plane  is  a  constant.  From  these  facts,  some 
jump  to  the  conclusion  that  bleaching  occurs  at  the  same  rate  in 
the  focus  plane  as  it  does  outside  this  plane.  While  this  may  be 
true  for  short  distances  above  and  below  the  focus  plane  and  when 
using  lenses  of  low  NA,  it  is  not  true  once  the  height  of  a  stack  of 
images  becomes  comparable  with  the  width  of  the  scanned  area. 
Figure  28.3(B)  in  Chapter  28  shows  an  xz-image  through  such  a 
bleach  pattern  made  with  an  NA  1 .4  in  a  specimen  of  fluorescent 
plastic.  It  confirms  that  there  is  a  pyramid  of  bleached  dye  expand¬ 
ing  above  and  below  the  plane  of  focus.  As  the  illumination 
intensity  at  any  level  is  proportional  to  the  cross-sectional  area  of 
the  light  cone  at  that  level,  one  can  see  that,  with  a  large  NA  lens, 
the  illumination  level  and,  therefore,  the  bleach  rate  drops  off  quite 
quickly  with  distance  away  from  the  focus  plane.  The  illumination 
level  at  these  planes  is  not  constant  within  the  cone  but  is  less 
strong  near  the  edges. 

Figure  35.1  is  a  diagram  to  help  visualize  this  process.  We 
assume  that  the  BFP  of  the  objective  is  uniformly  illuminated  and, 
therefore,  light  is  focused  into  the  focused  spot  from  all  angles 
equally.  As  a  result,  the  total  number  of  photons  passing  through 
any  particular  level  within  the  cone  of  the  beam  is  a  constant.  The 
intensity  itself  decreases  roughly  with  the  square  of  the  distance 
away  from  the  focus  plane  but,  as  the  beam  scans  over  the  raster, 
this  effect  is  counterbalanced  by  the  fact  that  most  points  in  non¬ 
focus  planes  will  be  illuminated  for  a  longer  time.  Although  points 
that  are  only  illuminated  by  the  outer  rays  of  the  cone  when  the 
beam  is  scanning  near  the  edge  of  the  raster  will  only  be  illumi¬ 
nated  once  per  scan,  those  near  the  center  of  the  raster  will  be  illu¬ 
minated  for  a  time  that  is  longer  in  direct  proportion  to  the 
reduction  in  the  illumination  intensity  at  any  instant.  All  of  the 
points  for  which  this  is  true  for  a  particular  focus  position  will  lie 
inside  a  bleaching  octahedron  having  the  scanned  area  as  its  base 
and  a  defining  angle  equal  to  the  acceptance  half-angle  of  the 
objective  (a).  Inside  this  octahedron  the  bleach  rate  is  constant, 
while  outside  it  the  bleach  rate  decreases  slowly  until  it  becomes 
zero  in  the  region  where  even  the  outer  rays  of  a  beam  scanning 
the  edge  of  the  raster  never  reach. 

As  the  focus  plane  moves  up  or  down  in  the  specimen  during 
a  z-series,  the  total  bleaching  will  be  proportional  to  the  super¬ 
position  of  the  octahedra  of  the  individual  planes.  The  result  is  that 
points  inside  the  octahedron  associated  with  the  central  focus  plane 
will  receive  a  maximum  dose  while  those  farther  from  the  center 


z  axis 


FIGURE  35.1 .  Pattern  of  bleaching  in  the  confocal  microscope.  A  stationary, 
focused  light  beam  will  bleach  a  conical  volume  in  a  uniform  fluorescent  spec¬ 
imen.  The  severity  of  the  bleaching  at  any  one  level  in  the  cone  is  inversely 
proportional  to  the  area  of  the  cone  at  that  level  and  is  most  severe  at  the  focus 
plane.  However,  as  the  beam  is  scanned  over  the  focus  plane,  points  in  this 
plane  are  only  illuminated  a  few  times  while  points  in  adjacent  planes  will  be 
illuminated  for  a  longer  time.  As  the  beam  is  scanned  over  a  line  in  the  focus 
plane,  the  convergence  angle  of  the  beam  (a)  defines  a  triangle  inside  which 
reside  points  at  which  the  lower  intensity  of  the  flux  associated  with  not  being 
in  the  focus  plane  is  exactly  compensated  for  by  the  increased  amount  of  time 
they  are  illuminated  [black  diamond  (A)].  As  this  line  scans  to  create  a  2D 
image,  the  triangle  of  equal  total  illumination  becomes  an  octahedron  (B).  As 
the  2D  focus  plane  is  scanned  in  to  create  a  3D  image,  the  octahedrons  add 
up  to  produce  more  damage  near  the  center  of  the  raster,  where  the  planar  con¬ 
stant  bleach  pattern  is  thickest  (C  ).  It  should  be  noted  that,  because  of  the  rather 
slow  response  of  the  mechanical  scanning  mirrors,  a  considerably  higher  level 
( 1 0x-20x)  of  bleaching  will  occur  to  either  side  of  the  imaged  area  of  the  focus 
plane  where  the  horizontal  motion  of  the  mirrors  slows  down  and  reverses.  In 
some  confocal  instruments,  the  beam  is  blanked  during  this  retrace  period. 


will  receive  less.  The  reason  for  mentioning  these  simple  conse¬ 
quences  of  geometrical  optics  here  is  to  emphasize  the  difficulty 
of  devising  a  computer  program  to  correct  for  bleaching  artifacts 
even  if  the  bleach  rate  is  assumed  to  be  directly  proportional  to  the 
energy  deposited  and  the  dye  stays  in  one  place.  The  best  plan  is 
to  be  careful  not  to  waste  photons  and  thereby  minimize  the 
amount  of  bleaching  that  occurs. 

Although  the  bleaching  problem  may  be  no  worse  in  confocal 
microscopy  than  in  normal  fluorescence  microscopy,  it  is  still  a 
very  serious  problem.  In  fact,  it  is  not  unlikely  that,  particularly 
when  viewing  living  cells,  one’s  success  at  using  either  method 
will  depend  on  how  successful  one  is  at  collecting  and  recording 
as  great  a  fraction  as  possible  of  the  fluorescent  photons  produced 
and  at  recording  only  the  information  that  one  really  needs  while 
using  no  more  than  the  minimum  necessary  exposure.  We  explain 
how  to  do  this  in  the  sections  that  follow. 
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FIGURE  35.2.  Alignment  of  the  laser  in  the  back-focal  plane  (BFP)  of  the  objective  as  seen  using  a  45°  prism  that  allows  one  to  view  a  circular  target  situated 
in  the  BFP.  (A)  Properly  centered,  (C)  misaligned.  The  two  lower  images  show  the  bundle  of  light  rays  projected  into  the  specimen  under  each  alignment  con¬ 
dition.  These  images  were  made  by  photographing  the  fluorescent  light  emanating  from  a  cube  of  uranium-doped  glass  placed  in  the  specimen  plane  of  a  1 0x/0.45 
objective.  The  angles  involved  are  somewhat  smaller  than  expected  because  the  glass  has  a  very  high  index  of  refraction. 


GETTING  A  GOOD  CONFOCAL  IMAGE 

To  obtain  an  optimal  confocal  image,  a  fluorescent  specimen  must 
first  be  in  focus  under  the  right  type  of  objective:  that  is,  one  that 
is  designed  to  focus  into  a  medium  having  the  refractive  index  of 
your  specimen,  thereby  avoiding  problems  with  spherical  and 
chromatic  aberration  (see  Chapters  7  and  20,  this  volume).  Because 
of  the  optical  sectioning  property  on  the  confocal  microscope, 
specimens  that  are  far  out  of  focus  will  produce  no  signal!  If  you 
are  having  trouble  finding  the  specimen,  open  the  pinhole  all  the 
way,  focus  on  the  specimen,  then  close  it  down  again. 

Next,  the  scanhead  must  be  properly  aligned.  When  the  beam 
scans  over  the  center  of  the  raster  pattern,  the  light  from  the  laser 
should  be  both  on  axis  and  fill  the  BFP  of  the  objective,  and  the 
light  passing  up  from  the  focused  spot  to  the  PMTs  must  be  aligned 
to  pass  through  the  pinhole.  Once  alignment  has  been  achieved, 
the  pinhole  size  should  be  set  for  the  best  compromise  between 
signal  strength  and  xy-  and  z-resolution,  (for  details  see  Chapter 
22,  this  volume)  the  PMTs  should  be  adjusted  to  produce  an  appro¬ 
priate  signal  level  and  the  monitors  should  be  adjusted  so  that  the 
full  range  of  signal  can  be  viewed. 


Proper  alignment1  of  the  scanhead  first  requires  adjustment  of 
the  mirror(s)  that  direct(s)  the  laser  light  down  the  optical  path 
of  the  microscope.  A  45°  angle  prism  (or  even  just  a  small  piece 
of  lens-cleaning  tissue)  can  be  positioned  in  place  of  an  objective 
so  that  the  laser  light  path  can  be  viewed  with  respect  to  the  BFP 
(Fig.  35.2).  Although  the  angle  at  which  it  passes  the  BFP  is  pro¬ 
portional  to  the  instantaneous  position  of  the  beam  in  the  imaged 
plane,  the  laser  beam  should  always  pass  through  the  center  of  the 
BFP  and  then  be  focused  by  the  objective  to  form  a  symmetrical 
spot  at  the  specimen  plane  [Fig.  35.2(A,B)].  Misalignment  at  the 
BFP  may  mean  that  the  light  does  not  fill  the  pupil  of  the  objec¬ 
tive,  reducing  its  effective  resolution  and  causing  it  to  pass  through 
the  specimen  at  an  angle  [Fig.  35.2(C,D)]. 

Next,  a  standard  fluorescent  or  reflective  test  specimen  must 
be  used  to  align  the  scanhead  so  that  the  central  maximum  of  the 


1  Many  newer  confocal  microscopes  come  pre-aligned  and  there  are  no  user 
controls  for  adjusting  the  alignment  as  described  in  this  section.  However, 
some  of  the  tests  used  to  check  performance  can  still  be  used  to  see  if  you 
need  a  service  visit. 
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diffraction-limited  spot  (Airy  disk)  in  the  specimen  is  focused 
through  the  pinhole  where  it  can  be  detected  by  the  PMT.  When 
aligning  for  peak  signal,  it  may  be  helpful  to  change  the  display 
to  have  a  colorized  lookup  table  (LUT).  A  colorized  LUT  empha¬ 
sizes  small  changes  in  signal  intensity  because  the  eye  is  more  sen¬ 
sitive  to  absolute  hue  than  to  absolute  gray  level.  While  scanning, 
adjust  the  scanhead  mirrors  or  pinhole  location  to  maximize  the 
signal  received  by  the  PMT.  For  the  time  being,  you  need  only  be 
concerned  with  obtaining  the  brightest  image,  not  one  of  maximum 
quality.  It  is  advisable  to  make  the  mirror  adjustments  using  the 
objective  you  plan  to  use  for  your  final  high-quality  image. 
Because  wedge  errors  in  the  dichroic  beam-splitter  can  displace 
the  apparent  position  of  the  pinhole  (Chapter  9,  this  volume),  older 
instruments  will  require  re-alignment  of  the  scanhead  whenever 
the  dichroic  is  changed. 

Once  one  has  a  reasonable  image,  it  is  probably  worthwhile  to 
make  a  note  of  the  settings  of  all  the  user  controls.  These  can  later 
be  used  to  monitor  instrument  (and  operator!)  performance  and  to 
serve  as  starting  conditions  after  one  has  implemented  major 
service  or  development  changes. 

With  basic  alignment  complete,  you  may  find  it  interesting  to 
view  the  Airy  disk  in  the  pinhole  plane.  On  older  scopes  this  can 
be  accomplished  by  placing  a  small  CCD  camera  in  the  intermedi¬ 
ate  image  plane,  but  if  your  instrument  uses  a  larger,  iris-diaphragm 
aperture  rather  than  a  real  pinhole,  you  may  be  able  to  see  the  light 
directly  by  viewing  it  scattered  by  a  piece  of  paper  placed  in  front 
of  the  pinhole.  Figure  35.3  shows  the  sort  of  image  you  can  expect 
to  see  if  you  do  this  by  placing  a  piece  of  paper  in  the  second  filter 
well  of  a  Bio-Rad  MRC  series  scope  that  has  been  set  up  to  measure 
BSL  from  a  plane  mirror  surface  that  is  either  in  focus  [Fig. 
35.3(A)]  or  just  out  of  focus  [Fig.  35.3(B)],  The  in-focus  image 
shows  some  astigmatism,  probably  caused  by  imperfections  in  the 
beam-splitter.  It  is  informative  to  see  how  the  image  of  the  Airy 
disk  degrades  if  one  reduces  the  zoom  magnification,  a  process  that 
reveals  the  deleterious  effect  of  off-axis  aberrations  such  as  astig¬ 
matism,  curvature  of  field,  and  coma  (see  also  Chapter  11,  this 
volume).  One  can  also  use  this  setup  to  view  the  asymmetrical 
increase  in  Airy  disk  size  (and  decrease  in  resolution)  caused  by 
misaligning  the  illumination  in  the  objective  BFP  (as  in  Fig.  35.2). 

Actually,  viewing  the  Airy  disk  helps  to  make  clear  the  idea 
that  each  objective  has  an  optimum  pinhole  size  because  it 
becomes  evident  that  the  light  at  the  pinhole  plane  is  simply  a  mag¬ 
nified  view  of  the  light  in  the  image  plane  and  that,  therefore,  at  a 
fixed  NA,  the  size  of  the  image  of  the  spot  will  be  proportional  to 
the  magnification  of  the  objective.  A  pinhole  aperture  equal  to  the 
diameter  of  the  first  minimum  in  the  Airy  disk  (i.e.,  1  Airy  unit) 
will  pass  about  80%  of  the  in-focus  signal  and  still  have  a  bit  better 
xy-resolution  than  a  widefield  microscope.  Once  this  benchmark 
detector  aperture  is  known  for  a  given  optical  setup  (wavelength, 
NA,  magnification),  it  is  possible  to  make  informed  choices  about 
the  most  appropriate  aperture  for  any  other  objective. 

Specimen  preparation,  objective  lens,  pinhole  size  and  align¬ 
ment,  and  focus  plane  all  affect  the  amount  of  light  collected  from 
the  specimen.  Adjustment  of  the  display  monitor  and  the  PMT 
affect  the  image  viewed.  The  black  level  of  the  PMT  amplifier 
must  be  set  so  the  full  range  of  signal  can  be  detected.  This  can  be 
done  by  scanning  while  light  is  blocked  from  entering  the  PMT 
and  adjusting  the  black  level  until  the  rastered  area  becomes  just 
visible  compared  to  the  unscanned  part  of  the  monitor  screen.2 3 


2  When  the  black-level  control  is  set  correctly,  black  areas  will  register  as  2  to 

3  ADU  levels  out  of  255. 
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FIGURE  35.3.  The  Airy  disk  reflected  back  from  a  mirror  test  specimen  can 
be  viewed  directly  at  the  pinhole  plane  of  a  Bio-Rad  MRC  confocal  micro¬ 
scope  by  removing  the  second  filter  block  and  placing  a  piece  of  paper  into  the 
bottom  of  the  well.  (Turn  the  PMT  all  the  way  down  before  doing  this!)  (A) 
In  focus,  but  showing  slight  astigmatism,  probably  because  of  distortion  by  the 
dichroic;  (B)  slightly  out  of  focus.  Random  points  of  light  well  away  from  the 
optical  axis  defined  by  the  Airy  disk  represent  stray  light.  This  is  less  evident 
in  (A)  because  of  the  higher  relative  brightness  of  the  central  spot  and  because 
the  beam  dump  in  first  filter  block  had  been  improved  by  the  addition  of  a  piece 
of  black  velvet. 


Note  that  if  the  black  level  is  set  too  high,  some  intensity  values 
may  fall  below  the  voltage  corresponding  to  the  zero  value  of  the 
analog-to-digital  converter  (ADC)  and  so  not  be  recorded.  If,  after 
setting  the  black  level  in  this  way,  the  image  on  the  display  has 
insufficient  brightness,  this  can  usually  be  best  adjusted  by  modi¬ 
fying  the  display  LUT  in  the  computer. 

The  gain  of  the  detector  circuitry  (i.e.,  how  much  it  amplifies) 
is  changed  by  adjusting  the  accelerating  voltage  of  the  PMT.  In 
round  figures,  a  50  V  increase  in  the  accelerating  voltage  doubles 
the  gain.  Practically  speaking,  the  gain  controls  both  the  bright¬ 
ness  and  the  contrast  of  the  image.  When  it  is  properly  adjusted, 
the  pixels  in  the  brightest  areas  of  the  image  should  be  bright  white 
in  order  to  utilize  the  full  gray-scale  range  of  the  data  handling 
system.  However,  one  must  also  be  careful  not  to  saturate  the 
ADC  and  this  may  be  more  difficult  to  avoid  than  expected 
because,  in  fluorescence  confocal  microscopy,  the  brightest  signal 
may  represent  only  <16  counts/pixel.  Because  of  statistical  factors 
discussed  later  in  this  chapter,  an  average  value  of  16  counts/pixel 
really  implies  16  ±  4  counts/pixel.  Therefore,  if  one  is  to  avoid 
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truncating  data  representing  such  small  numbers  of  quantum 
events,  one  must  be  careful  that  the  highest  value  of  the  ADC 
output  (usually  255)  represents  not  the  average  value  of  the 
brightest  pixel  but  rather  an  unusually  high  measurement  of  this 
bright  value  (at  least  16  +  4  =  20  in  this  example).  In  other  words, 
if  the  gain  is  set  properly,  the  value  of  the  brightest  pixel  of  a  flu¬ 
orescent  image  resulting  from  the  average  of  several  frames3 
should  probably  be  no  more  than  200  to  220  and  not  255.  One  can 
check  the  gain  setting  by  plotting  a  histogram  of  pixel  intensities 
or  by  using  a  colorized  LUT  to  verify  that  the  brightest  pixels  in 
a  live  (not  averaged)  image  are  below  the  saturation  level  of  the 
digitizer. 

It  should  now  be  possible  to  obtain  a  reasonably  good  image 
of  the  specimen.  If,  however,  the  images  are  not  satisfactory,  one 
should  check  the  following  parameters: 

No  Signal 

•  Can  you  see  laser  light  of  the  appropriate  color  emerging  from 
the  objective?  Is  a  shutter  or  a  prism  obscuring  the  beam? 

•  Is  the  specimen  near  the  focus  plane? 

•  Is  the  computer  system  adjusted  to  display  the  signal  from  the 
PMT  that  is  actually  receiving  the  light  signal? 

•  Never  panic  and  turn  up  the  PMT  all  the  way  in  a  vain  search 
for  a  signal  that  does  not  exist  only  to  find  that  you  have 
damaged  the  PMT  by  overloading  it  when  you  finally  reset  the 
control  that  had  been  obscuring  the  signal! 

Low  Signal 

•  Do  small  adjustments  of  the  alignment  knobs  reduce  the 
signal  level  as  they  should  if  the  beam  is  properly  aligned? 

•  Is  the  laser  power  sufficient?  Are  neutral  density  filters  atten¬ 
uating  the  light  too  much?  Does  the  laser  cavity  need  retun¬ 
ing?  Are  the  optics  used  to  launch  the  laser  light  into  the  fiber 
correctly  aligned? 

•  Is  the  laser  line  appropriate  for  exciting  your  dye  and  are  the 
correct  filters  in  place? 

•  Is  the  specimen  stained  properly  and  has  an  anti-fade  agent 
been  included  in  the  mounting  media?  Check  by  viewing 
the  same  specimen  in  the  non-confocal  fluorescence  mode  if 
available. 

•  Is  the  sample  too  opaque  for  the  laser  light  to  reach  the  focus 
plane  and  for  the  emitted  light  to  exit  the  specimen  without 
excessive  scattering  losses? 

•  Are  the  settings  of  pinhole  size,  PMT  gain,  and  computer 
display  system  correct? 

Simultaneous  Detection  of  Backscattered  Light 
and  Fluorescence 

The  exercises  that  will  be  described  in  the  following  sections  are 
easier  to  perform  if  one  can  arrange  the  system  so  that  it  can  detect 
both  backscattered  light  (BSL  or  reflected  light)  and  fluorescent 
light  simultaneously  (Pawley  et  al.,  1993).  Such  a  system  is  dia¬ 
grammed  in  Figure  2.7  and  provides  the  optimal  removal  of  spec¬ 
ular  reflection  artifacts  from  optical  surfaces  above  the  objective. 
A  1/2-wave  plate  in  front  of  the  laser  is  used  to  rotate  the  polar¬ 
ization  plane  of  the  beam  so  that  a  suitable  fraction  of  it 
(0.8%-5.0%)  is  reflected  down  the  microscope  axis  by  an 


3  Averaging  several  frames  reduces  statistical  noise  and  means  that  the  number 
recorded  is  closer  to  the  real  intensity  value  for  each  pixel. 


uncoated,  clear  glass  beam-splitter  in  the  first  filter  block.  The  light 
then  passes  through  a  1/4-wave  plate  between  the  ocular  and  the 
objective  before  reaching  the  specimen.  Light  either  reflected 
and/or  emitted  from  the  specimen  returns  via  this  same  pathway, 
through  the  1/4-wave  plate  and  the  dichroic  beam-splitter  to  the 
second  filter  set.  This  system  is  constructed  to  pass  the  fluores¬ 
cence  signal  to  PMT  1  and  the  reflectance  signal  to  PMT  2  via  a 
rotatable  1/4-wave  plate,  analyzer,  and  laser-line  filter.  During 
setup,  the  rotatable  elements  of  the  second  filter  block  are  adjusted 
to  eliminate  axial  specular  reflections  and  the  lower  1/4- wave  plate 
is  rotated  to  obtain  a  maximum  signal  from  a  biological  specimen 
(Fig.  35.12  shows  a  stereo,  BSL  image  obtained  with  this  system). 
The  operation  of  a  similar  system  is  described  in  more  detail  in 
Chapter  17.  Because  this  system  uses  a  different  beam-splitter,  the 
mirrors  of  the  scanhead  will  need  to  be  re-adjusted  as  described 
above. 

NEW  CONTROLS 

Getting  the  best  out  of  a  confocal  fluorescence  microscope  requires 
consideration  of  several  matters  that  do  not  affect  normal  light  mi¬ 
croscopy.  In  order  to  extract  as  much  information  as  possible  from 
every  photon  emitted  by  the  specimen,  the  user  must  set  two  unfa¬ 
miliar  parameters  properly:  pinhole  size  and  zoom  magnification. 

Photon  Efficiency 

If  either  the  pinhole  size  or  the  zoom  setting  are  set  incorrectly, 
the  specimen  may  be  subjected  to  much  greater  levels  of  exciting 
light  than  should  be  necessary  to  produce  an  image  of  a  given 
quality.  This  will  increase  bleaching  unnecessarily. 

This  is  particularly  important  when  examining  living  speci¬ 
mens  containing  fluorescent  substances  because,  in  this  case,  the 
exciting  illumination  inevitably  produces  both  bleaching  and  cyto¬ 
toxicity  (Macintosh  et  al.,  1990;  Chapter  39,  this  volume ).  The 
bleaching  itself  will  be  more  severe  than  with  fixed  specimens 
because  on  living  specimens,  one  generally  cannot  use  anti-fade 
agents  (except,  perhaps,  oxygen  deprivation  or  Trolox).  In  general, 
the  biological  reliability  of  the  data  is  likely  to  be  inversely  pro¬ 
portional  to  the  laser  power  used.  In  other  words,  even  though 
using  more  light  may  make  the  image  appear  more  distinct,  it 
may  also  damage  the  object  that  the  image  is  supposed  to  repre¬ 
sent  so  seriously  that  the  data  is  merely  a  better  image  of  a  worse 
specimen  (see  Chapters  38  and  39,  this  volume ). 

Pinhole  Size 

The  advantage  of  the  confocal  method  of  image  formation  is  that 
light  emitted  or  scattered  into  the  objective  from  planes  other  than 
the  focus  plane  will  be  out  of  focus  when  it  reaches  the  pinhole 
plane  [Fig.  35.4(A)].  As  a  result,  most  of  it  will  not  pass  through 
the  pinhole  and,  to  the  extent  that  this  happens,  any  image  pro¬ 
duced  from  the  detected  signal  will  be  characteristic  of  only  a 
single  plane.  For  this  reason,  the  confocal  microscope  is  said  to 
make  optical  sections. 

The  diameter  of  the  aperture  in  front  of  the  photodetector  is 
usually  adjustable  so  that  it  can  be  matched  to  the  size  of  the  image 
of  the  Airy  disk  (rAlry)  that  the  objective  lens  projects  from  the 
excited  spot  in  the  specimen  to  the  pinhole  plane  [Fig.  35.4(B)]. 
The  diameter  of  this  disk  is  directly  proportional  to  the  total  mag¬ 
nification  between  the  object  and  the  pinhole  plane  (see  Chapter 
2,  this  volume)  and  through  the  Abbe  equation. 
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FIGURE  35.4.  Confocal  schematic.  (A)  Block  diagram  of  the  optical  compo¬ 
nents  of  a  fluorescent,  laser-scanning  confocal  microscope:  Short  wavelength 
blue  light  from  the  laser  passes  through  an  exciter  pinhole  and  filter  and  is  then 
reflected  by  the  dichroic  mirror  into  the  objective  lens.  The  lens  focuses  it  at  a 
spot  within  the  specimen.  Some  of  the  green  fluorescent  light  from  the  speci¬ 
men  returns  through  the  objective.  Because  of  its  longer  wavelength,  it  can  pass 
through  the  dichroic  and  barrier  filters,  and  it  comes  to  a  focus  at  the  plane  of 
the  detector  pinhole.  That  fluorescent  light  which  originated  from  the  plane-of- 
focus  passes  through  the  pinhole  to  the  detector  while  that  from  other  planes 
is  out-of-focus  at  the  pinhole  plane  and  so  is  selectively  excluded  from  the 
detector.  (B)  The  relationship  between  the  hole  in  the  pinhole  and  the  Airy  disk 
focused  at  this  plane. 


rAbbe=  0.6  WNA  (1) 

it  is  proportional  to  the  wavelength  X  and  inversely  proportional 
to  the  NA  of  the  objective.  The  area  of  the  central  spot  of  the 
Airy  disk  at  the  pinhole  plane  increases  by  a  factor  of  over  12 
if,  for  example,  a  lOx  NA  0.5  lens  is  replaced  by  a  lOOx  NA  1.3 
lens  (assuming  that  both  are  used  in  such  a  way  that  their  BFPs 


are  fully  illuminated).  Because  the  signal  that  passes  through 
the  aperture  from  a  non-planar  specimen  is  generally  propor¬ 
tional  to  the  pinhole  area,  an  incorrect  adjustment  of  the  pinhole 
size  can  significantly  reduce  the  effective  sensitivity  of  the 
microscope. 

However,  there  is  a  second  constraint  on  the  choice  of  pinhole 
size.  Because  almost  all  of  the  light  originating  from  the  plane  of 
focus  will  pass  through  a  properly-aligned  pinhole,  1  Airy  unit  in 
size,  one  might  expect  that  there  could  be  no  reason  for  ever 
wanting  to  use  any  other  aperture  size.  This  might  be  the  case  if 
the  diameter  of  the  pinhole  did  not  also  affect  the  spatial  resolu¬ 
tion  of  the  microscope  in  both  the  xy-plane  and,  to  a  lesser  extent, 
in  z.  If  the  pinhole  is  made  very  small  (<0.1  Airy  units),  the  xy- 
resolution  of  the  instrument  is  improved  by  -40%  over  that  set  by 
the  Abbe  limit,  but  only  at  the  cost  of  reducing  the  signal  level  by 
95%.  As  the  pinhole  is  made  larger,  it  begins  to  accept  more  light 
while  the  xy-resolution  is  reduced.  When  it  equals  1  Airy  unit,  80% 
of  the  light  originating  from  the  focus  plane  is  accepted,  while  a 
10%  resolution  gain  is  still  being  realized.  On  the  other  hand,  when 
the  pinhole  is  opened  still  more,  any  extra  light  that  it  accepts  must 
be  that  originating  from  either  above  or  below  the  focus  plane,  and 
this  reduces  the  optical  sectioning  effect  as  well  as  providing  more 
photons. 

Therefore,  depending  on  the  particulars  of  the  experiment,  it 
may  be  preferable  to  either  close  the  pinhole  down  to  improve  xy 
spatial  resolution  or  open  it  up  in  order  to  collect  more  signal  at 
somewhat  reduced  z-resolution.  The  latter  is  particularly  true  when 
viewing  sensitive  living  specimens  because,  as  these  will  usually 
not  tolerate  the  multiple  exposures  needed  to  record  many,  closely- 
spaced  optical  sections  in  any  case,  the  z-resolution  is  bound  to  be 
compromised  by  inadequate  sampling  (see  below)  unless  the 
optical  section  is  broadened.  In  this  case,  lower  z-resolution  will 
actually  improve  the  image. 

For  most  specimens  protected  by  anti-fade  agents  (Chapters  18 
and  39,  this  volume),  the  best  compromise  between  sensitivity  and 
spatial  resolution  is  found  by  setting  the  aperture  to  be  equal  to  0.5 
Airy  units,  the  diameter  at  which  the  Airy  disk  reaches  50%  of  its 
peak  intensity.  At  this  setting,  about  60%  of  the  light  from  the  plane 
of  focus  will  reach  the  detector,  and  the  xy-resolution  will  still  be 
a  bit  better  than  that  produced  by  the  same  optics  when  used  in  a 
non-confocal  manner. 

It  should  be  emphasized  that,  as  the  presence  of  epi- 
fluorescence  and  DIC  attachments  on  an  older  non-infinity  micro¬ 
scope  may  increase  the  effective  magnification  of  the  objective 
lens,  these  figures  should  only  be  used  as  a  guide. 


Stray  Light 

Stray  light  can  reach  the  pinhole  plane  in  many  ways  but.  when 
imaging  in  BSL  mode,  the  most  common  is  laser  light  reflected 
back  by  the  beam  dump  and  then  up  the  optical  axis  by  the  upper 
surface  of  the  beam-splitter  or  scattered  in  all  directions  by  dust 
on  optical  and  mirror  surfaces.  Most  of  the  diffuse  off-axis  scat¬ 
tering  visible  in  Figure  35.3  is  from  this  source,  but  one  must  also 
be  wary  of  room  light  entering  through  the  objective  or  through 
holes  in  the  light-tight  box  surrounding  the  detection  optics.  Stray 
light  can  sometimes  be  reduced  by  improving  the  efficiency  of  the 
beam  dump  (see  Appendix  1,  this  volume)  but,  in  general,  one  can 
only  control  for  it  by  measuring  the  PMT  output  for  different 
pinhole  sizes  without  a  reflective  specimen  in  the  focus  plane  and 
then  subtracting  these  values  from  the  corresponding  readings 
made  with  the  reflecting  surface  in  focus. 
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Is  the  Back-Focal  Plane  Filled? 

Some  early  confocal  microscopes  applied  the  laser  output  directly 
to  the  BFP  without  first  expanding  it  (Chapter  9,  this  volume). 
Though  the  exact  size  of  the  beam  depends  on  the  details  of  the 
design  of  the  laser,  it  is  usually  ~1  mm.  While  this  is  a  fairly  good 
match  to  a  lOOx  NA  0.8  objective,  it  is  too  small  by  a  factor  of  10 
for  a  lOx  NA  0.45  objective.  If  such  a  beam  is  used  with  such  a 
lens,  the  effective  NA  on  the  illumination  side  will  be  only  10% 
of  that  marked  on  the  lens,  and  the  xy-resolution  will  be  lOx  larger. 
On  air  lenses,  one  can  check  the  filling  of  the  BFP  by  turning  on 
the  microscope  with  no  specimen  present  and  either  parking  the 
beam  on  axis  or  going  to  the  highest  possible  zoom  and  then 
observing  the  pattern  that  the  light  leaving  the  objective  makes 
when  it  falls  on  a  screen  (a  piece  of  lens  tissue  will  do)  held  a  few 
centimeters  away  from  the  front  of  the  lens.  If  the  BFP  is  over¬ 
filled,  the  boundary  of  the  bright  ring  of  light  will  have  a  sharp 
edge  to  it.  (For  a  listing  of  the  dimensions  of  the  BFP  for  repre¬ 
sentative  objectives  see  Table  9.2,  this  volume.) 

Pinhole  Summary 

•  Make  the  pinhole  a  little  smaller  than  the  first  dark  ring  of  the 
Airy  disk  (-0.8  Airy  unit). 

•  On  a  non-planar  (bulk)  specimen,  the  signal  level  is  gener¬ 
ally  proportional  to  the  area  of  the  detector  pinhole. 

•  At  a  fixed  NA,  the  optimal  pinhole  diameter  is  proportional  to 
the  magnification  of  the  objective,  that  is,  the  diameter  appro¬ 
priate  for  40x  NA  1.3  is  only  40%  as  large  as  that  proper  for 
lOOx,  NA  1.3. 

•  Not  adjusting  for  this  fact  can  reduce  signal  6.25x. 

Circumstances  may  often  require  deviations  from  the  bench¬ 
mark  settings  of  the  pinhole  size  given  above,  but  this  should  only 
be  done  after  considering  the  potential  cost  in  terms  of  biological 
reliability  (smaller  pinholes  give  better  ^-resolution  but  less  signal, 
meaning  the  specimen  must  endure  more  excitation  light). 

Statistical  Considerations  in 
Confocal  Microscopy 

The  visibility  of  small  objects  viewed  in  a  confocal  microscope 
depends  not  only  on  the  optics  of  the  beam-forming  system  but 
also  on  detecting  a  sufficient  number  of  photons  to  permit  the 
signal  from  the  feature  to  be  statistically  discriminated  from  that 
of  the  background.  The  matter  of  visibility  is  complicated  by  the 
fact  that  the  images  of  small  features  are  transmitted  through  the 
optical  system  with  less  contrast  than  are  the  images  of  large  fea¬ 
tures.  In  other  words,  one  must  detect  and  record  an  image  with 
more  statistically-defined  gray  levels  if  one  expects  to  make  small 
features  visible. 

A  confocal  image  is  recorded  by  detecting  the  light  signal  from 
each  point  in  the  scanned  raster  and  storing  a  number  proportional 
to  this  light  intensity  in  a  computer.  The  image  can  be  seen  by  dis¬ 
playing  these  numbers  as  distinct  gray  levels  at  the  appropriate 
location  on  a  monitor  (see  Chapter  4,  this  volume). 

The  PMT  detector  used  in  most  laser  confocal  microscopes  is 
extremely  sensitive.  It  is  capable  of  producing  one  recordable 
event  (a  photoelectron,  PE)  for  every  5  to  10  photons  striking  it, 
and  as  long  as  it  does  not  get  too  warm  and  is  shielded  from  all 
sources  of  extraneous  light,  it  has  almost  no  dark  current.  Using 
the  fast  photon-counting  mode  found  on  some  instruments,  useful 
images  can  be  recorded  in  which  the  brightest  pixel  may  represent 


only  10  PE  (-100  photons)  while  the  dimmest  pixel  may  average 
only  0.1  PE  (1  photon).4 

Even  though  this  performance  is  impressive,  it  is  important  to 
remember  that  the  visibility  of  features  in  any  image  is  limited  by 
a  relationship  between  three  factors:  the  contrast  intrinsic  to  the 
signal  from  the  feature,  the  transfer  of  this  contrast  by  the  optics 
to  the  detector,  and  the  statistical  uncertainty  associated  with  the 
number  of  photons  detected  from  it. 

When  one  records  an  image  photographically,  the  number  of 
photons  involved  is  so  much  greater  and  the  chance  of  detecting 
an  individual  photon  is  so  much  less,  that  the  topic  of  photon  sta¬ 
tistics  is  seldom  explicitly  considered.  In  the  discussion  that 
follows,  contrast  refers  to  a  measure  of  the  variation  of  the  signal 
intensity  within  the  image. 

The  term  “photon  statistics”  recognizes  the  fact  that  a  beam  of 
light  is  actually  composed  of  photons,  which,  as  elementary  parti¬ 
cles,  are  governed  only  by  statistical  laws.  If,  after  many  mea¬ 
surements,  the  mean  brightness  of  a  given  pixel  is  found  to  be  25 
PE,  random  statistical  variations  ensure  that  the  actual  number 
counted  on  any  given  measurement  will  be  somewhere  in  the  range 
of  ±1  standard  deviation  (SD)  from  this  mean  value  only  67%  of 
the  time.  As  photons  obey  Poisson  statistics,  the  standard  devia¬ 
tion  is  simply  the  square  root  of  the  mean,  so  67%  of  the  mea¬ 
surements  will  be  in  the  range  of  20  to  30  counts/pixel,  and  33% 
of  the  measurements  will  be  outside  this  range! 

As  a  practical  matter,  a  feature  that  is  one  pixel  in  size  will 
only  be  recognizable  or  visible  above  the  statistical  noise  present 
in  the  surrounding  pixels  if  it  differs  from  the  mean  of  the  back¬ 
ground  by  5  SD  (Rose,  1948).  In  the  case  noted  above,  the  25- 
count  single-pixel  feature  would  only  be  visible  if  the  average 
background  signal  level  was  about  1  count/pixel.  In  other  words, 
in  order  to  produce  a  visible  feature  in  the  image,  the  staining  of 
an  isolated  bright  pixel  must  be  sufficiently  intense  that,  even  after 
its  contrast  has  been  reduced  by  the  optical  system,  it  can  still 
produce  a  detectable  signal  that  is  ~25x  larger  than  that  of  its  sur¬ 
roundings.  The  effects  of  this  statistical  limitation  can  be  seen  in 
Figure  35.5,  in  which  part  of  the  test  specimen  has  been  imaged 
with  an  average  of  2,  4,  8,  and  16  counts/pixel  in  the  bright  fea¬ 
tures.  We  must  emphasis  that  the  improved  visibility  per  se  of  the 
bars  in  the  middle  two  patches  is  not  caused  by  improved  optical 
resolution,  but  merely  by  counting  enough  particles  so  that  the  fea¬ 
tures  of  smaller  size  can  be  recognized. 


4  Novice  confocal  users,  having  made  an  intensity  histogram  of  one  of  their 
images  and  having  found  that  at  least  a  few  pixels  have  every  one  of  the  256 
intensities  possible  in  an  8-bit  image,  may  jump  to  the  conclusion  that  the 
stored  number  is  equal  to  the  number  of  detected  photons.  Although  not  quite 
impossible,  this  situation  is  extremely  unlikely  to  be  true.  The  misunder¬ 
standing  is  related  to  the  role  of  the  multiplicative  noise  that  characterizes  the 
performance  of  all  PMTs.  Once  a  photon  is  absorbed  and  a  free  photoelec¬ 
tron  (PE)  produced,  the  PE  is  attracted  to  the  first  dynode  where  it  collides 
producing,  say,  four  secondary  electrons  (SE),  which  then  proceed  to  the 
second  dynode.  As  Poisson  statistics  controls  this  process,  on  many  repeti¬ 
tions,  4  ±  2SE  will  be  produced.  Because  a  PE  that  produces  6SE  that  strike 
the  next  dynode  is  likely  to  produce  a  larger  final  pulse  at  the  far  end  of  the 
dynode  chain  than  one  that  produces  only  2  SE,  it  becomes  clear  that  all  PE 
are  not  amplified  equally.  As  a  result,  given  suitable  signal  levels  and  con¬ 
trast  settings,  even  an  image  that  actually  consists  of  only  two  signal  intensi¬ 
ties  (0  and  1  +  1)  can  yield  an  image  histogram  showing  all  possible  stored 
values  from  1  to  255.  Multiplicative  noise  masks  the  fact  that  fluorescent  con¬ 
focal  images  usually  represent  the  detection  of  very  few  photons/pixel.  If  it 
did  not,  and  if  the  system  gain  was  scaled  so  that  1  PE  =  1  unit  in  the  memory, 
the  posterized  nature  of  the  image  would  warn  the  viewer  against  having  great 
confidence  in  the  values  actually  recorded. 
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FIGURE  35.5.  Confocal  images  made  with  few  photons.  Part  of  the  test  specimen  has  been  imaged  with  an  average  of  (A)  2,  (B)  4,  (C)  8,  and  (D)  16  counts/pixel. 
The  improved  visibility  of  the  bars  in  the  smaller  patches  is  not  caused  by  improved  optical  resolution  but  merely  by  counting  enough  particles  so  that  these 
low-contrast  features  can  be  recognized. 


Fortunately,  as  is  shown  in  Chapter  4,  with  Nyquist  sampling 
(see  also  next  section)  all  real  features  cover  at  least  16  to  25  pixels 
in  a  two-dimensional  (2D)  image.  Under  these  circumstances,  it  is 
the  total  number  of  counts  recorded  for  the  whole  feature  that  is 
relevant  to  its  visibility.  Images  containing  six  gray  levels  (corre¬ 
sponding  to  0,  1,  4,  9,  16,  and  25  events)  can  be  usefully  recorded 
with  ~25  counts  in  the  brightest  pixel.  The  number  of  photons 
needed  to  see  the  bars  in  Figure  35.5  is  less  than  the  25  PE  used 
in  the  example  above  because  the  features  are  much  more  than  one 
pixel  in  size. 

The  limitations  imposed  by  counting  statistics  on  feature  vis¬ 
ibility  are  immutable  and  are  not  affected  by  linear  image  pro¬ 
cessing  (i.e.,  contrast  and  brightness  settings).  Therefore,  when 
imaging  specimens  that  have  a  stain  contrast  that  is  lower  than 
25  : 1,  it  is  often  necessary  to  average  a  number  of  scans  to  produce 
a  useful  image.  This  process  does  not  literally  “remove  the  noise,” 
it  merely  permits  more  photons  to  be  utilized,  and  this  reduces 
the  statistical  variations  in  each  of  the  thousands  of  the  intensity 
measurements  that  make  up  the  image.  Because  this  process 
obeys  Poisson  statistics  and  the  ratio  of  signal-to-noise  (S/N), 
S/N  =  «/Vn  =  Vn,  and  counting  4x  as  many  photons,  will  double 
the  S/N  making  it  possible  to  detect  features  in  the  image  having 
only  half  as  much  contrast. 

The  Importance  of  Pixel  Size 

A  pixel  is  that  part  of  a  digital  image  whose  intensity  can  be  accu¬ 
rately  represented  by  a  single  number.  Normally,  each  horizontal 
line  of  the  rectangular  scanning  raster  is  divided  into  either  512, 
768,  or  1024  subdivisions,  and  each  one  of  these  subunits  repre¬ 
sents  one  pixel.  The  area  of  the  specimen  that  can  be  properly  rep¬ 
resented  by  a  single  pixel  varies  with  the  magnification  and  NA  of 


the  objective  and  on  the  adjustment  of  the  zoom  magnification. 
The  zoom  setting  controls  the  magnitude  of  the  current  waveforms 
supplied  to  the  scanning  mirrors  and  hence  the  size  of  the  raster 
scanned  on  the  specimen.  Smaller  currents  make  the  light  beam 
scan  over  smaller  areas  of  the  specimen,  producing  a  higher  mag¬ 
nification  in  the  final  image  as  displayed  on  the  computer  screen; 
larger  scan  currents  produce  lower  magnification  as  the  raster 
scanned  on  the  object  is  larger  in  area. 

Although  the  ability  to  arbitrarily  change  the  magnification  in 
this  way  is  usually  seen  as  a  great  convenience  in  terms  of  being 
able  to  fit  the  scanned  area  to  the  size  of  the  object  of  interest,  there 
can  be  hidden  dangers  associated  with  the  incautious  use  of  this 
control.  As  there  is  usually  a  fixed  ratio  between  the  size  of  the 
raster  and  the  size  of  an  individual  pixel,  adjusting  the  zoom 
control  changes  the  area  of  the  specimen  represented  by  a  single 
intensity  value.  However,  all  possible  pixel  sizes  are  not  equally 
suitable  for  recording  a  set  of  digital  data  that  retains  all  of  the 
spatial  information  present  in  the  original  analog  image. 

The  Nyquist  Sampling  Theorem  states  that,  when  a  continu¬ 
ous,  analog  image  is  digitized,  the  information  content  of  the  signal 
will  be  retained  only  if  the  diameter  of  the  area  represented  by  each 
pixel  (referred  to  the  specimen)  is  at  least  2.3x  smaller  in  linear 
dimension  than  the  resolution  limit  of  the  optical  system  (in  x,  y, 
and  z!).5 


5  In  various  chapters  in  this  book,  the  optimal  number  of  samples/resolution- 
element  varies  between  2.3  and  4.0.  Although  reasonable  arguments  can  be 
made  for  all  these  numbers,  it  seems  safe  to  say  that  the  larger  ones  are  appro¬ 
priate  when  the  noise  level  in  the  image  is  low  and  when  viewing  small  peri¬ 
odic  objects.  The  smaller  numbers  are  suitable  for  most  confocal  fluorescence 
images,  because  in  these  images,  the  high  noise  level  prevents  one  from 
seeing  small,  low-contrast  features.  Consequently,  the  actual  resolution  may 
be  considerably  less  than  that  predicted  by  the  Abbe  equation. 
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The  optical  resolution  limit  in  the  xy-plane  is  set  by  the  Abbe 
equation  (Eq.  1).  This  implies  that,  for  each  X  and  objective  lens 
used,  there  is  an  optimal  setting  for  the  zoom  control.  In  the  same 
way,  the  interplane  sampling  interval  in  a  3D  data  set  should 
ideally  be  a  bit  less  than  half  the  z-resolution.  Taking  200  nm  as  a 
typical  value  for  rAbbe  for  a  high-resolution,  NA  1.4  objective,  a 
pixel  width  of  200/2.3  =  80  nm  is  needed  to  properly  sample  the 
data  in  the  xy-plane,  while  the  interplane  spacing  for  the  same 
optical  conditions  should  be  about  3x  larger,  or  240  nm.  This 
means  that  the  blob  in  the  image  that  represents  a  point  object  will 
always  be  at  least  5  pixels  wide  and  5  planes  in  height.  It  also 
means  that  the  diameter  of  the  probing  beam  is  ~5  times  larger 
than  the  interline  spacing. 

Measuring  Pixel  Size 

Increasingly,  the  control  software  of  the  confocal  microscope  cal¬ 
culates  and  displays  the  pixel  size  currently  in  effect.  If  this  is  not 
so  on  your  instrument,  it  is  easy  to  calculate  the  pixel  size  as  long 
as  you  know  the  dimensions  of  your  raster.  lust  image  a  stage 
micrometer  (available  from  major  microscope  manufacturers  —  a 
hemacytometer  makes  an  inexpensive  substitute  for  magnification 
calibrations),  determine  the  actual  width  of  the  field  of  view  in 
micrometers  by  matching  it  to  the  calibrated  lines  on  the  test  spec¬ 
imen,  and  divide  this  number  by  the  number  of  pixels  in  a  line. 

Alternatively,  assuming  that  the  service  engineer  calibrated  the 
x-  and  y-magnification  when  the  instrument  was  installed  and  that 
you  have  entered  into  the  computer  the  NA  and  magnification  of 
the  objective  in  use,  you  can  display  a  scale  bar,  and  by  measur¬ 
ing  both  it  and  the  entire  screen  with  a  ruler,  you  can  again  find 
the  field  width  and  make  the  same  calculation. 

Finally,  if  your  system  allows  you  to  measure  the  length  of 
the  scale  bar  in  pixels,  you  can  use  this  information  to  calculate 
the  dimensions  of  the  pixel.  To  meet  the  requirements  set  by  the 
Nyquist  criterion  for  a  NA  1.4  objective,  a  5  pm  bar  should  be 
5  pm/0.08  pm  =  62  pixels  long. 

Adjust  the  zoom  setting  until  this  criterion  is  met.  Note  that 
pixel  size,  referred  to  the  object,  is  inversely  proportional  to 
zoom  magnification. 

It  should  be  noted  that,  contrary  to  much  legend,  satisfying  the 
sampling  criterion  often  requires  a  higher  zoom  setting  for  low 
magnification  objectives  than  for  those  with  high  magnification. 
The  reason  for  this  is  that  although  a  lOOx  NA  1.2  lens  has  three 
times  the  resolution  of  a  lOx  NA  0.4  lens,  the  latter  produces  an 
image  in  which  all  features  are  lOx  smaller,  so  the  smallest  visible 
feature  (an  Airy  disk)  will  be  3.3x  smaller  in  any  image  plane  when 
using  the  lOx  lens. 

The  exact  mirror  movements  that  produce  a  given  pixel  size 
will  vary  somewhat  with  the  field  of  view  that  characterizes  the 
microscope  being  used,  with  the  way  the  instrument  has  been 
adjusted,  and  (on  older  instruments)  with  the  presence  of  acces¬ 
sories,  such  as  epi-fluorescence  or  differential  interference  contrast 
(DIC)  attachments,  that  increase  the  effective  magnification  of 
the  objective  lens.  As  a  result,  the  scanning  system  must  be 
re-calibrated  if  it  is  moved  from  one  type  of  microscope  to 
another. 

Over-Sampling  and  Under-Sampling 

Now  that  we  have  determined  the  correct  zoom  setting  needed  to 
record  the  full  resolution  in  your  image  from  the  sampling  point 
of  view,  we  should  point  out  that  there  may  be  circumstances 
where  other  settings  may  be  more  appropriate. 


Under-sampling 

Using  a  lower  zoom  setting  has  the  advantage  of  reducing  the 
bleach  rate  and  increasing  the  field  of  view:  bleaching  rate  = 
fc(zoom  factor)2.  On  living  specimens,  therefore,  one  may  wish  to 
work  at  a  lower  zoom  setting  to  preserve  the  specimen.  Under 
these  circumstances,  you  will  lose  little  more  in  terms  of  image 
resolution  if  you  also  open  the  detector  aperture  somewhat.  This 
will  make  the  z-resolution  somewhat  worse,  but  as  you  will  prob¬ 
ably  also  have  decided  to  image  fewer  optical  sections,  the  lower 
z-resolution  will  actually  assist  in  coming  closer  to  the  Nyquist  cri¬ 
terion  for  sampling  in  z.  It  will  also  permit  more  signal  to  be 
recorded  at  each  pixel.  As  a  result  of  this  under-sampling,  you  can 
get  sufficient  image  quality  using  less  laser  power  (i.e.,  use  a 
darker  neutral  density  (ND)  filter). 

Over-sampling 

Using  a  zoom  setting  higher  than  that  needed  to  satisfy  the  Nyquist 
criterion  leads  to  over-sampling.  Although  over-sampling  implies 
a  smaller  field  of  view  and  causes  more  bleaching  in  fluorescent 
specimens,  it  provides  a  larger  image  that  may  be  easier  to  view 
and  it  will  provide  data  that  are  marginally  more  suitable  for  image 
deconvolution,  a  digital  image  processing  technique  that  can  be 
used  to  increase  image  contrast  and  reduce  noise  (see  next  section). 
Over-sampling  is  also  needed  when  imaging  a  periodic  specimen 
having  features  near  the  resolution  limit. 

Nyquist  Reconstruction  and  Deconvolution 

It  is  often  forgotten  that  Nyquist  digital  sampling  has  two  parts: 
first  digitizing  and  recording  the  digital  data  and  then  reconstruct¬ 
ing  the  analog  output  from  this  data.  The  latter  process  imposes 
important  limits  on  digital  microscopy.  Because  the  Abbe  equation 
places  an  absolute  limit  on  the  resolution  of  microscopical  image 
data,  it  is  relatively  easy  to  calculate  the  proper  size  of  a  pixel  if 
you  know  X  and  NA.  However,  Nyquist  reconstruction  requires 
that  these  same  resolution  bandwidth  limits  also  be  applied  when 
the  digital  data  is  turned  back  into  an  image.  Of  course,  it  is  easy 
to  assume  that,  because  no  information  regarding  features  smaller 
than  the  resolution  limit  of  the  optical  system  can  emerge  from  the 
microscope,  such  information  cannot  possibly  be  present  in  the 
data  and  therefore,  that  it  need  not  be  removed.  Sadly,  this  is  not 
so.  Because  pixel-to-pixel  intensity  variations  associated  with 
measurement  and  Poisson  noise  are  added  to  the  signal  before  it 
is  digitized  and  because  these  are  often  big  enough  to  produce  con¬ 
trast  comparable  to  that  produced  by  variations  in  dye  concentra¬ 
tion  in  the  specimen,  Nyquist-sampled  confocal  data  sets  always 
include  single-pixel  noise  that  appears  to  represent  features  4x  to 
5x  smaller  than  the  optical  system  could  possibly  have  imaged. 

This  spurious  data  can  only  be  eliminated  by  deconvolving  the 
raw  digital  data  using  any  reasonable  point  spread  function  (PSF). 
Although  deconvolution  is  usually  thought  of  primarily  as  a  means 
of  removing  out-of-focus  blur  from  3D  widefield  data  sets,  the 
process  has  many  other  useful  functions.  In  the  present  context, 
the  chief  among  these  is  that  deconvolution  effectively  suppresses 
spatial  frequencies  above  the  bandwidth  of  the  microscope 
optics. 

This  idea  is  important  enough  to  be  repeated:  Poisson  noise 
can  introduce  features  into  the  data  that  appear  to  be  -1  pixel 
in  size  when  the  smallest  Nyquist-sampled  feature  should  be 
>5  pixels  across.  These  noise  features  can  and  should  be  elim¬ 
inated  by  deconvolving  the  data,  preferably  in  3D. 

As  an  aside,  because  the  confocal  PSF  can  be  approximated  as 
a  3D  Gaussian  blob,  3D  Gaussian  filtering  has  almost  the  same 
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FIGURE  35.6.  The  relationship  between  resolution  and  the  number  of  voxels 
needed  to  Nyquyist-sample  the  point-spread  function.  According  to  standard 
light  microscopy  theory,  the  Abbe  resolution,  8,  is  defined  as  the  radius  of  the 
first  dark  ring  of  the  Airy  disk.  In  confocal  microscopy,  this  defines  a  blob  with 
a  diameter  of  28.  If  we  assume  that  a  pixel  is  8/2,  then  the  2D  image  of  a  point 
object  will  put  signal  into  at  least  12  to  16  bright  pixels  in  the  focus  plane.  The 
3D  image  of  a  point  requires  collecting  signal  from  at  least  four  planes,  sepa¬ 
rated  by  a  distance  equal  to  one  half  of  the  z-resolution.  As  a  consequence  of 
these  factors,  the  3D  image  of  a  point  object  requires  measuring  signal  in  50 
to  100  voxels.  Averaging  signal  over  these  voxels  (e.g.,  by  deconvolution)  does 
not  reduce  spatial  resolution  but  does  greatly  improve  the  S/N  of  the  resulting 
data.  (See  Chapter  25,  this  volume.) 


effect  as  a  full-scale  3D  deconvolution,  as  least  in  terms  of  sup¬ 
pressing  noise  features. 

Filtering  the  data  in  this  way  has  another  benefit:  it  effectively 
averages  the  voxel  intensity  data  over  the  number  of  voxels  con¬ 
taining  significant  counts  in  the  Gaussian  blob  that  represents  the 
PSF.  If  we  estimate  that  the  blob  is  ~5  pixels  wide  in  x ,  y,  and  z 
(see  Fig.  35.6),  then  it  will  have  significant  counts  in  5  x  5  x  5  = 
125  voxels.  Therefore,  3D  deconvolution  will  effectively  average 
out  Poisson  noise  over  —125  voxels:  a  significant  factor  (Chapters 
19  and  25,  this  volume)\ 

Pixel  Size  Summary 

To  preserve  all  of  the  information  in  your  data,  you  must  use  a  pixel 
(or,  in  three  dimensions,  a  voxel)  size  at  least  2.3x  smaller  than  the 
Abbe  resolution  limit  of  your  optical  system  in  ( x ,  y,  and  z!!).  This 
means  that,  for  each  objective,  there  is  an  optimal  setting  for  both 
the  zoom  control  and  the  interplane  spacing  of  3D  data  sets. 


•  The  amount  of  light  passing  through  the  specimen  is  not 
affected  by  the  zoom  setting. 

•  At  high  zoom  settings,  the  confocal  microscope  can  easily  illu¬ 
minate  the  specimen  with  an  intensity  (and  a  bleaching  rate!) 
that  is  lOOx  greater  than  in  a  normal  fluorescence  microscope 
using  a  Hg  arc  source. 

•  Fortunately,  because  the  PMT  detector  is  more  sensitive  than 
film,  less  illumination  is  actually  required  to  record  a  usable 
confocal  image. 

•  Set  the  zoom  to  produce  a  pixel  size  satisfying  the  sampling 
criterion  for  the  lens  in  use. 

•  Don’t  forget  to  deconvolve  (or  3D  Gaussian  filter)  your  data 
before  viewing  it. 

USING  A  TEST  SPECIMEN 
Why  Use  a  Test  Specimen? 

The  tendency  to  use  tried-and-true  specimens  when  testing  or 
learning  how  to  operate  the  confocal  microscope  is  so  marked  that 
it  would  be  fruitless  to  recommend  against  it.  However,  it  does 
have  some  disadvantages,  and  the  chief  of  these  is  that  one  prob¬ 
ably  really  does  not  know  what  a  good  confocal  image  of  such  a 
specimen  would  look  like.  This  is  because  any  stained  biological 
specimen  is  probably  too  complex  to  understand  so  well  that  one 
can  predict  what  a  3D  image  of  it  should  look  like  and  then  adjust 
the  instrument  until  this  result  is  attained.  Although  we  may  have 
prepared  hundreds  of  similar  specimens  and  may  recognize  some 
as  successes  and  others  as  failures,  we  seldom  really  know  in  a 
quantitative  way  either  the  exact  size  of  any  of  the  specific  fea¬ 
tures  that  they  contain,  how  much  they  have  been  stained,  or  how 
much  this  stain  may  have  faded  or  bleached  (Chapters  16  and  39, 
this  volume).  We  may  believe  that  such  a  specimen  possesses  a 
particular  3D  structure  when,  if  fact,  this  structure  has  been  lost 
when  it  was  inadvertently  flattened  during  specimen  preparation 
(Chapter  18,  this  volume).  In  short,  such  a  specimen  is  not  a  test 
specimen,  and  one  cannot  really  use  it  to  measure  either  your  own 
skill  or  the  performance  of  the  microscope. 

Description  of  the  Test  Specimen 

Microscope  manufacturers  have  long  produced  a  variety  of  test 
specimens  for  measuring  the  performance  of  their  instruments.  The 
most  common  of  these  is  the  stage  micrometer,  which  usually  con¬ 
sists  of  a  graticle  etched  into  a  metal  film  on  the  lower  side  of  a 
coverslip.  Such  patterns  can  be  very  useful  for  checking  magnifi¬ 
cation  but,  because  they  are  made  using  light  optics,  the  finest 
spacings  that  can  be  produced  on  them  are  ~1  to  2  pm,  and  this  is 
not  fine  enough  to  really  test  the  ultimate  resolution  of  a  good 
optical  microscope.  To  fill  this  void,  we  followed  the  lead  of  Old- 
enbourg  and  Inoue  (Chapter  1,  this  volume)  and  designed  a  test 
specimen  that  we  then  fabricated  at  the  National  Nanofabrication 
Facility  at  Cornell  University  using  electron-beam  lithography 
(Pawley  et  al.,  1993). 6  The  patterns  were  etched  into  a  50nm  A1 
film  on  the  lower  side  of  a  1  x  1cm  piece  of  #1.5  coverslip  and 


6  Similar  test  specimens  may  soon  be  available  again  from  Louie  Kerr,  at  the 
Marine  Biological  Laboratory,  Woods  Hole,  MA.  Other  test  specimens  are 
discussed  in  Chapter  36. 
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FIGURE  35.7.  Scanning  electron  micro¬ 
graphs  of  a  confocal  test  specimen  fabri¬ 
cated  at  the  National  Nanofabrication 
Facility  at  Cornell  University  using  electron- 
beam  lithography.  The  pattern  is  etched  into 
a  50  nm  A1  film  on  the  lower  side  of  a  1  x  1 
cm  piece  of  #1.5  coverslip  and  laid  out  as  a 
square  about  500  pm  on  a  side  (left).  There  is 
an  L-shaped,  resolution  test  pattern,  40  pm 
along  each  arm,  at  each  corner  with  one  10  x 
10  pm  square  of  1  pm  period  at  the  comer  and 
six  5  x  10pm  rectangles  of  0.7,  0.5,  0.35, 
0.25, 0.20  and  0.17  pm  period  along  each  arm 
(right). 
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were  laid  out  as  a  square  about  500pm  on  a  side  [Fig.  35.7(A)]. 
There  is  an  L-shaped  resolution  test  pattern,  40  pm  along  each  arm, 
at  each  corner  with  one  10  x  10  pm  square  of  1pm  period  at  the 
corner  and  six  5  X  10pm  rectangles  of  0.7,  0.5,  0.35,  0.25,  0.20, 
and  0.17pm  period  along  each  arm  [Fig.  35.7(B)].  The  periods  of 
the  spacings  are  listed  next  to  the  pattern  in  the  top  right  corner 
when  the  coverslip  is  mounted,  Al-side  down,  so  that  the  writing 
is  legible.  The  procedure  for  measuring  the  optical  transfer  func¬ 
tion  of  your  microscope  from  images  of  such  patterns  is  explained 
at  the  end  of  Chapter  1. 

Such  a  test  specimen  can  be  used  to  record  images  using  trans¬ 
mitted  light,  reflected  light,  or  fluorescence.  For  the  latter,  the  cov¬ 
erslip  is  mounted  over  a  well  into  which  dye-laced  immersion  oil 
has  been  introduced.  As  the  test  pattern  is  predominantly  opaque, 
light  can  only  reach  the  dye  through  the  clear  areas  of  the  pattern 
[Fig.  35.8(A,B)].  Although  the  dye  itself  is  essentially  infinite  in 
thickness,  as  long  as  the  NA  is  reasonably  high,  most  of  the  exci¬ 
tation  is  confined  to  a  thin  triangular  region  just  under  a  clear  space 
in  the  pattern  and  next  to  the  coverslip  [Fig.  35.8(C)].  Although 
some  excitation  light  does  reach  the  dye  through  adjacent  clear 
lines  in  the  pattern,  this  is  only  a  minor  effect  as  can  be  seen  in 
the  xy-  and  Jtz-images  of  such  a  specimen,  shown  in  the  upper  half 
of  Figure  35.8(A,B).  Because  the  dye  is  a  liquid,  any  bleaching 
that  may  occur  is  masked  by  the  diffusion  of  new  dye  into  the 
imaged  area. 


Using  the  Test  Specimen 

To  image  the  test  specimen  in  reflected  light,  treat  it  as  you  would 
any  other.  Flowever,  you  may  find  it  difficult  to  adjust  the  plane  of 
focus  to  exactly  coincide  with  the  metal  film.  This  is  because,  as 
is  mentioned  in  Chapters  14  and  15,  the  position  of  maximum 
signal  can  be  determined  to  a  small  fraction  of  the  z-resolution, 
and  in  the  case  of  reflected  light,  the  stage-motion  control  on  your 
instrument  may  not  have  sufficient  precision  to  allow  you  to  focus 
exactly  on  the  metal  surface. 

Because  you  are  attempting  to  view  a  periodic  object,  the  zoom 
setting  should  be  set  about  2x  higher  than  that  normally  required 


by  the  Nyquist  criterion  (i.e.,  4  to  6  samples/resolution  element). 
This  is  because  the  Nyquist  analysis  is  based  on  information  theory 
and,  according  to  information  theory,  a  periodic  object  contains 
only  two  items  of  information:  the  frequency  and  the  phase. 
Although  these  can  be  determined  from  data  sampled  at  the 
Nyquist  rate,  the  image  can  look  very  poor.  Figure  35.9  shows  how. 


focused  beam  of  excitation  light 


“sea”  of  fluorescent  dye  in  immersion  oil 


FIGURE  35.8.  (A)  Diagram  of  the  fluorescent  test  object.  Because  it  is  dif¬ 
ficult  to  fabricate  a  planar  fluorescent  test  object  that  does  not  bleach,  this 
device  works  by  placing  a  pool  of  fluorescent  immersion  oil  just  below  the 
etched  metal  pattern  on  the  bottom  of  the  coverslip.  As  long  as  the  BFP  of  the 
objective  is  filled  uniformly,  much  of  the  illumination  will  approach  the  speci¬ 
men  at  relatively  large  angles,  selectively  exciting  a  triangular  prism  of  dye 
beneath  each  etched  line.  (B)  As  the  focus  plane  moves  into  the  dye,  some  of 
the  dye  deeper  in  the  specimen  is  excited  by  rays  that  pass  through  adjacent 
etched  lines.  However,  this  is  only  a  minor  effect  that  makes  the  z-response 
somewhat  asymmetrical. 
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FIGURE  35.9.  Images  of  the  test  specimen  made  using  fluorescence  (upper  pair)  or  backscattered/reflected  light  (lower  pair).  The  images  were  collected  simul¬ 
taneously  and  show  mirror  symmetry  because  the  fluorescent  light  emerges  from  those  areas  not  covered  with  the  metal  film  that  provides  the  reflected  signal. 
The  upper  member  of  each  pair  shows  an  .n- image  and  the  lower  member  shows  an  vz- image  that  makes  more  evident  the  asymmetry  of  the  fluorescent 
signal.  Nikon  60x/1.4  on  a  Bio-Rad  600/Optiphot.  The  vertical  spacings  have  the  following  spacings  left  to  right:  0.35,  0.5.  and  0.7pm  (vertical)  and  1.0pm 
(horizontal). 


when  sampling  at  twice  the  frequency  of  the  analog  data  but  doing 
so  just  as  the  signal  crosses  zero,  it  is  possible  to  miss  the  varia¬ 
tions  in  a  periodic  signal  entirely.  As  a  result,  one  should  choose 
0.05  pm  pixels  or  about  35  pm  field  width  for  a  768  pixel  line  for 
use  with  an  NA  1.3  to  1.4  objective. 

If  you  have  done  things  correctly,  the  picture  that  you  see 
should  look  like  that  in  Figure  35.10(A).  In  these  images,  it  can 
be  seen  that  the  edges  of  the  vertical  lines  have  a  high-frequency 
wiggle  (period  about  4-5  raster  lines  =  16.6-20  ms),  indicating  the 
presence  of  mains-frequency  electronic,  magnetic,  or  mechanical 
interference.  In  Figure  35.10(B),  the  sets  of  horizontal  lines  above 
and  below  the  axis  of  the  computer-calculated  Fourier  transform 
(FT)  below  each  image  clearly  demonstrate  the  presence  of  some 
other  periodic  instability  present  in  these  single-scan  images.  As 
the  vertical  spacings  of  the  two  finer  periods  are  near  the  cut-off 
frequency  of  the  contrast-transfer  function,  only  a  single  spot  is 
shown  along  the  horizontal  axis  on  either  side  of  the  vertical  axis, 
but  the  spacing  of  all  the  first-order  spots  can  be  seen  to  be  inversely 
related  to  the  actual  spacing  of  the  features  in  each  segment  of  the 
image.  The  horizontal  bars  in  the  FTs  to  the  left  of  Figure 
35.10(C,D)  show  the  effect  of  Kalman  averaging.  Not  only  do  the 
spots  corresponding  to  the  fine  serrations  of  the  edge  get  averaged 


out,  but  the  S/N  of  the  FT  is  markedly  higher.  The  FT  in  the  lower 
right  of  Figure  35.10(E)  was  made  from  a  larger  patch  of  the  spec¬ 
imen  having  a  0.8  pm  spacing.  Three  harmonics  can  be  seen  easily, 
but  the  fourth,  representing  0.8/4  =  0.2  pm  data,  is  barely  visible. 

If  you  have  the  equipment  for  simultaneous  fluorescent/BSL 
imaging,  the  specimen  can  be  imaged  in  fluorescence  light  simply 
by  switching  to  the  other  channel.  Otherwise  the  microscope  must 
first  be  reconfigured  for  fluorescent  light  imaging.  The  image  that 
you  see  should  look  like  that  in  Figure  35.11. 

The  Diatom:  A  Natural  3D  Test  Specimen 

The  test  specimens  discussed  have  the  disadvantage  of  being 
planar.  Making  a  test  specimen  having  precise  structural  features 
in  the  third  dimension  is  more  difficult. 

The  best  solution  so  far  is  to  immerse  diatom  frustules  (North 
Carolina  Biological)  in  fluorescent  oil  and  view  them  as  negative 
objects,  as  suggested  by  Roger  Tsien  (University  of  California,  San 
Diego).  Figure  35.12  shows  a  stereo  view  of  such  a  preparation, 
imaged  with  BSL  using  the  system  mentioned  above.  Although 
careful  attention  to  the  exact  species  of  diatom  can  result  in  highly 
reproducible  spacings,  this  specimen  is  not  without  its  problems. 


FIGURE  35.1 0.  Data  sampling  of  periodic  objects.  Although  the  Nyquist  criterion  states  that  only  ~2  samples  are  required  for  each  period  of  the  highest  spatial 
frequency  in  the  data,  this  does  not  always  work  for  periodic  objects.  Although  a  fortuitous  phase  relationship  between  the  sampling  instant  and  the  positive  and 
negative  peaks  of  the  highest  frequency  can  lead  to  an  accurate  digitization  (A),  a  phase  relationship  that  sampled  the  signal  only  at  the  instant  that  the  signal 
crossed  the  axis  would  not  record  this  contrast  at  all  (B).  For  this  reason,  it  is  safer  to  digitize  images  of  periodic  objects  with  at  least  four  samples/period  (C). 


FIGURE  35.1 1 .  (A)  High  resolution  reflected  light  image  of  test  specimen  recorded  using  a  Nikon  60x/l  .4  lens  on  a  Bio-Rad  600/Optiphot.  Spacing,  left  to  right: 
0.5,  0.35,  and  0.25  pm.  In  (B),  the  sets  of  horizontal  lines  above  and  below  the  axis  of  the  computer-calculated  FT  below  each  image  clearly  demonstrate  the  pres¬ 
ence  of  a  periodic  instability  in  these  single-scan  images.  As  the  two  finer  spacings  are  near  the  cut-off  frequency  of  the  contrast-transfer  function,  only  a  single 
spot  is  shown  along  the  horizontal  axis  on  either  side  of  the  vertical  axis,  but  the  spacing  of  all  the  first-order  spots  can  be  seen  to  be  inversely  related  to  the  actual 
spacing  of  the  features  in  each  segment  of  the  image.  Panels  (C,  D)  are  reflected  light  images  of  0.7  pm  horizontal  spacings  made  either  live  (C)  or  after  averag¬ 
ing  20  scans  (D).  The  horizontal  bars  in  the  FT  plots  to  the  left  of  each  image  show  the  effect  of  averaging.  Not  only  do  the  spots  corresponding  to  the  fine  ser¬ 
rations  of  the  edge  get  averaged  out,  but  the  S/N  of  the  lower  FT  is  markedly  higher.  The  FT  in  the  lower  right  (E)  was  made  from  an  image  of  a  larger  patch  of 
the  specimen  having  an  0.8  pm  spacing.  Three  harmonics  can  be  seen  easily,  but  the  fourth,  representing  0.8/4  =  0.2  pm  data,  is  barely  visible. 
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FIGURE  35.12.  Stereo  image  of  part  of  a 
diatom  immersed  in  immersion  oil  made  with 
BSL  using  a  Nikon  60x/1.4  lens  on  a  Bio- 
Rad  600/Optiphot  equipped  with  the  opti¬ 
mized  BSL  system  described  in  the  text  and 
diagrammed  in  Figure  2.3.  The  diatom  is 
resting  on  a  microscope  slide  and  is  viewed 
from  the  top  through  the  coverslip  and  two 
layers  of  immersion  oil.  The  signal  reflected 
from  the  slide  surface  can  be  seen  around  the 
edge  of  the  image  but  the  presence  of  the 
diatom  distorts  the  focus  plane  away  from 
the  surface  in  the  center  of  the  field  of  view. 
(Specimen  kindly  provided  by  Nelson 
Navaro,  C.U.P.R.,  Puerto  Rico.) 


Because  the  amorphous  silica  out  of  which  the  frustule  is  made 
has  an  index  of  refraction  different  from  that  of  either  water  or 
immersion  oil,  it  refracts  and  also  scatters  considerable  light. 
While  this  is  convenient  for  BSL  imaging,  it  produces  aberrations 
whenever  the  focus  plane  penetrates  too  far.  This  effect  can  be  seen 
in  Figure  35.12  where,  around  the  edge  of  the  image,  one  can  see 
the  signal  reflected  back  from  the  flat  surface  of  the  glass  slide  on 
which  the  diatom  is  resting.  However,  in  the  center  of  the  field  the 
beam  must  pass  the  through  the  frustule  to  reach  the  surface  and 
its  presence  distorts  the  focus  plane  away  from  the  surface  of  the 
glass  making  it  appear  dark. 

REASONS  FOR  POOR  PERFORMANCE 
Sampling  Problems 

As  mentioned  above,  when  imaging  periodic  objects,  it  is  neces¬ 
sary  to  sample  the  image  data  at  higher  than  the  Nyquist  rate. 
Failure  to  do  so  will  result  in  the  aliasing  artifacts  shown  in 
Figure  35.13. 

Optical  Problems 
Aberrations 

As  is  mentioned  above  and  covered  in  Chapters  1,  7,  8,  and  20, 
one  can  only  obtain  diffraction-limited  resolution  if  all  optical 
aberrations  are  absent.  The  major  aberrations  are  spherical  aberra¬ 
tion  and  chromatic  aberration.  In  a  high  NA  objective,  spherical 
aberration  is  usually  corrected  for  a  number  of  wavelengths  but 
only  for  one  immersion  medium.  To  get  the  recommended  perfor¬ 
mance,  you  must  use  the  immersion  liquid  for  which  the  lens  was 
designed  or  use  a  lens  having  a  correction  collar  to  adjust  for  dif¬ 
ferent  media. 

As  the  focal  length  of  any  objective  changes  slightly  with 
wavelength,  its  magnification  also  changes  with  wavelength,  only 
in  the  opposite  sense.  As  a  result,  when  operating  in  fluorescence, 
an  off-axis  ray  at  the  excitation  wavelength  will  cross  the  inter¬ 
mediate  image  plane  at  a  different  point  from  the  longer  wave¬ 
length  emission  ray  coming  from  the  same  point  on  the  specimen. 
As  a  result,  the  emission  ray  may  partially  or  totally  miss  the 
pinhole,  resulting  in  loss  of  signal  (Fig.  35.14)!  Two  other  points 
are  worth  mentioning  in  regard  to  this  second  point: 


FIGURE  35.1 3.  The  effect  of  under-sampling  on  a  periodic  image.  The  upper 
image  was  recorded  using  reflected  light  with  a  Nikon  60x/1.4  lens  on  a 
Bio-Rad  600/Optiphot  with  0.04  pm/pixel  (twice  the  Nyquist  rate  for  0.2|tm 
structures)  and  then  printed  on  a  dye  sublimation  printer  having  pixels  the 
equivalent  of  0.018pm  in  size.  In  it,  both  the  0.25pm  (left)  and  0.2pm  (right) 
spacings  are  clearly  seen.  The  lower  image  was  made  with  identical  optical 
conditions  but  the  pixel  size  was  0.08  pm  (exactly  the  Nyquist  limit)  and  the 
image  was  recorded  photographically  from  a  video  monitor  having  a  screen 
pitch  equivalent  to  0.05  pm/elements.  To  some  extent,  the  reduced  contrast  in 
the  0.25  pm  spacing  may  be  due  to  the  bandwidth  limitations  of  the  monitor  or 
the  vagaries  of  the  photographic  process,  but  the  virtual  absence  of  the  0.2  pm 
spacings  is  probably  due  to  sampling  problems,  as  is  evidenced  by  the  pres¬ 
ence  of  a  faint  aliasing  pattern  at  about  twice  the  period  of  the  actual  features 
in  the  image. 
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FIGURE  35.14.  Lateral  chromatic  aberration.  The  effect  of  chromatic  magni¬ 
fication  error  on  signal  level  in  the  confocal  microscope.  The  focal  length  and, 
hence,  the  magnification  of  any  optical  system  varies  with  wavelength.  Con¬ 
sequently,  an  off-axis  ray  of  short  wavelength  light  will  not  follow  the  same 
path  as  that  of  a  ray  of  longer  wavelength  light  originating  from  the  same  point. 
In  the  laser-scanning  confocal  microscope,  the  scanning  mirrors  are  supposed 
to  deflect  both  the  source  and  the  detector  pinhole  off-axis  by  the  same  amount. 
However,  this  will  not  happen  if  the  optical  system  has  high  chromatic  aber¬ 
ration  and  the  system  is  used  for  fluorescence,  because  the  exciting  and  emit¬ 
ting  wavelengths  are  magnified  by  different  amounts.  As  a  result,  the  signal 
light  will  miss  the  pinhole  and  the  image  will  become  darker  away  from  the 
axis.  The  problem  can  be  reduced  only  by  using  optical  systems  that  are  highly 
corrected  for  chromatic  aberration  and  by  placing  the  field-of-view  as  near  to 
the  optical  axis  as  possible. 


•  In  many  older  microscopes,  it  was  common  to  correct 
chromatic  aberration  in  the  eyepiece.  Objectives  from  such 
microscopes  will  demonstrate  totally  unacceptable  levels  of 
chromatic  aberration  if  they  are  not  used  in  conjunction  with 
the  appropriate  correcting  eyepieces.  Generally  one  should  use 
only  modern,  highly-corrected  objectives  for  fluorescence  con- 
focal  microscopy  and  use  them  only  with  other  components 
that  are  properly  matched. 

•  The  magnitude  of  the  displacement  caused  by  the  chromatic 
magnification  error  is  generally  proportional  to  the  distance 
that  the  point  in  the  image  is  away  from  the  optical  axis.  There¬ 
fore,  the  effect  will  be  less  severe  if  imaging  is  restricted  to  an 
area  near  the  optical  axis. 

Curvature  of  Field 

Unless  concrete  steps  are  taken  to  prevent  it,  simple  lenses  will 
focus  a  plane  surface  onto  a  segment  of  a  sphere.  Objectives  des¬ 
ignated  “plan”  are  supposed  to  have  flatter  fields  of  view  than 
most,  but  this  feature  is  usually  incorporated  at  the  cost  of  addi¬ 
tional  elements  and  often  lower  light  transmission.  Figure  35.15 
shows  two  images  of  a  plane  mirror,  one  made  with  a  non-plan 
lens  (upper)  and  one  with  a  planapo-correction  (lower).  The 
images  have  been  posterized  to  emphasize  the  variation  in  signal 
caused  by  field  curvature.  The  variation  in  signal  strength  across 
the  field  is  shown  with  greater  precision  in  the  .r-profile  plots  at 
the  bottom.  We  should  emphasize  that  these  plots  show  variation 
in  the  measured  intensity  of  reflected  light  passing  the  confocal 
pinhole,  not  the  actual  geometrical  curvature  of  the  field!  Although 
the  upper  image  displays  a  variation  in  focus  that  could  degrade 
the  image  of  a  flat  test  object  such  as  that  mentioned  above,  one 
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FIGURE  35.15.  Curvature  of  field.  In  general,  lenses  focus  a  plane  surface 
onto  a  spherical  one.  In  objectives  not  specifically  corrected  for  plan  operation, 
the  curvature  of  field  has  the  effect  that  an  image  of  a  plane  surface  loses  inten¬ 
sity  off-axis.  This  figure  shows  two  images  of  a  plane  mirror,  one  made  with 
an  uncorrected  lens  (A)  and  one  with  a  planapo  correction  (B).  The  variation 
in  signal  strength  across  the  field  is  shown  with  greater  precision  in  the  two  x- 
profile  plots  (C).  (We  should  emphasize  that  these  plots  show  variation  in  the 
measured  intensity  of  reflected  light  passing  the  confocal  pinhole,  and  are  not 
a  measure  of  the  actual  geometrical  curvature  of  the  field!)  Although  field  cur¬ 
vature  can  be  a  serious  problem  when  viewing  planar  specimens,  it  is  often  less 
serious  when  making  3D  fluorescent  images  of  biological  specimens  because 
in  the  latter  case  there  is  no  signal  loss  but  simply  a  slight  dish-shaped  distor¬ 
tion  in  the  final  3D  data.  However,  if  this  distortion  is  to  be  kept  symmetrical, 
it  is  important  not  to  pan  the  scanned  area  away  from  the  center  of  the  field  of 
view  when  using  such  a  lens. 


should  remember  that,  with  a  z-resolution  of  ~0.6  Jim,  even  an 
intensity  drop  of  90%  between  the  center  and  the  edge  of  the  field 
would  only  indicate  a  field  curvature  of  <l|im.  However,  this 
image  does  show  the  penalty  of  panning  the  zoomed  area  away 
from  the  optical  axis  (e.g.,  to  avoid  reflection  artifacts  in  BSL 
imaging)  when  using  a  non-plan  lens. 
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Not  Filling  the  Objective  Pupil 

We  have  mentioned  above  that  one  can  only  expect  an  optical 
system  to  reach  its  diffraction-limited  performance  if  it  is  operated 
at  its  full  NA.  Check  that  the  size  of  the  laser  beam  at  the  BFP 
(upper  part  of  Fig.  35.2)  is  equal  to  the  diameter  of  the  hole  in  the 
rear  of  the  objective  that  you  are  using.  If  you  are  in  doubt, 
compare  its  x-y  resolution  performance  with  an  objective  of  the 
same  NA  but  a  higher  magnification,  as  this  will  have  a  smaller 
BFP.  If  the  performance  is  better  with  the  higher  magnification 
objective,  the  lower  magnification  one  was  probably  being 
under-filled. 

Dirty  Objective 

Common  blemishes  found  on  optical  elements,  such  as  dust, 
smudges,  or  old  immersion  oil,  significantly  reduce  their  optical 
performance  and  contribute  to  stray  light.  In  fact,  any  smudge  or 
smear  that  shifts  the  phase  of  the  light  passing  through  it  by  as 
little  as  \  X  will  prevent  those  rays  from  contributing  to  a  diffrac¬ 
tion-limited  image  (see  Chapter  11,  this  volume). 

Especially  disturbing  are  blemishes  at  or  near  the  conjugate 
image  planes  or  on  the  front  surface  of  the  objective.  Specks  of 
dust  or  fibers  can  be  carefully  removed  by  lightly  sweeping 
the  surface  with  a  soft  camel-hair  brush  or  by  blowing  them  off 
with  a  puffer  made  of  a  large  rubber  bulb  fitted  with  a  plastic 
micropipette  tip.  Canned-air  blowers  should  be  used  with  caution 
as  these  sometimes  deposit  oil  or  condensed  impurities  from  the 
propellant  onto  optical  surfaces. 

Material  that  is  more  adherent  may  require  wiping  with  a 
cleaning  solution  or  solvent.  The  type  of  cleaning  solution  required 
depends  on  the  specific  material  involved.  Water,  glycerol,  and  salt 
crystals  can  easily  be  removed  using  double-distilled  water,  while 
immersion  oil  and  some  mounting  media  respond  well  to  the 
household  glass  cleaner  Sparkle  (A.J.  Funk  and  Company,  Elgin, 
IL),  which,  despite  its  humble  origin,  is  recommended  by  several 
optical  manufacturers.  This  cleaner  is  effective  at  removing  oil, 
evaporates  fairly  quickly,  and  does  not  contain  harsh  solvents  that 
might  attack  the  cement  holding  the  lens  elements  together. 
Because  it  is  non-flammable  and  non-toxic,  it  can  be  stored  right 
beside  the  microscope. 

One  should  not  resort  to  harsher  solvents  such  as  ether, 
benzene,  xylene,  or  acetone  until  the  more  benign  ones  have 
proved  ineffective.  Although  these  solvents  can  damage  some  lens 
cements,  the  bigger  danger  is  that  they  may  infiltrate  the  lens  to 
wet  the  interior  glass/air  surface,  where  they  will  then  deposit 
impurities  when  they  dry.  When  using  such  solvents,  clean  the  sur¬ 
faces  quickly  so  that  the  solvent  evaporates  before  it  has  a  chance 
to  leak  inside  the  lens.  Never  soak  an  objective  in  anything!  If 
organic  solvents  must  be  used,  take  proper  safety  precautions,  such 
as  working  in  a  fume  hood.  Whenever  wiping  is  necessary,  one 
must  use  clean  lens  tissue  (i.e.,  one  that  has  not  been  exposed  to 
dust  deposit  from  the  air),  free  from  finger  oils,  and  never  use  the 
same  area  of  the  lens  tissue  twice. 

With  hard  surfaces  such  as  oculars  and  camera  lenses,  it  may 
be  sufficient  to  fog  the  lens  with  your  breath  and  gently  wipe  the 
surface  with  clean  lens  tissue.  Either  use  the  folded  edge  of  the 
tissue  or,  if  needed,  apply  slight  finger  pressure.  Be  sure  to  use 
several  thicknesses  of  tissue  so  that  finger  oils  do  not  penetrate  to 
the  glass. 

To  clean  more  delicate  lens  surfaces,  a  non-contact  method 
is  preferred.  Apply  a  drop  of  the  cleaning  solution  of  choice  at 
the  near-edge  of  a  piece  of  lens  tissue  held  horizontally  (be  sure 
this  area  was  not  previously  touched  by  your  fingers).  Lower 


FIGURE  35.16.  How  to  clean  objectives.  (A)  Apply  a  drop  of  the  cleaning 
solution  of  choice  at  the  near-edge  of  a  piece  of  lens  tissue  held  horizontally 
(be  sure  this  area  was  not  previously  touched  by  your  fingers).  (B)  Lower  the 
drop  of  cleaning  solution  to  the  surface  of  the  lens  and  gently  draw  the  lens 
tissue  over  the  surface  as  shown.  There  is  no  need  to  apply  pressure.  Repeat 
this  procedure  as  necessary. 


the  drop  of  cleaning  solution  to  the  surface  of  the  lens  standing 
upright  on  the  tabletop  and  gently  draw  the  lens  tissue  over  its 
surface  (Fig.  35.16).  There  is  no  need  to  apply  pressure.  Repeat 
this  procedure  three  or  four  times.  As  the  tissue  is  drawn  across 
the  surface,  most  of  the  cleaning  solution  and  the  oil  and  dirt 
are  drawn  off  by  capillary  action,  and  the  remaining  solution 
evaporates. 

Use  an  inverted  ocular  as  a  magnifier  to  check  that  the  oil  or 
dirt  has  been  removed  and  that  there  are  no  streaks  on  the  outer 
surface  of  the  objective.  Surface  films  are  most  easily  seen  if  you 
orient  the  objective  so  that  you  are  viewing  an  overhead  light 
reflecting  directly  off  the  surface  of  interest.  When  a  film  of  solvent 
dries  onto  a  surface,  any  impurities  in  it  will  be  deposited  onto  the 
surface.  Use  very  clean  solvents! 

This  cleaning  method  works  well  for  all  types  of  objective 
lenses,  even  those  with  recessed  surfaces.  To  clean  oculars,  CCD 
camera  sensors,  or  the  back  surface  of  the  objective,  you  must 
make  a  swab  of  lens  tissue.  Wrap  the  end  of  a  wooden  applicator 
stick  with  lens  paper  or  carefully  washed  cotton  batting.  Do  not 
use  a  commercial  cotton  swab,  even  if  wrapped  with  lens  tissue, 
because  it  may  contaminate  the  lens  with  the  glue  used  to  secure 
the  cotton  fibers.  Wet  the  end  of  the  applicator  with  cleaning  solu¬ 
tion  and,  starting  from  the  center  of  the  lens,  gently  sweep  from 
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the  center  to  the  edge  of  the  surface  in  a  continuous  spiral  while 
twirling  the  applicator.  Using  a  new  lens  tissue  swab  each  time, 
repeat  this  procedure  until  the  lens  surface  has  been  cleaned. 

If  one  is  using  an  inverted  microscope  with  an  oil-  or  water- 
immersion  lens,  there  is  the  potential  for  oil  or  water  to  seep 
down  the  barrel  and  into  the  body  of  the  lens.  This  can  be  an 
annoying  problem  and  objectives  can  be  expensive  to  clean 
(US$1,000-US$2,000).  Some  simple  devices  can  prevent  this 
problem.  Placing  a  rubber  O-ring  around  the  upper  part  of  the  lens 
will  act  as  an  oil  dam.  As  long  as  there  are  no  cracks  in  the  O-ring 
and  excess  oil  is  blotted  away  periodically,  this  remedy  works 
exceptionally  well.  Some  people  replace  the  O-ring  with  a 
“scrunchie,”  an  elasticated  device  normally  used  to  confine  hanks 
of  human  hair!  Other  remedies  to  the  problem  can  compromise  the 
spring  shock  absorber  that  is  a  feature  of  most  short-working-dis- 
tance  lenses.  One  can  also  wrap  the  inner  barrel  of  the  lens  with 
Teflon  tape  to  seal  the  point  where  the  oil  enters  the  inside  of  the 
lens  or  place  a  rubber  sleeve  over  the  entire  lens.  Such  a  sleeve 
can  be  made  by  cutting  one  of  the  digits  from  a  rubber  surgical 
glove  and  making  a  small  hole  at  the  tip.  The  sleeve  is  stretched 
over  the  entire  objective  so  that  only  the  front  element  pokes 
through  the  small  hole. 

Air  Bubbles 

An  air  bubble  in  the  immersion  oil  will  act  as  a  small,  yet 
extremely  powerful  lens,  which  can  have  a  pronounced  and  dele¬ 
terious  effect  on  the  optical  performance  of  the  microscope.  For¬ 
tunately,  such  bubbles  can  be  detected  by  imaging  the  oil  layer 
directly.  It  is  possible  to  focus  on  a  plane  within  the  oil  by  adjust¬ 
ing  either  an  internal  Bertrand  lens  (also  called  a  phase  lens  as  it 
is  used  to  observe  the  process  of  centering  the  phase-contrast  rings) 
or  an  external  phase-telescope  fitted  in  place  of  one  of  the  oculars. 

The  best  plan  is  to  try  to  avoid  the  formation  of  the  air  bubbles 
in  the  first  place  by  making  sure  that  any  bubbles  have  a  chance 
to  float  up,  away  from  the  tip  of  the  dispenser  before  you  express 
a  drop  or,  perhaps  even  better,  dip  a  drop  of  oil  out  of  the  oil  con¬ 
tainer  using  a  glass  rod. 

Imaging  Depth 

An  important  aspect  of  any  microscope  used  for  making  3D 
images  is  how  far  below  the  surface  of  the  specimen  it  can  obtain 
useful  data.  The  most  obvious  limitation  is  the  mechanical  one 
imposed  by  the  finite  working  distance  of  the  objective  (Chapter 
7,  Table  7.3,  this  volume).  Once  the  objective  touches  the  speci¬ 
men  surface,  it  clearly  cannot  be  focused  to  yet  deeper  planes. 

Unfortunately,  the  opacity  and  optical  inhomogeneity  of  the 
specimen  usually  impose  more  stringent  limits  (for  more  details, 
see  Chapters  17  and  20,  this  volume).  To  the  extent  that  the  illu¬ 
minating  beam  is  either  scattered  or  absorbed  as  it  passes  through 
the  upper  layers  of  the  specimen,  it  will  fail  to  reach  any  plane  of 
focus  located  farther  into  the  specimen.  Likewise,  signal  light  that 
is  scattered  or  absorbed  between  the  focus  plane  and  the  detector 
cannot  be  measured  (except  when  non-descanned  detection  is  used 
with  two-photon  excitation).  Finally,  any  optical  inhomogeneity  in 
the  specimen  between  the  focus  plane  and  the  objective  will  tend 
to  defocus  both  the  beam  of  incident  illumination  and  the  return¬ 
ing  signal.  This  will  increase  the  effective  size  of  the  focused  spot 
that  is  returned  to  the  pinhole,  so  that  much  of  the  light  that  should 
pass  through  it  does  not  do  so  (Pawley,  2000;  Chapter  21.  this 
volume). 

All  three  of  these  effects  reduce  the  fraction  of  the  signal  that 
is  detected  as  the  focus  plane  is  pushed  progressively  farther  into 


the  specimen.  In  addition,  the  last  mechanism  also  reduces  the 
spatial  resolution.  Together,  these  three  factors  usually  place  a  prac¬ 
tical  limit  on  the  effective  penetration  depth  of  the  confocal  light 
microscope,  and  all  of  them  depend  on  the  optical  properties  of  the 
specimen. 

When  using  an  oil-immersion  objective  on  a  biological  speci¬ 
men  that  is  only  lightly  stained  and  has  been  cleared  by  replacing 
the  water  with  an  imbibing  medium  having  an  index  of  refraction 
close  to  both  that  of  the  solid  components  of  the  cell  and  that  of 
immersion  oil,  then  the  useful  penetration  depth  can  be  several 
hundred  micrometers,  but  on  less  ideal  specimens  it  is  usually 
much  less.  On  the  other  hand,  even  deeper  penetration  is  some¬ 
times  possible  if  the  microscope  is  used  at  lower  magnification 
with  an  immersion  objective  having  an  NA  of  <0.8,  because  the 
defocusing  effects  of  the  specimen  are  usually  less  serious  under 
these  circumstances  (Chapter  20,  this  volume). 

Singlet-State  Saturation 

The  intensity  of  the  light  at  the  focus  of  a  high-NA  objective  lens 
is  so  high  that,  if  the  beam  power  is  more  than  -1  mW,  a  large  frac¬ 
tion  of  the  fluorescent  molecules  within  the  focal  region  are  in  the 
excited  state  at  any  given  time.  As  excited  molecules  are  unlikely 
to  absorb  a  second  photon,  they  represent  a  reduction  in  the  effec¬ 
tive  stain  concentration. 

Saturation  can  be  avoided  by  using  less  input  power.  However, 
as  the  rate  at  which  data  is  produced  from  the  specimen  is  pro¬ 
portional  to  the  excitation  intensity,  dye  saturation  places  an 
absolute  limit  on  the  rate  at  which  fluorescence  information  can 
be  obtained.  This  limit  is  most  severe  when  the  instrument  is  oper¬ 
ating  at  high  spatial  resolution  because  this  implies  a  smaller 
focused  spot  and,  therefore,  a  higher  light  flux  density.  Fortunately, 
many  fluorescent  specimens  produce  adequate  images  using  as 
little  as  a  few  microwatts  (Pawley  and  Centonze,  1997)  or  even 
nanowatts  (Fig.  19.2,  this  volume )  of  laser  power.  At  these  levels 
it  is  often  possible  to  make  thousands  of  images  before  the  speci¬ 
men  bleaches. 

Because  of  fluorescence  saturation,  one  cannot  expect  to  scan 
a  single  beam  rapidly  [e.g.,  at  television  (TV)  scan  rate]  and  also 
produce  a  statistically  well-defined,  high-resolution,  fluorescent, 
confocal  image  unless  one  also  reduces  the  number  of  pixels  and, 
therefore,  the  raster  size  of  each  image. 

•  Because  the  degree  of  saturation  is  directly  proportional  to  the 
fluorescent  lifetime,  operating  at  or  near  saturation  means  that 
signal  intensity  may  reflect  environmental  conditions  that 
affect  this  parameter  as  well  as  reflecting  the  active  concen¬ 
tration  of  the  dye. 

•  As  saturation  is  a  function  of  the  dye  molecule,  rather  than,  for 
example,  the  detector,  it  affects  all  parts  of  the  image  equally. 
As  a  result,  areas  with  more  dye  will  still  appeal'  brighter  than 
areas  with  less,  and  the  image  does  not  look  saturated  in  the 
same  way  as  an  over-exposed  photograph  does. 

•  As  most  photodamage  processes  do  not  saturate,  operating 
near  saturation  produces  relatively  more  photodamage  for  the 
amount  of  data  that  is  obtained. 

•  As  the  degree  of  saturation  depends  on  the  intensity  of  the  inci¬ 
dent  light  flux,  saturation  losses  are  highest  near  the  focus 
plane  where  the  light  beam  is  narrowest.  As  a  result,  an  image 
recorded  from  a  dye  that  is  saturated  tends  to  record  relatively 
more  signal  representing  light  from  planes  that  are  out  of 
focus.  This  reduces  the  --resolution. 

•  The  saturation  problem  decreases  with  the  square  of  the  reso¬ 
lution  of  the  optical  system. 
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WHICH  3D  METHOD  IS  BEST? 

Elsewhere  in  this  book,  you  will  find  chapters  describing  all  the 
major  methods  of  3D  light  microscopy:  widefield/deconvolution, 
confocal,  and  two-photon  (the  latter  two,  in  both  single-beam  and 
multi-beam  versions).  You  will  also  find  chapters  pointing  out 
the  advantages  of  combining  deconvolution  with  the  other  two 
methods  and  comparing  their  performance  from  a  theoretical  point 
of  view.  However,  because  each  of  these  chapters  is  written  by 
skilled  practitioners  of  the  particular  method  described,  there  is 
little  discussion  of  some  of  the  fundamental  differences  between 
them.  This  section  sets  out  to  fill  this  gap. 

As  making  comparisons  of  any  type  is  always  invidious,  we 
must  start  by  explaining  what  we  will  not  do.  We  will  not 
recommend  one  method  over  another  nor  will  we  provide  more 
than  a  rudimentary  description  of  the  operating  principles. 
What  we  will  do  is  point  to  the  features  of  each  one  that  makes  it 
particularly  suitable  for  certain  studies,  particularly  studies  of 
living  specimens.  Our  discussions  will  cover  differences  in 
the  shape  of  the  bleach  pattern,  the  power  density  of  the  excita¬ 
tion,  the  differences  in  the  quantum  efficiency  and  noise  level  of 
the  detector,  and  how  these  affect  resolution  and  S/N  of  the  final 
image. 

Because  widefield  techniques  collect  data  in  parallel  and  use 
CCDs  with  high  effective  quantum  efficiency  (QE),  while  single¬ 
beam  scanning  techniques  are  serial  devices  and  use  photomulti¬ 
plier  tubes  with  5x  to  lOx  lower  QE,  one  can  both  elicit  and  detect 
many  more  photons/second  with  widefield  techniques.  Figure 
35.17  shows  the  excitation  aspect  of  this  difference.  Assuming  a 
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FIGURE  35.1 7.  Variation  in  peak  brightness  of  the  excitation  light  in  the  three 
common  forms  of  3D  fluorescence  microscopy.  Assuming  that  one  needs  to 
excite  a  fixed  number  of  photons  from  the  focus  plane  in  the  specimen  in  order 
to  record  a  1000  x  1000  image  of  a  given  quality,  the  level  of  excitation  at  the 
focus  plane,  in  photons/square  micrometer,  must  be  much  higher  in  a  scanning 
microscope,  where  only  about  12  pixels  are  excited  at  any  one  time,  than  with 
a  widefield  microscope  in  which  106  pixels  are  illuminated  simultaneously.  The 
low  duty-cycle  of  femtosecond  two-photon  excitation  produces  a  peak  power 
that  is  about  105  to  106  times  higher  again.  It  is  now  clear  that  many  dyes  bleach 
more  rapidly  at  these  extremely  high  excitation  levels  (see  Chapters  38  and  39, 
this  volume). 
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FIGURE  35.18.  Excitation  duty  cycle  for  80MHz,  pulsed  illumination. 
Because  the  time  between  the  excitation  pulses  of  an  80  MHz  pulsed  laser  is 
long  (12  ns)  compared  to  the  decay  time  of  most  fluorescent  dyes,  such  dyes 
are  driven  into  singlet-state  saturation  at  a  signal  level  about  lOx  lower  than 
they  can  produce  with  continuous  illumination. 


1024  x  1024  pixels  CCD,  widefield  will  be  exciting  and  recording 
data  from  106  pixels,  while  the  single  beam  of  the  confocal  will  be 
highly  exciting  about  12  Nyquist  pixels  and  collecting  data  from 
most  of  these.7 

To  collect  an  image  in  the  same  time  period,  the  total  flux  of 
detected  photons/s  must  be  the  same  in  both  cases  and  this  can 
only  occur  if  the  laser  is  105x  more  intense  (in  photons/pm2  at 
the  focus  plane)  than  the  Hg  illumination  used  in  widefield 
microscopy.8  When  one  moves  to  two-photon  microscopy,  there  is 
another  major  increase  in  the  peak  flux  required  and  this  is  pro¬ 
vided  by  the  fact  that  the  duty  cycle  of  a  lOOfs  pulsed  laser  oper¬ 
ating  at  100  MHz  is  a  10~5  and  this  means  that  the  peak  intensity 
is  105x  higher  than  confocal,  for  the  same  average  beam  power.  In 
fact,  two-photon  systems  are  commonly  operated  with  an  average 
power  about  lOx  to  50x  higher  than  that  used  for  confocal,  but  it 
is  also  true  that,  with  a  wavelength  that  is  2x  longer,  the  area  of 
the  two-photon  spot  is  4x  larger  in  area  and  the  intensity  is  con¬ 
sequently  4x  lower.  Because  excitation  is  proportional  to  the 
square  of  the  intensity,  the  area  from  which  the  fluorescent  light 
emerges  is  about  1.4x  larger  in  diameter  and  2x  larger  in  area  than 
would  be  the  case  with  single-photon  excitation. 

A  second  factor  relates  to  the  fact  that  using  a  100  MHz  laser 
excites  organic  dyes  inefficiently.  Figure  35.18  shows  that  even  if 
each  pulse  excites  essentially  all  the  dye  in  the  focal  volume, 
because  the  decay  time  of  most  common  dyes  is  short  compared 
to  the  time  between  pulses,  the  system  is  essentially  turned  off  for 
about  90%  of  the  time. 


7  Assuming  that  the  pinhole  is  set  to  1  to  2  Airy  units,  some  of  the  photons 
emitted  from  all  these  12  pixels  on  the  specimen  will  be  stored  as  the  value 
for  a  single  pixel  in  the  computer  (see  Fig.  24. A6  in  the  appendix  to  Chapter 
24,  this  volume,  for  more  on  this). 

8  Strictly  speaking,  to  account  for  the  lower  detector  QE,  it  might  require  106x 
more  brightness.  However,  because  better  image  data  is  needed  if  one  must 
later  remove  the  out-of-focus  light  by  deconvolution,  things  get  complex  and 
we  will  leave  this  argument  to  other  chapters. 


Tutorial  on  Practical  Confocal  Microscopy  and  Use  of  the  Confocal  Test  Specimen  •  Chapter  35  645 


FIGURE  35.19.  Dependence  of  total  photodamage  on 
damage/emitted-photon  and  specimen  thickness  in 
single-  and  two-photon  excitation  fluorescence  imaging. 
One  of  the  signature  advantages  of  two-photon  excita¬ 
tion  is  that  excitation/damage  is  normally  confined  to  the 
focus  plane  (A,  D).  However,  if  as  seems  likely,  in  many 
cases,  the  damage/excitation  produced  by  two-photon 
excitation  is  greater  than  with  single-photon  excitation 
by  a  factor  y,  then  two-photon  excitation  will  only  be 
advantageous  when  the  specimen  is  so  thick  that  the 
number  of  single-photon  excitations  produced  as  the 
beam  passes  through  the  entire  specimen  exceeds  the 
two-photon  excitations  produced  in  the  focus  plane  by  a 
factor  of  >y  (C,  F). 


single-photon 


It  would  be  surprising  indeed  if  excitation  of  the  dye  was 
entirely  unaffected  by  these  very  marked  differences  in  photon 
flux.  At  the  very  least,  one  can  see  that  one  must  operate  ~105  x 
closer  to  singlet-state  saturation  with  single-beam  confocal  than 
with  widefield  and  at  least  another  factor  of  lOx  closer  with  two- 
photon  microscopy.  The  two  simplest  responses  to  this  problem 
are:  (1)  to  take  more  time  to  collect  data  from  a  single  plane  in 
confocal  and  two-photon  microscopy,  and  (2)  to  settle  for  detect¬ 
ing  lOOx  to  lOOOx  fewer  photons/pixel  than  is  common  in  wide- 
field/deconvolution.9 

Numerous  studies  now  connect  phototoxicity  to  excitation  of 
the  dye.  By  themselves  either  the  dye  or  the  light  are  relatively 
innocuous.  Together,  energy  is  deposited  and  some  of  this  energy 
produces  damaging  compounds.  Although  it  is  often  said  that  both 
widefield  and  confocal  systems  do  the  same  amount  of  bleaching 
above  and  below  the  plane  of  focus,  this  is  not,  strictly  speaking, 
true.  Figure  35.1(B)  shows  the  situation  for  confocal  in  which  the 
worst  bleaching  is  confined  to  two  pyramids,  above  and  below  the 
central  focus  plane.  On  the  other  hand,  if  widefield  microscopy  is 
performed  using  a  light  source  that  does  fill  the  objective  BFP  and 
with  the  field  diaphragm  set  to  confine  the  illumination  to  a  circle 
just  large  enough  to  contain  the  area  imaged  by  the  CCD,10  then 
the  excitation  away  from  the  focus  plane  depends  only  on  defocus. 
Damage  is  not  peaked  towards  the  center  of  the  out-of-focus 
planes.  On  the  other  hand,  probably  the  best  known  feature  of  two- 
photon  imaging  is  that  excitation  is  confined  close  to  the  focus 
plane.  What  complicates  the  choice  is  that,  for  many  dyes,  the 
damage/excitation  produced  by  two-photon  excitation  is  higher 
than  when  the  same  dye  is  excited  by  a  single  photon  (see  Chap¬ 
ters  38  and  39,  this  volume).  This  differing  dye-damage  sensitiv¬ 


9  These  estimates  are  just  that.  Different  users  viewing  specimens  with  widely 
different  staining  levels  may  find  that  they  can  produce  acceptable  results  with 
recorded  signals  that  vary  over  a  range  at  least  this  large.  The  numbers  used  are 
meant  to  be  illustrative,  not  definitive.  It  is  also  important  to  remember  that, 
because  of  the  difference  in  detector  QE,  this  may  only  amount  to  a  lOxto  lOOx 
difference  in  the  number  signal  photons  detected  from  near  the  plane  of  focus. 

10  This  setup  is  recommended  both  to  reduce  damage  to  those  parts  of  the 
preparation  not  being  imaged  and  also  to  reduce  stray  and  scattered  light  to 
a  minimum. 


ity  is  almost  undoubtedly  related  to  the  marked  differences  in  peak 
excitation  intensity  noted  above. 

Although  the  magnitude  of  the  nonlinear  damage  effect  is 
poorly  understood,  let  us  consider  the  implications  if  the 
damage/excitation  is  lOx  higher  for  two-photon  than  it  is  for  single¬ 
photon  microscopy.  Figure  35.19(A,B)  diagrams  the  two-photon 
excitation  of  a  tissue-culture  cell  on  a  coverslip.  The  damage  at  the 
plane  of  focus  is  worse  locally  but  no  damage  occurs  away  from 
it.  Figure  35.19(D,E)  shows  the  situation  for  single-photon  excita¬ 
tion:  damage  is  less  at  any  point  but  extends  throughout  the  cell. 
Clearly,  if  the  cell  is  less  than  lOx  thicker  than  the  two-photon  exci¬ 
tation  plane,  the  total  amount  of  photodamage  from  a  single  scan 
will  be  more  with  two-photon  excitation.  Assuming  a  damage  plane 
thickness  of  about  1  pm,  single -photon  excitation  would  have  the 
advantage  for  most  common  tissue  culture  cell  preparations. 

However,  given  our  assumptions,  if  the  stained  volume  of  the 
preparation  is  more  than  lOx  thicker  than  the  two-photon  damage 
plane,  there  will  be  less  damage  with  two-photon  excitation  [Figs. 
35.19(C,F)]. 

If  the  damage/excitation  discrepancy  is  less  than  lOx,  the 
advantage  will  switch  to  two-photon  excitation  at  a  specimen 
thickness  that  is  somewhat  less  than  10pm. 

Finally,  there  is  the  matter  of  how  much  power  is  needed  for 
two-photon  microscopy.  In  Chapter  38,  Karsten  Koenig  points  out 
that  about  5  mW  of  femtosecond  pulsed  laser  light  is  sufficient  to 
prevent  cells  from  dividing  if  it  is  focused  into  a  small  spot  by  a 
high-NA  objective  [Fig.  35.20(A)].  On  the  other  hand,  the  popular 
titanium :  sapphire  (Ti :  Sa)  lasers  only  work  properly  if  they  are 
producing  about  a  watt  of  optical  output.  Although  some  users  have 
taken  advantage  of  this  surplus  power  to  make  many-beam  two- 
photon  excitation  systems,  it  has  also  been  utilized  to  overcome  the 
losses  imposed  by  imaging  thick,  scattering  specimens  where  hun¬ 
dreds  of  milliwatts  must  be  used  to  get  sufficient  intensity  to 
produce  two-photon  effects  in  an  aberrated  spot  located  some  hun¬ 
dreds  of  micrometers  from  the  surface  of  the  specimen.  This  topic 
is  worth  talking  about  here  because  a  number  of  new  femtosecond 
mode-locked  lasers  are  now  in  development  that  couple  great  oper¬ 
ating  convenience  with  potentially  much  lower  cost  but  that 
produce  comparatively  modest  amounts  of  power  (10-20mW).  It 
seems  likely  that  these  systems  will  find  applications  in  which  oper¬ 
ation  on  specimens  50  to  100|im  thick  is  routine. 
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Optimal  3D  Light  Microscopy  Summary 

After  all  these  words,  important  questions  remain: 

•  How  important  are  “dose-rate  effects”? 

—  Are  dye  molecules  that  have  been  excited  to  the  singlet  state 
more  likely  to  absorb  a  second,  damaging  photon  in  laser 
confocal  microscopy?  In  two-photon  microscopy?  Why 
does  this  problem  seem  worse  with  some  dyes  than  others? 
—  How  fast  does  the  risk  of  bleaching/emitted  photon  increase 
as  the  excitation  gets  more  intense? 

—  Does  crossing  to  the  triplet  state  significantly  reduce  the 
effective  dye  concentration  in  laser  confocal  microscopy? 
In  two-photon  microscopy? 

—  If  so,  how  does  this  depend  on  the  type  of  dye  and  the  local 
environment? 

•  When  does  staining  geometry  and  density  become  a  serious 
problem  for  widefield/deconvolution? 

•  Are  there  aspects  of  dye  performance  under  two-photon  exci¬ 
tation  that  may  limit  the  use  of  certain  dyes? 

—  Absorption  spectra 
—  Absorption  cross-sections 
—  Quantum  efficiency 
—  Bleaching/phototoxicity 

—  New  dye  molecules  developed  for  two-photon  excitation 

•  Do  confocal  and  two-photon  excitation  bleach  dyes  by  dif¬ 
ferent  mechanisms?  (Probably!) 

Things  to  Remember  About  Deconvolution 

•  Deconvolution  effectively  averages  signal  over  data  stored  in 
>64  voxels,  reducing  Poisson  noise. 

•  Deconvolution  separates  in-focus  from  out-of-focus  light  by 
imposing  PSF,  non-negativity,  and  other  constraints. 

•  Deconvolution  has  more  problems  if  there  is  too  much  out-of¬ 
focus  light. 

•  Deconvolution  should  be  applied  to  both  confocal  and  wide- 
field  data. 

•  Blind  deconvolution  techniques  can  be  subject  to  bootstrap 
errors.  (If  you  start  with  a  “bad”  first  estimate,  or  iterate  too 
long,  you  may  “create”  structure.) 

•  Deconvolution  requires  very  high  positioning  accuracy  and  a 
relatively  constant,  known  PSF.  Remember,  the  widefield  PSF 
varies  with  wavelength,  NA,  field  and  aperture  diaphragm 
diameters,  alignment  of  the  arc  (or  use  of  ground  glass),  and 
the  immersion  medium. 


FIGURE  35.20.  Total  beam  power  required  varies  with  specimen  thickness  in 
two-photon  excitation.  (A)  When  focused  near  the  coverslip,  scattering  and 
aberrations  are  low  and  most  of  the  laser  light  passing  through  the  objective  is 
focused  into  a  small,  high-intensity  spot.  (B)  As  the  focus  plane  moves  farther 
into  the  specimen,  more  light  is  scattered  out  of  the  beam  and  aberrations 
increase  beam  diameter  (8),  further  reducing  peak  intensity.  Maintaining  signal 
level  under  these  circumstances  can  require  lOx  to  lOOx  more  beam  power. 


Decision  Time 

Premise:  All  3D  microscopical  data  should  be  deconvolved  before 
viewing  and  that  the  object  is  to  get  the  best  2D  (optical  section) 
or  3D  data  possible  from  a  fixed  number  of  excitations  in  the 
specimen. 


MULTI-PHOTON  VERSUS  SINGLE-PHOTON 
EXCITATION 

The  advantages  of  multi-photon  excitation  include: 

•  Only  the  focus  plane  subjected  to  short  wavelength  damage: 
—  Less  damage  as  long  as  specimen  is  thick. 

—  Less  absorption  of  the  excitation  beam  by  heavily-stained 
specimens. 

•  Optical  sectioning  without  use  of  a  pinhole: 

—  On  thick,  scattering  specimens,  non-descanned  detection  of 
scattered  signal  can  be  up  to  300%  more  efficient  than 
normal  semi-confocal  detection. 

•  Longer  wavelengths  are  scattered  less  by  the  specimen:  better 
penetration  of  excitation. 

•  Can  excite  ultraviolet  (UV)  dyes  without  needing  UV  lasers  or 
UV  optics. 

•  System  acts  as  if  the  pinhole  is  infinitely  small,  and  perfectly 
aligned. 

•  Huge  Stokes  shift  means  that  dichroic  beam-splitters  and  filters 
can  be  more  efficient. 

•  Laser  often  tunable  to  optimal  excitation  wavelength. 

The  disadvantages  of  MP  include: 

•  Longer  wavelength,  somewhat  lower  resolution:  not  2x  worse, 
more  like  20%  to  40%. 

•  Excitation  duty  cycle  only  -10%  forces  one  to  work  closer  to 
singlet-state  saturation. 

•  Hard  to  arrange  >1  distinct  excitation  wavelengths. 

•  Requires  expensive,  pulsed  laser:  compare  the  productivity  of 
one  two-photon  versus  two  confocals. 

•  Higher  average  beam  power  destroys  cells  that  absorb  infrared 
(IR)  light. 

•  Many  dyes  emit  >10x  fewer  photons/molecule  before  being 
destroyed. 

•  Hard  to  use  with  fiber-optics  and  acousto-optical  deflectors. 

•  Laser-tweezer  effects  may  displace  some  small  particles. 
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•  Nonlinear  dependence  of  both  signal  and  damage  on  laser 
power  makes  it  crucially  important  to  set  this  parameter 
accurately. 

Widefield  Versus  Beam  Scanning 

Advantages  of  widefield  scanning  include: 

•  Well-optimized  systems  available. 

•  High  detector  QE  and  optical  simplicity  means  more  fluores¬ 
cent  photons  are  detected. 

Disadvantages  of  widefield  scanning  include: 

•  Cannot  view  optical  section  directly  (must  collect  several 
planes,  then  deconvolve). 

•  Image  noise  from  out-of-focus  fluorescence  eventually 
swamps  the  advantage  of  higher  detector  QE. 

•  Rapid  2D  imaging  difficult. 

•  CCD  read  noise  rises  with  square-root  of  the  readout  speed. 

The  question  between  widefield/deconvolution  and  confo¬ 
cal  is:  Assuming  that  you  have  set  up  both  instruments  in  an 
optimal  manner,  to  what  extent  does  the  statistical  noise  produced 
by  out-of-focus  light  from  your  WF  specimen  offset  the  increased 
QE  of  the  CCD  detector  used  in  widefield  scanning? 

•  Thin,  sparsely-stained  specimens  are  good  for  widefield/ 
deconvolution. 

•  Confocal  is  good  if  you  only  need  to  see  one  plane. 

The  questions  between  confocal  and  two-photon  excitation 

are: 

•  Do  you  need  to  look  more  than  20  pm  into  your  specimen? 

•  Are  you  content  to  excite  at  only  one  wavelength  at  a  time? 

•  Will  you  get  more  successful  live-cell  experiments  done  on 
one  two-photon  setup  than  on  two  confocals? 

•  Are  your  specimens  free  from  important  structures  that  absorb 
in  the  near  IR  or  that  might  be  damaged  by  three-photon  exci¬ 
tation  at  the  wavelengths  you  plan  to  use? 

•  Do  you  have  enough  room  for  a  two-photon  system? 

•  Can  you  get  along  without  video-rate  imaging  or  rapid  wave¬ 
length  switching? 


•  Do  you  have  the  technical  support  needed  to  keep  the  pulsed 
laser  happy  or  can  you  live  with  one  of  the  automated  fem¬ 
tosecond  lasers  with  a  reduced  wavelength  range?  (See  Chap¬ 
ters  21  and  40,  this  volume,  for  a  discussion  of  the  advantages 
of  using  wavelengths  >1000  nm  for  multi-photon  excitation.) 

If  YES  to  all  of  these,  then  the  greater  signal  collection  effi¬ 
ciency  and  reduced  bleaching  volume  should  make  two-photon 
your  best  bet.  These  characteristics  are  summarized  in  Table  35.1 

SUMMARY 

As  with  any  type  of  sophisticated  instrument,  the  best  results  can 
only  be  obtained  if  the  operator  uses  a  confocal  microscope  cor¬ 
rectly.  Many  aspects  of  proper  practice  are  either  similar  or  iden¬ 
tical  to  those  that  govern  the  operation  of  the  conventional  light 
microscope.  However,  there  are  two  controls,  one  contrast  effect 
and  one  optical  effect,  that  are  unique  to  the  confocal  microscope. 
The  limitations  imposed  on  confocal  imaging  by  these  four  factors 
—  pixel  size,  pinhole  size,  fluorescence  saturation,  and  chromatic 
magnification  error  —  must  be  understood  if  one  is  to  make  best 
use  of  this  highly  productive  instrument.  In  addition,  an  apprecia¬ 
tion  of  the  effects  of  photon  statistics  will  enable  users  to  get  the 
most  possible  information  from  their  specimens. 

Remember  that  when  using  a  confocal  microscope,  the  reso¬ 
lution  visible  in  the  final  image  can  be  limited  by  three  consider¬ 
ations  in  addition  to  the  optics  defining  the  volume  of  the  specimen 
that  is  sampled  by  both  the  laser  beam  and  the  confocal  pinhole. 
These  three  considerations  are: 

1.  The  choice  of  pixel  size  (referred  to  the  specimen):  Pixel  size 
is  a  function  of  the  zoom  setting  and  the  NA  and  magnifica¬ 
tion  of  the  objective  lens. 

2.  Deconvolving,  or  at  least  filtering,  the  3D  data  before  dis¬ 
playing  or  measuring  suppresses  artifactual  single-pixel  noise 
features  and  improves  S/N  by  averaging  the  photon  signal  over 
many  voxels. 

3.  The  S/N  of  the  intensity  signal  recorded  can  be  optimized  by 
maximizing  dye  contrast  and  counting  as  many  of  the  photons 
produced  by  the  specimen  as  possible. 


TABLE  35.1.  Three-Dimensional  Microscopy  Methods:  Summary 


Widefield/Deconvolution 

Confocal  Single-Beam 

Confocal  Disk  Scanner 

Two-Photon 

Effective  detector  QE 

60%-80%  (CCD) 

3%-12%  (PMT) 

30%-40%  (EM-CCD) 

3%-12%  (PMT) 

Detector  noise  (RMS 

±4 

<0.01 

0.01 

<0.01 

electrons  pixel) 

Peak  signal" 

>20,000 

20-100 

1-3000 

20-100  photons/pixel 

Acquistition 

Depends  on 

0.33-3 

0.002-10 

0.33-10 

time  (s/frame) 

CCD  readout 

Excitation  intensity 

10nW 

0.1  mW 

l-10gW 

10W 

power/jim2 

Excitation  wavelengths 

350-650nm 

Laser  lines  (fiber-coupled) 

2-3  laser  lines 

700-900  nm 

(Hg  arc) 

(dichroic  hard  to  change) 

Ti :  Sa 

Major  limitation 

Poisson  noise  from 

Singlet  saturation  limits 

Less  sectioning  than  single-beam 

Singlet  saturation 

out-of-focus  light 

data  rate 

Scattering  samples  defocus  signal 

Hard  to  excite  only  one 

Scattering  samples  defocus 

member  of  FRET  pair 

signal 

Damage/ excitation 

Galvo  limits  scan  speed 

greater  for  many  dyes 

"These  figures  assume  data  collected  from  same  volume  of  the  specimen  in  same  time  period.  Ideally,  the  raw  data  from  all  methods  should  be  deconvolved  (see  Chapter 
25,  this  volume). 
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Determine  and  use  the  optimal  pixel  and  pinhole  sizes  for  each 
objective  and,  if  the  confocal  bleaching  rate  seems  higher  than  that 
you  have  come  to  expect  from  normal,  photographic  epi-fluores- 
cent  microscopy,  you  are  probably  doing  something  wrong! 

While  the  points  that  we  have  tried  to  emphasize  in  this  chapter 
were  chosen  because  they  seemed  to  us  to  be  those  most  often 
misunderstood  by  novice  users  of  the  confocal  microscope,  they 
certainly  do  not  cover  all  of  the  possible  problem  situations  that 
can  occur  in  this  held.  A  better  understanding  of  more  complex 
problems  of  a  general  and  theoretical  nature  can  be  obtained  by 
reading  the  other  50  chapters  in  this  book  while  specific  problems 
that  apply  to  a  certain  type  of  instrument  or  dye  are  best  forwarded 
either  to  the  appropriate  supplier  or  to  discussion  on  the  Confocal 
(LISTSERVER)  E-mail  Network.  This  network  supports  active, 
informal,  and  informative  discussions  of  current  topics  in  confocal 
microscopy  by  over  two  thousand  scientists  having  a  wide  variety 
of  experience.  Anyone  on  the  Internet  can  subscribe  to  this  service 
by  sending  the  message:  “subscribe  confocal  <your  name>”  to  the 
address  “LISTSERV@UBVM.CC.BUFFALO.EDU”.  You  will 
then  receive  a  message  describing  the  rules  and  purpose  of  the 
group  as  well  as  future  postings  to  it. 
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APPENDIX 

Normal  Kohler  Illumination 

Proper  alignment  of  a  light  microscope  begins  with  the  micro- 
scopist  setting  up  the  optical  components  for  Kohler  illumination 
(see  also  Figs.  6.3  and  6.7,  this  volume).  This  mode  of  illumina¬ 
tion  establishes  two  important  sets  of  conjugate  image  planes. 
Conjugate  planes  are  sets  planes  perpendicular  to  the  optical  path 
that  focus  onto  each  other,  that  is,  the  same  features  will  be  visible 
in  each  plane  of  a  set  of  conjugate  planes,  although  the  size  may 
vary.  There  are  an  infinite  number  of  such  sets  of  conjugate  planes 
in  any  optical  system  but  only  two  of  these  sets  are  of  particular 
interest  in  microscopy:  aperture  planes  and  image  planes.  In 
Kohler  illumination,  the  aperture  planes  are  at  the  light  source,  the 
iris  diaphragm  of  the  condenser,  the  BFP  of  the  objective,  and 
the  eyepoint  (i.e.,  where  you  are  supposed  to  place  your  cornea). 
The  image  planes  are  the  field  diaphragm,  the  focus  plane  in  the 
specimen,  the  field  stop  in  the  eyepiece  (located  1  cm  inside  the 
tube  holding  the  occular),  and  the  retina  of  the  eye  or  the  camera 
film  plane.  These  two  sets  of  planes  are  reciprocally  related  to  one 
another  in  the  following  way:  light  passing  through  any  location 
(point)  in  any  image  plane  will  be  found  to  have  passed  at  a  spe¬ 
cific  angle  through  any  aperture  plane  and  vice  versa. 

To  produce  maximum  resolution,  light  must  leave  (or 
approach)  the  specimen  from  as  many  angles  as  permitted  by  the 
size  (or  NA)  of  the  objective.  This  means  that,  for  full  perfor¬ 
mance,  all  aperture  planes  must  be  filled  to  the  full  NA  of  the 
system.  In  addition,  the  optical  components  for  phase  and  DIC 
contrast  enhancement  need  to  be  placed  in  aperture  planes. 
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Setting  up  for  Kohler  illumination  ensures  that  these  condi¬ 
tions  are  met.  If  all  specimens  had  exactly  the  same  thickness  (+1 
pm)  and  refractive  index,  it  would  be  possible  to  preset  the  micro¬ 
scope  for  Kohler  illumination.  As  they  do  not,  one  must  set  it  up 
manually  every  time  a  new  specimen  is  mounted. 

To  achieve  Kohler  illumination  for  transmitted  light  micro¬ 
scopy,  one  must  first  center  and  focus  the  illuminator  (this  step  is 
usually  not  required  in  modern  microscopes  in  which  the  illumi¬ 
nator  is  built  into  the  microscope  base).  Remove  all  filters  from 
the  lamp  housing,  close  down  the  field  diaphragm,  and  focus  the 
collector  lens  to  obtain  a  sharp  image  of  the  filament  on  a  target 
placed  at  a  distance  of  ~25cm  away.  The  lamp  centering  screws 
should  then  be  adjusted  to  insure  that  the  collector  lens  and  the 
source  are  aligned  and  the  image  of  the  filament  is  centered.  [Note: 
If  using  a  Hg-arc  source  it  is  better  to  align  the  source  while 
viewing  an  aperture  plane  in  the  microscope  (and  using  a  lot  of 
neutral  density  filters!)  in  order  to  avoid  exposure  to  UV  light.] 

Next,  the  condenser  must  be  aligned.  Using  a  low-power 
objective,  focus  on  a  specimen  on  the  microscope  stage,  close 
down  the  field  diaphragm  (the  field  diaphragm  is  usually  in  the 
base  of  the  microscope  while  the  condenser  aperture,  described 
below,  is  actually  incorporated  into  the  condenser  that  is  mounted 
under  the  stage),  and  adjust  the  height  of  the  condenser  until  you 
obtain  a  sharp  image  of  the  edge  of  the  field  diaphragm  superim¬ 
posed  on  the  focus  plane  in  the  specimen.  Move  the  condenser 
adjustment  screws  to  center  the  opening  of  the  field  diaphragm  in 
the  field  of  view.  If  the  condenser  is  badly  misaligned,  you  may 
not  see  any  light  if  you  initially  close  the  field  diaphragm  down 
all  the  way.  In  this  case,  do  it  a  little  bit  at  a  time,  focusing  and 
centering  at  each  step. 
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The  field  diaphragm  should  then  be  opened  to  the  edges  of  the 
field  of  view  or  at  least  until  it  surrounds  the  area  of  interest. 
Opening  it  more  than  necessary  will  let  more  light  into  the  system 
than  you  need,  and  the  extra  light  will  cause  reflections  that 
produce  flare  and  hot  spots  that  reduce  image  contrast. 

The  working  NA  of  an  objective  used  in  transmission  is  set  by 
the  aperture  diaphragm.  Closing  the  condenser  iris  reduces  the 
angle  of  the  illumination  striking  the  specimen,  lowering  the  effec¬ 
tive  NA  of  the  system.  Opening  the  condenser  increases  the 
working  NA,  increasing  the  resolution  but  usually  reducing  the 
depth  of  field,  the  contrast  and  sometimes  introducing  flare.  Con¬ 
sequently,  final  adjustments  to  the  condenser  iris  are  a  compromise 
between  resolution  and  image  contrast. 

Because  the  condenser  is  often  fixed  to  the  stage,  the  objective 
and  condenser  usually  do  not  maintain  their  spacing  as  the  focus 
is  adjusted  or  as  additional  optical  components  such  as  filters  are 
introduced.  Therefore,  one  must  readjust  for  Kohler  and  reset  the 
field  diaphragm  and  the  condenser  iris  to  obtain  the  best  image 


possible  each  time  the  optical  system  is  changed  (e.g.,  by  chang¬ 
ing  objectives).  When  the  microscope  is  properly  adjusted  for 
Kohler  illumination,  the  result  is  maximum  lateral  and  axial  reso¬ 
lution,  optimum  contrast,  a  uniform  level  of  illumination  across 
the  field,  and  a  minimum  amount  of  flare  from  internal  reflections. 

Strictly  speaking,  conventional,  transmitted  light,  Kohler  illu¬ 
mination  is  not  necessary  for  epi-fluorescent  confocal  microscopy 
because  both  excited  and  emitted  light  travel  through  the  same 
objective.  However,  one  must  set  up  the  epi-illumination  system 
for  Kohler  in  order  to  ensure  even  illumination  and  to  use  the  epi- 
field  diaphragm  to  restrict  the  excitation  to  just  the  area  being 
imaged,  a  habit  that  greatly  reduces  non-specific  flare. 

Transmitted  Kohler  is  also  necessary  if  one  needs  an  optimal 
phase  contrast  or  bright-field  image  to  locate  the  most  interesting 
part  of  one’s  specimen  before  beginning  confocal  work.  In  addi¬ 
tion,  the  non-confocal  transmitted-light  detectors  provided  with 
many  instruments  will  only  work  correctly  if  the  condenser  has 
been  properly  adjusted  for  Kohler  illumination. 
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THE  ART  OF  IMAGING  BY 
CONFOCAL  MICROSCOPY 

This  chapter  is  aimed  at  giving  a  practical  overview  of  using  con¬ 
focal  microscopes  and  monitoring  their  performance.  Confocal 
microscopy  is  a  multi-skilled  endeavor  that  requires  a  considerable 
level  of  competence  in  sample  preparation,  knowledge  of  poten¬ 
tial  sources  of  artifact  and  appropriate  controls  samples,  instru¬ 
mentation  evaluation  and  operation,  computational  post¬ 
processing,  as  well  as  knowledge  of  a  multitude  of  fluorescent 
probes  (Hibbs,  2004). 

Do  not  overlook  the  specimen  itself  as  a  component  of  the 
imaging  process.  As  living  cells  are  often  particularly  optically 
active,  it  is  important  to  understand  the  mechanisms  of  intentional 
as  well  as  unintentional  contrast  generation  in  confocal  micro¬ 
scopy.  Appropriate  controls  are  crucial  to  the  production  of 
reliable  multi-dimensional  image  data. 

Balancing  Multiple  Parameters 

The  goal  of  instrument  adjustment  during  image  collection  on  a 
confocal  microscope  is  to  balance  a  number  of  potentially  con¬ 
flicting  parameters  on  the  instrument.  The  main  paradox  involves 
achieving  adequate  spatial  resolution  while  keeping  cells  alive 
and  healthy  or  at  least  unbleached.  Recording  dynamic  cellular 
processes  with  minimal  light  exposure  may  require  a  significant 
compromise  in  the  degree  of  resolution  that  can  be  obtained. 

Every  image  should  be  acquired  using  parameters  that  maxi¬ 
mize  its  information  content  within  the  limitations  of  the  selected 
mode  of  microscopy.  Post-acquisition  processing,  such  as  decon¬ 
volution  or  three-dimensional  (3D)  Gaussian  filtering,  histogram 
stretching,  or  projections  of  multiple  planes,  can  be  employed  to 
reveal  the  important  details  under  photon-limited  conditions  (see 
Chapters  32  and  48,  this  volume). 

MONITORING  INSTRUMENT  PERFORMANCE 

The  ability  to  determine  relative  dye  concentrations,  colocaliza¬ 
tion,  Forster  resonance  energy  transfer  (FRET),  shifts  in  emission 
spectra,  and  identity  of  ambiguous  fluorescent  signals  is  highly 
dependent  on  the  accuracy  and  precision  of  a  multitude  of  instru¬ 


ment  functions.  To  ensure  consistent  peak  performance,  one  must 
monitor  performance  at  regular  intervals  and  keep  good  docu¬ 
mentation.  Benchmarks  of  functionality  can  be  organized  around 
four  major  subsystems  common  to  scanning  microscopes:  (1)  illu¬ 
mination  source,  (2)  signal  detection  subsystem,  (3)  scan  raster 
mechanisms,  and  (4)  lenses  and  other  optical  elements.  In  order 
for  accurate  and  precise  data  to  be  recorded,  not  only  must  each 
subsystem  perform  well  on  its  own  merits,  but  they  must  also  be 
carefully  orchestrated  to  work  together  in  synergy. 

Illumination  Source 

Power  Measurement 

Obtaining  consistent  results  from  a  confocal  microscope  requires 
knowledge  of  the  actual  laser  power  level  at  the  specimen  and  its 
temporal  stability.  A  well-equipped  microscopy  facility  needs  a 
reliable,  accurate  power  meter  (a  photometer)  suitable  for  making 
measurements  at  the  specimen  plane.  Otherwise,  low  signal  level 
can  be  wrongly  attributed  to  poor  specimen  preparation  or  prob¬ 
lems  in  the  detection  channel  when  the  actual  fault  is  low  excita¬ 
tion  power.  Such  devices  may  be  acquired  from  optics  companies 
such  as  Newport  (Irvine,  CA),  Melles  Griot  (Carlsbad,  CA),  and 
Linos  Photonics  (Milford,  MA),  among  others.  Adjustments  such 
as  the  re-alignment  of  the  fiber  to  the  laser  output  may  occasion¬ 
ally  be  needed.  Changes  in  the  illumination  stability  often  reflect 
the  gradual  deterioration  of  lasers  and  arc  bulbs  over  the  long  term. 
As  a  laser  cavity  nears  the  end  of  its  lifetime,  it  may  also  display 
a  reduced  output  level  that  may  be  abrupt  and  very  obvious.  In 
lasers  with  multiple  lines,  it  is  common  for  one  line  to  lose  power 
before  the  others. 

Tracking  long-term  changes  in  power  output  requires  the  use 
of  standardized  settings.1  The  appropriate  dichroic  mirror  should 
be  used  for  each  laser  line.  Because  objectives  vary  in  terms  of 
their  transmission  and  pupil  diameter,  it  is  important  to  use 
the  same  objective  for  standard  laser  power  measurements.  To 
measure  laser  power,  use  the  beam-parking  or  point-bleaching  fea¬ 
tures  to  place  a  diffraction-limited  point  near  the  center  of  the  field 
of  view,  and  turn  on  the  beam  for  an  exposure  of  ~30  s.  As  the  laser 


'On  instruments  where  the  beam  expander  is  under  software  control,  the 
control  should  be  disabled  or  it  should  be  removed  from  the  optical  path. 
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power  is  not  modulated  during  beam-parking  it  is  ideal  for  power 
measurement.  On  older  scanners  that  do  not  have  acousto-optic 
tuning  filter  (AOTF)  beam  blanking  during  retrace,  use  the 
maximum  zoom  setting. 

The  AOTF  settings  (or  neutral  density  filters)  should  be  con¬ 
figured  to  provide  maximum  power,  and  if  there  is  a  coarse  adjust¬ 
ment  for  the  laser  power,  this  should  be  maximized  as  well.  The 
power  meter  sensor  should  be  carefully  positioned  over  the  field 
of  view  near  the  focal  plane  of  a  dry  objective  and  perpendicular 
to  the  optical  axis.  Because  most  sensors  are  not  equally  sensitive 
across  the  entire  surface,  use  a  stable  sensor  mounting  system  so 
as  to  use  the  same  area  of  the  sensor  surface.  Be  sure  the  laser 
power  meter  is  set  to  the  appropriate  detection  wavelength.2 

To  measure  the  power,  start  the  point  bleach  sequence  for  a 
single  laser  line  and  record  the  meter  readout.  Repeat  for  each  laser 
line  in  turn.  For  accurate  readings  the  room  should  be  dark,  the 
lasers  adequately  warmed  up,  and  several  measurements  should  be 
taken  to  check  data  precision. 

Similar  readings  taken  in  the  standard  scanning  mode  will  be 
lower  than  this  value  if  the  laser  beam  is  blanked  by  the  AOTF 
during  the  flyback  and  during  the  interval  between  consecutive 
frames. 

Laser  Stability 

Although  laser  stability  on  the  order  of  seconds  can  be  measured 
with  a  photometer,  rapid  oscillations  of  laser  output  are  beyond  the 
temporal  resolution  of  the  power  meter.  To  measure  high  frequency 
stability,  image  a  bright  standardized,  stable  fluorescent  sample 
such  as  a  coverslipped  piece  of  fluorescent  plastic  (Chroma  Tech¬ 
nologies  Inc.,  Rockingham,  VT;  Applied  Precision,  Issaquah,  WA), 
or  a  high  concentration  of  fluorescent  dye  in  oil  (fluorescent  seas; 
see  Chapters  2  and  35,  this  volume ;  also  Model  and  Burkhardt, 
2001).  To  reduce  photobleaching,  use  a  freshly  prepared,  stan¬ 
dardized  dilution  of  fluorochrome  in  an  index-matched  solvent.3 

On  inverted  microscopes,  use  chambered  coverslips  with  mul¬ 
tiple  wells  (Nalge  Nunc  International,  Rochester,  NY )  to  hold  the 
fluorescent  liquid.  Carefully  prepared  concavity  slides  (Electron 
Microscopy  Sciences,  Hatfield,  PA)  can  be  used  as  an  alternative 
on  upright  microscopes.  It  is  a  good  idea  to  centrifuge  the  test  solu¬ 
tion  in  order  to  remove  particulates  prior  to  use.  Focus  the  micro¬ 
scope  into  the  bulk  fluorescent  standard  near  (but  not  at)  the 
dye-coverslip  interface. 

Set  up  a  two-dimensional  (2D)  time  series  with  an  appropriate 
acquisition  interval  and  overall  duration.  The  scan  area  can  be 
reduced  to  only  a  few  lines  in  order  to  reduce  the  acquisition  inter¬ 
val  for  greater  temporal  resolution.  Settings  should  be  configured 
such  that  the  laser  power  is  standardized  to  a  value  high  enough 
to  provide  a  reasonably  noise-free  image,  yet  low  enough  that 
bleaching  is  not  a  problem,  and  the  detectors  adjusted  such  that  the 
measured  mean  pixel  intensity  is  within  the  linear  response  range 
(see  Chapter  2,  this  volume ).  Analyze  the  acquired  dataset  by  eye 
for  obvious  indications  of  instability,  such  as  some  lines  being 


2  Most  meters  actually  measure  the  number  of  photons/second.  As  red  photons 
carry  less  energy  than  blue  ones,  the  meter  needs  to  know  the  wavelength  in 
order  to  calculate  the  energy/second  or  power. 

3  Bleaching  can  be  removed  as  a  variable  by  using  a  coverslipped,  first-surface 
mirror  as  the  test  specimen.  When  viewed  using  reflected  light,  such  a  spec¬ 
imen  produces  a  very  high  signal  level  (and  consequently  a  very  low  level  of 
Poisson  noise),  although  system  vibrations  are  more  likely  to  account  for  any 
variations  that  you  see  due  to  the  surface  going  in  and  out  of  focus,  than  by 
laser  instability.  This  vibration  sensitivity  can  be  reduced  by  using  an  objec¬ 
tive  with  low  NA. 


brighter  than  others,  etc.,  or  by  graphing  the  mean  intensity  of  each 
time  point.  Small,  periodic  variations  will  be  made  more  obvious 
in  a  Fourier  transform  of  the  data. 

Scan  Raster  and  Focus  Positioning 

Spatial  measurements  and  accurate  morphometric  classification 
rely  on  the  accuracy  and  precision  of  the  mechanisms  used  to 
move  the  focal  volume  through  the  specimen  in  the  x-,  and  z- 
dimensions.  From  a  practical  standpoint,  measures  of  the 
lateral  and  axial  resolution  on  a  laser-scanning  instrument  are 
meaningless  unless  the  accuracy  and  precision  of  pixel-to-pixel 
spacing  in  the  x,  y,  and  z  can  be  verified.  A  non-uniform  scan 
speed  will  also  result  in  differential  exposure  of  localized  areas 
within  the  field  of  view  (Carlsson,  1991).  This  confounds  accurate 
photometry  and  can  result  in  increased  phototoxicity  when  living 
specimens  are  being  imaged  (Fig.  36.1). 

x  and  y  Galvanometers 

Standards  for  verifying  lateral  scan  accuracy  are  easily  purchased 
or  fashioned  from  readily  available  components.  Because  the 
response  of  the  galvanometer  to  a  change  in  either  zoom  or  scan 
speed  frequently  departs  from  an  ideal  linear  relationship,  it  is  not 
adequate  to  confirm  scan  calibrations  at  a  single  magnification  and 
horizontal  scan  frequency,  and,  thus,  one  needs  a  standard  speci¬ 
men  that  can  be  used  at  high  as  well  as  low  zoom  settings.  A  grid 
standard  provides  an  efficient  means  of  evaluating  both  x-  and  y- 
scan  rasters  simultaneously.  Suitable  standards  include  a  2000- 
mesh  transmission  electron  microscope  (TEM)  grid  mounted  in 
appropriate  media  (Electron  Microscopy  Sciences,  Hatfield,  PA), 
an  etched  silicon  10  pm  grid  standard  for  reflected  light  microscopy 
(Electron  Microscopy  Sciences),  a  carbon  replica  of  a  diffraction 


FIGURE  36.1.  Differential  phototoxicity  as  a  result  of  uneven  scan  speed 
across  the  field  of  view.  In  this  image,  metabolically-active  cells  are  stained 
with  fluorescein  diacetate  (green);  the  onset  of  propidium  iodide  (red)  indicates 
compromise  of  the  plasma  membrane  associated  with  cell  death.  The  speed  of 
the  scan  raster  was  slower  at  the  edges  than  in  the  center  of  the  field  of  view 
in  this  case. 
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FIGURE  36.2.  A  well-calibrated  system.  (A)  The  standard  in  this  case  is  a  reflective  etched  silicon  standard  designed  for  reflected  light  microscopy.  Each  square 
is  10  microns  per  side;  (B)  a  graph  of  intensity  as  a  function  of  position  on  the  black  line  in  (A).  Quantitative  measurements  confirm  the  accuracy  and  precision 
of  the  x-  and  v-galvanometers  in  this  example. 


grating  or  MBL-NNF  Test  Slide4  (see  Chapter  35,  this  volume),  and 
related  products  such  as  the  Microscopic  Image  Analysis  Microm¬ 
eter  (Edmunds  Optics,  Barrington,  NJ)  and  the  Richardson  test 
slide  (Electron  Microscopy  Sciences).  If  a  TEM  grid  (or  other 
uncharacterized  standard)  is  used,  the  accuracy  and  precision  of 
center-to-center  spacing  between  grid  bars  can  be  accurately  deter¬ 
mined  using  a  properly  calibrated  widefield  system. 

Measurements  of  the  center-to-center  (side  of  one  grid  bar  to 
the  same  side  of  the  next)  distance  at  different  points  within  the 
field  of  view  should  be  conducted  for  both  x-  and  y-axes  in  con¬ 
ventional  xyz-imaging  mode.  It  is  important  to  realize  the  effect 
that  under-sampling  can  have  on  such  measurements;  sampling 
intervals  for  the  scan  resolution  should  satisfy  the  Nyquist  criteria 
(see  Chapters  2,  4,  and  35,  this  volume).  A  well-calibrated  instru¬ 
ment  will  have  accuracy  within  1%  of  the  known  value  and  less 
than  1%  variability  between  measurements.  This  should  hold  at 
both  high  and  low  zoom  values,  and  across  all  scan  speeds.  Results 
from  an  instrument  that  is  well  calibrated  can  be  seen  in  Figure 
36.2,  and  the  opposite  is  shown  in  Figure  36.3. 

z-Drive  Mechanism 

The  performance  of  the  z-stepping  or  z-scanning  system  can  be 
evaluated  using  large  diameter  beads,  for  example,  10.2  |im  (see 
Table  36.1),  carefully  mounted  in  an  index-matched  [i.e.,  refrac¬ 
tive  index  (RI)  =  1.515;  see  Chapters  8  and  20,  this  volume \  semi¬ 
solid  or  high-viscosity  mountant.  High  axial  resolution  requires 
proper  refractive  index  with  a  high  numerical  aperture  (NA)  oil- 
immersion  objective  (designed  for  RI  of  1.51;  polystyrene  beads 
have  RI  between  1.55  and  1.59).  Do  not  use  a  dry  objective 
(Carlsson,  1991). 

Commercially  prepared  fluorescent  bead  calibration  slides  can 
be  purchased  from  Spherotech  and  Molecular  Probes  (Table  36.1). 
z-Calibration  measurements  should  be  performed  at  both  high  and 
low  magnification  by  making  xz-images  across  the  center  of  a 
bead.  The  height  of  several  beads  from  several  levels  within  the  z- 
plane  should  be  measured.  The  diameter  of  the  bead  with  respect 


4  Louie  Kerr  (Marine  Biological  Laboratory,  Woods  Hole,  MA)  has  fabricated 
suitable  test  slides  in  the  past  using  electron  beam  lithography  at  the  National 
Nanofabrication  Facility  (NNF)  at  Cornell  University.  He  expects  to  do  so  in 
the  future. 


to  z  is  taken  as  the  distance  between  half-maximum  intensity 
values  on  the  steep-sloped  rising  and  falling  intensity  gradients  that 
indicate  the  top  and  bottom  of  the  bead.  The  accuracy  and  preci¬ 
sion  of  the  values  obtained  are  then  compared  to  the  stated  toler¬ 
ances  of  the  calibration  beads.  Unless  the  calibration  beads  have 
been  severely  compressed,  the  mean  value  and  standard  deviation 
taken  from  a  sample  should  be  the  same  in  the  xz-image  as  in  the 
xy-image  [Fig.  36.4(A,B)].  A  large  systematic  departure  from  the 
values  as  measured  with  respect  to  the  xy-plane  likely  represents 
poor  performance  of  the  z-movement  mechanism,  miscalibration 
of  the  control  mechanisms,  or  the  use  of  a  mounting  or  immersion 
liquid  having  the  wrong  RI  for  the  lens  in  use. 

z-Positioning  Stability 

In  both  volumetric  imaging  and  2D  time-lapse  imaging,  it  is  impor¬ 
tant  that  the  relationship  between  the  position  of  the  sample  and 
the  optics  is  stable.  As  noted  in  the  earlier  footnote,  this  can  be 
assayed  using  a  mirror  standard  viewed  in  reflected  light 
(described  below  under  tests  for  axial  resolution)  over  a  time 
course.  Motion  of  structures  in  an  xy-image  of  the  mirror  surface 
shows  vibration,  while  collecting  a  time-lapse  series  of  3D  images 
through  the  reflective  interface  will  reveal  stage  or  focus  drift. 
Rapid  xz-scanning  over  time  can  assist  in  resolving  high  frequency 
drift.  After  acquisition,  the  position  of  the  reflection  intensity 
maximum  with  respect  to  z  can  be  found  at  each  time  point  and 
plotted  as  a  function  of  time. 

Vibration  can  be  reduced  by  using  a  vibration-isolation  table 
and  making  sure  that  all  the  wires  or  hoses  connecting  the  table  to 
the  outside  world  are  bent  to  reduce  their  ability  to  transmit 
mechanical  forces.  The  most  likely  cause  of  drift  is  temperature 
change,  often  caused  by  the  air  conditioning  going  on  and  off,  or 
in  actual  use,  by  components  involved  in  heating  the  specimen  and 
the  objective  for  living-cell  studies.  The  solution  is  to  insulate  your 
setup  from  changes  in  room  air  and  run  stage-heater  controllers  in 
the  open-loop  mode  or  place  the  entire  microscope  in  a  heated 
enclosure  (see  Chapter  19,  this  volume). 

Optical  Performance  and  Objective  Lenses 

Because  the  performance  of  the  objective  is  crucial  to  the  overall 
performance  of  an  imaging  system  (see  Chapter  11,  this  volume). 
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FIGURE  36.3.  A  malfunctioning  system.  (A)  Observation  at  low  magnification  indicates  problems  with  both  accuracy  and  precision  between  scans.  Two  scans 
taken  in  sequence  are  indicated  by  the  red  and  green  channels  of  the  image.  (B)  A  scan  taken  at  higher  magnification  on  the  same  instrument.  The  standard  in 
this  case  is  milled  into  a  coverslip  surface  with  a  focused  ion  beam  (kind  gift  of  Dr.  Carlos  Martinez,  NIST,  Gaithersburg,  MD).  Again,  red  and  green  compo¬ 
nents  of  the  image  reflect  the  lack  of  precision.  Wavering  in  the  scan  raster  appears  as  distortion  in  the  grid  standard.  The  square  pattern  appears  rectangular 
(narrower  in  the  x-direction)  because  of  poor  calibration  of  the  x-scan  galvanometer. 


TABLE  36.1.  Sources  of  Fluorescent  Beads 


Manufacturer 

Address 

Telephone 

Web  site/E-Mail 

Bangs  Laboratories,  Inc. 

9025  Technology  Drive 

Fishers,  IN  46038-2886 

USA 

Phone:  +1  317-570-7020 

USA:  +1  800-387-0672 

Fax:  +1  317-570-7034 

www.bangslabs.com 
info  @  bangslabs.com 

Brookhaven  Instruments,  Ltd. 

Chapel  House,  Stock  Wood 
Redditch,  Worcestershire 

B96  6ST,  UK 

Phone:  +44  (0)  1386-792727 

Fax:  +44  (0)  1386-792720 

www.brookhaven.co.uk 

enquiries@brookhaven.co.uk 

Duke  Scientific  Corporation 

2463  Faber  Place 

Palo  Alto,  CA  94303 

USA 

Phone:  +1  650-424-1177 

USA:  +1  800-334-3883 

Fax:  +1  650-424-1158 

www.dukescientific.com 

info@dukescientific.com 

G.Kisker  GbR 

Produkte  f.d.Biotechnologie 

Postfach  1329 

48543  Steinfurt 

Germany 

Phone:  +49  2551-864310 

Fax:  +49  2551-864312 

www.kisker-biotech.com 

contact  @  kisker-biotech.com 

Molecular  Probes,  Inc. 

2985 1  Willow  Creek  Road 

Eugene,  OR  97402 

USA 

Phone:  +1  541-465-8300 

Fax:  +1  541-335-0504 

Order:  +  1  800-438-2209 

www.probes.invitrogen.com 
order  @  probes  .com 

Poly  sciences,  Inc. 

400  Valley  Road 

Warrington,  PA  18976 

USA 

Phone:  +1  215-343-6484 

USA:  +1  800-523-2575 

Fax:  +1  800-343-3291 

www.polysciences.com 

info@polysciences.com 

Sigma- Aldrich  Corp. 

3050  Spruce  Street 

St.  Louis,  MO  63103  USA 

Phone:  +1  314-771-5765 

USA:  +1  800-325-3010 

Fax:  +1  314-771-5757 

www.sigmaaldrich.com 
OC_DOM_HC  @  sial.com 

Spherotech,  Inc. 

1 840  Industrial  Dr. 

Suite  270  Libertyville,  IL 
60048-9467 

USA 

Phone:  +1  847-680-8922 

USA:  +1  800-368-0822 

Fax:  +1  847-680-8927 

service  @  spherotec.com 
j  wsphero  @  aol .  com 

These  vendors  offer  beads  down  to  subresolution  sizes  in  single  and  multiple  wavelengths.  Many  offer  larger  beads  in  a  variety  of  configurations.  Larger  beads  may  be 
solidly  fluorescent,  contain  surface  rings,  overlapping  concentric  rings,  concentric  rings  that  do  not  overlap,  and  combinations  of  solid  and  ring  labels.  Some  vendors  in 
this  list  also  offer  silica,  glass,  or  melamine  beads  that  possess  different  physical  properties  compared  to  the  commonly  used  polystyrene  or  latex. 

This  list  does  not  include  vendors  of  fluorescent  beads  where  the  bead  preparations  are  specifically  designed  for  flow  cytometry. 
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all  objectives  should  be  tested  for  performance.  Objectives  do  vary 
within  brand  and  type  and  should  be  carefully  compared  before 
purchase  and  then  regularly  monitored  (see  Chapter  11,  this 
volume,  for  more  details).  The  most  common  tests  of  imaging 
optics,  those  for  lateral  and  axial  resolution  and  chromatic  aberra¬ 
tion,  should  be  performed  periodically  as  objective  lens  perfor¬ 
mance  may  deteriorate  over  time. 

Spherical  aberration  created  by  the  specimen  is  a  serious  con¬ 
straint  in  the  creation  of  high-quality  images,  particularly  when 
imaging  living  biological  material.  As  described  in  detail  in  Chap¬ 
ters  2,  7,  and  20,  when  using  an  oil-immersion  lens  to  image  living 
biological  material  there  will  be  a  serious  mismatch  in  RI  between 
the  design  parameters  of  the  objective  and  the  aqueous  sample. 
Even  using  an  oil-immersion  objective  with  a  sample  mounted  in 
a  media  with  a  relatively  high  RI,  such  as  glycerol  (RI  =  1.47), 
there  is  still  be  a  significant  level  of  spherical  aberration  present. 

The  best  plan  is  to  use  objective  lenses  designed  for  the 
imaging  conditions  under  which  they  are  being  used.  Some  objec¬ 
tives  are  equipped  with  a  collar  to  correct  for  routine  variations  in 
coverslip  thickness,  as  noted  in  Table  36.2.  However,  RI  mis¬ 
matches  can  also  be  partially  corrected  by  coverslip-thickness 
correction  collars  or  using  the  automated  spherical-aberration  cor¬ 
rector  (SAC)  now  offered  by  Intelligent  Imaging  (Denver,  CO). 
This  will  greatly  diminish  spherical  aberration,  and  result  in 
improved  resolution  and  signal  level,  particularly  when  imaging 
deep  into  the  sample.  The  effect  of  spherical  aberration  on  signal 
intensity  is  readily  demonstrated  by  collecting  an  .xz-image  into  a 
dilute  fluorescent  solution  such  as  0.01%  coumarin  in  water  (Fig. 
36.5).  In  Figure  36.5(A,B),  a  40x  NA  0.60  non-immersion  dry 
objective  lens  has  been  used  to  image  into  a  dilute  coumarin  solu¬ 
tion  contained  in  a  sealed  imaging  chamber.  The  base  of  the 
chamber  is  a  0.17  mm  thick  coverslip,  with  the  correction  collar 
on  the  objective  set  for  0.17  mm  [Fig.  36.5(A)]  or  0.5  mm  [Fig. 
36.5(B)].  Even  when  the  spherical  aberration  collar  is  correctly 
matched  to  the  thickness  of  the  coverslip,  the  intensity  profile 


TABLE  36.2.  Coverslip  Thickness  and  Refractive  Index 


Designated  Size 

Tolerance  (mm) 

#00 

0.06-0.08 

#0 

0.08-0.12 

#1 

0.13-0.16 

#1.5 

0.16-0.19 

#2 

0.19-0.23 

#3 

0.28-0.30 

#4 

0.38-0.42 

#5 

0.50-0.60 

Refractive  Index  at  20°C  for  Commonly  Used  Glass 

Desag  D-263 

(546nm)=  1.5255 

(588  nm)=  1.5230 

Corning  0211 

(589  nm)  =  1.5230 

diminishes  significantly  as  the  focus  plane  moves  away  from  the 
coverslip  [Fig.  36.5(A)].  This  drop-off  in  intensity  is  due  to  the 
RI  mismatch  between  the  immersion  media  (in  this  case,  air) 
and  the  sample  (water).  If  the  spherical  aberration  correction  collar 
is  adjusted  to  0.50  mm,  while  retaining  a  coverslip  thickness  of 
0.17  mm,  then  the  zone  of  maximum  intensity  in  the  image 
is  moved  farther  into  the  sample  [Fig.  36.5(A)].  This  simple 
adjustment  can  be  used  to  collect  good  images  over  a  limited  range 
of  depths  within  living  tissue  samples  using  a  non-immersion  air 
lens. 

Figure  36.5(C)  shows  the  performance  of  a  40x  NA  1.20 
water-immersion  lens  (with  0.17  mm  coverslip  correction). 
Because  the  objective  is  used  with  water  on  both  sides  of  the  cov¬ 
erslip,  the  total  thickness  of  water  and  glass  between  the  front  of 
the  objective  and  the  focus  plane  does  not  change  with  focus  posi¬ 
tion  and  therefore,  intensity  of  the  signal  in  this  xz-image  does  not 
diminish  throughout  the  full  depth  of  the  scan  (200  pm). 


FIGURE  36.4.  Checking  ^-positioning  calibration  using  fluo¬ 
rescent  beads.  (A)  10.2  |im  polystyrene  spheres  in  immersion  oil. 
The  data  maintain  the  correct  aspect  ratio,  and  accurate  mea¬ 
surements  in  the  z-axis  are  possible  with  an  oil-immersion  lens. 
(B)  Measurement  of  calibration  beads  in  the  z-axis  under  prop¬ 
erly  index-matched  conditions  permits  evaluation  of  the  z-drive 
calibration.  The  midpoint  on  each  of  the  steep-sloped  lines 
depicting  the  edge  response  of  the  top  and  bottom  limits  of  the 
bead  are  found  (green/red  dotted  lines).  The  distance  between 
these  points  on  the  x-axis  of  the  intensity  plot  is  the  height  of  the 
bead.  Note  that  the  intensity  of  the  bead  is  not  constant  through 
the  diameter  of  the  bead  due  to  absorption  of  excitation  light  by 
the  bead.  For  this  reason,  the  halfway  points  of  the  bottom  and 
top  edge  response  are  located  at  different  intensity  levels.  In  this 
case  the  .--drive  (a  --galvanometer  stage)  is  performing  well,  and 
measurement  of  the  axial  dimensions  of  the  calibration  beads  are 
both  accurate  and  precise.  (C)  The  same  beads  in  a  classic  9 : 1 
glycerol/PBS  buffer  mountant  of  RI  =  1.43  viewed  with  an  oil- 
immersion  lens.  Significant  distortion  in  the  --axis  is  evident  due 
to  spherical  aberration  as  well  as  self-lensing.  Accurate  mea¬ 
surements  with  respect  to  z  are  not  possible  under  these  condi¬ 
tions.  (D)  10.2  micron  polystyrene  spheres  in  water  (RI  =  1.32) 
viewed  with  an  oil-immersion  lens.  Distortion  in  the  z-axis  is 
exacerbated  in  accordance  with  Snell’s  law. 
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FIGURE  36.5.  Correcting  for  spherical  aberration,  xz-images 
were  collected  on  a  Zeiss  510  META  confocal  microscope  while 
imaging  into  (1)  a  dilute  aqueous  fluorescent  coumarin  solution 
(approximately  0. 1  m M)  and  (2)  green  fluorescent  plastic.  The 
488  nm  blue  laser  line  from  an  argon-ion  laser  was  used  for  exci¬ 
tation,  and  the  fluorescent  light  between  510  and  520  nm  was 
collected  into  a  single  detection  channel.  Each  image  represents 
a  single  line  collected  at  increasing  depth  ( 1  pm  deeper  per  scan) 
within  the  solution  or  plastic.  The  position  of  the  coverslip  is 
marked  with  an  arrow  (<— ).  All  images  were  obtained  using  a 
0.17  mm  thick  coverslip,  with  various  adjustments  of  coverslip 
correction  collars  as  described  below.  All  images  were  collected 
using  a  pinhole  size  of  1  Airy  disk.  (1)  Imaging  into  fluorescent 
aqueous  media:  a  Zeiss  LD-Achroplan  40x  0.60  NA  dry  lens 
was  used  to  collect  images  into  the  fluorescent  aqueous  media 
with  the  coverslip  thickness  correction  collar  set  to  (A)  0.17  mm 
and  (B)  0.50  mm.  A  Zeiss  C-Apochromat  40  x  1.2  NA  water- 
immersion  lens  with  coverslip-correction  collar  set  to  0.17  mm 
(C),  and  a  Zeiss  Plan-Apochromat  63  x  1.4  NA  oil-immersion 
lens  with  0.17  mm  coverslip  correction  (D)  were  used  to  collect 
xz-images  in  the  fluorescent  coumarin  solution.  (2)  Imaging  into 
fluorescent  plastic:  a  0.17  mm  thick  coverslip  was  placed  on  the 
surface  of  a  small  sheet  of  green  fluorescent  plastic  using  a  small 
amount  of  immersion  oil  between  the  coverslip  and  the  plastic, 
xz-images  were  collected  using  a  Zeiss  Plan-Apochromat 
63  x  1.4  NA  oil-immersion  lens  with  0.17  mm  coverslip  cor¬ 
rection  (E)  and  a  Zeiss  C-Apochromat  40  x  1.2  NA  water- 
immersion  lens  with  the  coverslip  thickness  correction  collar  set 
to  0.17  mm  (F). 
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In  Figure  36.5(D),  a  63x  NA  1.40  oil-immersion  lens  (cor¬ 
rected  for  0.17  mm  thick  coverslip)  has  been  used  to  image  into 
the  dilute  coumarin  solution.  The  severe  RI  mismatch  between 
the  immersion  media  (oil)  and  the  sample  (aqueous  solution  of 
coumarin)  results  in  a  significant  reduction  in  the  intensity  of  the 
fluorescent  signal  with  increased  imaging  depth.  The  j-position  of 
maximum  signal  detection  is  nearly  always  located  close  to  the 
coverslip  when  using  an  oil  lens  to  image  into  samples  mounted 
in  aqueous  media.  Oil-immersion  lenses  do  not  have  spherical 
aberration  correction  collars. 

Lateral  Resolution 

The  lateral  resolution  of  a  digital  microscope  is  usually  measured 
with  either  a  subresolution  point  source  standard  or  a  special  test 
slide  (see  Chapter  35,  this  volume ;  Stark  et  al,  2003). 

Subresolution  Beads 

The  3D  diffraction  pattern  characteristic  of  a  microscopic  imaging 
system  is  known  as  a  point  spread  function  (PSF;  see  Fig.  36.6; 
Chapters  7,  11,  and  20,  this  volume ;  also  Hiraoka  et  al.,  1990; 
Shaw  and  Rawlins,  1991;  Wilson  and  Juskaitis,  1995;  Scalettar  et 
al.,  1996;  Booth  and  Wilson,  2001;  Wallace  et  al,  2001;  Cox  and 
Sheppard,  2004).  The  PSF  defines  the  spatial  resolution  of  a  par¬ 
ticular  optical  train.  Where  deconvolution  software  is  employed, 
one  should  record  a  library  of  PSF  images  for  each  objective  lens 
and  wavelength  (see  Chapter  11,  this  volume). 

Fluorescent  polystyrene  beads  measuring  less  than  200  nm  (or 
even  less  than  lOOnm)  can  be  purchased  prelabeled  with  dyes  suit¬ 


able  for  measuring  resolution  at  different  wavelengths  (see  Table 
36.1  for  a  list  of  suppliers).  They  are  conveniently  prepared  for 
imaging  by  allowing  a  few  microliters  of  a  bead  suspension  to 
spread  over,  and  dry  onto,  the  surface  of  a  coverslip.5  This  ensures 
that  all  the  beads  are  in  the  same  focal  plane  and  also  eliminates 
Brownian  motion.  Most  aqueous  mounting  agents  used  for  fluo¬ 
rescent  samples  can  be  used  with  beads,  but  take  care  when  using 
solvent-based  mountants  or  immersion  oil  as  the  dyes  may  leach 
out  or  the  beads  may  even  dissolve. 

The  PSF  is  acquired  by  collecting  a  '-scries  extending  about 
6  to  8  pm  axially  above  and  below  the  bead.  The  pixel  size  and 
focus  interval  should  be  appropriate  for  the  objective  (i.e.,  0.08  pm 
pixels  and  0.3  pm  focus  step  for  NA  1.4,  and  0.16  pm  pixels  and 
1.2  pm  focus  step  for  NA  0.7). 

Select  single  beads  that  are  well  separated  from  others  and  near 
the  center  of  the  field.  Because  Poisson  noise  imparts  a  great 
degree  of  ambiguity  to  such  images,  either  select  a  slow  scan  speed 
or  Kalman  average  at  least  three  scans  during  image  collection. 
Most  deconvolution  software  packages  capable  of  utilizing  a  mea¬ 
sured  PSF  also  have  a  provision  for  averaging  several  individual 
PSF  volumes  together  to  reduce  from  the  effects  of  noise,  drift, 
focusing  errors,  and  intensity  fluctuations  in  the  illumination. 


5  The  coverslip  should  be  first  cleaned  by  soaking  in  1%  HC1  in  70%  ethanol 
for  5  min,  then  rinsing  thoroughly  with  H20  and  air  drying.  Application  of  the 
beads  in  70%  ethanol  will  aid  in  dispersion,  although  prolonged  exposure  may 
extract  the  dye. 
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FIGURE  36.6.  Determining  xy-  and  --resolution  using  fluorescent  beads.  The 
x- ,  v-.  and  --resolution  of  a  confocal  microscope  can  be  determined  by  imaging 
latex  beads  or  quantum  dots  that  are  smaller  than  the  resolution  limit  of  the 
instrument.  The  beads  will  need  to  be  immobilized  by  attaching  them  to  the 
glass  coverslip  (e.g.,  by  coating  the  coverslip  with  poly-L-lysine).  The  image 
shown  here  is  a  single  xz  optical  slice  taken  through  a  small  subresolution  flu¬ 
orescent  latex  bead  (0.03  pm  diameter)  attached  to  a  coverslip  using  poly-L- 
lysine  and  mounted  in  aqueous  media.  The  image  was  collected  on  a  Leica  SP2 
laser-scanning  confocal  microscope  using  a  Leica  60  X  1.20  NA  water-immer¬ 
sion  objective.  The  FWHM,  derived  from  the  plot  of  the  intensity  profile  in  the 
x-  and  --directions  can  be  used  to  determine  the  resolution  limit  of  the  instru¬ 
ment.  The  scale  bar  represents  300nm. 


as  the  Richardson  Test  Slide  Gen  III  (Richardson  Technologies 
Inc.,  Toronto,  Canada),  the  Microscopic  Image  Analysis  Microm¬ 
eter  (Edmund  Optics,  Barrington,  NJ),  and  the  MBL-NNF  Test 
Slide  (Marine  Biological  Laboratory,  Woods  Hole,  MA).  The 
aforementioned  slides  have  periodic  line  gratings  of  various  spatial 
frequencies  that  are  usually  imaged  in  the  reflected  light  mode, 
although  some  varieties  are  printed  using  fluorescent  phosphors. 
The  spatial  frequency  at  which  the  image  contrast  disappears  is 
taken  as  the  resolution  limit.  This  test  emphasizes  the  relationship 
between  contrast  and  resolution.  This  relationship  is  described  by 
the  contrast  transfer  function  (CTF;  see  Chapters  1,  2,  4,  and  8, 
this  volume',  also  van  der  Voort  et  al.,  1988;  Pawley,  1994;  van  den 
Doel  et  al.,  1998;  Young,  2000). 

Axial  Resolution 

Because  --resolution  is  affected  by  the  setting  of  the  adjustable 
pinhole  (or  pinholes  as  the  case  may  be),  as  well  as  by  the  NA  and 
wavelength,  it  is  prudent  to  take  measurements  over  the  full  range 
of  pinhole  settings  (Fig.  36.7).  Doing  so  will  also  reveal  anomalies 
in  the  pinhole  alignment.  Misaligned  pinholes  will  yield  poor 
results  for  axial  resolution  and  the  expected  relationship  between 
pinhole  diameter,  signal  intensity,  and  axial  resolution  will  be  dis¬ 
turbed  (see  Chapters  2,  3,  8,  11,  and  22,  this  volume ;  also  Cox  and 
Sheppard,  2004). 

Two  methods  are  frequently  employed  to  judge  --resolution. 
The  first  is  the  use  of  subresolution  fluorescent  particles  to  gener¬ 
ate  a  PSF  as  above  (Ho  and  Shao,  1991;  Shaw  and  Rawlins,  1991). 
The  second  approach  involves  making  an  xz  reflected  light  image 
of  the  surface  of  a  mirror  slide  as  described  below  (Ho  and  Shao, 
1991;  Visser  et  al.,  1991;  Cox,  1999;  Zucker  and  Price,  2001a). 

Axial  Resolution  Using  a  Mirror 

A  front-face  mirror  slide  can  be  purchased  (Electron  Microscopy 
Sciences;  Spherotech,  Libertyville,  IL)  or  made  from  a  high- 
quality  coverslip  onto  which  a  reflective  metal  surface  has  been 
deposited  on  one  side  using  either  sputter  coating  or  vacuum  depo- 


The  lateral  resolution  of  a  PSF  is  taken  as  full  width  at  half 
maximum  (FWHM,  the  width  of  the  intensity  peak  at  50%  of  its 
maximum  intensity;  see  Fig.  36.6).  The  FWHM  is  measured  by 
first  finding  half  the  maximum  peak  height  of  the  intensity  profile 
above  background  (/half  max,  Eq.  1)  and  then  determining  the  width 
of  the  intensity  profile  at  7half  max  (denoted  by  the  double  ended 
arrow  in  Fig.  36.6). 
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A  projection  along  the  --axis  and  through  the  center  of  a  vol¬ 
umetric  dataset  of  a  point  source  will  ensure  that  the  centroid  of 
the  intensity  distribution  is  measured. 

Some  drawbacks  to  using  fluorescent  polystyrene  spheres 
include  the  fact  that  they  may  bleach  under  high  zoom  or  high  laser 
intensity  conditions  and,  after  a  time  the  dye  may  leach  into 
organic  mountants  such  as  immersion  oil. 


Resolution  Test  Slides 

As  an  alternative  to  point  source  standards,  lateral  resolution  can 
be  tested  using  specially  designed  microscopy  standard  slides  such 


FIGURE  36.7.  Axial  resolution  as  a  function  of  pinhole  diameter.  Measure¬ 
ments  were  conducted  using  a  mirror  standard  with  a  63x  oil-immersion  objec¬ 
tive  with  NA  1.32. 
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FIGURE  36.8.  Determining  ^-resolution  using  a  mirror. 
Backscatter  images  of  a  front  surface  silvered  mirror  using  a 
Nikon  lOOx  1.40  NA  oil-immersion  lens  on  a  Bio-Rad  MRC- 
1024  confocal  microscope  with  the  488nm  blue  line  from  an 
argon-ion  laser  and  a  pinhole  size  of  1  Airy  disk.  The  front¬ 
surfaced  mirror  was  imaged  by  placing  a  small  drop  of  immer¬ 
sion  oil  directly  on  the  mirror  surface  and  then  placing  a 
coverslip  onto  the  oil.  Immersion  oil  was  then  placed  on  top  of 
the  coverslip  to  create  a  continuum  with  the  objective  lens.  (A) 
Backscatter  image  of  the  mirror  surface  (xy-image),  (b) 
backscatter  image  through  the  mirror  surface  (xz-image),  and 
(C)  confocal  zoom  of  a  small  section  of  the  xz-image  in  (B) 
showing  an  intensity  profile  graph  (D)  at  the  position  of  the 
dotted  line  in  image  (C).  The  full-width  at  half-maximum 
(FWHM)  of  the  intensity  profile  shown  in  (D)  (measured  in 
microns)  is  a  measure  of  the  z-resolution  of  the  instrument.  The 
slight  asymmetry  of  the  profile  is  due  to  a  small  amount  of 
spherical  aberration  of  the  optical  system.  To  establish  whether 
this  is  due  to  an  imperfection  in  the  objective  lens  you  would 
need  to  rigorously  establish  that  the  preparation  of  the  sample 
has  not  introduced  minor  optical  aberrations.  The  asymmetry  of 
the  plot  shows  the  presence  of  spherical  aberration.  The  scale 
bar  in  (A)  and  (B)  represents  1.6  pm  and  in  (C)  represents 
0.4  pm. 


sition  (Fig.  36.8).  The  coverslip  should  be  mounted  on  the  slide 
with  the  metallic  side  away  from  the  objective.  Purchased  mirrors 
must  be  coverslipped  (immersion  oil  makes  a  good  mountant). 

When  determining  axial  resolution  using  a  reflective  surface, 
use  a  neutral  density  filter  or  the  AOTF  to  reduce  laser  power  to 
as  low  as  possible  (less  than  lOOpW  should  be  adequate)  and 
choose  dichroic  and  filter  settings  that  allow  the  wavelength  of  the 
laser  line  to  reach  the  detector.  Adjust  the  focus  until  you  begin  to 
detect  a  signal  with  the  pinhole  fully  open.  Then  reduce  the  pinhole 
size  as  the  reflection  signal  increases.  At  the  focus  position  pro¬ 
ducing  the  highest  reflected  signal,  adjust  the  PMT  gain  to  create 
a  bright  image  that  does  not  quite  saturate  any  pixels  [Fig. 
36.8(A)]. 

The  mirror  should  then  be  moved  to  an  area  lacking  major 
defects  and  an  xz-image  (vertical  section)  through  the  surface  of 
the  mirror  is  collected  [Fig.  36.8(B)].  The  upper  and  lower  focus 
limits  should  be  set  sufficiently  far  apart  to  collect  all  the  diffrac¬ 
tion  maxima.  The  z-step  should  be  chosen  to  allow  for  more 
than  Nyquist  sampling  through  the  profile  of  the  mirror  [Fig. 
36.8(C)].  An  xz-intensity  profile  through  the  mirror  can  also  be 
obtained  by  collecting  a  stack  of  xy-images,  and  then  extracting 
the  xz-image  using  suitable  software.  The  field  curvature  from  any 
lens  can  be  tested  using  either  a  mirror  or  a  thin  fluorescent  film 
by  collecting  xy-scans  or  a  z-series.  Collect  a  3D  data  stack  that 
includes  the  full  depth  of  the  reflection  image  over  the  entire  field 
at  zoom  =  1 .  An  xz-  or  yz-profile  through  this  stack  will  reveal  the 
degree  of  curvature  [Fig.  36.8(C);  also  see  Chapter  11,  this 
volume]. 

Asymmetry  in  the  xz-profile  through  the  mirror  is  usually 
indicative  of  spherical  aberration.  In  the  example  shown  in  Figure 
36.8,  a  slight  asymmetry  is  evident.  The  use  of  incorrect  immer¬ 
sion  media  or  incorrect  settings  on  the  spherical  aberration  cor¬ 
rection  collar  (often  marked  as  a  coverslip  thickness  or  immersion 
media  correction  collar)  will  induce  spherical  aberration. 


Chromatic  Registration 

For  multi-channel  laser  scanning  microscopy,  it  is  important  that 
the  spatial  registration  of  all  channels  can  be  confirmed.  This  is 
particularly  true  where  experiments  involve  colocalization  and/or 
ratiometric  measurements.  Probable  sources  of  lateral  chromatic 
registration  error  are  found  in  situations  where  lasers  delivered 
through  separate  fibers  (or  direct  couplings)  are  used  together, 
and/or  when  different  optics  such  as  dichroic  beam-splitters  or 
beam  expanders  are  used  for  separate  channels  in  an  automated 
sequential-scanning  strategy.  Lens  aberrations  or  mismatched 
optics  can  also  be  a  source  of  lateral  chromatic  aberration  (see 
Chapter  7,  this  volume).  Although  the  laser  launch  into  the  scan- 
head  is  aligned  when  the  instrument  is  installed,  measurements 
should  be  conducted  immediately  following  installation  or  just 
before  the  end  of  any  service  call.  Widefield  systems  also  suffer 
from  alignment  problems  when  filter  cubes  and  dichroic  mirrors 
are  moved.  In  addition,  the  relay  lenses  in  some  camera  couplers 
impart  chromatic  aberration. 

Lateral  Chromatic  Registration 

This  aberration  can  cause  points  in  the  image  to  be  shifted  radi¬ 
ally  in  a  wavelength-dependent  manner  that  becomes  progres¬ 
sively  larger  towards  the  periphery  of  the  field  (see  Chapter  7,  this 
volume).  The  relatively  large  (4-15  pm  in  diameter)  multi-labeled 
Focal  Check  beads  marketed  by  Molecular  Probes  (Eugene,  OR) 
are  useful  for  evaluating  the  lateral  registration  of  multiple  chan¬ 
nels.  These  beads  are  made  of  a  series  of  fluorescent  shells,  each 
of  which  is  excitable  at  a  different  wavelength.  A  multi-channel 
image  through  the  lateral  central  plane  of  these  spheres  shows  a 
different  colored  ring  for  each  channel.  An  overlay  of  all  channels 
should  depict  colocalized  signals  (Fig.  36.9). 

Alternatively,  one  can  use  a  mixture  of  subresolution  beads 
(Zucker  and  Price,  2001a)  that  fluoresce  over  a  wide  range  of 
wavelengths  providing  that  individual  channels  are  excited 
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FIGURE  36.9.  Lateral  view  of  Focal  Check  beads  in  three  chan¬ 
nels.  In  this  case,  the  lateral  registration  of  channels  excited  by 
488  nm,  543  nm,  and  633  nm  laser  wavelengths  is  in  agreement.  All 
wavelengths  are  delivered  through  the  same  fiber-optic  to  the  scan- 
head  and  a  triple  dichroic  beam-splitter  was  used,  thereby  elimi¬ 
nating  two  of  the  primary  causes  of  problems  with  lateral 
registration. 


sequentially.  When  testing  laser  lines  that  are  delivered  to  the  scan¬ 
ning  optics  through  separate  couplings,  it  is  advantageous  to  use  a 
multi-wavelength  dichroic  beam-splitter  to  reduce  the  potential  for 
ambiguity  regarding  the  source  of  any  displacement  (e.g.,  dichroic 
vs.  laser  coupling  alignment).  By  the  same  token,  when  one  sus¬ 
pects  misalignment  in  the  beam-splitter  or  the  dichroics,  laser  lines 
should  be  delivered  through  the  same  fiber  or  use  beads  that  offer 
multi-channel  emissions  from  a  single  excitation  line. 

Evaluation  of  multi-channel  registration  is  most  easily  facili¬ 
tated  when  the  intensity  levels  are  closely  matched  for  each 
channel.  Use  an  intensity  profile  plot  through  the  image  to  deter¬ 
mine  the  degree  of  misalignment,  rather  than  relying  on  the  multi¬ 
colored  image  shown  on  the  screen. 

Axial  Chromatic  Registration 

This  type  of  aberration,  in  which  the  image  channels  shift  in  the  z- 
axis,  is  described  in  Chapter  7.  It  is  as  important  for  2D  images  as 
for  3D  images.  The  large  beads  described  above  are  not  the  best 
solution  for  evaluating  chromatic  aberration  in  the  z-dimension. 
Due  to  a  refractive  index  mismatch  between  the  core  of  the  bead 
and  the  fluorescent  shell,  the  beads  act  as  ball  lenses  and  produce 
impressive  (not  to  mention  alarming)  artifacts  in  the  ^-dimension 
(Fig.  36.10;  also  see  Pawley,  2002).  Axial  chromatic  aberration  is 
best  evaluated  utilizing  the  same  front-face  mirror  test  used  to  deter¬ 
mine  axial  resolution,  except  in  this  case  the  data  from  multiple  laser 
lines  are  collected  in  separate  channels.  This  test  is  very  sensitive 
to  chromatic  aberration  in  the  optical  system,  and  even  very  small 
axial  displacements  of  the  focal  plane  between  laser  wavelengths 
can  be  quantified  (see  Chapter  15,  this  volume ;  also  Akinyemi  etal., 
1992;  Browne  et  al.,  1992;  Maly  and  Boyde,  1994).6  Axial  chro- 


6  However,  although  this  measurement  gives  some  measure  of  lens  perfor¬ 
mance,  one  should  note  that  it  does  not  measure  lens  performance  at  wave¬ 
lengths  between  the  laser  lines  where  the  fluorescent  signal  is  actually 
recorded. 


matic  aberration  is  evaluated  by  plotting  intensity  as  a  function  of 
z-position  for  multiple  channels  simultaneously  (Fig.  36. 1 1 ).  A  mis¬ 
match  between  the  refractive  index  of  the  lens  immersion  media  and 
the  specimen  can  exacerbate  longitudinal  chromatic  aberration 
because  the  spherical  aberration  produced  by  the  mismatch  has  a 
chromatic  component.  Such  discrepancies  can  be  a  major  concern 
in  resolution-sensitive  multi-channel  experiments. 

Field  Illumination 

One  cannot  assume  a  correlation  between  signal  intensity  and  flu- 
orochrome  concentration  unless  the  illumination  pattern  is  uniform 
across  the  field  of  view.  This  can  be  checked  using  a  fluorescent 
sea  (see  above),  fluorescent  plastic  slides  (van  den  Doel  et  al., 
1998;  Zucker  and  Price,  1999)  or  using  reflected  light  from  a 
mirror  standard.  At  a  sequence  of  increasing  zoom  levels,  record 
the  fluorescence  intensity  near  to  the  coverslip  interface  over  the 
field  of  view  (FOV).  Adjust  the  mean  pixel  intensity  to  fall  within 
the  linear  response  range  of  the  digitizer. 

To  evaluate  the  data,  draw  a  diagonal  region  of  interest  (ROI) 
from  one  corner  to  the  opposite  corner  and  graph  the  intensity 
values  as  a  function  of  position  along  this  line  (Fig.  36.12).  At  low 
zoom,  the  distribution  of  pixel  intensities  should  be  symmetrical, 
and  the  hot  spot  should  be  located  at  the  center  of  the  field  of  view. 
At  higher  zoom  levels,  the  intensity  distribution  should  flatten  out 
such  that  the  values  of  the  sampled  pixels  approach  the  mean  pixel 
intensity  integrated  over  the  entire  FOV.  When  this  is  not  the  case, 
it  is  likely  that  there  is  an  alignment  problem  in  the  optical  train 
(van  den  Doel  et  al.,  1998). 7 


7  Recently,  thin-film  (150  to  200  nm)  fluorescent,  specimens  have  been  devel¬ 
oped  by  Brakenhoff  to  measure  the  performance  of  optical-sectioning 
imaging  systems  (Zwier,  et  al.,  2004).  These  films  permit  standardized  eval¬ 
uation  of  illumination  and  signal-collection  uniformity,  axial  resolution; 
spherical  aberration,  and  off-axis  chromatic  aberrations  in  the  form  of  sec¬ 
tioned  imaging  property  (SIP)  charts  (Brakenhoff,  et  al.,  2005). 
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FIGURE  36.10.  Self-lensing  artifacts.  (A)  Axial  view 
through  center  of  Focal  Check  bead.  Lensing  by  the  fluo¬ 
rescent  bead  shell  causes  the  artifact  illustrated  in  this 
example.  (B)  Axial  view  through  a  10pm  polystyrene  sphere 
in  both  backscatter  (green)  and  fluorescence  (red)  channels. 
Despite  reasonable  refractive  index  matching,  enough  of  a 
difference  exists  to  cause  the  bead  to  act  as  a  ball  lens.  Con¬ 
centrated  patches  of  scattered  light  (green)  are  evident  as  a 
result  of  self-focusing. 


FIGURE  36.1 1 .  Chromatic  aberration.  In  this  example,  a  mirror  standard  is  imaged  simultaneously  in  two  channels;  the  peak-to-peak  distance  between  the  first- 
order  intensity  maximum  for  each  respective  channel  is  242  nm  (about  half  the  axial  resolution  of  the  lens).  The  wavelengths  for  this  image  were  488  and  543 
nm,  and  a  63x  NA  1.32  plan-apochromatic  (oil-immersion)  lens  was  used. 


FIGURE  36.12.  Flatness  of  field.  Flatness 
of  field  is  evaluated  by  plotting  intensity  as  a 
function  of  position  on  a  line  drawn  diago¬ 
nally  across  the  field  of  view  (in  this  case  the 
same  63x  objective  used  above,  at  zoom  =1). 
The  intensity  values  have  been  normalized 
for  purposes  of  illustration  in  this  case. 
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FIGURE  36.13.  Instrument  noise.  Instrument  noise 
may  result  in  a  speckled  appearance  within  an  image  col¬ 
lected  on  a  confocal  microscope,  although  this  will  nor¬ 
mally  only  become  apparent  if  the  instrument  is  used  at 
near  maximum  gain  or  PMT  voltage  settings.  In  this 
example,  an  image  was  collected  on  a  Zeiss  510  META 
confocal  microscope  with  no  laser  light  and  no  fluo- 
rophore  present,  while  the  PMT  voltage  was  set  to 
maximum  (1250  V)  and  the  amplifier  gain  set  to  1  (the 
lowest  setting).  Panel  (A)  shows  a  512  x  512  pixel  image 
collected  in  a  single  pass,  and  panel  (B)  shows  a  digi¬ 
tally  enlarged  view  of  a  small  region  of  the  image  shown 
in  panel  (A).  Minimal  room  lighting  and  a  dark  enclo¬ 
sure  over  the  objective  lens  were  used  during  image 
acquisition  to  eliminate  the  possibility  of  stray  light 
entering  the  scanhead  of  the  instrument.  Note  the  rela¬ 
tively  few  high  intensity  pixels  adjacent  to  a  large 
number  of  very  low  intensity  pixels.  In  some  cases  the 
noise  is  only  detected  in  a  single  pixel,  but  in  others  the 
noise  is  distributed  across  2  or  even  3  pixels  in  the  hor¬ 
izontal  direction  (the  direction  of  scanning). 


Signal  Detection 

The  Signal-to-Noise  Ratio  and  Coefficient 
of  Variation 

The  sensitivity  in  an  imaging  system  depends  on  the  ratio  of  the 
signal  generated  by  the  stain  and  registered  by  the  detector  to  the 
sum  of  all  the  noise  signals  (i.e.,  detector  noise,  stray  light,  non¬ 
specific  staining,  etc.),  as  recorded  under  imaging  conditions  (see 
Chapters  2,  4,  22,  and  35,  this  volume ;  also  Pawley,  1994;  Tran, 
2005).  This  is  referred  to  as  the  signal-to-noise  ratio  (S/N).  The 
S/N  capability  of  a  microscope  is  reflected  in  the  intensity  distri¬ 
bution  of  pixels  corresponding  to  a  constant  signal  level.  Thus,  for 
routine  evaluation,  the  S/N  can  be  conveniently  gauged  from 
image  data  in  terms  of  a  percentage  known  as  the  coefficient  of 
variation  or  CV  (Eq.  2)  measured  from  a  known  and  homoge¬ 
neous  specimen  using  a  fixed  amount  of  excitation.  (Zucker  and 
Price,  2001b): 

CV  =  —100%  n, 

I  (2) 

av 

where  is  the  mean  pixel  intensity  and  o  is  the  standard  devia¬ 
tion  of  the  pixel  intensities  in  the  measured  region. 

Many  factors  influence  the  signal  measured  by  the  detector  on 
a  laser-scanning  confocal  microscope.  In  order  to  attribute  any 
variation  between  measurements  to  the  detection  subsystem,  all 
of  these  other  factors  must  be  held  constant  (Pawley,  2000). 
Furthermore,  the  noise  level  in  a  confocal  microscope  has  two 
fundamentally  different  components,  the  noise  level  of  the  instru¬ 
mentation  itself  (dark  noise)  and  photon  (shot  or  Poisson)  noise. 
PMTs  vary  in  their  quantum  efficiency  (QE),  gain  response,  and 
dark-count  rate,  even  within  the  same  model  and  manufacturing 
lot.  More  to  the  point,  some  tubes  break  down  when  used  with  a 
high  gain  voltage  and  this  increases  their  noise  output  substan¬ 
tially.8  Strategies  for  evaluating  and  documenting  the  relative 


8  Because  the  rule  of  thumb  states  that  PMT  gain  increases  by  a  factor  of  2  for 
every  40  to  50  volts  of  acceleration,  one  expects  noise  pulses  to  become 
bigger  and  more  visible  as  the  PMT  voltage  is  increased.  Breakdown  is  dif¬ 
ferent  in  that  it  is  characterized  by  the  relatively  sudden  onset  of  greatly 
increased  noise:  not  only  bigger  pulses,  but  many  more  of  them. 


contributions  of  dark  noise  and  shot  noise  under  standardized 
conditions  are  described  below. 

Instrument  Dark  Noise 

To  determine  the  level  of  dark  noise  first  turn  on  the  confocal  micro¬ 
scope  and  associated  equipment  and  allow  it  to  reach  the  normal 
operating  temperature.  Prevent  stray  light  from  entering  the  detec¬ 
tors  by  turning  off  room  lights  and  shielding  the  objective  from  any 
light  from  computer  monitors  and  other  sources  of  ambient  light¬ 
ing  [the  laser(s)  may  also  need  to  be  turned  off  as  there  can  be  some 
light  leakage  through  an  AOTF  filter  even  when  set  to  0%  trans¬ 
mission].  Using  standardized  settings  for  PMT  gain  and  offset, 
collect  a  single-scan  image  (Fig.  36.13).  If  the  microscope  is 
working  properly,  the  small  image  features  will  actually  be  PMT 
dark  noise  counts.  PMT  noise  will  be  displayed  as  a  single  high- 
intensity  pixel  (or  sometimes  2  or  3  bright  pixels,  always  oriented 
in  the  direction  of  the  scan  line)  scattered  randomly  throughout  the 
image.  Changing  the  confocal  zoom  setting  will  usually  not  alter 
the  presence  or  size  of  these  high-intensity  noise  pixels. 

When  using  standardized  settings  for  scan  speed  and  sampling 
resolution  (pixel  size),  it  may  be  wise  to  record  the  PMT  dark 
counts  for  a  series  of  increasing  PMT  gain  voltages  so  that  a  base¬ 
line  relationship  between  PMT  gain  and  dark  noise  is  character¬ 
ized.  The  mean  pixel  value  and  standard  deviation  of  pixel  values 
from  images  of  dark  noise  counts  can  be  used  to  calculate  the  CV 
associated  with  standard  PMT  gain  settings.  Dark  noise  should  be 
re-evaluated  whenever  one  suspects  problems  with  the  imaging 
electronics. 

Photon  (Shot)  Noise 

One  might  expect  that  imaging  a  solution  of  a  small  molecular 
weight  fluorophore  would  show  a  perfectly  even  intensity  image 
(no  variation  between  pixels).  However,  when  imaging  a  fluores¬ 
cent  specimen  of  low  quantum  yield,  a  great  deal  of  speckle  is 
evident  within  the  image  (Fig.  36.14).  Assuming  that  the  PMT  gain 
is  set  below  the  level  at  which  electrical  breakdown  occurs,  this 
non-uniformity  mainly  reflects  the  statistical  uncertainty  inherent 
in  the  detection  of  any  photon  signal.  This  uncertainty  is  called 
photon  noise  (shot  noise;  see  above  and  Chapters  2,  4,  22,  and  35, 
this  volume). 
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FIGURE  36.14.  Demonstrating  noise  in  images.  Photon  or  shot  noise  can  be  seen  in  images  collected  on  a  confocal  microscope  as  speckle  within  that  part  of 
the  image  where  fluorescent  light  is  being  emitted.  To  demonstrate  the  effect  of  averaging  on  the  level  of  shot  noise  a  dilute  solution  of  coumarin  (in  a  sealed 
imaging  chamber)  was  imaged  on  a  Zeiss  510  Meta  confocal  microscope  using  a  60x  1.20  NA  water-immersion  objective.  Image  (A)  shows  a  single  Is  scan 
with  an  image  collection  box  size  of  512  X  512  pixels.  The  whole-image  intensity  histogram  (A')  demonstrates  the  wide  spread  of  the  signal  (large  standard 
deviation)  due  both  to  statistical  variation  in  the  arrival  of  photons  at  the  detector  and  multiplicative  noise  in  the  PMT  (see  also  Chapter  2,  this  volume).  Image 
(B)  was  averaged  by  16  line  scans  taken  at  the  same  scan  speed  as  that  shown  in  image  (A).  The  whole  image  intensity  histogram  (B')  now  has  a  much  smaller 
standard  deviation  (narrower  histogram  profile),  indicative  of  less  variation  between  individual  pixels  within  the  image. 


The  histogram  of  the  intensity  levels  within  the  image  (Fig. 
36.14)  provides  an  intuitive  way  to  visually  assess  this  noise  level 
(a  broader  spread  in  intensities  results  in  a  broad  histogram).  As 
long  as  the  signal  source  is  uniform,9  the  FWHM  of  this  intensity 
histogram  is  a  measure  of  the  noise  in  the  image.  The  width  of  the 
distribution  can  be  expressed  quantitatively  by  calculating  the  CV 
for  the  image. 

A  metric  that  tests  the  detection  efficiency  of  imaging  instru¬ 
mentation  is  valuable  for  determining  whether  problems  with 
image  quality  are  due  to  poor  specimen  preparation,  or  to  less  than 
optimal  instrument  performance.  With  this  in  mind,  it  is  useful  to 
measure  and  record  CV  values  over  a  range  of  signal  intensities 
(Fig.  36.15)  as  a  reference  of  relative  detector  performance  at  a 
particular  point  in  time.  Special  calibration  slides  covered  with 
microspheres  having  discrete  intensity  levels  are  available  (log  and 
linear  scales;  Rainbow  Particle  Slide,  Spherotech,  Libertyville, 
IL).  Fresh  dilutions  of  fluorochrome  are  a  convenient  alternative. 
Such  standards  could  be  used  to  record  CV  over  a  range  of  inten¬ 
sity  values  using  a  fixed  laser  power. 

An  analogous  test  uses  a  sample  having  a  standardized 
quantum  yield  and  monitors  CV  as  a  function  of  the  laser  power 
(Zucker  and  Price,  2001b).  Fluorescent  plastic  slides  are  useful  in 
this  context.  Most  plastics  behave  as  a  highly  viscous  solution  of 
fluorescent  dye,  which  has  the  advantage  that  any  photobleached 
areas  will  no  longer  be  distinguishable  from  the  surrounding 
plastic  after  several  hours  of  storage.  Flowever,  the  fact  that  it  is 
difficult  to  confirm  that  the  quantum  yield  of  fluorescent  plastics 
is  stable  over  long-term  use  introduces  uncertainty  into  quantita¬ 
tive  comparisons  of  instrument  performance  over  time.  In  addi¬ 
tion,  as  the  RI  of  the  plastic  is  not  one  for  which  objective  lenses 
are  normally  corrected,  spherical  aberration  will  reduce  the 
recorded  signal  significantly  as  one  focuses  further  into  it. 


9  That  is,  not  only  no  visible  features  but  not  even  any  shading  such  as  might 
be  caused  by  signal  loss  at  the  edges  of  the  field  of  view.  This  condition  is 
more  easily  met  by  using  a  medium  zoom. 


PMT  Linearity 

To  establish  that  the  relationship  between  the  signal  digitized  from 
the  PMT  responds  linearly  with  the  fluorescence  signal  produced 
by  the  specimen,  first  prepare  a  fresh  standardized  dilution  of  flu- 


FIGURE  36.15.  Detector  response  to  decreased  signal.  Coefficient  of  varia¬ 
tion  as  a  function  of  fluorochrome  concentration.  CV  values  were  taken  from 
the  spectral  image  that  corresponds  to  the  peak  emission  range  (568-573  nm) 
at  minimal  bandwidth  (5  nm)  using  a  spectral  detection  system.  The  series  is  a 
10-fold  dilution  of  Rhodamine  6G;  the  laser  power  was  held  constant  and  the 
PMT  gain  was  increased  at  each  reduction  in  concentration  to  maintain  a  mean 
signal  of  130A.U.  (scale  0-255).  The  PMT  gain  was  maximized  (1000  V)  at  a 
10-6  dilution.  At  the  lowest  concentration  (10-7),  the  system  is  unable  to  produce 
a  mean  signal  above  50  A.U.  As  the  PMT  gain  is  already  maximized  at  a  dilu¬ 
tion  of  10-6,  the  decreased  signal  is  responsible  for  the  increase  in  apparent 
noise  at  the  lowest  concentration. 
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orochrome  in  the  appropriate  refractive  index  solvent.  Then  adjust 
the  PMT  gain  setting  to  give  a  signal  within  the  linear  response 
range  and  finally  record  the  mean  signal  as  a  function  of  the  illu¬ 
mination  intensity.  The  mean  brightness  should  be  proportional  to 
laser  power  until  the  laser  becomes  so  bright  that  the  molecules  in 
the  focused  spot  are  driven  into  singlet-state  saturation  (i.e.,  so 
many  of  the  molecules  are  in  the  excited  state  that  there  is  no  dye 
left  to  excite;10  see  Chapters  2,  16,  and  35,  this  volume).  Approach¬ 
ing  saturation  will  produce  a  less  than  proportional  increase  in 
signal  for  an  increase  in  excitation.  Although  the  effect  of  satura¬ 
tion  can  be  reduced  by  going  to  a  lens  with  lower  NA  because  this 
makes  the  spot  bigger  and  the  light  less  intense,  you  then  have  a 
dry  lens  and  all  the  SA  problems  associated  thereto.  One  should 
validate  the  performance  of  the  detector  subsystem  periodically  as 
the  instrument  gets  older. 

Spectral  Systems 

An  exciting  development  in  laser-scanning  microscopy  is  the 
increasing  availability  of  signal  detection  systems  capable  of 
recording  crude  emission  spectra  at  each  pixel  in  an  image.  Evi¬ 
dence  of  the  ability  of  such  systems  to  solve  problems,  such  as  dis¬ 
tinguishing  the  identity  of  overlapping  emission  profiles,  has  been 
established  (Dickinson  et  al.,  2001;  Lansford,  et  al.,  2001; 
LaMorte  et  al.,  2003;  Huth  et  al.,  2004;  Hutter,  2004).  The  accu¬ 
racy  of  such  extrapolations  is  dependent  on  the  accuracy  and  reli¬ 
ability  of  the  underlying  system  (Garini  et  al.,  1999;  Neher  and 
Neher,  2004;  Zucker  and  Lerner,  2004). 

Spectral  detection  devices  can  be  made  in  several  ways.  The 
advantages  of  a  particular  system  will  depend  on  the  scope  of 
applications  an  instrument  is  expected  to  handle.  As  all  such 
systems  are  complex  (and  expensive!)  compared  to  traditional 
filter-based  systems,  finding  a  convenient  strategy  to  evaluate  their 
performance  is  of  prime  importance. 

Three  major  aspects  of  the  spectral  imaging  subsystem  should 
be  periodically  evaluated:  (1)  calibration  accuracy  of  the  system 
(wavelength  accuracy),  (2)  resolution  in  terms  of  the  minimum 
resolvable  bandwidth,  and  (3)  variation  of  sensor  efficiency  with 
wavelength. 

Spectral  Accuracy 

The  best  way  to  test  accuracy  is  to  image  a  standard  that  displays 
narrow  peaks  in  known  locations.  When  doing  these  tests,  all 
sources  of  extraneous  light  (overhead  lamps  and  computer  moni¬ 
tors)  that  might  contaminate  the  spectrum  of  the  illumination 
source  should  be  masked  (often  a  black  velvet  shroud  or  a  black 
box  can  be  carefully  placed  over  the  imaging  area).  Using  the 
reflected  light  from  a  mirror  test  slide  and  a  high  zoom  to  avoid 
off-axis  aberrations,  the  lasers  installed  on  the  system  can  be  used 
as  a  convenient  source  of  monochromatic  light.  In  order  to  check 
the  accuracy  of  spectral  registration,  it  is  advantageous  to  use  as 
many  laser  lines  as  possible.  PMT  gain  voltage  should  be  adjusted 
such  that  the  magnitude  of  the  intensity  peak  reflected  from  the 
strongest  laser  line  is  within  the  linear  response  range  of  the 
PMT/digitizer.  Acquire  a  high-resolution  spectral  scan  over  the 
visible  spectrum,  and  observe  the  location  of  the  spectral  features 
corresponding  to  the  laser  wavelengths.  This  can  often  be  recorded 
as  a  plot  of  intensity  versus  wavelength  [Fig.  36.16(A)].  If  the 


10  Because  the  degree  of  saturation  varies  throughout  the  PSF,  this  saturation  is 
not  a  “hard”  effect  in  the  sense  that  an  absolute  limit  is  reached  and  no  further 
increase  is  possible.  Rather  the  signal  increases  asymptotically  as  the  degree 
of  saturation  increases  at  ever  greater  distances  from  the  center  of  the  PSF. 


Wavelength  (nm) 

FIGURE  36.16.  Spectral  scan  over  laser  lines  using  reflected  light.  (A)  A 
typical  plot  for  a  spectral  imaging  instrument  working  correctly.  The  signature 
for  the  laser  lines  are  centered  about  the  appropriate  wavelength  and  the  spec¬ 
tral  resolution  (FWHM)  is  close  to  lOnm.  (B)  A  malfunctioning  system.  The 
wavelength  selection  mechanism  for  PMT  1  is  inoperable,  hence  the  low  signal 
for  that  channel.  The  spectral  reading  for  each  channel  is  centered  about  a  dif¬ 
ferent  wavelength  and  the  spectral  resolution  approaches  18nm  in  places. 


peaks  are  not  where  they  are  expected  [Fig.  36.16(B)],  either  the 
detector  is  miscalibrated  or  one  or  more  of  the  system  components 
has  failed. 

Alternatively,  if  the  laser  lines  available  don’t  properly  cover 
the  full  wavelength  range,  one  can  use  an  NIST-certified  multi-ion 
discharge  calibration  lamp  (MIDL,  LightForm  Inc.,  Hillsborough, 
NJ)  in  the  manner  described  by  Zucker  and  Lerner  (2004).  In  this 
approach,  a  lamp  with  very  well-characterized  peaks  in  its  spec¬ 
tral  output  is  placed  directly  above  the  objective  being  used  for 
measurement,  and  a  spectral  reading  is  taken  using  the  acquisition 
subsystem  of  the  microscope.  The  NIST  calibration  lamp  has 
numerous  spectral  features  that  form  a  sophisticated  spectral  fin¬ 
gerprint.  In  theory,  all  instruments  should  reproduce  the  charac¬ 
teristic  set  of  relative  heights  of  peaks  and  valleys  provided  by  the 
standard.  The  effect  of  under-sampling  in  the  wavelength  dimen¬ 
sion  on  the  accuracy  and  precision  of  the  recorded  spectra  is  easily 
demonstrated  using  this  standard  (see  Chapter  4,  this  volume',  also 
Zucker  and  Price,  2001b). 

Spectral  Resolution 

The  resolution  is  indicated  by  the  FWHM  of  peaks  in  spectra 
derived  from  subresolution  spectral  features  (such  as  monochro¬ 
matic  laser  output).  Presently,  most  systems  are  designed  for  2nm 
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to  lOnm  bandwidth  resolution,  with  the  value  for  each  particular 
system  available  from  the  manufacturer.  When  spectral  resolution 
values  are  less  than  expected  [Fig.  36.16(B)],  the  boundaries  of 
virtual  filter  bandpass  settings  may  be  offset  to  some  degree 
unknown  to  the  operator  (in  the  case  of  the  Leica,  the  moveable 
mirrors  may  be  miscalibrated  or  stuck  in  one  position).  Such  errors 
produce  much  higher  background  than  expected  because,  under 
imaging  conditions,  scattered  light  from  the  laser  can  reach  the 
detector.  Data  collected  with  poor  or  shifted  spectral  resolution  can 
also  produce  anomalous  fluorochrome  localization  results  when 
treated  using  spectral  deconvolution  algorithms. 

Wavelength  Response 

The  reflected  laser  light  test  paradigm  outlined  above  can  be  used 
to  provide  an  expedient  measure  of  sensor  response  in  different 
wavelength  regimes;  anomalous  sensor  response  with  respect  to 
frequency  will  seriously  affect  the  accuracy  of  spectral  un-mixing 
schemes.  Fingerprints  or  other  contamination  on  certain  mirrors 
within  the  detection  system  can  easily  cause  diminished  response 
in  part  of  the  spectrum.  In  this  test,  the  relative  laser  powers  for 
all  the  individual  lines  used  are  known.  First,  the  laser  lines  are 
individually  adjusted  to  some  standard  value  (e.g.,  15  pW,  emerg¬ 
ing  from  the  same  dry  objective)  then  the  PMT  gain  and  offset  are 
configured  so  that  the  entire  intensity  range  is  within  the  linear 
response  range  of  the  PMT(s).  A  spectral  scan  is  taken  across  the 
part  of  the  spectrum  containing  the  laser  lines.  On  the  resultant  plot 
of  intensity  versus  wavelength,  the  intensity  values  measured  for 
the  laser  lines  should  be  of  roughly  the  same  magnitude,11  and  the 
mean  value  of  the  noise  level  should  remain  constant  in  the 
domains  between  laser  lines.  On  systems  with  multiple  spectral 
detectors,  each  detector  should  be  checked  in  turn.  A  method  for 
performing  the  equivalent  test  using  the  NIST  standard  lamp  is  dis¬ 
cussed  in  Zucker  and  Lerner  (2004). 

The  uniformity  of  the  response  of  a  spectral  detector  can  be 
assayed  by  collecting  a  spectrum  from  the  light  transmitted 
through  a  microscope  slide  from  the  transmitted  light  source. 
Assuming  that  this  source  uses  a  tungsten/halogen  bulb,  apply  the 
correct  voltage  to  heat  the  filament  to  3400  K,  and  set  the  con¬ 
denser  for  Kohler  illumination.  If  you  now  collect  a  spectrum  with 
the  laser  turned  off,  it  should  slope  from  higher  in  the  red  to  lower 
in  the  blue.  When  recorded  on  the  Zeiss,  META,  valleys  in  this 
spectrum  probably  correspond  to  individual  micro-PMTs  that  have 
lower  than  average  gain  or  QE.  Although  this  is  to  be  expected,  it 
is  the  reason  why  it  is  better  to  collect  reference  spectra  using  the 
specific  setup  that  will  be  used  for  the  experiment,  rather  than 
relying  on  stored  spectra  that  were  obtained  at  the  factory.  This 
same  transmitted  light  setup  can  be  use  with  a  Hg-arc  source  that 
provides  additional  calibration  lines  (see  Chapter  4,  this  volume). 

OPTIMIZING  MULTI-LABELING  APPLICATIONS 

Confocal  microscopy  is  ideally  suited  to  multi-labeling  applica¬ 
tions  due  to  the  ease  and  flexibility  with  which  multiple  fluo- 
rophores  can  be  viewed  and  recorded  as  separate  images.  This 


11  There  are  two  offsetting  factors  at  work  here.  If  the  photometer  is  calibrated 
for  the  measured  wavelength,  then  15  (iW  will  correspond  to  more  red 
photons  than  blue  photons.  As  the  output  of  the  PMT  is  proportional  to  the 
number  of  electrons  produced  at  the  photocathode,  one  might  expect  that  the 
PMT  would  produce  more  signal  from  the  red  beam  because  it  has  more 
photons.  However,  this  tendency  is  usually  more  than  cancelled  out  by  the 
fact  that  the  QE  of  the  PMT  drops  at  longer  wavelengths. 


includes  the  more  traditional  spectral  separation  of  fluorescence 
from  different  dyes  as  well  as  the  ability  to  separate  highly- 
overlapping  fluorophores  using  spectral  unmixing. 

Control  Samples  Establish  the  Limits 

The  relationship  between  the  fluorescence  intensity  data  content  of 
an  image  and  the  distribution  of  the  fluorophore  of  interest  in  the 
sample  can  only  be  verified  by  collecting  reference  images  from 
the  correct  controls.  Controls  are  also  essential  for  establishing  the 
detection  limits. 

Establishing  the  Level  of  Autofluorescence 

Fluorescence  emission  from  a  sample  to  which  you  have  not  yet 
added  a  fluorophore  is  termed  autofluorescence  (for  further  dis¬ 
cussion,  see  Chapters  8,  21,  44,  this  volume).  Many  naturally 
occurring  compounds  in  plant  and  animal  tissues  will  fluoresce, 
and  various  cellular  treatments,  including  the  media  in  which  they 
are  grown  and  the  physical  manipulation  (including  the  method  of 
fixation),  may  influence  the  intensity  and  wavelengths  of  emission. 

An  unlabeled  control  is  essential  for  establishing  the  level  of 
autofluorescence  in  the  sample.  Additional  images  of  the  unlabeled 
control  collected  with  increased  detector  gain,  pinhole  size,  or 
excitation  power  can  be  used  to  establish  the  upper  limit  of  detec¬ 
tion  of  your  fluorophore.  However,  even  a  high  level  of  autofluo¬ 
rescence  is  not  necessarily  a  problem,  as  long  as  the  intensity, 
location,  or  the  spectral  characteristics  of  the  probe  can  be  distin¬ 
guished  from  autofluorescence. 

Is  Reflected  Light  Contributing  to  Your 
Fluorescence  Image? 

The  level  of  signal  intensity  from  an  unlabeled  control  is  also 
essential  for  establishing  the  level  of  backscattered  or  reflected 
light  that  is  contributing  to  the  image  (Pawley,  2002).  Any  excita¬ 
tion  light  that  is  reflected  or  backscattered  back  toward  the  detec¬ 
tor  will  be  registered  as  a  signal  unless  the  barrier  filter  stops  it. 
Even  the  very  best  barrier  filters  are  not  100%  efficient;  and 
because  the  intensity  of  illumination  is  several  orders  of  magni¬ 
tude  higher  than  that  of  the  fluorescence  signal  even  very  small 
percentages  of  reflected  light  passing  through  a  barrier  filter  may 
be  of  similar  intensity  to  that  of  a  fluorescent  signal.  Small  refrac- 
tile  features  that  produce  much  backscattered  light  may  also  appear 
to  show  specifically  labeled  subcellular  detail.  A  useful  test  for  dis¬ 
tinguishing  contributions  from  backscattered  illumination  com¬ 
pared  to  autofluorescence  is  whether  or  not  one  detects  a  uniform 
background  signal  level  as  the  focus  plane  coincides  with  the 
surface  between  the  coverslip  and  the  aqueous  media.  While  such 
a  signal  can  be  a  handy  marker  for  the  location  of  the  glass  surface, 
it  is  also  an  indication  that  some  of  the  signal  recorded  in  this 
channel  stems  from  backscattered  light. 

How  Much  Is  Bleed-Through  Contributing 
to  the  Image? 

Bleed-through  is  the  appearance  of  fluorescence  from  one  fluo¬ 
rophore  in  the  detection  channel  set  up  to  detect  another  fluo¬ 
rophore.  The  amount  of  bleed-through  is  determined  by  imaging 
control  slides  in  which  only  one  dye  is  present  while  using  the 
dual-channel  settings  you  plan  to  use  for  your  dual-labeled 
samples.  Images  collected  at  various  gain,  pinhole,  and  laser  inten¬ 
sity  levels  may  be  useful  for  establishing  the  amount  of  bleed- 
through  at  various  settings.  This  information  should  be  used  to  set 
limits  for  the  various  settings  that  will  result  in  an  acceptably  low 
bleed-through  level. 


664  Chapter  36  •  A.R.  Hibbs  et  al. 


How  Much  Is  the  Secondary  Conjugate  in 
Immunolabeling  Contributing  to  the  Signal? 

Secondary  conjugates  (including  avidin/biotin  systems  as  well  as 
secondary  antibodies)  often  contribute  to  background  signal  in 
immunolabeling  procedures.  Application  of  fluorescently-labeled 
secondary  conjugate,  in  the  absence  of  its  target  primary  antibody, 
is  an  essential  control  in  all  immunolabeling  experiments.  The  con¬ 
tribution  of  the  secondary  conjugate  to  the  fluorescent  signal  is 
determined  by  imaging  a  sample  labeled  using  the  secondary 
conjugate  alone  under  the  same  conditions  as  employed  for  the 
experimental  sample.  In  addition,  as  discussed  above  for  auto¬ 
fluorescence,  images  collected  at  increased  detector  gain,  pinhole 
size,  and  laser  power,  will  be  useful  for  establishing  the  limits  to 
detection  for  the  binding  of  your  primary  antibody. 

A  Positively  Labeled  Sample 

It  is  a  great  asset  to  have  a  cell  or  tissue  sample,  or  an  environ¬ 
mental/metabolic  condition  that  you  know  will  produce  a  positive 
response  for  your  target  molecule.  This  can  assist  in  establishing 
the  probe  concentrations,  imaging  conditions,  and  the  instrument 
settings  that  can  be  used  on  your  experimental  sample.  A  positive 
control  closely  related  to  your  test  sample  (e.g.,  the  same  cell  line, 
but  stimulated  to  produce  a  known  positive  response)  is  an  ideal 
model  of  the  labeling  and  imaging  conditions  used  for  the  exper¬ 
imental  sample.  A  known  positive  is  particularly  important  when 
troubleshooting  problems  with  an  immunolabeling  protocol. 

Separation  of  Fluorescence  into 
Spectral  Regions 

Traditionally,  one  images  multiple  fluorophores  by  using  interfer¬ 
ence  filters  to  separate  the  fluorescent  light  into  different  spectral 
regions.  Although  the  use  of  dichroic  mirrors  and  optical  filters  has 
the  advantage  of  being  a  robust  and  relatively  simple  technology, 
the  regions  of  the  spectrum  directed  to  individual  channels  is  con¬ 
strained  by  the  availability  of  suitable  mirrors  and  filters. 

Another  way  to  separate  the  fluorescent  light  into  different 
spectral  regions  is  to  use  a  prism  or  diffraction  grating  to  separate 
the  light  spatially  and  to  then  use  either  a  mirror/slit  mechanism 
or  photomultiplier  tube  array  to  select  the  region  of  the  spectrum 
that  will  be  digitized  in  each  channel.  This  type  of  spectral  sepa¬ 
ration  has  the  advantage  that  the  operator  has  full  control  of  the 
region  of  the  light  spectrum  directed  to  each  channel,  without 
being  constrained  by  the  available  optical  filters.  A  variety  of  light 
separation  approaches  are  used  by  different  confocal  microscope 
manufacturers,  sometimes  more  than  one  approach  being  incorpo¬ 
rated  into  a  single  instrument. 

Spectral  unmixing,  described  in  detail  below,  is  a  method  of 
separating  the  signals  from  various  fluorophores  based  on  their 
spectral  profile,  rather  than  by  their  presence  or  absence  in  a  spec¬ 
ified  spectral  window.  Spectral  unmixing  has  the  important  advan¬ 
tage  that  fluorophores  with  highly  overlapping  spectral  emission 
profiles  can  be  separated  into  individual  channels. 

Sequential  Channel  Collection  to  Minimize 
Bleed-Through 

Bleed-through  occurs  predominantly  when  multiple  fluorophores 
are  excited  simultaneously  and  portions  of  their  emission  spectra 
overlap  with  the  spectral  region  accepted  by  more  than  one 
channel.  Simply  exciting  and  collecting  each  channel  sequentially, 
sometimes  called  multi-tracking,  is  a  very  practical  means  of  sub¬ 


stantially  reducing  bleed-through.  Although  this  can  be  done  on 
older  confocal  systems  that  use  slow,  motor-driven  filter  wheels  to 
switch  laser  lines,  newer  systems  using  acoustic-optical  devices 
(AOD)  are  far  more  flexible  and  allow  nearly  instantaneous  laser 
line  selection.  This  allows  sequential  channel  acquisition  to 
become  practical  for  routine  use.  Some  instruments  are  capable  of 
line-by-line  multi-tracking,  where  one  wavelength  is  used  to  excite 
the  sample  as  the  beam  travels  in  one  direction  and  the  second 
marker  is  excited  by  a  different  wavelength  on  the  return  scan. 
Although  faster,  one  must  adjust  the  instrument  (sometimes  called 
phase  control)  to  ensure  that  the  digitizer  stays  in  synch  with  the 
position  of  the  galvanometer  mirror.  Otherwise  the  two  sets  of  data 
can  be  displaced  horizontally  from  each  other. 

Spectral  Unmixing 

Spectral  unmixing  is  a  computational  method  of  separating  the 
signals  from  various  fluorophores  based  on  their  spectral  profile 
rather  than  by  their  presence  or  absence  in  a  specified  spectral 
window.  Spectral  unmixing  has  the  important  advantage  that  the 
fluorescence  from  fluorophores  with  highly  overlapping  spectral 
emission  profiles  can  be  reliably  separated. 

In  spectral  unmixing,  the  emission  spectrum  of  each  fluo- 
rophore  is  characterized  in  the  form  of  a  plot  of  its  fluorescence 
emission  intensity  as  a  function  of  wavelength.  Because  the  shape 
of  this  emission  profile  is  assumed  to  be  fairly  stable,12  one  may 
infer  the  magnitude  of  signal  that  would  be  detected  on-peak  using 
a  portion  of  the  emission  signal  that  was  measured  using  a  spec¬ 
tral  window  located  on  the  tail  of  the  spectrum.  Multiple  samples 
of  the  signal  intensity  taken  over  the  entire  the  emission  profile 
may  provide  an  even  more  accurate  estimate  of  the  peak  signal. 

It  is  then  possible  to  use  a  deconvolution  operation  to  separate 
overlapped  spectra  in  a  manner  analogous  to  the  way  in  which 
spatial  deconvolution  is  used  to  separate  overlapped  structural  fea¬ 
tures.  The  process  starts  with  the  collection  of  a  reference  spec¬ 
trum  from  each  positive  control  sample.13  Next,  spectral  data  is 
collected  at  each  voxel  in  a  2D  or  3D  image.  To  unmix  the  data, 
the  computer  fits  the  measured  data  to  a  weighted  sum  of  the  ref¬ 
erence  spectra.  Spectral  unmixing  can  be  performed  on  as  few  as 
two  spectral  windows  of  the  emission  spectra  at  each  pixel,  but 
using  data  from  multiple  spectral  windows  at  each  pixel  greatly 
improves  the  degree  of  separation  that  can  be  achieved. 

Spectral  Unmixing  to  Separate 
Overlapping  Fluorophores 

In  Figure  36.17,  live  cells  labeled  with  acridine  orange  have  been 
imaged  using  two  conventional  channels  on  a  Zeiss  Meta  confo¬ 
cal  microscope.  The  green  fluorescent  light  (500-530  nm)  has  been 
directed  to  one  channel  [Fig.  36.17(A)]  and  the  red  fluorescent 
light  (565-615  nm)  has  been  directed  to  a  second  channel  [Fig. 
36.17(B)].  Although  the  two  channels  do  show  some  differences, 
the  high  degree  of  overlap  is  evident  by  the  yellowish  color  that 


12  There  are  lots  of  counterexamples,  such  as  carboxyfluorescein,  that  changes 
its  output  spectrum  in  response  to  the  environment,  such  as  changes  in  pH. 

13  Although  the  manufacturer  often  provides  such  spectra  for  common  dyes, 
these  will  only  be  accurate  to  the  extent  that  (1)  the  QE/wavelength  curves 
of  the  detectors  used  in  the  factory  are  the  same  as  those  on  your  machine; 
(2)  the  dye  spectra  has  not  been  modified  by  the  local  environment;  and  (3) 
your  system  remains  aligned. 
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FIGURE  36.17.  Spectral  separation  using  two  spectral  images.  The  process  of  linear  unmixing  can  be  used  with  as  few  as  two  spectral  images  to  improve  the 
separation  between  the  two  channels.  In  this  example  HEK496  cells  have  been  labeled  with  Acridine  Orange  (final  concentration  0.5  |tg/mL)  for  1  h  at  room 
temperature  and  images  were  collected  on  a  Zeiss  510  Meta  confocal  microscope  using  two  “conventional”  optical  filter  separation  channels.  Images  were  col¬ 
lected  from  (A)  the  green  (500-530nm)  region  of  the  light  spectrum  and  (B)  the  red  region  of  the  spectrum  (565-615nm).  A  merged  two-color  image  of  the 
green  and  the  red  channel  is  shown  in  (C).  In  (D)  a  fluorogram  of  the  two  channels  is  shown.  The  corresponding  images  after  spectral  reassignment  are  shown 
in  (A')  green  (500-530nm),  f  IT)  red  (565-615)  nm.  and  both  channels  merged  (C).  (IT)  A  fluorogram  of  the  reassigned  merged  image  is  shown  in  (C')-  There 
is  considerable  overlap  between  these  two  original  imaging  channels,  which  is  shown  graphically  in  the  scattergram  shown  in  (D).  Applying  the  process  of  linear 
unmixing  (using  the  Zeiss  LSM  510  Meta  software)  the  separation  between  these  two  channels  has  improved  [(A)  and  (B)],  which  is  shown  by  comparing  the 
fluorogram  before  (D)  and  after  linear  unmixing  (D').  The  scale  bar  represents  20  pm. 


predominates  in  the  merged  image  [Fig.  36.17(C)].  This  overlap  is 
also  shown  by  the  tight  central  location  of  the  dots  in  the  whole- 
image  intensity  histogram  [Fig.  36.17(D)].  However,  after  linear 
unmixing  of  these  two  images,  the  difference  between  the  two 
images  becomes  more  distinct  [Fig.  36.17(A',B')].  A  merged  two- 
color  image  of  the  two  reassigned  channels  [Fig.  36.17(C')]  clearly 
shows  that  a  population  of  vesicles  has  now  been  separated  from 
the  bulk  of  the  staining.  Spectral  reassignment  has  resulted  in  a 
significant  movement  of  red  and  green  pixels  towards  their  respec¬ 
tive  single  label  axes  [see  the  greater  spread  of  the  pixel  intensity 
histogram  in  Fig.  36.17(D')  compared  with  Fig.  36.17(D)]. 
However,  the  separation  is  by  no  means  complete  as  the  majority 
of  structures  are  still  being  detected  in  both  channels. 

Figure  36.18  demonstrates  that  multiple  spectral  images  permit 
an  even  more  refined  separation  of  subcellular  structures.  In  this 
experiment,  10  individual  spectral  images  (each  with  a  spectral 
window  of  21.4  nm)  [Fig.  36.18(A)]  were  collected  using  two 
sequential  scans  on  a  Zeiss  510  Meta  confocal  microscope  (up  to 
eight  spectrally  distinct  images  can  be  collected  simultaneously). 
These  10  images  are  then  analyzed  as  a  “spectral  stack”  [Fig. 
36.18(B)],  and  four  separate  regions  of  the  image  stack  are  high¬ 
lighted  with  a  numbered  cross.  The  spectral  information  from  each 
of  these  regions  is  shown  graphically  in  Figure  36.18(C).  Using 
this  graphical  display  of  the  four  different  spectra  found  within  the 
sample,  two  spectral  regions,  green  (504-569  nm),  which  predom¬ 


inantly  covers  the  spectral  emission  of  ROI  2  to  4,  and  red 
(591-719nm),  covering  ROI  1  have  been  chosen  for  the  extrac¬ 
tion  of  two  channels  [Fig.  36.18(D)].  The  population  of  small  vesi¬ 
cles  with  a  spectral  emission  consisting  predominantly  of  red 
fluorescence  can  be  clearly  distinguished  from  the  structures  emit¬ 
ting  in  the  green  region  of  the  spectrum.  However,  the  separation 
of  the  red  vesicles  from  the  rest  of  the  cell  is  not  complete;  the 
majority  of  the  red  vesicles  also  contain  green  pixels  [seen  as 
yellow/orange  pixels  in  the  merged  image  of  Fig.  36.18(D)], 

The  spectral  information  shown  graphically  in  Figure  36.18(C) 
can  be  used  to  locate  and  extract  other  regions  of  the  specimen 
having  a  similar  spectral  fingerprint  using  linear  unmixing.  In  this 
case,  the  software  has  separated  the  stack  into  four  separate  chan¬ 
nels  [Fig.  36.18(E)]  plus  a  residual  channel  (regions  of  the  stack 
that  cannot  be  assigned  using  spectral  information  from  the 
selected  areas).  The  merged  two-color  image  of  channels  1  and  3 
[Fig.  36.18(E)]  shows  two  distinct  populations  of  vesicles  clearly 
separated  from  any  background  fluorescence.  The  vesicles  that 
were  partially  separated  from  the  other  cellular  labeling  using  con¬ 
ventional  optics  [Fig.  36.17(A,B)]  or  channel  extraction  from  the 
spectral  stack  [Fig.  36.18(D)],  as  described  above,  have  now  been 
separated  into  two  distinct  subpopulations  that  are  also  separated 
from  other  subcellular  structures.  This  example  shows  that  spec¬ 
tral  imaging  can  be  used  both  to  separate  the  fluorescence  from 
different  overlapping  signals  into  separate  channels,  and  also  to 
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FIGURE  36.18.  Spectral  separation  using  multiple  spectral  images.  Linear  unmixing  using  multiple  spectral  images  results  in  better  separation  of  fluorophores 
with  overlapping  emission  spectra.  HEK496  cells  were  labeled  with  Acridine  Orange  (final  concentration  0.5  pg/mL)  for  1  h  at  room  temperature  and  multiple 
spectral  images  were  collected  (A)  from  21.4nm  wide  regions  of  the  light  spectrum  using  the  Meta  channel  on  a  Zeiss  510  Meta  confocal  microscope.  The 
10  spectral  images  were  analyzed  as  a  spectral  stack  (B)  and  the  spectra  extracted  from  four  specified  single-pixel  regions  of  interest  (ROI,  marked  with  +  in 
the  spectral  stack)  are  shown  in  (C).  Two  defined  spectral  regions  (504-569  and  591-719  nm)  were  selected  and  the  spectral  information  extracted  into  separate 
channels  (D).  Four  spectrally-reassigned  channels  plus  a  residual  channel  after  linear  unmixing  are  shown  in  (E).  The  scale  bar  represents  5  pm. 
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collect  and  recognize  the  spectral  fingerprint  of  specific  structures 
within  the  sample. 

Removing  Autofluorescence  Using 
Spectral  Unmixing 

Two  different  approaches  to  separating  autofluorescence  from 
the  signal  produced  by  an  added  label  can  be  employed.  The  first 
approach  is  to  treat  the  autofluorescence  as  an  endogenous  fluo- 
rophore  that  will  be  placed  in  a  separate  channel  after  linear  unmix¬ 
ing.  This  approach  works  well  when  the  autofluorescence  is 
dominated  by  a  single  endogenous  fluorophore.  Unfortunately,  this 
is  rarely  an  accurate  assumption.  Another  approach  to  unmixing 
autofluorescence  from  specific  labels  is  to  use  spectral  profiles 
from  the  fluorophore  positive  controls  for  spectral  unmixing,  and 
to  assign  any  measured  fluorescence  emission  that  does  not  fit 
these  reference  spectra  to  a  “residual  channel”  [Fig.  36.18(E)],  In 
this  way  complex  spectral  emission  due  to  autofluorescence  can 
often  be  separated  from  the  more  specific  emission  associated  with 
the  particular  probes  added.  However,  caution  is  needed  to  prevent 
“seeing  what  one  wants  to  see,”  especially  when  the  level  of 
Poisson  noise  is  high. 

Limitations  to  Spectral  Unmixing 

Even  though  spectral  unmixing  has  a  remarkable  ability  to  sepa¬ 
rate  similar  fluorescence  emission  spectra,  there  are  disadvantages 
that  should  not  be  overlooked.  Spatial  deconvolution  (see  Chap¬ 
ters  25,  this  volume )  can  help  improve  the  integrity  of  results  from 
spectral  unmixing.  The  procedure  would  be  to  spatially  decon¬ 
volve  or  at  least  Gaussian  filter14  the  data  from  each  wavelength 
segment  prior  to  unmixing. 

Poisson  uncertainty  (see  Chapters  2,  4,  and  35,  this  volume) 
places  severe  restraints  on  the  accuracy  with  which  spectral  unmix¬ 
ing  can  be  carried  out.  Supposing  that  a  particular  dye  produces 
100  +  10  detected  photons/voxel,  if  these  are  then  shared  among 
only  four  channels  of  a  spectral  detector,  the  signal  drops  to  25  ± 
5  or  20%  noise.  It  is  also  notable  that  the  effective  QE  of  the  mini- 
PMT  arrays  used  in  some  systems  is  lower  than  that  found  on  many 
standard  PMT  tubes  used  in  confocal  microscopes.  Furthermore, 
the  layout  of  the  very  small  electron  multipliers  that  they  contain 
precludes  taking  actions  that  would  reduce  multiplicative  noise. 
Thus,  contributions  from  both  Poisson  noise  and  multiplicative 
noise  to  the  uncertainty  of  spectral  unmixing  may  be  exacerbated 
by  detector  design. 

The  collection  of  spectral  information  frequently  necessitates 
the  collection  of  several  images  in  series,  which  may  subject  the 
specimen  to  damaging  doses  of  light. 

Another  potential  problem  is  that  the  composite  spectrum  may 
be  distorted  because  any  segment  of  the  spectrum  that  overlaps  a 
dichroic  mirror  blocking  range  will  exhibit  a  prominent  dip  in  the 
signal  intensity.  In  addition,  if  the  spectra  being  collected  overlaps 
with  an  active  laser  line,  excessive  backscattered  light  may  con¬ 
taminate  the  signal. 

Environmentally  caused  variation  in  the  spectral  characteris¬ 
tics  of  a  fluorophore  may  lead  to  the  inappropriate  signal  separa¬ 
tion.  Many  fluorophores,  such  as  Acridine  Orange,  as  used  in  the 
example  above  (see  also  Chapter  39,  this  volume),  can  respond  dif¬ 
ferentially  to  environmental  influences. 


14  Because  the  data  is  so  noisy  and  the  PSF  of  a  confocal  or  multi-photon  micro¬ 
scope  is  so  simple,  3D  Gaussian  filtering  and  deconvolution  have  much  the 
same  effect.  Gaussian  filtering  can  also  be  applied  to  2D  data,  a  point  that  is 
important  because  much  unmixing  work  is  carried  out  on  2D  data. 


COLOCALIZATION 

Colocalization  is  a  tool  for  quantifying  the  degree  of  association 
or  codistribution  of  labeled  structures  between  any  two  channels 
in  an  image.  Although  colocalization  is  a  very  powerful  tool  for 
determining  subcellular  associations,  even  under  ideal  imaging 
conditions  the  resolution  limit  of  light  microscopy  severely  con¬ 
strains  the  validity  of  any  colocalization  analysis. 


Image  Collection  for  Colocalization 

Colocalization  by  counting  overlapping  signals  is  very  sensitive  to 
intensity  imbalances  between  each  channel,  as  are  methods  utiliz¬ 
ing  channel  subtraction  and  cross-correlation  analysis  (for  discus¬ 
sion  see  Manders  et  al.,  1993;  Costes  et  al.,  2004).  As  the  initial 
assessment  of  the  degree  of  colocalization  often  relies  on  a  quick 
visual  inspection  of  a  merged  two-channel  image,  it  is  advisable 
to  always  collect  relatively  balanced  signal  intensities  for  each 
channel.  Signal  balance  is  also  important  so  that  spread  of  the 
signal  outwards  from  the  highest  intensity  areas  does  not  bias  the 
results.  This  phenomenon  is  particularly  noticeable  in  small  struc¬ 
tures  with  one  or  more  dimensions  close  to  the  resolution  limit  of 
the  microscope. 

Fluorophore  selection  is  an  important  part  of  any  colocaliza¬ 
tion  study,  and  is  discussed  in  detail  by  Smallcombe  (2001).  Suit¬ 
able  pairs  of  fluorophores  should  have  a  wide  separation  between 
their  emissions  or  bleed-through  between  collection  channels  will 
be  registered  as  colocalization,  no  matter  which  method  of  analy¬ 
sis  is  used.  Filter  selection  (see  Chapter  3,  this  volume)  and  proper 
controls  (discussed  above)  are  crucial  for  the  validity  of  any  colo¬ 
calization  analysis.  Single  image  planes  from  any  volume  should 
be  examined  carefully  before  any  attempt  at  quantification.  Noise 
and  all  sources  of  background  (e.g.,  blur,  autofluorescence,  non¬ 
specific  labeling,  reflections,  and  bleed-through)  have  a  major 
impact  on  the  quantification  of  colocalization  (Manders  et  al., 
1993).  Optical  misalignment  of  the  instrument  will  also  have  a  big 
impact  on  the  apparent  degree  of  colocalization.  As  a  general  rule, 
problems  of  image  quality  should  be  addressed  at  the  time  of  data 
collection,  rather  than  by  postprocessing.  Providing  the  guidelines 
for  Nyquist-pixel  sampling  have  been  observed  and  the  raw  data 
has  been  3D  spatially  deconvolved  or  2D  filtered  to  remove 
“single-pixel  noise”  (see  below  and  Chapter  4,  this  volume),  the 
validity  of  colocalization  analysis  is  constrained  by  the  resolution 
limit  of  light  microscopy. 

Complex  images  should  be  examined  carefully,  and  specific 
regions  or  structures  within  the  image  analyzed  in  detail  before 
attempting  to  quantify  the  degree  of  colocalization.  Information  on 
the  biology  of  the  sample  will  be  of  great  value  in  determining 
the  approaches  taken  to  quantifying  the  degree  of  colocalization. 
Projection  images  from  optical  volumes  cannot  be  utilized  for 
colocalization  because  pixels  mapped  to  the  projection  image  may 
have  originated  in  planes  displaced  axially  by  several  microns. 

Temporal  resolution  should  not  be  overlooked  when  using 
sequential  imaging  on  living  cells.  This  is  particularly  important 
when  sequential  frame  scanning,  but  even  when  sequential  line 
scanning,  or  using  a  CCD  camera  on  high-speed  Nipkow  disk 
systems,  a  delay  of  as  little  as  100  ms  between  channels  can  allow 
a  structure  to  change  position  in  resting  cells,  this  includes  the 
normal  movement  of  vesicles,  organelles,  filopodia,  or  neuronal 
growth  cones.  The  temporal  sampling  rate  becomes  more  critical 
when  attempting  colocalization  of  more  dynamic  processes.  The 
result,  of  course,  is  that  instead  of  observing  a  possible  colocal- 
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ization  event,  one  observes  the  structure  as  single-labeled  and  spa¬ 
tially  separated  in  two  separate  channels. 


Quantifying  Colocalization 

The  2D  histogram,  sometimes  called  a  fluorogram,  is  a  powerful 
tool  for  exploring  the  relationship  of  the  intensities  between  two 
channels  in  an  image  (or  within  a  defined  region  of  the  image). 
The  2D  histogram  is  a  scatter  plot  with  the  intensities  of  all  the 
pixels  in  one  image  channel  plotted  along  the  jc-axis  versus  the 
intensities  of  the  same  pixels  in  the  other  channel  plotted  along  the 
y-axis  (Demandolx  and  Davoust,  1995,  1997).  For  example,  in 
Figure  36.20,  the  green  channel  is  plotted  along  the  y-axis  and  the 
red  channel  is  plotted  along  the  x-axis,  with  0  at  the  origin.  Given 
equal  background  levels  in  the  channels,  pixels  possessing  nearly 
equal  intensities  in  both  channels  would  plot  along  a  45°  line 
passing  through  the  origin,  similar  to  the  distribution  in  the  fluo¬ 
rogram  in  Figure  36.17(D).  Custom  color  tables  can  be  useful  to 
highlight  potential  colocalizations  when  working  with  rare  events 
or  imbalances  in  channel  intensities  (Demandolx  and  Davoust, 
1997;  Agnati  et  al. ,  2005). 

The  most  commonly  employed  algorithms  for  quantifying 
colocalization  measurement  are  derived  from  Pearson’s  correlation 
coefficient,  first  applied  for  this  purpose  by  Manders  and  col¬ 
leagues  (1992).  The  Pearson’s  coefficient,  R ,  describes  the  corre¬ 
lation  between  the  intensity  distribution,  or  pattern  overlap,  in  two 
channels  in  terms  of  a  least-squares  fit  (Eq.  3).  This  value  can  be 
between  -1  and  1,  where  R  =  —  1  indicates  complete  exclusion  and 
R  =  1  indicates  complete  correlation  between  the  two  channels. 
Values  between  0  and  1  indicate  a  proportional  degree  of  overlap 
between  the  two  channels.  Values  between  -1  and  0  are  ambigu¬ 
ous,  indicating  some  form  of  inverse  relationship  between  the 
channels,  as  may  occur  when  one  channel  is  very  dim  and  the  other 
is  bright,  or  even  when  non-overlapping. 

X(51i-SlJx(S2,.-S2j 

R~  SL{s\-s^J^{s2-siJ  (3) 

V  i  i 

where  51,  is  the  intensity  of  the  ;th  pixel  in  channel  1;  51„Vg  is  the 
average  intensity  of  all  pixels  in  channel  1 ;  52,  is  the  intensity  of 
the  ;th  pixel  in  channel  2;  and  S2avg  is  the  average  intensity  of  all 
pixels  in  channel  2. 

Other  coefficients  have  been  subsequently  derived  to  provide 
additional  information  regarding  the  nature  of  the  colocalization, 
as  well  as  providing  less  sensitivity  to  channel  imbalances 
(Manders  et  al.,  1993).  The  overlap  coefficient,  r,  describes  the 
degree  of  overlap  between  the  signals  on  a  range  of  0  to  1  (Eq.  4) 

1(^1,  x  52,) 

r=jxmrxi(v,y  <4) 

V  i  i 

This  coefficient  provides  a  significant  result  only  when  the 
number  of  colocalizing  objects  in  both  channels  is  greater  than 
zero.  The  advantage  of  this  coefficient  is  that,  unlike  R,  it  is  not  as 
affected  by  differences  in  intensities  between  the  channels,  reduc¬ 
ing  the  impact  of  channel  imbalance.  Although  the  overlap  coeffi¬ 
cient  has  these  advantages,  the  Pearson’s  coefficient  is  less 
sensitive  to  image  background  levels.  Both  of  these  coefficients,  R 
and  r,  are  affected  by  the  relative  number  of  labeled  objects  (or 
voxels)  in  each  channel  of  the  image. 

The  colocalization  coefficients  Ml  (Eq.  5)  and  M2  (Eq.  6)  are 
often  the  most  biologically  useful,  representing  the  proportion  of 


pixels  from  each  channel  that  contribute  to  the  colocalized  area. 
They  can  be  determined  regardless  of  the  balance  between  the 
number  of  pixels  colocalizing  in  each  channel  or  between  the 
average  intensities  of  the  two  channels.  The  only  limitation  is  that 
there  must  be  some  colocalization  present. 


2X 


Ml  =  ■ 


£51,. 


(5) 


where  5 1  Uoloc  =  51,  if  S2t  >  0  and  51,c„toc  =  0  if  S2t  =  0 

£  52 i^oloc 


M2  =  - 


£52, 


(6) 


where  S2Ucoloc  =  5 1 ,  if  5 1  ,  >  0  and  S2icoloc  =  0  if  SI i  =  0. 

Each  numerator  represents  colocalized  pixels  —  the  sum  of 
intensities  from  pixels  within  one  channel  that  also  contain  some 
intensity  component  from  the  other  channel.  The  denominators 
constitute  the  summed  intensities  of  all  pixels,  above  threshold,  in 
the  channel.  Some  software  applications  use  upper-  and  lowercase 
to  distinguish  colocalization  coefficients  calculated  from  the  global 
image  from  those  calculated  from  a  thresholded  image  or  a  region 
selected  from  the  fluorogram. 


Setting  Thresholds 

Quantitation  in  colocalization  studies  is  usually  dependent  upon 
the  subjective  application  of  thresholds.  Establishing  the  threshold 
to  eliminate  noise  or  background  is  sometimes  earned  out  by  sub¬ 
tracting  the  threshold  from  the  image,  or  by  setting  limits  in  the 
2D  histogram,  which  many  find  to  be  more  intuitive.  One  may 
have  to  test  multiple  thresholds  before  finding  the  levels  that  are 
supported  by  correlative  methods  and  the  controls.  When  software 
recommends  threshold  settings  (or  setting  background  levels),  it  is 
important  to  know  how  those  settings  are  being  calculated.  A  back¬ 
ground  derived  from  the  darkest  pixels  within  the  image  more  cor¬ 
rectly  represents  the  noise  floor  of  the  image  rather  than  the  amount 
of  background  from  bleed-through,  autofluorescence,  or  non-spe¬ 
cific  labeling.  A  new  approach  has  been  developed  that  estimates 
thresholds  for  both  channels  simultaneously  over  a  range  of 
decreasing  thresholds  until  the  probability  of  correlation  equals 
zero  (Costes  et  al.,  2004).  This  approach  promises  to  be  less  sub¬ 
jective,  but  can  be  still  skewed  by  an  imbalance  in  average  channel 
intensities. 


Spatial  Deconvolution  in  Colocalization  Studies 

Noise  and  background  in  colocalization  can  be  reduced  by  em¬ 
ploying  restorative  deconvolution  (Van  Steensel  et  al,  1996; 
Landemann,  2002;  Landmann  and  Marbet,  2004).  Even  analysis 
intended  for  2D  images  can  benefit  from  deconvolution  if  the  data 
is  collected  as  a  shallow  z-series  and  the  single  2D  image  selected 
following  deconvolution  of  the  z-series.  Alternatively,  2D  confo- 
cal  images  can  be  Gaussian  filtered  if  sampled  adequately. 

In  the  example  shown  in  Figure  36.19,  a  field  of  peripheral 
nerves  in  rat  tooth  pulp  is  shown  as  a  single  optical  slice  from  a 
z-stack  collected  at  low  resolution  [Fig.  36.19(A)],  with  a  small 
region  of  the  sample  [small  box  in  panel  (A)]  shown  at  higher 
optical  resolution  [Fig.  36.19(B)].  The  arrowhead  points  to  the 
same  fiber  in  both  panels.  Panel  (B)  indicates  that  there  are  actu¬ 
ally  two  populations  of  closely  intertwining  nerve  fibers,  larger 
fibers  predominantly  labeled  red  (arrowhead),  and  the  smaller, 
somewhat  beaded  fibers  labeled  green  (arrows).  Closer  examina- 
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FIGURE  36.19.  Colocalization.  Nerve  fibers  labeled  for  peripherin  (red)  and 
CGRP  (green)  in  tooth  pulp  from  rats  3  weeks  of  age.  Initial  observations  with 
a  20  X  0.75  NA  Fluor  lens  (Nikon)  (A)  suggest  a  population  of  nerves  co- 
immunostained  for  both  proteins,  appearing  yellow.  A  60  X  1.4  NA  PlanApo 
lens  (Nikon)  with  zoom  =  2  (B)  reveals  two  distinct  populations  of  fibers:  a 
thicker  population  labeled  with  peripherin  (arrowhead),  and  smaller,  beaded 
fibers,  labeled  with  CGRP  (small  arrows ).  Both  images  are  single  planes  from 
."-slacks.  High  magnification  images  were  undersampled  60%  laterally  (150  nm 
vs.  S3  nm)  for  this  sample.  Image  (B)  has  been  deconvolved  by  a  maximum 
likelihood  estimate  algorithm  (MLE).  Scale  bars  represent  25  pm  (A)  and 
10pm  (B). 


tion  shows  that  the  two  labels  may  be  co-existing  (as  noted  by 
yellow)  only  where  the  libers  cross,  or  in  some  of  the  thickened 
regions  along  the  small  green  fibers.  Given  that  the  ^-resolution  of 
the  confocal  microscope  is  significantly  less  than  the  xy-resolution, 
such  overlaps  may  simply  be  due  to  the  diminished  z-resolution. 

The  2D  histograms  in  Figure  36.20  are  derived  from  the  high- 
resolution  volume  shown  in  Figure  36.19(B),  and  graphically 
demonstrate  the  impact  on  colocalization  of  background  removal 
through  deconvolution.  The  2D  histogram  of  the  original  (non- 
deconvolved)  volume  [Fig.  36.20(A)]  displays  two  putative  corre¬ 
lations,  suggesting  that  each  fiber  is  predominantly  labeled  for  one 
protein,  but  containing  the  other  protein  in  reduced  amount.  Pixels 
plotting  close  to  the  x-axis  represent  low  intensity  green  label,  such 
as  the  line  of  pixels  very  close  to  the  x-axis  (below  the  threshold 
line)  indicating  low-intensity  background  fluorescence  in  the  green 
channel  that  associates  with  all  levels  of  red  fluorescence.  Pixels 
plotting  near  the  y-axis  represent  low  intensity  red  label. 

The  2D  histogram  changes  dramatically  after  restora¬ 
tive  deconvolution  by  maximum  likelihood  estimation  (MLE, 
using  Huygens,  Scientific  Volume  Imaging)  of  the  z-stack 


FIGURE  36.20.  Fluorogram  analysis  of  colocalization.  2D  histograms  of  the 
volumes  shown  in  Figure  36.19.  Green  intensities  are  plotted  on  the  y-axis 
(0-255),  and  red  intensities  are  plotted  on  the  x-axis  (0-255).  The  fluorogram 
before  deconvolution  (A)  shows  low  intensity  background  in  the  green  channel 
indicated  by  a  layer  of  points  lying  above  the  x-axis  and  two  correlations  of 
unequal  intensity  possibly  existing  between  the  two  channels.  After  restorative 
deconvolution  the  2D  histogram  (B)  has  less  background  and  shows  the  effect 
of  background  removal  on  putative  colocalization.  While  (A)  suggests  a  pos¬ 
sible  weak  correlation  in  a  bimodal  distribution,  (B)  suggests  a  lack  of  colo¬ 
calization.  The  white  lines  in  both  plots  represent  threshold  levels  applied  to 
both  channels  for  subsequent  analysis  on  the  volume  (Table  36.3),  and  shown 
by  white  masks  in  panel  (C)  and  (D).  Panel  (C)  is  the  same  field  as  that  shown 
in  Figure  36. 19(B),  but  taken  from  the  volume  before  deconvolution,  and  panel 
(D)  is  the  same  field  after  deconvolution.  The  scale  bar  represents  10  pm. 


[Fig.  36.20(B)].  The  low-intensity  background  from  the  green 
channel  has  been  removed  and  the  two  regions  of  correlation 
observed  in  Figure  36.20(A)  have  been  sharply  reduced,  suggest¬ 
ing  that  they  were  due  to  blur,  as  does  the  fact  that  most  pixels  are 
now  shifted  towards  the  x-  or  y-axis. 

The  impact  of  MLE  on  the  correlation  coefficients  is  shown  in 
Table  36.3.  The  red  and  green  labels  show  very  high  correlations 
in  the  original,  undeconvolved  low  resolution  volume  [Fig. 
36.19(A)],  with  values  of  0.817  and  0.896  for  R  and  respectively, 
in  Table  36.3,  Line  1.  The  colocalization  coefficients  Ml  and  M2 
indicate  that  all  voxels  are  colocalized  because  100%  of  voxels  in 
each  channel  are  contributing  to  the  colocalization.  The  higher 
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TABLE  36.3.  Effect  of  MLE  and  Threshold  on  Colocalization 


Sample 

MLE 

Threshold 

R 

r 

Ml 

M2 

1 

Fig.  36.19(A) 

No 

No 

0.817 

0.896 

1.00 

1.00 

2 

Fig.  36.19(B) 

No 

No 

0.624 

0.803 

1.00 

1.00 

3 

Fig.  36.19(B) 

No 

37/37 

0.457 

0.461 

0.496 

0.789 

4 

Fig.  36.19(B) 

Yes 

No 

0.345 

0.475 

0.807 

0.833 

5 

Fig.  36.19(B) 

Yes 

20/20 

0.298 

0.303 

0.254 

0.556 

resolution  volume  [Fig.  36.19(B)]  displays  some  reduction  in  both 
Pearson’s  coefficient  and  overlap  coefficient  (Table  36.3,  Line  2) 
as  the  two  populations  of  fibers  are  now  resolved  into  separate 
objects.  However,  the  colocalization  coefficients  for  both  channels 
are  still  at  unity  suggesting  either  perfect  correlation  or,  more 
likely,  significant  levels  of  background. 

MLE  processing  also  changes  the  colocalization  values,  as 
shown  in  Table  36.3,  Line  4.  The  amount  of  correlation  from 
Pearson's  coefficient  drops  to  34.5%  and  the  overlap  coefficient  is 
reduced  to  47.5%.  The  proportion  of  contribution  to  colocalization 
by  the  two  channels  (Ml  and  M2)  are  reduced  to  80.7%  for  the  red 
channel  (Ml)  and  83.3%  for  the  green  channel  (M2). 

The  threshold  is  important  for  determining  the  level  of  fluo¬ 
rescence  that  is  to  be  considered  significant.  Thresholds  were  set 
for  both  channels  by  measurements  of  presumed  background  from 
the  darker  regions  between  the  labeled  fibers.  These  thresholds  are 
represented  by  the  white  lines  in  Figure  36.20(A),  at  a  threshold 
of  37  for  the  original  image  stack  and  at  a  threshold  of  20  in  Figure 
36.20(B),  the  fluorogram  of  the  volume  after  MLE.  Table  36.3  pro¬ 
vides  the  corresponding  coefficients  for  the  volumes  following 
application  of  thresholds.  The  field  in  Figure  36.20(C)  is  the  same 
field  as  in  Figure  36.19(B),  but  taken  from  the  high  resolution 
volume  before  deconvolution.  The  white  mask  represents  the 
pixels  above  threshold,  as  applied  to  this  image  plane.  Figure 
36.19(D)  is  the  same  field  as  Figure  36.19(B),  taken  from  the 
volume  after  deconvolution.  Pixels  above  the  threshold  applied  to 
this  image  after  deconvolution  are  represented  by  the  white  mask. 
The  differences  between  Figure  36.20(C)  and  Figure  36.20(D)  not 
only  include  the  difference  in  number  of  colocalized  pixels,  but 
extend  to  the  apparent  thicknesses  of  the  nerve  fibers  and  back¬ 
ground  around  the  fibers. 

Clearly,  the  apparent  colocalization  in  the  low  magnification 
image  and  the  high-resolution  image  was  due  to  blending  of  the 
two  channels  by  blur,  noise  and  bleed-through.  Restorative  decon¬ 
volution  reduced  these  contributions  in  a  far  more  meaningful 
manner  than  was  possible  with  simple  thresholding.  The  remain¬ 
ing  colocalization  in  this  example  is  likely  due  to  bleed-through, 
blur  at  the  intersections  of  fibers,  as  well  as  blur  from  fibers  in 
adjacent  volume  planes  (not  shown). 

Using  deconvolution,  direct  inspection  of  the  image,  and  the 
2D  histogram  in  conjunction  with  the  quantitative  analysis  tools 
available  for  colocalization  can  provide  far  greater  insight  into  the 
relationships  between  the  two  labels  in  a  volume  than  can  any 
single  tool  alone.  While  we  may  be  left  with  questions  regarding 
the  exact  nature  and  extent  of  colocalization  in  this  example,  these 
tools  can  be  used  together  to  indicate  directions  that  might  be  taken 
to  determine  the  relationship  between  these  two  proteins  more 
closely.  Careful  use  of  adequate  controls  to  establish  acquisition 
parameters  that  avoid  bleed-through,  noise,  blur  and  autofluores¬ 
cence,  as  described  in  detail  by  Costes  et  al.  2004,  is  essential  for 
obtaining  sound  datasets  for  colocalization. 


DISCUSSION 

Recent  advances  spanning  only  the  past  20  to  30  years  have  fueled 
extremely  rapid  development  and  popular  adoption  of  laser¬ 
scanning  microscopy  as  a  practical  and  uniquely  powerful  tool  for 
scientific  discovery.  The  capabilities  of  these  instruments  have  two 
faces  —  on  one  hand,  they  can  gather  light  with  unprecedented 
sensitivity,  lack  of  noise,  and  resolution  so  that  we  can  work  at  the 
limits  of  physics  on  biological  questions  that  were  unapproachable 
in  an  earlier  era.  On  the  other  hand,  they  can  also  generate  exquis¬ 
itely  stunning  images  of  artifact  or  elevate  the  most  mundane 
source  of  background  into  signals  that  overwhelm  all  meaningful 
information  from  the  sample.  Adequate  controls  are  an  absolute 
requirement  to  understand  the  interplay  of  the  instrument  controls 
and  your  samples.  The  regular  application  of  standard  confidence 
tests  applied  to  both  the  instrument  and  the  biological  samples 
serve  to  demonstrate  the  outstanding  capabilities  of  a  well-main¬ 
tained  imaging  system  as  well  as  providing  meaningful  informa¬ 
tion  on  the  limitations  of  such  equipment. 

To  ensure  maximum  productivity,  it  is  advantageous  to 
monitor  the  progress  of  declining  performance  so  that  the  system 
may  be  restored  to  peak  performance  before  the  potential  for  griev¬ 
ous  artifact  or  frank  system  failure  becomes  a  reality.  Murphy’s 
Law  dictates  that  there  is  a  disproportionate  chance  such  problems 
will  be  discovered  at  the  onset  of  an  important  experiment  or  when 
results  are  needed  in  the  face  of  looming  deadlines.  Quantitative 
measures  of  performance  may  serve  to  aid  remote  diagnosis  by  a 
field  engineer  and  expedite  subsequent  ordering  of  the  necessary 
parts.  It  is  hoped  that  the  tests  outlined  in  this  chapter  will  aid  you 
in  realizing  the  goals  of  data  integrity,  peak  performance,  and  high 
instrument  reliability  when  using  laser-scanning  confocal  micro¬ 
scopes  in  a  variety  of  environments. 
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INTRODUCTION 

In  this  chapter  we  present  an  alternative  way  of  optically  sectioning 
the  sample  in  fluorescence  microscopy.  By  illuminating  the  sample 
from  the  side  with  a  sheet  of  light,  and  viewing  the  light  emitted  or 
scattered  by  this  layer  with  a  widefield  microscope  oriented  on  an 
axis  perpendicular  to  the  sheet,  a  sectioning  effect  that  is  similar  to 
that  in  confocal  microscopy  can  be  produced.  Such  an  approach  has 
several  advantages  over  confocal  scanning  microscopy  and  these 
make  the  technology  especially  well  suited  for  relatively  large 
samples  such  as  embryos.  However,  the  principle  is  universal  and 
can  also  be  applied  to  micrometer-sized  samples. 

This  chapter  introduces  light-sheet  microcopy  by  providing  an 
overview  of  microscopy  techniques  for  large  samples  and  the 
history  of  techniques  employing  plane  illumination.  The  light- 
sheet  microscope  that  we  have  developed  at  the  EMBL  is  illus¬ 
trated  and  explained,  its  characteristics  are  described  and  evaluated 
in  the  context  of  confocal  microscopy  and  its  performance  demon¬ 
strated  by  a  few  applications. 

The  stereo  microscope  is  the  most  commonly  used  microscope 
in  developmental  biology.  It  provides  a  stereoscopic  image  of  the 
sample  while  keeping  it  under  ideal  conditions  for  sorting  and 
selecting  embryos  (e.g.,  in  a  Petri  dish  filled  with  an  appropriate 
medium).  A  camera  can  be  used  to  record  images  or  even  movies 
of  the  developing  embryo  for  later  reference.  However,  no  volu¬ 
metric  quantification  can  be  made  because  the  stereo  microscope 
offers  little  depth  discrimination  and  no  optical  sectioning.  The 
same  holds  true  for  conventional  widefield  microscopes,  which  in 
addition  often  suffer  from  short  working  distances  that  are  inade¬ 
quate  for  imaging  large  samples. 

Quantitative  analysis  of  three-dimensional  (3D)  structures 
requires  sectioning  (optical  or  otherwise).  Out-of-focus  light  is 
rejected  in  confocal  microscopy  by  coupling  point  excitation  with 
a  laser  beam  to  point  detection  through  a  pinhole,  so  that  in  general 
only  in-focus  light  is  detected.  This  principle  works  well  in  rela¬ 
tively  thin  samples  (up  to  ca.  100 |im);  however,  in  large  and  scat¬ 
tering  objects,  the  signal  is  lost  since  most  of  the  fluorescent  light 
is  scattered  and  therefore  rejected  by  the  pinhole.  The  confocal 
microscope,  therefore,  suffers  from  a  limited  penetration  depth, 
and  cannot  be  applied  for  in  toto  studies  of  most  embryos.  In  such 
samples,  the  best  resolution  is  obtained  by  physically  sectioning 
the  sample  (Weninger,  2002),  which  is  slow,  laborious,  destroys 
the  sample  irretrievably,  and  is  not  applicable  for  studies  of  living 
preparations. 

The  many  methods  that  have  recently  been  developed  to 
enhance  the  resolution,  e.g.,  4Pi-confocal  (Hell,  1992),  confocal 
theta  microscopy  (Stelzer,  1994),  I5M  microscopy  (Gustafsson, 
1999),  and  Stimulated  Emission  Depletion  (STED)  microscopy 
(Klar,  2000)  do  not  address  the  challenges  encountered  in  large  and 


scattering  samples  such  as  embryos.  Their  improved  resolution  is 
easily  degraded  by  sample-induced  wavefront  distortions,  and  in 
addition,  they  generally  require  quite  complex  instrumentation. 
Chapter  34  presents  an  overview  of  imaging  techniques  for  large 
samples,  such  as  Optical  Projection  Tomography  (OPT;  Sharpe, 
2002)  and  Micro  Magnetic  Resonance  Imaging  ( |J.MRI ;  Louie, 
2000),  which  to  some  extent  attempt  to  overcome  these  challenges. 

COMBINING  LIGHT  SHEET  ILLUMINATION 
AND  ORTHOGONAL  DETECTION 

“Light-sheet”  microscopy  now  provides  an  alternative  approach 
for  imaging  large  samples  at  high  resolution.  Like  confocal 
microscopy,  techniques  based  on  sheet  illumination  have  been 
“invented”  a  number  of  times  and  all  owe  much  to  the  “Ultra- 
mikroskop,”  an  orthogonal,  darkfield  illuminator  invented  by 
Siedentopf  and  Zsigmondy  in  1903,  to  visualize  gold  particles 
much  smaller  than  the  resolution  limit  by  detecting  light  scattered 
perpendicular  to  the  illumination. 

The  light-sheet  concept  has  been  used  in  several  other  micro¬ 
scopes  since  then.  For  decades  the  slit-lamp  was  used  for  viewing 
the  human  eye  (Campbell  et  al.,  1974)  and  similar  techniques 
were  reviewed  by  Webb  in  Chapter  34  of  the  second  edition  of 
this  Handbook.  More  recently,  Voie  used  the  high  coherence  of 
laser  light  to  develop  orthogonal-plane  fluorescence  sectioning 
(OPFOS)  and  applied  it  to  the  cochlea  (Voie,  1993,  Voie  et  al., 
1995). 1  Although  initially  operating  at  a  fixed  magnification  and  a 
resolution  set  by  the  close-focus  camera  lens  used  to  convey 
the  image  to  the  CCD  camera,  more  recent  implementations 
(Voie,  2002)  used  a  4x  optical  system  and  a  rotating  specimen 
immersed  in  index-matching  medium  to  record  details  about  1 0pm 
in  size. 

In  2002,  Fuchs  described  the  Thin  Laser  Light  Sheet  Micro¬ 
scope  (TLLSM)  (Fuchs,  2002)  to  detect,  discriminate  and  docu¬ 
ment  microbes  passing  through  the  light  sheet.  Although  it  has  the 
same  optical  arrangement  as  Siedentopf  and  Zsigmondy,  TLLSM 
uses  fluorescence,  which  was  not  practical  100  years  ago.  Because 
in  this  system,  the  stage  is  stationary  relative  to  the  light  sheet, 
only  the  slice  of  the  sample  that  happens  to  intercept  the  sheet  is 
imaged,  making  the  system  unsuitable  for  generating  three- 
dimensional  images. 

Another  technique  with  orthogonal  illumination  and  detection 
is  3D  Light  Scanning  Macrography  (Huber,  2001),  which  is  used 
for  generating  macroscopic  reconstructions  of  samples  using  scat- 


1  More  information  on  this  system  can  be  obtained  at  www. 
spencertechnologies.com 
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tered  light.  The  orthogonal  arrangement  of  illumination  and  detec¬ 
tion  was  also  used  in  theta  microscopy  (Stelzer,  1994),  although 
no  light  sheet  is  formed  in  this  variant  of  the  single-point  confo- 
cal  scanning  microscope. 

Our  current  implementation  of  the  technique  is  the  latest  of 
three  versions  of  the  selective  plane  illumination  microscope 
(SPIM).  It  is  embodied  in  a  compact,  stable,  and  economical 
system  based  on  three  operating  principles:  illumination  of  the 
sample  with  a  light  sheet,  observation  of  light  coming  from  the 
sample  in  a  direction  perpendicular  to  the  illumination  axis,  and, 
optionally,  rotation  of  the  sample  about  an  axis  parallel  to  gravity. 
A  3D  image  is  recorded  by  scanning  the  sample  through  the  sta¬ 
tionary  light  sheet  and  recording  the  fluorescence  or  scattered  light 
with  a  CCD  camera.  The  sample,  which  can  be  as  small  as  a  few 
micrometers  or  as  large  as  several  millimeters  depending  on  the 
working  distance  of  the  objective  lenses,  can  be  embedded  in  a 
gel.  immersed  in  a  liquid  or  held  in  air.  As  it  is  mounted  on  a  stage 
that  can  be  rotated  as  well  as  translated,  one  can  record  3D  images 
from  multiple  directions,  and  these  data  sets  can  be  combined  into 
a  single  3D  data  set  with  a  spatial  resolution  dominated  by  the 
lateral  resolution  of  the  detection  system. 


SELECTIVE  PLANE  ILLUMINATION 
MICROSCOPY  SETUP 

Figure  37.1  shows  the  main  components  of  SPIM.  A  series  of 
lasers  [several  helium-neon  (He-Ne)  and  a  multi-line  argon  (Ar)- 
ion  laser]  provide  a  range  of  excitation  wavelengths,  which  can  be 
focused  by  a  cylindrical  lens  to  form  a  thin  light  sheet.  The  sample 
is  mounted  in  a  transparent  cylinder,  typically  made  of  agarose 
prepared  using  an  aqueous  solution  suitable  for  the  sample  (e.g., 
phosphate-buffered  saline,  PBS).  The  agarose  cylinder  is  then 
immersed  in  a  bath  of  this  solution,  virtually  eliminating  refractive 
index  artifacts  at  the  surface.  The  cylinder  is  held  from  above  by 
a  micropositioning  device  with  four  degrees  of  freedom  (three 
translational,  one  rotational),  which  allows  positioning  the  sample 
such  that  the  excitation  light  illuminates  any  plane  of  interest.  An 
objective  lens  placed  perpendicular  to  the  illumination  plane,  a 
detection  filter,  and  a  tube  lens  are  used  to  image  fluorescent  light 
excited  by  the  light  sheet  onto  a  CCD  camera.  A  variety  of  objec¬ 
tive  lenses  can  be  used  (preferably  those  designed  for  imaging  in 
water  without  a  coverslip),  with  magnifications  ranging  from  2.5x 
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FIGURE  37.1.  Schematic  setup  of  a  selective  plane  illumination  microscope.  The  light  sources  (back),  illumination  path  (front,  left),  and  detection  path  (front, 
right)  are  shown.  The  beams  of  two  lasers  are  superimposed  and  the  acousto-optical  tunable  filter  (AOTF)  is  used  to  select  and  to  attenuate  the  desired  laser 
line.  (The  use  of  the  optical  fiber  to  deliver  the  laser  light  to  the  microscope  helps  to  reduce  mechanical  vibrations,  but  is  actually  optional.)  The  laser  light  is 
focused  by  the  cylindrical  lens  to  form  a  light  sheet  that  traverses  the  chamber.  The  sample  is  placed  into  the  chamber  in  the  path  of  the  light  sheet.  Widefield 
detection  is  performed  with  the  objective  lens,  filter,  tube  lens,  and  CCD  camera.  (Inset)  Detailed  view  of  the  chamber  with  sample,  illumination  beam,  and 
detection  lens.  The  light  sheet  enters  the  aqueous  medium-filled  chamber  (shown  cut  open)  through  a  thin  glass  window.  The  sample  is  embedded  in  a  cylinder 
of  agarose,  which  is  supported  from  above  and  can  be  moved  by  a  translating  and  rotating  stage.  The  objective  lens  is  sealed  into  the  chamber  with  a  rubber  O- 
ring,  and  can  be  moved  axially  to  focus  on  the  plane  of  fluorescence  excited  by  the  light  sheet.  The  sample  is  then  moved  through  the  light  sheet  to  acquire  a 
3D  data  set. 
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FIGURE  37.2.  Theoretical  intensity  point  spread  functions  (PSF)  in  the  SPIM.  The  system  PSF  (right)  is  obtained  by  multiplying  the  illumination  (left)  and 
detection  PSFs  (center).  /,|U  =  488nm,  =  520nm,  NAffl  =  0.15,  NAde,  =  0.8,  n  =  1.33.  Contour  lines  are  plotted  for  intensities  of  0.9,  0.7,  0.5,  0.3,  0.2,  0.1, 

0.05,  0.03,  0.02.  0.015,  0.01,  0.005,  0.002,  and  0.001.  The  intensities  are  shown  with  a  lookup  table  (LUT)  of  y  =  0.25,  in  blue  for  excitation,  in  green  for  detec¬ 
tion,  and  in  red  for  the  system. 


to  lOOx,  numerical  apertures  (NAs)  from  0.16  to  1.0,  and  working 
distances  of  1  to  12  mm. 

LATERAL  RESOLUTION 

The  lateral  resolution  of  SPIM  is  determined  by  the  NA  of  the 
detection  lens,  as  is  generally  the  case  in  widefield  microscopy. 
However,  if  the  images  are  under-sampled  by  the  CCD,  the  lateral 
resolution  of  the  detection  lens  is  not  fully  exploited,  and  the  pixel 
size  of  the  camera  determines  the  lateral  resolution.  This  is  a 
general  problem  in  digital  microscopy  using  currently  available 
objective  lenses  and  cameras,  and  is  not  unique  to  SPIM.  Figure 
37.2  shows  the  point  spread  functions  (PSF)  of  illumination  and 
detection  in  SPIM,  as  well  as  the  product  of  the  two,  that  is,  the 
system  PSF,  for  typical  NAs.  The  light  sheet  illumination  does  not 
affect  the  lateral  resolution. 

LIGHT  SHEET  THICKNESS  AND 
AXIAL  RESOLUTION 

There  are  two  distinct  yet  related  effects  of  the  light  sheet  dimen¬ 
sions  that  are  relevant  to  SPIM.  First,  selective  illumination  by  the 
light  sheet  provides  optical  sectioning  in  SPIM,  and  the  extent  of 
this  capability  depends  on  the  thickness  of  the  light  sheet.  Second, 
the  light  sheet  improves  the  axial  resolution  of  the  SPIM  if  it  is 
thinner  than  the  axial  extent  of  the  detection  PSF  (Fig.  37.2).  For 
both  of  these  reasons  the  optimal  light  sheet  for  SPIM  is  made  as 


thin  as  possible,  while  maintaining  its  properties  approximately 
uniform  across  the  field  of  view.  The  relations  between  the  light 
sheet  dimensions  and  the  detection  field  of  view  are  schematically 
illustrated  in  Figure  37.3. 

The  minimum  thickness  of  the  light  sheet  is  inversely  propor¬ 
tional  to  the  NA  of  the  cylindrical  illumination  lens,  while  the 
width  of  this  thin  waist  (i.e.,  its  extent  along  the  illumination  axis) 
is  inversely  proportional  to  its  NA2.  While  the  importance  of  the 
thickness  of  the  light  sheet  is  obvious  (for  the  reasons  given 
above),  that  of  its  width  is  perhaps  less  so.  However,  when  one 
considers  that  it  is  desirable  to  have  the  extent  of  the  optical  sec¬ 
tioning  uniform  across  the  field  of  view,  it  becomes  clear  that  it  is 
necessary  to  have  the  light  sheet  width  at  least  as  large  as  the  detec¬ 
tion  field  of  view  (see  Fig.  37.3).  Thus,  the  optimal  SPIM  design 
involves  a  compromise  between  strong  optical  sectioning  (requir¬ 
ing  a  large  illumination  NA)  and  a  large  field  of  view  (requiring  a 
small  illumination  NA). 

The  simplest  means  of  balancing  these  two  parameters  is  to 
change  the  focal  length  of  the  cylindrical  lens  or  the  size  of  its 
aperture.  As  an  example  of  a  configuration  suitable  for  relatively 
large  samples,  with  a  10x/0.30  detection  objective  lens  the  SPIM 
has  a  field  of  view  of  660  pm.  A  Gaussian  light  sheet  can  be  formed 
with  a  thickness  that  varies  between  6  pm  and  9  pm  across  this  dis¬ 
tance  by  using  a  cylindrical  lens  of  focal  length  100  mm  and  aper¬ 
ture  6  mm.  In  addition  to  providing  optical  sectioning,  this 
considerably  improves  the  axial  resolution  of  the  system,  which 
would  be  ~14pm  with  conventional  illumination. 

If  SPIM  is  to  be  used  for  higher-resolution  imaging  of  smaller 
samples,  a  100x/1.0  with  an  axial  PSF  width  of  1.08  pm  and  a  field 


FIGURE  37.3.  Light  sheet  illumination  (blue)  and  widefield 
detection  (green)  in  the  SPIM.  Illumination:  CL,  cylindrical 
lens;  NAdi  =  n  sin  t,  light  sheet  thickness;  w,  light  sheet 
width.  Detection:  OL,  objective  lens;  NAdet  =  n  sin  ad;  fov,  field 
of  view;  wd,  working  distance. 
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of  view  of  66|J.m  is  more  appropriate.  For  this  detection  lens,  a 
light  sheet  thickness  of  2  pm  yields  a  reasonably  uniform  illumi¬ 
nation  across  the  field  of  view  (thickness  variation  <70%).  In  this 
case,  although  the  light  sheet  does  not  significantly  decrease  the 
axial  extent  of  the  widefield  PSF  [as  defined  by  its  full-width  at 
half-maximum  (FWHM)],  it  can  still  contribute  to  the  image 
quality  in  a  thick  fluorescent  sample  by  providing  optical  section¬ 
ing,  and  thereby  increasing  image  contrast  (Stelzer,  1998).  When 
imaging  a  200  pm  thick  sample  in  the  SPIM  with  a  2  pm  wide  light 
sheet,  the  contrast  of  features  in  the  focus  plane  will  be  improved 
by  a  factor  of  up  to  lOOx  compared  to  imaging  with  uniform 
illumination. 

In  addition  to  the  use  of  light  sheet  illumination,  the  resolution 
in  SPIM  can  be  significantly  improved  by  multi-view  reconstruc¬ 
tion  (see  section  below  on  Processing  SPIM  Images/Multi- View 
Reconstruction;  Shaw,  1989;  Swoger  et  al.,  2003;  Huisken  et  al., 
2004).  When  this  is  done,  the  lateral  resolution  of  the  individual 
views  determines  the  overall  resolution  in  the  fused  dataset,  that 
is,  the  axial  resolution  becomes  as  good  as  the  lateral. 


APPLICATIONS 

SPIM  technology  can  be  readily  applied  to  a  wide  range  of  spec¬ 
imens,  from  living,  fully  intact  organisms  to  single  cells.  Because 
the  method  of  embedding  the  sample  in  a  low  concentration  (0.5%) 
agarose  cylinder  is  non-disruptive,  SPIM  is  well  suited  for  high- 
resolution  visualization  of  spatiotemporal  patterns  of  gene  and 
protein  expression.  To  demonstrate  this,  we  imaged  the  transgenic 
Medaka  line  Arnie,  which  expresses  green  fluorescent  protein 
(GFP)  in  somatic  and  smooth  muscles.  A  4-day-old  fixed  Arnie 
embryo  is  shown  in  Figure  37.4.  SPIM  was  capable  of  resolving 
the  internal  structures  of  the  organism  with  high  resolution  (better 


than  6  pm)  as  deep  as  500  pm  inside  the  fish,  a  penetration  depth 
that  cannot  be  reached  using  confocal  laser-scanning  microscopy. 
Figure  37.4  also  includes  a  comparison  of  widefield  and  SPIM  illu¬ 
mination.  The  optical  sectioning  capability  of  SPIM  is  apparent. 

Figure  37.5  shows  the  summary  of  an  experiment  in  which  a 
relatively  opaque  Drosophila  embryo  was  observed  over  a  period 
of  17  h.  Even  after  being  illuminated  to  generate  11,480  two- 
dimensional  (2D)  images,  the  embryo  completed  embryogenesis 
normally.  SPIM  is  ideally  suited  for  tracking  the  development  of 
intact  embryos:  internal  structures  are  clearly  identifiable  and 
traceable  and  even  subcellular  resolution  can  be  obtained  in  living 
samples  kept  in  a  biologically  relevant  environment  within  the 
organism  or  in  culture. 

PROCESSING  SELECTIVE  PLANE 
ILLUMINATION  MICROSCOPY 
IMAGES/MULTI-VIEW  RECONSTRUCTION 

One  of  the  advantages  of  SPIM  is  that  it  is  possible  to  generate 
improved  sample  reconstructions  by  taking  multiple  3D  views  of 
the  sample.  As  discussed  above,  multiple  3D  images  of  the  sample 
are  generated  from  varying  directions.  These  different  views  can 
contain  information  about  different  aspects  of  the  sample:  they  can 
cover  different  regions  of  the  sample,  either  in  physical  space  (in 
a  semi-opaque  sample  where  only  part  of  it  can  be  imaged  from  a 
given  direction)  or  in  Fourier  space  (because  of  the  anisotropy  of 
the  system  resolution).  The  goal  of  the  multi- view  reconstruction 
is  to  combine  the  best  components  of  each  view  into  a  single  3D 
representation  of  the  sample. 

The  data  processing  steps  used  for  the  fusion  of  multi-view 
SPIM  images  are  similar  to  those  described  by  Shaw  (1989),  and 
outlined  below: 


max.  intensity  projections 

lateral  dorsal-ventral 


FIGURE  37.4.  A  Medaka  embryo  imaged  with  SPIM  by  two  different  modes  of  illumination.  Lateral  (A,  C)  and  dorsal-ventral  (B,  D)  maximum  projections 
are  shown.  In  (A)  and  (B),  the  sample  was  illuminated  uniformly,  that  is,  without  the  cylindrical  lens,  as  in  a  conventional  widefield  microscope.  There  is  no 
optical  sectioning;  the  elongation  of  fluorescent  features  along  the  detection  axis  is  clearly  visible  in  (B).  In  contrast,  selective  plane  illumination  (C,  D)  pro¬ 
vided  optical  sectioning.  Both  image  stacks  were  taken  with  a  5x,  0.25  objective  lens.  The  embryo  shows  GFP  expression  in  the  eyes  and  muscles  (sample  pro¬ 
vided  by  J.  Wittbrodt). 
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FIGURE  37.5.  Time-lapse  imaging  of  Drosophila  melanogaster  embryogenesis:  56  planes  were  recorded  at  205  time  points,  creating  a  four-dimensional  (4D) 
dataset  30GB  in  size.  (A)  A  single  plane  at  six  selected  time  points.  (B)  A  corresponding  central  vertical  plane,  that  is,  one  parallel  to  the  detection  axis  z,  showing 
the  deep  penetration  of  SPIM.  Despite  the  optically  dense  structure  of  the  Drosophila  embryo,  features  are  well  resolved  in  the  sample  (see  insets)  and  the  overall 
morphogenetic  movements  during  embryonic  development  can  be  followed  (a  4D  movie  is  available  on  this  volume’s  Web  site).  The  images  were  normalized 
to  exhibit  the  same  overall  intensity,  thus  compensating  for  the  continuous  production  of  GFP-moesin.  lOx,  0.30W  objective  lens  (sample  provided  by 
F.  Jankovics). 


1.  Preprocessing  includes  cropping  of  the  region  of  interest  in 
all  three  dimensions,  rescaling  along  the  detection  axis  (to 
make  the  lateral  and  axial  voxel  dimensions  equal),  and  rotat¬ 
ing  the  images  to  a  common  orientation. 

2.  Registration  is  the  process  of  aligning  the  different  views  so 
that  features  visible  in  more  than  one  view  overlap.  The  stacks 
are  high-pass  filtered  to  reduce  background-induced  registra¬ 
tion  artifacts2  and  cross-correlated.  The  position  of  the  result¬ 
ing  correlation  peak  determines  the  translation  that  is  applied 


2  Note  that  the  high-pass,  3D,  Fourier  tranform-Gaussian  filtering  (with  the 
standard  deviation  set  to  one  half  of  the  maximum  spatial  frequency  possible 
for  this  pixel  size)  is  only  applied  for  the  purposes  of  creating  the  cross¬ 
correlations:  the  subsequent  processing  is  done  with  the  non-filtered  data  sets. 


to  register  the  preprocessed  (but  otherwise  raw,  unfiltered) 
images. 

3.  Image  fusion  is  the  final  stage,  in  which  a  single,  optimal 
image  is  produced.  The  high-resolution  features  are  extracted 
from  each  view  and  combined  to  form  a  single  data  set.  The 
data  sets  are  fused  by: 

a.  Fourier  transforming  the  individual  views  to  generate 
(complex-valued)  spatial  frequency  representations. 

b.  For  each  spatial  frequency  (i.e.,  each  voxel  in  Fourier 
space),  one  selects  the  (complex)  value  from  the  view  with 
the  largest  magnitude,  and  inserts  it  into  the  new,  fused 
dataset.  This  effectively  combines  the  images  by  selecting 
the  component  with  the  highest  information  content  at  each 
spatial  frequency. 
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c.  Inverse  Fourier  transforming  this  selected,  fused  Fourier 
transform  of  the  data  to  obtain  the  final,  fused  image. 

4.  In  addition,  if  the  single-view  PSFs  are  known,  the  fused  image 
can  also  benefit  from  the  effects  of  deconvolution.  In  such 
cases  we  use  a  multi-view  deconvolution  algorithm  similar  to 
that  described  in  Bertero  and  Boccacci  (2000),  adapted  to 
allow  processing  of  3D  microscopy  datasets. 

If  the  datasets  overlap  sufficiently  in  the  multi-view  recon¬ 
struction,  the  lateral  resolution  dominates  and  the  multi-view 


reconstruction  compensates  the  poor  axial  resolution  in  any  single 
view  with  information  from  others,  providing  a  nearly  isotropic 
resolution  as  demonstrated  with  subresolution  bead  data  in  Swoger 
and  colleagues  (2003). 

We  performed  a  multi-view  reconstruction  with  four  views 
of  a  Medaka  embryo  (see  Fig.  37.6).  Combining  these  stacks 
yielded  a  complete  view  of  the  sample,  and  the  decrease  in  image 
quality  with  penetration  depth  is  corrected  by  the  fusion  process. 
In  such  a  sample,  image  fusion  compensates  for  spatially  varying 
image  quality  caused  by  aberrations  and  scattering.  As  long  as 


FIGURE  37.6.  Volume  renderings  representing  the  result  of  fusing  four  datasets  taken  with  0°,  90°,  180°,  and  270°  orientations  of  the  sample  (same  sample  as 
in  Fig.  37.4).  (Top)  Ventral  view  of  the  fused  data.  (Bottom)  Anterior  view  of  the  single  views  and  fusion.  The  multi-view  reconstruction  combines  information 
from  the  single  views  while  preserving  the  high-resolution  parts. 
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FIGURE  37.7.  For  an  almost  transparent  sample,  a  multi-view 
reconstruction  provides  isotropic  resolution  because  imaging 
from  multiple  directions  provides  overlapping  datasets.  (A)  A 
single  dataset  of  the  autofluorescence  from  a  grain  of  paper  mul¬ 
berry  pollen  suffers  from  poor  axial  resolution.  (Seventeen 
similar  datasets  were  taken  with  10°  angular  shifts,  not  shown.) 
(B)  The  multi- view  reconstruction  incorporates  all  of  the  high- 
resolution  information  available  in  the  individual  views.  Single 
slices  parallel  to  the  detection  axis  are  shown.  Objective  lens, 
63x;  NA,  0.9  W. 


high-resolution  information  is  available  in  at  least  one  view  of  a 
given  region  of  the  sample,  the  reconstruction  algorithm  will  favor 
this  information  over  the  low-resolution  information  of  the  same 
area  found  in  other  views.  The  outcome  is  high-resolution  data 
over  a  much  larger  volume  than  in  a  single  unprocessed  stack. 
Using  this  approach,  the  whole  sample  can  be  viewed  at  high 
resolution  as  long  as  the  penetration  depth  is  at  least  half  its 
thickness. 

When  used  to  image  transparent  samples  (i.e.,  those  in  which 
scattering/absorption  are  negligible),  multi-view  SPIM  can 
provide  isotropic  resolution.  This  is  demonstrated  in  Figure  37.7, 
which  shows  single-view  and  multi-view  processed  (optical)  slices 
through  a  pollen  grain.  Unlike  in  the  Medaka  embryo  shown  in 
Figure  37.6,  in  the  individual  pollen  views  (only  one  is  shown  in 
Fig.  37.7)  the  issue  is  not  spatially  varying  resolution,  but  rather 
anisotropic  resolution.  However,  the  algorithm  presented  here  for 
fusion  of  multiple  SPIM  views  is  seen  to  compensate  for  both  of 
these  artifacts. 

While  multi-view  data  can  be  processed  to  yield  a  further 
increase  in  image  quality  and  axial  resolution  compared  to  a  single 
stack,  in  many  cases  a  single,  unprocessed,  3D  SPIM  stack  alone 
provides  sufficient  information.  This  can  be  particularly  important 
in  live  imaging  of  transient  phenomena,  where  data  acquisition 
speed  can  be  of  primary  importance. 

SUMMARY 

If  one  is  solely  interested  in  near-surface  features,  a  confocal  laser¬ 
scanning  microscope  (CLSM)  can  be  ideal  because  of  its  optical 
sectioning  and  compatibility  with  high-NA  objective  lenses. 
However,  this  absolutely  precludes  imaging  the  entirety  of  many 
thick  samples,  because  currently  available  high-NA  lenses  do  not 


have  sufficient  working  distance.  In  addition,  the  severe  drop  in 
signal  intensity  caused  by  aberrations  and  scattering  with  increas¬ 
ing  depth  in  heterogeneous  specimens  is  more  pronounced  when 
working  with  a  high  NA  lens.  Because  it  uses  planar  illumination, 
SPIM  does  not  suffer  from  scattering  as  much  as  the  confocal 
microscope  does:  scattering  of  the  illumination  within  the  plane  of 
the  light  sheet  does  not  deteriorate  the  image  at  all,  although  scat¬ 
tering  out  of  the  plane  will  widen  the  thickness  of  the  light  sheet. 

In  a  CLSM,  fluorescence  light  is  collected  by  a  photomulti¬ 
plier  during  the  short  time  the  focal  spot  rests  at  each  pixel  (0.1- 
I  Ops).  In  contrast,  the  1344  x  1024  pixel  Hamamatsu  ORCA  ER 
CCD  camera  used  in  SPIM  allows  integration  times  of  0. 1  to  1  s, 
which  has  the  result  that  the  local  illumination  intensity  can  be 
decreased,  reducing  the  likelihood  of  fluorophore  saturation.  To 
reduce  the  dose  to  the  sample  even  further,  we  have  also  used  an 
EM-CCD  and  a  result  made  using  this  camera  is  shown  in  Figures 
10.12,  10.15  and  43.5  in  other  Chapters,  this  volume). 

Like  multi-photon  microscopy,  SPIM  illumination  provides 
intrinsic  optical  sectioning  because  no  out-of-focus  light  is  gener¬ 
ated.  This  is  in  contrast  to  CLSM,  where  out-of-focus  light  is  gen¬ 
erated  but  rejected  by  the  detection  pinhole.  In  CLSM,  the  entire 
volume  of  the  sample  is  illuminated,  even  if  only  a  single  plane  is 
being  imaged  generating  a  3D  image  with  a  confocal  fluorescence 
microscope  can  therefore  result  in  excessive  photobleaching. 
Because  of  these  two  factors,  imaging  a  3D  sample  in  SPIM 
requires  a  much  smaller  total  number  of  fluorophore  excitations 
than  does  any  widefield  or  confocal  microscope.  This  is  a  partic¬ 
ular  advantage  when  imaging  thick  specimens. 

Samples  mounted  in  agarose  gel  for  SPIM  enjoy  conditions 
that  are  certainly  much  closer  to  physiological  relevance  than 
samples  flattened  on  coverslips.  The  environment  is  more  appro¬ 
priate  for  embryos,  3D  cell  cultures,  stem  cell  research,  and  many 
other  areas  of  biological  research.  Fragile  objects  can  be  carefully 
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immersed  in  the  liquid  agarose  gel3 *  and  remain  in  their  natural 
form  when  the  gel  sets. 

SPIM  is  a  3D  optical  technique  capable  of  imaging  deep  inside 
both  fixed  and  living  organisms.  Selective  illumination  of  a  single 
plane  provides  efficient  optical  sectioning  and  reduces  photodam¬ 
age  in  samples  large  or  small.  The  method  of  sample  mounting 
allows  positioning  and  rotation  to  orient  the  sample  for  optimal 
imaging  conditions.  Image  fusion  can  combine  data  from  different 
SPIM  views  to  produce  an  optimal  3D  representation  of  the  sample. 
Because  the  system  is  also  inexpensive,  compact,  fast,  optically 
stable,  and  easy  to  use,  it  is  clear  that  there  are  many  applications 
that  are  ideally  suited  to  be  investigated  with  SPIM  techniques. 
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INTRODUCTION 

Conventional  fluorescence  microscopes  are  based  on  the  excita¬ 
tion  of  exogenous  and  endogenous  fluorophores  by  ultraviolet 
(UV)  and  visible  (VIS)  radiation.  The  fluorescence  excitation 
radiation  in  non-laser  microscopes  is  provided  by  high-pressure 
50W/100W  mercury  lamps  with  typical  emission  lines  at  365  nm, 
505  nm,  436 nm,  etc.,  and  broadband  75  W  xenon  lamps.  Most 
laser-scanning  microscopes  possess  the  argon  (Ar)-ion  laser  at  364 
nm,  488  nm,  and  515nm  emission,  the  frequency-doubled  and 
-tripled  Nd:YAG  laser  at  532  nm  and  355  nm,  and  the 
helium-neon  (He-Ne)  laser  at  543  nm  and  633  nm. 

Typically,  the  radiation  power  of  these  microscopes  is  on  the 
order  of  10  to  100|iW.  This  corresponds  to  light  intensities  in  the 
range  of  kilowatts  per  square  centimeter  when  focused  to 
diffraction-limited  spots  by  objectives  with  a  numerical  aperture 
(NA)  larger  than  1.  These  light  intensities  are  not  sufficient  to 
induce  multi-photon  effects. 

In  spite  of  the  low  radiation  power,  photodestructive  effects 
occur.  In  particular,  UVA  radiation,  including  365  nm  lamp  radia¬ 
tion,  364  nm  argon  laser  radiation,  and  337  nm  laser  radiation  of  a 
nitrogen  laser  has  a  relatively  high  damage  potential  compared  to 
VIS  radiation.  Although  not  directly  absorbed  by  DNA,  UVA  radi¬ 
ation  may  also  induce  DNA  strand  breaks  (Konig  et  al.,  1996a). 

In  contrast  to  one-photon  fluorescence  microscopes,  multi¬ 
photon  microscopes  are  based  on  the  application  of  low  energy 
photons  in  the  near  infrared  (NIR)  between  700  nm  and  1200nm. 
This  spectral  range  is  also  called  the  “optical  window  of  cells  and 
tissues”  where  the  one-photon  absorption  and  scattering  coeffi¬ 
cients  of  unstained  cells  and  tissues  are  relatively  low  and  where 
the  light  penetration  depth  is  high  (Fig.  38.1).  In  the  view  of  one- 
photon  processes,  most  cells  appear  transparent  and  do  not  expe¬ 
rience  destructive  effects  due  to  the  lack  of  efficient  absorption. 
Exceptions  are  pigmented  cells  (e.g.,  cells  containing  melanin, 
hemoglobin,  or  chlorophyll)  where  linear  absorption  is  likely  to 
occur. 

In  the  case  of  non-resonant  two-photon  absorption,  NIR  radi¬ 
ation  may  induce  UVA  effects  —  including  photodamage  —  in 
particular  when  using  wavelengths  below  800  nm.  However,  the 
destructive  UV  effect  is,  in  principle,  confined  to  the  tiny  focal 
volume  where  the  nonlinear  absorption  occurs  and  does  not  affect 
the  out-of-focus  regions. 

When  four  photons  or  more  are  involved  in  the  nonlinear 
process,  ionization  of  the  target  combined  with  the  formation  of 
free  or  quasi-free  electrons  may  occur.  This  can  lead  to  optical 
breakdown  and  plasma  formation.  These  multi-photon  processes 
result  in  immediate  photodamage. 


Nonlinear  effects  such  as  non-resonant  multi-photon  absorp¬ 
tion  require  high  light  intensities  in  the  range  from  megawatts  per 
square  centimeter  up  to  terawatts  per  square  centimeter.  In  order 
to  obtain  such  intensities,  higher  light  powers  than  are  used  in  one- 
photon  microscopy  are  required.  When  using  continuous  wave 
(CW)  NIR  radiation,  powers  of  at  least  100  mW  are  necessary 
(Hiinninen  et  al.,  1994;  Konig  et  al.,  1995,  1996b).  That  means 
about  3  orders  of  magnitude  higher  power  levels  or  more  have  to 
be  applied  in  multi-photon  microscopy. 

Power  levels  in  the  range  of  lOOrnW  of  NIR  light  have  been 
used  for  optical  micromanipulation  of  living  cells  using  laser 
tweezers.  In  the  case  of  multi-photon  scanning  microscopy  for  flu¬ 
orescence  imaging,  such  trapping  effects  may  disturb  the  specimen. 

In  order  to  avoid  trapping  effects  and  to  reduce  the  mean 
power,  ultrashort  laser  pulses  with  kilowatts  of  peak  power  but  a 
mean  power  in  the  microwatt  and  milliwatt  range  are  commonly 
used  in  multi-photon  microscopy. 

Almost  all  multi-photon  laser-scanning  microscopes  are  based 
on  the  application  of  ultrashort  laser  pulses  at  high  repetition  rate. 
Because  the  multi-photon  efficiency  of  an  n-photon  process 
depends  on  the  peak  power  P  according  to  a  Pn  relation,  the  use 
of  pulsed  laser  beams  enables  efficient  fluorophore  excitation  at  a 
fast  scanning  rate. 

The  major  application  of  multi-photon  microscopes  in  life  sci¬ 
ences  involves  the  excitation  of  a  variety  of  intracellular  fluores¬ 
cent  probes  that  lack  one-photon  absorption  bands  in  the  NIR  by 
the  simultaneous  absorption  of  two  or  three  NIR  photons  (Denk  et 
al.,  1990).  Typical  two-photon  and  three-photon  absorption  cross- 
sections  of  these  fluorophores  are  about  10"48  to  1(T50  cm4  s/photon 
and  10“75  to  10_84cm6  (s/photon)2  (Xu  et  al.,  1995;  Maiti  et  al., 
1997). 

In  a  first  approximation,  the  probability  of  two-photon  absorp¬ 
tion  decreases  with  the  distance  d  from  the  focal  plane  according 
to  a  d~4  relation.  When  using  high-NA  objectives  and  appropriate 
power  levels,  two-photon  effects  can  be  limited  to  a  minute  sub- 
femtoliter  volume.  By  contrast,  microscopy  with  UV  or  VIS 
lamp/laser  light  implies  possible  destructive  effects  by  one-photon 
absorption  in  out-of-focus  illuminated  regions.  This  problem  is 
often  encountered  in  conventional  confocal  laser-scanning 
microscopy  as  well  as  in  conventional  intracellular  laser  surgery 
performed  with  pulsed  UV  and  VIS  laser  beams  (Bems,  1998; 
Greulich,  1999). 

In  particular,  when  studying  thick  3D  objects  such  as  large 
cells,  multi-cell  aggregates,  embryos,  and  tissues  by  optical  sec¬ 
tioning,  multi-photon  microscopy  with  its  tiny  excitation  volume 
can  be  the  superior  method  compared  to  one-photon  confocal  scan¬ 
ning  microscopy  with  its  large  excitation  cones  (Fig.  38.2). 
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FIGURE  38.1.  Absorption  spectra  of  the  cellular  absorbers  water,  melanin, 
and  hemoglobin.  The  range  of  low  absorption  and  low  scattering  is  from  about 
600nm  up  to  1200nm  (optical  window).  At  higher  NIR  wavelength,  linear 
absorption  of  water  increases  and  may  induce  destructive  linear  heating  effects. 


Multi-photon  microscopy  on  living  specimens  can  be  per¬ 
formed  without  phototoxic  effects  under  certain  conditions.  This 
has  been  demonstrated  by  long-term  scanning  experiments  on 
(i)  single  hamster  ovarian  cells  with  a  peak  intensity  of  about 
200  GW/cm2,  where  laser  exposure  even  for  hours  had  no  impact 
on  cellular  reproduction  and  vitality  (Konig,  2000)  as  well  as  (ii) 
on  living  hamster  embryos.  The  embryos  were  exposed  for  24 h 
with  NIR  femtosecond  laser  pulses  of  a  multi-photon  fluorescence 
microscope  without  impact  on  embryo  development  in  contrast  to 
control  studies  performed  with  a  conventional  one-photon  VIS 
laser  scanning  microscope  (Squirrel  et  al.,  1999). 

Nevertheless,  NIR  multi-photon  microscopes  can  potentially 
induce  undesirable  destructive  multi-photon  effects  by  the  high 
light  intensity  within  the  focal  volume  (Konig  et  al.,  2000;  Diaspro 
and  Robello,  1999).  Indeed,  an  additional  optical  window  exists 
for  the  safe,  non-destructive  observations  of  the  specimens  of 
interest.  It  is  mainly  defined  by  light  intensity  thresholds  but  also 
has  a  dependence  on  NIR  wavelength.  Outside  this  window,  cells 
may  undergo  either  (i)  a  photothermal  damage  in  the  case  of  pig¬ 
mented  cells,  (ii)  a  slow  destructive  effect  related  to  multi-photon 


FIGURE  38.2.  Scheme  of  one-photon  excitation  versus  multi-photon  excitation  and  corresponding  scanning  microscopes.  Optical  sectioning  based  on  multi¬ 
photon  excitation  of  fluorophores  does  not  require  spatial  pinholes  and  can  avoid  out-of-focus  photodamage/bleaching. 
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FIGURE  38.3.  Two-photon  effects  with  intense  near-infrared  radiation  can 
induce  visible  fluorescence,  second  harmonic  generation,  and  photochemical 
reactions.  Photochemical  reactions  such  as  photo-oxidation  processes  may  lead 
to  slow  photodestructive  effects.  When  using  laser  intensities  in  the  range  of 
terawatts/cm2,  multi-photon  ionization  and  plasma  formation  may  occur  that 
can  result  in  immediate  cell  damage.  Maria  Goppert-Mayer  predicted  two- 
photon  effects  in  her  1930  Ph.D.  thesis. 


naturally  occurring  endogenous  absorbers  in  cells  and  tissues 
which  can  be  excited  with  NIR  radiation  via  two-  or  three-photon 
excitation  processes. 

For  example,  amino  acids  with  a  one-photon  absorption 
maximum  below  300  nm  can  be  excited  via  a  three-photon  process 
with  NIR  photons.  The  reduced  coenzymes  NAD(P)H  absorb 
intense  NIR  radiation  in  the  700  nm  to  800  nm  range  by  two-photon 
excitation.  It  is  known  from  one-photon  studies  that  excitation 
of  amino  acids  as  well  as  of  NAD(P)H  can  induce  irreversible 
DNA  damage  (Cunningham  et  al. ,  1985;  Tyrell  and  Keyse,  1990; 
Bertling  et  al.,  1996).  Excitation  of  flavins,  which  can  also  be 
excited  by  a  two-photon  process,  may  also  cause  cell  damage  by 
the  photo-induced  production  of  H202  and  its  metabolites,  which 
are  hydroxyl  radicals  (Hockberger  et  al.,  1999). 

In  addition,  the  excitation  of  cellular  porphyrins  leads  to  the 
formation  of  ROS.  Singlet  oxygen  is  produced  by  a  type  II  photo¬ 
oxidation  process  (photodynamic  reaction).  Metal-free  porphyrins, 
such  as  coproporphyrin  and  protoporphyrin,  are  such  efficient  ROS 
producers  that  they  are  termed  photosensitizers.  As  known  from 
photodynamic  porphyrin  therapy,  ROS  formation  in  mitochondria 
often  leads  to  apoptosis,  whereas  ROS  formation  in  lysosomes  or 
other  organelles  may  induce  either  necrosis  or  apoptosis  (Doiron 
and  Gomer,  1985;  Moor,  2000).  Photodynamic  reactions  are  also 
the  major  source  of  phototoxic  response  when  using  fluorescent 
markers  or  other  exogenous  absorbers. 


excitation  of  cellular  absorbers  and  subsequent  cytotoxic 
photochemical  reactions,  or  (iii)  an  instantaneous  destructive  opto¬ 
mechanical  effect  by  optical  breakdown  phenomena  and  plasma 
formation  due  to  multi-photon-mediated  ionization  of  molecules 
(Fig.  38.3). 

Interestingly,  the  highly  localized  destructive  multi-photon 
effects  can  be  used  to  realize  precise,  3D  nanoprocessing  of  bio¬ 
logical  structures  such  as  cell  membranes,  intracellular  organelles, 
and  human  chromosomes,  as  well  as  corneal  and  brain  tissues. 
Therefore,  multi-photon  microscopes  can  also  be  used  to  knock 
out  single  mitochondria,  to  realize  efficient  targeted  transfection 
(Tirlapur  and  Konig,  2002),  and  chromosome  dissection  as  well  as 
to  perform  intrastromal  nanosurgery  in  eyes.  Multi-photon- 
induced  minimum  cut  sizes  of  85  nm  full-width  at  half-maximum 
(FWHM)  have  been  realized  in  the  human  chromosome  1  using 
80  MHz  NIR  nanojoule  femtosecond  laser  pulses. 

This  chapter  provides  a  review  (i)  of  potential  photo-induced 
effects  that  may  cause  cellular  damage  and  (ii)  the  use  of  destruc¬ 
tive  multi-photon  effects  with  nanojoule  and  sub-nanojoule  fem¬ 
tosecond  laser  pulses  for  nanosurgery  and  gene  technology. 


Laser  Exposure  Parameters 

In  most  experiments  we  have  used  compact  solid-state  80  MHz 
titanium :  sapphire  lasers  (Vitesse,  Coherent;  MaiTai,  Spectra 
Physics)  as  turn-key  laser  sources;  however,  a  tunable  Ar-ion  laser- 
pumped  Mira  system  (Coherent)  was  employed  for  some  of  this 
work.  The  laser  was  coupled  to  an  adapter  module  (JenLab  GmbH, 
Jena,  Germany)  which  consisted  of  a  1 : 5  beam  expander,  a  step- 
motor  driven  beam  attenuator,  a  fast  photodiode  for  power  mea¬ 
surements,  a  trigger  pulse  supply,  a  laser  control  unit,  and  a  shutter. 
The  expanded  beam  at  tunable  power  was  introduced  into  a  mod¬ 
ified  laser-scanning  microscope  (LSM  410,  Carl  Zeiss  Jena  GmbH, 
Jena,  Germany)  with  baseport  detector  (Konig,  2000)  as  well  as 
into  a  compact  scanning  microscope  where  the  galvoscanners  were 
attached  to  the  side  port  of  an  inverted  Zeiss  Axiovert  microscope 
(TauMap,  JenLab  GmbH)  (Ulrich  et  al.,  2004).  The  beam  was 
focused  to  a  diffraction-limited  spot  by  conventional  high-NA 
objectives.  The  typical  NIR  transmission  of  the  objective  was 
found  to  be  50%,  whereas  the  overall  transmission  was  typically 


PHOTOCHEMICAL  DAMAGE  IN 
MULTI-PHOTON  MICROSCOPES 

Absorbers  and  Targets  in  Biological  Specimens 

Photochemical  damage  can  be  induced  by  the  nonlinear  excitation 
of  endogenous  and  exogenous  absorbers  followed  by  phototoxic 
reactions.  These  reactions  include  the  formation  of  reactive  oxygen 
species  (ROS),  which  result  in  oxidative  stress.  A  wide  variety  of 
endogenous  cellular  absorbers  can  be  excited  with  multi-photon 
microscopes.  Some  of  them  are  fluorescent  and  can  be  imaged 
by  autofluorescence  detection.  Table  38.1  represents  important 


TABLE  38.1.  Spectral  Characteristics  (One-Photon  Maxima) 
of  Important  Endogenous  Absorbers 


Absorber 

Absorption  Maximum 
(nm) 

Fluorescence  Maximum 
(nm) 

Water 

1500,  1900,  2900 

_ 

Hemoglobin 

410,  540,  580,  760 

— 

Tryptophan 

220,  280-290 

320-350 

NAD(P)H 

340 

450-470 

Flavins 

370,  450 

530 

Melanin 

<300 

575 

PPIX 

405,  505,  540,  575,  630 

635 
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FIGURE  38.4.  Cell  chamber  for  high-resolution  microscopy  with  two  thin 
170pm  thick  glass  windows  with  grid.  Cells  can  grow  as  monolayers  on 
the  windows.  Media  and  dyes  can  be  added  by  injection  through  the  gas- 
permeable  silicone  gasket. 


determined  using  a  sandwich  system  (Fig.  38.5;  Konig  et  al., 
1997a). 

Optical  dispersion  results  in  pulse  broadening  during  trans¬ 
mission  through  microscope  optics.  Typically,  the  pulse  width  at 
the  sample  was  about  150fs  to  250fs  (Konig,  2000).  At  lOmW 
mean  power,  the  peak  power  and  the  peak  intensities  were  0.8  kW 
and  1.2  x  1012W/cm2  (1.2TW/cm2),  respectively,  assuming  a 
FWHM  beam  size  of  X/2  NA  =  310nm.  A  typical  beam  dwell  time 
(time  of  exposure  per  pixel)  during  one  scan  was  about  30  ps, 
which  resulted  in  a  frame  rate  of  8  s/frame.  At  zoom  setting  2,  512 
X  512  pixels  covered  an  area  of  160  pm  X  160  pm.  For  cell  damage 
studies,  typically  one  or  more  cells  of  interest  were  scanned  10 
times  in  the  same  focal  plane. 


Evidence  for  Near  Infrared-Induced  Reactive 
Oxygen  Species  Formation 

In  order  to  know  if  multi-photon  NIR  laser-scanning  microscopy 
with  80MFIz  laser  radiation  above  certain  intensity  levels  is  able 
to  induce  ROS  formation  and  DNA  breaks  in  living  cells,  we  have 
performed  extensive  photodamage  studies  on  PTK2  cells. 

Using  170fs  laser  pulses  at  800  nm,  we  found  that  a  mean 
power  of  >7  mW  was  sufficient  to  evoke  ROS  in  PTK2  cells  within 
a  scanned  region  of  interest.  By  contrast,  no  ROS  formation  was 
detected  in  the  non-irradiated  adjacent  cells.  As  a  ROS  indicator, 
we  used  the  fluorescent  probe  Jenchrom  px  blue  (JenLab  GmbFl). 
The  punctate  fluorescence  of  the  Jenchrom  reaction  product  was 
found  to  be  exclusively  confined  to  the  irradiated  cells  (Tirlapur  et 
al.,  2001). 

In  addition,  we  have  used  the  membrane-permeable  dye  dihy¬ 
drofluorescein  (DFLF)  for  in  situ  detection  of  ROS  (Hockberger  et 
al.,  1999)  in  PTK2  cells.  We  exposed  one  half  of  the  DHF-loaded 
PTK2  cells  to  170 fs  pulses  at  800 nm  using  a  mean  power  of  2,  5, 
7,  10,  12,  or  15mW.  There  was  virtually  no  DF1F  fluorescence  in 
cells  exposed  to  mean  laser  power  of  <5mW.  However,  diffuse 
DHF  fluorescence  was  exclusively  present  in  cells  exposed  to  a 


in  the  range  of  30%.  When  studying  living  cells,  a  closed 
microchamber  made  of  two  170  pm  thick  glass  windows  (one 
with  an  etched  grid)  was  used  (MiniCeM-grid,  JenLab  GmbH) 
(Fig.  38.4). 

We  used  three  illumination  modes:  (i)  scanning  a  512  x  512 
pixel  area,  (ii)  scanning  a  defined  region  of  interest  (ROI),  or  (iii) 
single  point  exposure,  where  the  beam  is  parked  at  pixels  of  inter¬ 
est  within  the  living  cell. 

The  high  sensitivity  of  the  baseport  photomultiplier  enabled 
the  detection  of  two-photon  excited  intracellular  fluorophores  with 
high  fluorescence  quantum  yield  (e.g.,  DAPI  and  Hoechst)  using 
a  low  mean  power  at  the  sample  of  from  25  pW  to  1  mW  at  a  frame 
rate  of  1Hz  (512  x  512  pixels).  More  importantly,  detection  of 
endogenous  fluorophores,  such  as  fluorescent  coenzymes,  was  pos¬ 
sible  with  a  mean  excitation  power  of  less  than  2  mW  at  the  spec¬ 
imen  at  appropriate  NIR  wavelengths. 

The  laser  power  after  transmission  through  the  whole  micro¬ 
scope  was  measured  in  air  using  a  Fieldmaster  power  meter  with 
the  two  detector  heads  LM-2  and  LM-3.  In  order  to  correct  for  the 
true  photon  flux  through  oil  and  the  chamber  window  into  the 
medium  as  well  as  for  the  limited  detection  angle  of  the  power 
detector  heads,  correction  values  of  1.1  and  2.0  had  been 


laser  light  in 


FIGURE  38.5.  Diagram  of  the  system  used  to  measure  the  beam  power  actu¬ 
ally  reaching  the  specimen. 
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higher  mean  laser  powers  and  was  very  intense  in  cells  exposed  to 
a  mean  laser  power  of  15  mW. 

Evidence  for  Near  Infrared-Induced 
DNA  Strand  Breaks 

Shafirovich  and  colleagues  (1999)  have  demonstrated  in  vitro  that 
excitation  of  pBR322  supercoiled  plasmid  DNA  with  intense 
810nm  NIR  femtosecond  laser  pulses  having  a  pulse  width  of 
140  fs,  a  pulse  repetition  rate  of  76  MHz  and  an  average  intensity  of 
1.2mW/cm2  (GW/cm2  peak  intensities)  resulted  in  single  strand 
breaks  as  a  result  of  a  simultaneous  multi-photon  absorption  process. 

Using  the  well-established  TdT-mediated  dUTP-nick  end 
labeling  (TUNEL)  assay  (Gavrieli  et  al.,  1992;  Negri  et  al.,  1997), 
we  have  undertaken  a  more  detailed  in  situ  analysis  of  the  effect 
of  800  nm  NIR  femtosecond  laser  pulses  at  different  mean  laser 
powers  on  DNA  in  unlabelled  PTK2  cells.  Cells  were  irradiated 
with  the  800nm  (170fs  pulse  duration)  NIR  scanning  beam  10 
times  with  a  beam  dwell  time  of  ~60  or  ~120|ds  per  pixel  that  cor¬ 
responded  to  16  or  32s  per  frame  of  512  x  512  pixels  (at  zoom 
setting  2)  using  mean  powers  of  2,  5,  10,  and  20mW. 

Neither  control  PTK2  cells  (not  exposed  to  the  800  nm  laser 
beam)  nor  those  exposed  to  2  to  3  mW  mean  power  showed  any 
TUNEL-positive  green  fluorescence  in  their  nuclei.  This  indicates 
that  lower  mean  laser  powers  do  not  induce  any  DNA  strand  breaks 
in  situ  and  hence  are  not  deleterious. 

However,  most  of  the  PTK2  cells  exposed  to  a  mean  power  of 
5mW  demonstrated  TUNEL-positive  green  fluorescence,  mostly 
in  the  nuclear  area.  When  the  mean  laser  power  was  increased  to 
10  or  15  mW,  almost  all  of  the  cells  had  TUNEL-positive  nuclei. 
When  cells  were  exposed  to  20  mW,  green  fluorescence  was  seen 
in  all  nuclei  (Fig.  38.6). 

The  pattern  of  this  green  fluorescence,  indicative  of  DNA 
strand  breaks  in  cells  exposed  to  high  power  NIR,  is  strikingly 
similar  to  that  reported  for  apoptosis  in  other  mammalian  cells 
(Negri  et  al. ,  1997). 

Our  in  vivo  findings  of  NIR  laser-induced  DNA  fragmentation 
is  consistent  and  corroborates  those  of  Shafirovich  and  colleagues 
(1999).  Nuclear  events  during  programmed  cell  death  (apoptosis) 
are  known  to  begin  with  the  changes  in  the  chromatin  in  the 
nuclear  periphery  (Tirlapur  et  al.,  2000).  Because  the  structural 
changes  and  DNA  breaks  in  the  nuclei  of  NIR-exposed  cells  seem 
quite  similar  to  those  occurring  during  apoptosis  in  different  cell 
types  (Negri  et  al.,  1997;  Bedner  et  al.,  2000),  it  is  therefore  con¬ 
ceivable  that  NIR  laser  irradiation  of  PTK2  cells  may  under  certain 
exposure  conditions  lead  to  apoptosis-like  cell  death. 

Photodynamic-Induced  Effects 

We  studied  the  effect  of  two-photon  excitation  of  porphyrins  and 
their  subsequent  interaction  with  molecular  oxygen.  Light-excited 
porphyrin  photosensitizers  react  with  oxygen  by  a  type  II  photo¬ 
oxidation  process  where  energy  transfer  occurs  from  the 
metastable  triplet  state  to  molecular  oxygen.  This  results  in  the  for¬ 
mation  of  reactive  singlet  oxygen  which  induces  cytotoxic  effects. 

In  particular,  we  studied  the  effect  of  780nm-excited  proto¬ 
porphyrin  IX  that  was  synthesized  in  the  mitochondria  of  Chinese 
hamster  ovary  (CHO)  cells  after  external  administration  of  the  por¬ 
phyrin  precursor  aminolevulinic  acid  (ALA,  1.5mg/mL).  Four 
hours  after  incubation,  cells  were  exposed  to  170fs  pulses  at  a 
mean  power  of  2mW  and  a  frame  rate  of  0.0625s_1  (16s/frame). 
This  corresponds  to  a  typical  exposure  time  of  =400  ms  per  scan 


FIGURE  38.6.  Evidence  of  DNA  damage  in  PTK2  cells  exposed  to  intense 
NIR  femtosecond  laser  radiation.  The  green-colored  fluorescein  fluorescence 
images  indicate  TUNEL-positive  and  TUNEL-negative  nuclei  in  dependence 
on  mean  laser  power. 


on  a  single  cell.  Live/dead  assays  revealed  lethal  effects  and  fading 
of  the  red  PP  IX  fluorescence  in  most  of  the  cells  after  100  scans 
in  the  same  focal  plane.  Control  cells  without  ALA  could  be 
scanned  more  than  700  times  (up  to  3  h)  without  affecting  vitality 
(Konig  et  al.,  1999b)  (Fig.  38.7). 

Nonlinear  excitation  of  porphyrins  can  be  used  to  induce 
desired  photodynamic  reactions  in  tumor  tissue.  The  major  advan¬ 
tages  of  NIR  porphyrin  photodynamic  treatment  are  the  excitation 
of  higher  order  singlet  states  (Soret  band  excitation)  and  the  high 
light  penetration  depth.  Two-photon  excitation  can  also  be  used  to 
evoke  type  I  photo-oxidation  processes  (charge  transfer)  (Fisher 
et  al.,  1997). 
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FIGURE  38.7.  Impact  of  exogenous  absorbers  on  reproduction  of  laser- 
exposed  CHO  cells.  Two-photon  induced  photodynamic  reactions  lead  to  cell 
damage. 


PHOTOTHERMAL  DAMAGE 

As  we  lack  detailed  knowledge  of  either  the  precise  intracellular 
concentration,  location,  or  NIR-absorption  of  the  one-photon  or 
multi-photon  endogenous  absorbers  found  in  cells,  we  had  to 
assume  that  water,  with  a  typical  absorption  coefficient  of  less  than 
0.1  cm-1,  is  the  most  important  absorber  in  the  optical  window  of 
non-pigmented  cells  and  tissues.  However,  our  studies  using 
CHO  cells  exposed  to  femtosecond  and  CW  lasers  showed  that 
the  cell  damage  at  920  nm  was  less  pronounced  than  that  caused 
by  780  nm  radiation  even  though  water  absorbs  less  at  780  nm. 
It  therefore  seems  likely  that  the  photodamage  caused  by 
intense  femtosecond  laser  radiation  is  not  based  on  linear  water 
heating. 

We  measured  a  warming  of  only  l°K/100mW  in  CHO  cells 
exposed  to  CW  1064nm  microbeams  at  mW/cm2  intensity  and 
GJ/cm2  fluence  (Liu  et  al.,  1995).  The  temperature  measurements 
were  performed  by  microspectrofluorometry.  CHO  cells  were 
labeled  with  the  thermosensitive,  fluorescent  membrane  stain 
Laurdan,  which  manifests  a  red  shift  of  the  fluorescence  maximum 
with  increasing  temperature.  Calibration  enabled  the  determina¬ 
tion  of  a  mean  intracellular  temperature  with  an  accuracy  of 
about  0.1  °K.  This  is  in  line  with  temperature  calculations  for 
non-pigmented  cells  exposed  to  femtosecond  and  picosecond 
pulses,  which  predict  only  a  slight,  non-toxic  increase  in 
temperature  (0.1°K/100mW  at  850nm)  (Schonle  and  Hell, 
1998). 

Nevertheless,  pigmented  cells  such  as  erythrocytes  and  those 
containing  melanin  may  undergo  photothermal  destruction.  In  the 
optical  window  of  cells  and  tissues,  the  one-photon  absorption 
coefficients  of  these  pigments  are  much  higher  than  that  of  water. 
Whereas  the  absorption  of  melanin  decreases  with  increasing 
wavelength,  the  pigment  hemoglobin  has  an  absorption  maximum 
in  this  spectral  region.  Reduced  hemoglobin  possess  a  760  nm 
absorption  band  with  a  one-photon  molecular  absorption  co¬ 
efficient  of  about  lmM^'cm”1.  Our  studies  have  shown  that 
human  erythrocytes  (a  ~  30cm_1)  exposed  to  780nm  CW  micro¬ 
irradiation  at  room  temperature  undergo  hemolysis  within 
120  s  when  the  power  exceeds  the  value  of  60  mW.  A  heating  rate 
of  about  60°K/100mW  can  be  estimated  from  these  experimental 


data.  The  heating  rate  decreased  at  wavelengths  longer  than 
760  nm. 

Interestingly,  the  photoinduced  hemolysis  studies  revealed  that 
femtosecond  laser  pulses  at  high  repetition  rate  cannot  simply  be 
considered  as  quasi-CW  beams.  Damage  may  occur  at  a  much 
lower  mean  power  of  pulsed  radiation  at  high  repetition  frequency 
compared  to  power  values  of  CW  radiation.  Using  130fs  pulses 
(76MHz)  at  the  same  wavelength,  a  mean  power  of  less  than 
lOmW  was  found  to  be  sufficient  to  induce  hemolysis  (Konig 
et  al.,  1996c). 


DAMAGE  BY  OPTICAL  BREAKDOWN 

The  high  transient  powers,  in  particular  in  the  kilowatt  range,  com¬ 
bined  with  submicron  exposure  spots  result  in  enormous  light 
intensities  in  the  terawatt  per  square  centimeter  range.  These  can 
cause  optical  breakdown  phenomena  resulting  in  plasma  forma¬ 
tion  and  destructive  thermomechanical  effects.  Optical  breakdown 
induces  immediate  cell  damage. 

The  threshold  for  optical  breakdown  decreases  with  the  pulse 
width.  In  water,  the  threshold  for  lOOfs  pulses  was  found  to  be  2 
orders  lower  than  for  6ns  pulses  (Vogel  et  al.,  1999).  The  primary 
cause  for  optical  breakdown  in  the  case  of  femtosecond  pulses  is 
efficient  multi-photon  absorption.  Inverse  Bremsstrahlung  and  col¬ 
lision  processes  are  not  dominant. 

The  onset  of  optical  breakdown  depends  on  the  material.  For 
some  materials,  femtosecond  laser  pulses  at  light  intensities  of 
about  0. 1  TW/cm2  are  sufficient  to  induce  optical  breakdown. 
Optical  breakdown  and  plasma  formation  is  often  accompanied  by 
the  occurrence  of  a  nearly  white  luminescence  and  a  complex  sub¬ 
nanosecond  emission  decay  (Konig  et  al.,  1996d). 

We  have  observed  instantaneous  white  light  luminescence  and 
immediate  damage  of  intracellular  structures  in  CHO  cells  during 
scanning  with  76  MHz,  170fs  pulses  at  mean  powers  higher  than 
lOmW.  Interestingly,  the  optical  breakdown  occurred  preferen¬ 
tially  in  the  mitochondrial  region  and  did  not  start  in  either  the 
extracellular  medium  or  the  nucleus.  Evaluation  with  the  fluores¬ 
cent  live/dead  fluorophore  kit  (calcein  and  ethidium  homodimer 
from  Molecular  Probes,  Inc.,  Eugene,  OR)  confirmed  immediate 
death  in  scanned  cells  with  extended  plasma  formation.  However, 
when  the  phenomenon  of  optical  breakdown  was  highly  confined 
to  a  tiny  perinuclear  region,  the  cell  often  appeared  to  survive.  At 
even  higher  laser  powers,  shockwave-induced  morphological 
damage  was  manifested  by  total  cell  fragmentation  (Konig  et  al., 
1996d). 


MODIFICATIONS  OF  ULTRASTRUCTURE 

Our  ultrastructural  studies  on  CHO  cells  reveal  that  mitochondria 
are  the  major  targets  of  NIR  irradiation.  Femtosecond  laser  expo¬ 
sure  induced  swollen  mitochondria  accompanied  by  loss  of  chris- 
tae  and  the  formation  of  electron  dense  bodies  in  the  mitochondrial 
matrix  (Oehring  et  al.,  2000).  In  addition,  the  lumen  of  the  endo- 
plasmatic  reticulum  was  enlarged.  For  a  mean  laser  power  of 
more  than  12mW  at  800  nm,  the  complete  destruction  of  mito¬ 
chondria  and  their  transformation  to  electron  dense  structures  was 
observed  (Fig.  38.8).  At  higher  power  levels,  the  outer  membranes 
ruptured. 
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FIGURE  38.8.  Electron  microscopy  reveals  significant  damage  to  mitochondria  (left)  as  the  major  target  of  multi-photon  microscopy.  (Right)  Control  image  of 
non-exposed  cell. 


INFLUENCE  OF  ULTRASHORT  NEAR  INFRARED 
PULSES  ON  REPRODUCTIVE  BEHAVIOR 

Squirrel  and  colleagues  (1999)  intermittently  exposed  Chinese 
hamster  embryos  for  24  h  with  femtosecond  1047  nm  laser  pulses 
of  gigawatt  per  square  centimeter  peak  intensity  without  impact  on 
embryo  development,  in  contrast  to  visible  light  exposure  which 
the  embryos  did  not  survive.  We  have  studied  the  influence  of 
intense  NIR  exposure  on  the  reproductive  behavior  of  more  than 
3000  CHO  cells.  For  that  purpose,  adherent  CHO  cells  were  main¬ 
tained  in  a  sterile  cell  chamber  and  exposed  through  the  glass 
window.  Following  laser  scanning,  the  cell  division  of  the  exposed 
cell  as  well  as  of  its  daughter  cells  was  monitored  up  to  3  days.  If 
the  cell  was  not  affected  by  the  NIR  light,  division  occurred  every 
12 h  and  the  cell  produced  a  clone  of  up  to  64  cells  within  3  days. 

We  found  that  CHO  cells  can  be  scanned  with  an  800  nm 
80  MHz  femtosecond  laser  beam  at  2mW  mean  power  for  hours 
without  damage  to  the  exposed  cells  and  their  derivatives.  In  par¬ 
ticular  we  irradiated  single  CHO  cells  for  a  time  period  of  700 
scans  and  3h,  respectively  (16  s/frame).  However,  at  higher  mean 
powers  and  intensities,  a  significant  decrease  in  cloning  efficiency 
occurred.  In  order  to  study  the  dependence  on  power,  wavelength, 
and  pulse  width  we  used  tunable  laser  beams  which  passed  pulse 
stretching  units  based  on  prisms  pairs  and  grating  pairs.  Typically, 
single  CHO  cells  were  scanned  at  microsecond  pixel  dwell  times, 
10  times  in  the  same  focal  plane  (Konig  et  al.,  1997b,  1999a). 

We  found  less  damage  at  wavelengths  in  the  800  nm  to  900  nm 
range  compared  to  the  700 nm  to  800 nm  range.  Interestingly,  at 
the  same  average  excitation  power  photodamage  was  found  to  be 
more  pronounced  in  the  case  of  femtosecond  laser  pulses  com¬ 
pared  to  picosecond  laser  pulses  (1-5  ps,  same  energy  per  pulse). 
At  780 nm,  impaired  cell  division  was  found  for  50%  of  the  cells 
at  1 1  mW  mean  power  (P50)  in  the  case  of  2  ps  pulses  and  at  3  mW 
for  240  fs  pulses.  CW  scanning  beams  did  not  impair  cellular 
reproduction  up  to  35  mW  power.  Using  mean  laser  powers  of 
<2mW,  CHO  cells  could  be  scanned  with  picosecond  and 
femtosecond  pulses  for  hours  without  impact  on  reproduction.  This 


implies  that  below  certain  power  (intensity)  thresholds,  “safe” 
multi-photon  microscopy  is  possible. 

When  the  P50  value  is  plotted  against  the  square  root  of  the 
pulse  broadening,  a  nearly  linear  function  is  obtained  (Konig  et  al., 
1999a).  This  indicates  that  the  photodamage  process  is  likely  based 
on  a  two-photon  excitation  process  rather  than  on  one-photon  or 
three-photon  events.  This  relationship  was  confirmed  by  Koester 
and  colleagues  (1999)  based  on  calcium  measurements  in  neocor- 
tical  neurons  in  rat  brain  slices. 

In  order  to  compare  photodamage  in  NIR  multi-photon  micro¬ 
scopes  versus  UV  one-photon  microscopes,  the  same  cloning  assay 
was  performed  on  CHO  cells  using  a  364  nm  beam  from  a  con¬ 
ventional  Ar+  laser  scanning  microscope.  The  50%  cloning  effi¬ 
ciency  was  obtained  at  the  low  power  of  4|iW.  Therefore, 
UV-induced  damage  occurs  at  about  3  orders  of  magnitude  lower 
power  (4|TW)  compared  to  the  mean  values  of  NIR  femtosecond 
laser  pulses  (2-8  mW,  750nm-800nm). 


NANOSURGERY 

As  pointed  out  in  the  section  on  photodamage  by  optical  break¬ 
down  phenomena,  the  cell  can  survive  if  the  damage  induced  by 
the  high-intensity  beam  is  confined  to  a  minute  subcellular  volume. 
Exposure  of  a  small  region  of  interest  including  single  spot  expo¬ 
sure  may  therefore  be  useful  for  intracellular  processing  of  bio¬ 
logical  material. 

We  increased  the  mean  power  in  femtosecond  laser  microscopes 
with  high  numerical  aperture  objectives  to  values  of  30  mW  to 
50  mW  and  light  intensities  of  terawatts  per  square  centimeter, 
respectively,  to  knockout  intracellular  material,  to  perform  nano¬ 
dissections  (Konig  et  al.,  1999c;  Konig,  2000),  and  to  “drill”  holes. 

By  fine-tuning  the  laser  power,  only  the  central  part  of  the 
beam  spot  provided  sufficient  intensity  for  material  removal. 
Therefore,  we  were  able  to  drill  holes  and  to  cut  structures  with  a 
precision  below  the  diffraction  limit,  that  is,  the  cut  size  was  below 
the  size  of  the  submicron  beam  spot. 
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FIGURE  38.9.  Nanoprocessing  of  human  chromosomes  with 
80  MHz  NIR  femtosecond  laser  pulses  at  800  nm. 


For  example,  we  used  the  multi-photon-induced  ablation 
process  to  cut  human  chromosomes.  Measurements  with  the 
atomic  force  scanning  microscope  reveal  chromosome  dissection 
with  a  FWHM  cut  size  below  200  nm  under  conditions  where  the 
calculated  Abbe  diameter  of  the  beam  was  -600  nm.  In  addition, 
partial  removal  (incisions)  of  chromosome  material  with  a  preci¬ 
sion  of  llOnm  and  a  FWHM  of  85  nm  was  observed,  which  is 
likely  the  smallest  laser  cut  ever  performed  in  biological  material. 
Figure  38.9  demonstrates  examples  of  hole  drilling  and  line  cutting 
in  human  chromosomes. 

Multi-photon  laser  microscopes  can  therefore  be  used  as  a  non- 
contact  nanoscalpel  for  surgery  inside  the  cell,  inside  the  cell 
nucleus,  or  even  inside  an  organelle  without  affecting  other  cellu¬ 
lar  compartments.  Indeed,  we  were  able  to  cut  chromosomes 
within  a  living  PTK  cell  without  loss  of  vitality.  The  cells  finished 
cell  division  after  laser  surgery  (Fig.  38.10)  (Konig  et  al.,  2005). 


CONCLUSION 

Multi-photon  microscopes  can  be  used  in  live-cell  fluorescence 
microscopy  at  moderate  laser  powers  apparently  without  damag¬ 
ing  the  target.  Repeated  scanning  of  the  biological  object  of  inves¬ 
tigation  over  extended  time  periods  is  possible  without  affecting 
reproductive  behavior  or  producing  lethal  effects. 

Above  certain  intensity  thresholds,  which  depend  on  laser 
wavelength  and  pulse  width,  apoptosis-like  gradual  cell  damage 
by  two-photon  photochemistry  may  occur.  When  studying  pig¬ 
mented  cells  and  tissues,  such  as  erythrocytes  or  skin  with  a  high 
amount  of  melanin,  photothermal  damage  has  to  be  considered. 
Immediate  damage  occurs  at  high  light  intensities  in  the  range  of 
a  terawatt  per  square  centimeter  by  optical  breakdown  phenom¬ 
ena.  Plasma-induced  ablation  can  be  used  for  intracellular 
nanosurgery.  Femtosecond  lasers  at  high  repetition  rate  and  low 
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pulse  energies  of  0.1-10nJ  provide  novel  tools  for  laser  nanomed¬ 
icine  (Konig  et  al. ,  2005). 

Meanwhile,  the  first  femtosecond  clinical  laser  imaging 
device,  Dermalnspect  (JenLab  GmbH)  is  in  clinical  use.  The 
system  is  based  on  a  tunable  80/90  MHz  Ti:  sapphire  laser  and 
operates  in  the  safe  optical  window  with  low  <10  mW  laser  power 
on  the  human  skin  for  cancer  diagnosis  and  in  situ  screening  of 
pharmaceutical  and  cosmetic  compounds  (Konig  and  Riemann, 
2003;  Konig  et  al. ,  2006). 
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INTRODUCTION 

Thanks  to  the  wide  variety  of  applications,  fluorescence  micro¬ 
scopy  is  now  one  of  the  most  popular  imaging  techniques  in 
biology  (Weber,  1960;  Lakowicz,  1999;  Periasamy,  2001;  Michalet 
et  al.,  2003;  Tsien,  2003;  Bastiens  and  Hell.  2004;  Taroni  and 
Valentini,  2004;  Diaspro  et  al. ,  2005).  Fluorescence  microscopy 
utilizes  fluorescently  labeled  probes  of  high  biochemical  affinity 
to  image  the  molecular  composition  and  dynamics  of  biological 
structures.  Moreover,  the  use  of  probes  that  change  their  fluo¬ 
rescence  properties  in  response  to  specific  physiological  param¬ 
eters  enables  one  to  analyze  the  physiological  state  of  cells 
or  tissues  (Birks,  1970;  Emptage,  2001;  Zhang  et  al,  2002; 
Lippincott-Schwartz  and  Patterson,  2003;  Stephens  and  Allan, 
2003).  Fluorescence  is  highly  specific  either  as  an  exogenous  label 
(e.g.,  4,,6-diamidino-2-phenylindole  (DAPI)  bound  to  DNA)  or  an 
endogenous  tracker  [(e.g.,  autofluorescence  of  NADH,  or  visible 
fluorescent  proteins  such  as  green  fluorescent  protein  (GFP)]  pro¬ 
viding  spatial  and  functional  information  through  precise  photo¬ 
physical  properties  such  as  absorption,  emission,  lifetime,  and 
anisotropy.  Furthermore,  sample  preparation  is  relatively  simple, 
allowing  non-invasive  imaging  and  three-dimensional  (3D) 
mapping  within  cells  and  tissues  to  be  achieved  by  means  of 
computational  optical  sectioning,  confocal  laser-scanning  micro¬ 
scopy  (CLSM),  and  two-photon  excitation  microscopy  (TPEM) 
(Periasamy,  2001).  In  particular,  CLSM  and  TPEM  are  two  com¬ 
paratively  recent  fluorescence  microscopy  techniques  that  have 
improved  the  quality  of  biological  images  (Wilson  and  Sheppard, 
1984;  Denk  et  al.,  1990;  Pawley,  1995a;  Diaspro,  2002,  2004; 
Matsumoto,  2002;  Amos  and  White,  2003). 

The  main  value  of  both  CLSM  and  TPEM  is  their  ability  to 
produce  optical  section  images  through  very  thick  biological  spec¬ 
imens,  such  as  embryos  or  tissue  slices.  As  these  images  exclude 
optical  information  from  out-of-focus  planes,  they  can  be  com¬ 
bined  to  make  a  3D  reconstruction  of  the  specimen  (Kriete,  1992; 
Bonetto  et  al.,  2004).  The  ability  of  fluorescence  microscopy  to 
detect  specific  molecules  at  very  low  concentrations  depends 
strictly  on  the  performance  of  fluorophores  used.  Fluorescent  dyes 
featuring  high  extinction  coefficients,  high  quantum  yields,  and 
large  Stokes  shift  are  now  widely  available.  However,  one  must 
remember  that  their  photostability  is  another  attribute  of  crucial 
importance.  In  CLSM,  where  the  fluorescence  signal  is  collected 
from  only  a  thin  layer  of  the  sample,  the  laser  beam  must  be  intense 


enough  to  obtain  an  adequate  signal  to  form  an  image  with  a  good 
signal-to-noise  ratio  in  a  reasonable  scan  time.  Under  these 
imaging  conditions,  the  fluorescence  emitted  is  often  observed  to 
decrease  substantially  with  time,  a  phenomenon  referred  to  as  pho¬ 
tobleaching.  This  shows  that  in  CLSM  the  photostability  problem 
can  have  a  severe  impact  on  the  image  formation  process.  Even  in 
TPEM,  despite  the  tight  confinement  of  the  excitation  volume  and 
the  use  of  infrared  light,  the  high  peak  power  of  the  laser  pulses 
may  cause  photodamage  of  the  fluorescent  probes  being  used 
(Brakenhoff  et  al.,  1996;  Mertz,  1998;  Patterson  and  Piston,  2000; 
Dittrich  and  Schwille,  2001;  Konig  and  Tirlapur,  2002).  In  general, 
the  loss  of  fluorescence  that  comes  along  with  photobleaching  has 
the  undesirable  effect  of  reducing  the  signal-to-noise  ratio  and  the 
quality  of  the  collected  images  and  data  (Pawley,  1995b). 

On  the  other  hand,  it  is  worth  noting  that  photobleaching  has 
been  exploited  since  the  mid-1970s  (Peters  et  al.,  1974;  Poo  and 
Cone,  1974;  Edidin  et  al.,  1976;  Axelrod  et  al.,  1976)  for  produc¬ 
ing  valuable  information  about  biological  system  dynamics  and  the 
advent  of  GFPs  (Tsien,  1998)  has  led  to  renewed  interest  in  pho¬ 
tobleaching  experiments  including  the  utilization  of  selective  pho¬ 
tobleaching  for  information  encoding  (Cole  et  al.,  1996;  White  and 
Stelzer,  1999;  Reits  and  Neefjes,  2001;  Davis  and  Bardeen,  2002; 
Braeckmans  et  al.,  2003;  Braga  et  al.,  2004;  Delon  et  al.,  2004; 
Stavreva  and  McNally,  2004;  Straub,  2003).  Fluorescence  recov¬ 
ery  after  photobleaching  (FRAP)  and  fluorescence  loss  in  photo¬ 
bleaching  (FLIP)  are  the  most  popular  photobleaching-based 
techniques  (McNally  and  Smith,  2002)  and  are  technically  based 
on  the  ability  of  the  confocal  and  multi-photon  microscopes  to 
confine  photobleaching  to  an  arbitrary  pattern  or  region  of  inter¬ 
est.  Unfortunately,  the  mechanism  of  photobleaching  during  con¬ 
focal  or  multi-photon  imaging  is  not  fully  understood  and  some 
aspects  still  remain  obscure,  even  in  the  case  of  very  popular 
molecules  such  as  fluorescein  in  solution  (Song  et  al.,  1995). 

As  lack  of  signal  limits  the  precision  of  most  fluorescent  tech¬ 
niques  and  as  bleaching  and  phototoxicity  limit  the  available 
signal,  we  feel  that  photobleaching  is  one  of  the  most  important 
factors  restraining  future  developments  in  all  these  fields.  Although 
we  are  not  yet  in  the  position  of  being  able  to  develop  dyes  that 
do  not  bleach,  any  attempt  to  do  so  must  start  with  an  effort  to 
assemble  what  we  know  so  far.  This  chapter  attempts  to  do  just 
that.  We  will  attack  this  topic  using  both  microscopic  imaging  and 
single-molecule  studies  under  confocal  and  multi-photon  excita¬ 
tion  conditions. 
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PHOTOBLEACHING 

Fluorescence  emitted  by  almost  all  fluorescent  dyes  fades  during 
observation.  This  phenomenon  is  called  photobleaching  or  dye 
photolysis1  and  involves  a  photochemical  modification  of  the  dye 
resulting  in  the  irreversible  loss  of  its  ability  to  fluoresce.  Even 
though,  in  some  cases,  a  fluorescent  molecule  can  be  switched  on 
again  after  an  apparent  loss  of  emission  ability  or  from  a  natural 
initial  dark  state  (photocycling)  or  is  able  to  switch  on  and  off  by 
itself  (blinking)  within  a  short  timescale,  we  refer  to  photobleach- 
ing  as  an  irreversible  phenomenon.2  This  point  is  particularly  rel¬ 
evant  in  FRAP  and  FLIP  because  these  techniques  are  rigorously 
based  on  the  idea  that  fluorescence  recovers  only  because  fluores¬ 
cent  molecules  diffuse  into  the  bleached  sample  volume.  In  this 
case,  it  is  even  more  important  to  be  clear  that  the  photobleached 
state  cannot  be  reversed. 

Apart  from  this  warning,  we  focus  now  on  how  photobleach¬ 
ing  affects  the  image  formation  process.  Indeed,  fluorescent  dyes 
in  the  excited  state  may  undergo  chemical  or  biochemical  reac¬ 
tions  that  lead  to  their  rapid  degradation  and  destruction  and  con¬ 
sequent  loss  of  image  quality  during  measurement.  In  a  certain 
sense,  fluorophores  are  consumed  by  being  observed.  When  fluo¬ 
rescent  molecules  are  illuminated  at  a  wavelength  for  which  they 
exhibit  a  “good”  cross-section  (Weber  and  Teale,  1958;  Chen  and 
Scott,  1985;  Tsien  and  Waggoner,  1995;  Harper,  2001;  Xu,  2002), 
possibly  close  to  the  cross-section  maximum,  there  is  a  shift  from 
the  ground  energy  level  (S)  to  the  singlet-excited  energy  level  (S*). 
Such  a  temporary  excess  of  energy  can  be  dissipated  by  the  emis¬ 
sion  of  fluorescence,  or  in  radiationless  processes  such  as  internal 
conversion  and  intersystem  crossing  to  the  excited  triplet  state 
(T*).  The  decay  times  from  S*  and  T*  to  S  are  different  accord¬ 
ing  to  the  selection  rules,  and  are  of  the  order  of  1  to  10  ns  and 
10~3 * *  to  Ur6s,  respectively. 

However,  a  molecule  in  the  excited  singlet  or  triplet  state  can 
also  undergo  a  permanent  structural  change  that  often  annihilates 
its  ability  to  fluoresce,  thereby  becoming  a  photobleached  mole¬ 
cule.  Many  factors,  such  as  the  molecular  environment  and  the 
intensity  of  excitation  light,  may  affect  the  mechanism,  and  thus 
the  reaction  order  and  the  rates  of  photobleaching  (Bernas  et  al., 
2004).  In  fact,  spatial  variability  in  the  photobleaching  process  has 
been  demonstrated  within  individual  cells  (Benson  et  al.,  1985). 
Figure  39.1  demonstrates  an  example,  where  Acridine  Orange 
bound  to  DNA  and  RNA  (green  fluorescence  and  red  lumines¬ 
cence,3  respectively)  of  the  same  cell  bleaches  at  different  rates. 
This  dye  binds  to  double-stranded  nucleic  acids  by  intercalation, 
while  stacks  are  formed  on  single-stranded  nucleic  acids. 
Apparently  these  two  forms  of  the  same  dye  exhibit  different 
photostability  under  the  same  illumination  conditions. 


1  Photobleaching  is  the  process  whereby  a  molecule  is  rendered  non- 
fluorescent.  This  may  be  caused  by  photolysis  (lysis,  breaking  open)  or  other 
mechanisms  such  as  bonding  of  excited  fluorophores  to  nearby  molecules. 
Photo-oxidation  describes  the  first  stage  of  many  such  processes. 

2  We  should  also  note  here  another  confusing  parameter.  If  a  macromolecular 
target  is  over-labeled  in  the  sense  of  being  bound  to  more  than  the  optimal 
number  of  dye  molecules,  then  they  may  tend  to  quench  each  other  and  so 
reduce  the  total  emission.  Under  these  conditions,  bleaching  some  dye  mol¬ 
ecules  may  actually  increase  total  light  output  from  the  preparation  because 
the  bleached  molecules  no  longer  quench. 

3  Because  the  red  component  of  Acridine  Orange  (AO)  emission  has  a  long 

time  component,  it  is  more  properly  called  luminescence.  In  a  single-beam 

confocal,  only  the  fast  component  is  seen  because  the  signal  can  only  reach 

the  PMT  for  a  few  microseconds. 


FIGURE  39.1.  Loss  of  luminescence  intensity  of  Acridine  Orange  bound  to 
DNA  in  the  nucleus  and  RNA  in  the  cytoplasm  of  a  fixed  human  fibroblast 
during  acquisition  of  subsequent  confocal  frames.  The  initial  intensities  of 
luminescence  in  two  areas  of  nucleus  and  cytoplasm  were  normalized.  Images 
collected  on  Bio-Rad  MRC1024,  simultaneous  excitation  488nm,  0.5mW  at 
the  specimen,  dichroics  527DRLP,  565DRLP,  emission  filters  540/30nm  and 
RG630,  lens  60x  NA  1.4  PlanApo  (Zarebski  and  Dobrucki,  unpublished;  see 
also  Dobrucki  and  Darzynkiewicz,  2001;  Bernas  et  al.,  2004,  2005,  for  further 
information  on  the  kinetics  of  this  process  and  the  way  to  minimize  the  dif¬ 
ferential  photobleaching). 


Figure  39.2  shows  a  similar  effect  obtained  using  two-photon 
excitation  in  the  picosecond  regime.  Even  if  the  sample  is  rela¬ 
tively  thin,  significant  photobleaching  occurs  under  two-photon 
excitation  after  delivering  a  larger  dose  of  light. 

Photobleaching  Mechanisms 

Many  phenomena,  such  as  self-quenching  or  fluorescence  (or 
Forster)  resonance  energy  transfer  (FRET),  are  known  to  affect  the 
amount  of  light  emitted  by  an  excited  fluorescent  molecule.  The 
manner  in  which  these  processes  reduce  the  fluorescence  intensity 
is  relatively  well  understood.  In  contrast,  photodestruction  is  still 
a  poorly  understood  phenomenon  and  few  data  are  available  on  the 
bleaching  properties  of  most  common  fluorophores.  Dye  photo¬ 
lysis  is  a  very  complex  process. 

Several  theories  have  been  proposed  to  explain  photobleach¬ 
ing.  The  main  causes  seem  to  involve  photodynamic  interactions 
between  excited  fluorophores  and  molecular  oxygen  (02)  in  its 
triplet  ground  state  and  dissolved  in  the  sample  media.  If  the  dye 
has  a  relatively  high  quantum  yield  for  intersystem  crossing,  a  sig¬ 
nificant  number  of  dye  molecules  may  cross  from  a  singlet  excited 
state  S*  to  the  long-lived  triplet  excited  state  T*,  a  process  that 
permits  these  molecules  to  interact  with  their  environment  for  a 
much  longer  time  (milliseconds  instead  of  nanoseconds).  Interac¬ 
tions  between  02  and  dye  triplets  may  generate  singlet  oxygen 
according  to  T*  +  302  — >  S  +  02.  Singlet  oxygen  has  a  longer  life¬ 
time  than  the  excited  triplet  states  of  the  dyes.  Moreover,  several 
types  of  damaging  oxygen  free  radicals  can  be  created  when  it 
decays.  A  fluorophore  in  the  excited  triplet  state  is  also  highly  reac¬ 
tive  and  may  undergo  irreversible  chemical  reactions  involving 
other  intracellular  organic  molecules.  All  these  chemical  reactions 
depend  both  on  the  intracellular  singlet  oxygen  concentration  and 
on  the  distance  between  the  dye  and  intracellular  components  such 
as  proteins,  lipids,  etc.  Therefore,  the  number  of  photons  emitted 
before  a  dye  molecule  is  destroyed  depends  both  on  the  nature  of 
the  dye  molecule  itself  and  on  its  environment. 
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FIGURE  39.2.  Loss  of  fluorescence  intensity  of  three  probes,  bound  to  DNA  (DAPI,  blue),  mitochondria  (MitoTracker  Red  CMXRos,  red),  and  actin  filaments 
(BODIPY-FL  phallacidin,  green)  in  fixed  bovine  pulmonary  artery  endothelial  (BPAE)  cells.  Two-photon  excitation  was  provided  by  a  Leica  TCS  SP2  AOBS 
confocal  microscope  coupled  to  a  Chameleon-XR  (Coherent,  Santa  Clara,  CA)  ultrafast  tunable  titanium-sapphire  laser  through  the  IR  port.  Pulse  width  was 
increased  to  picoseconds  from  140fs,  760nm  wavelength,  20mW  average  power  at  the  entrance  of  the  scanning  head.  Temporal  series  was  acquired  at  a  rate  of 
729ms  per  image  every  6s.  Image  dimensions  are  512  X  512  pixels  over  a  79.76  X  79.76pm2.  (Images  collected  by  Paolo  Bianchini,  LAMBS-MicroScoBio, 
University  of  Genoa,  http://www.lambs.it.) 


Some  fluorophores  have  a  very  short  useful  lifetime,  fading 
after  the  emission  of  only  a  few  hundred  photons,  whereas  other 
molecules  in  other  surroundings  can  emit  a  very  large  number  of 
photons  (tens  of  millions)  before  being  bleached.  If  a  dye  is  pro¬ 
tected  from  reaction  with  environmental  molecular  oxygen,  the 
observed  rate  of  photobleaching  is  lower  (Bernas  et  al. ,  2004).  This 
occurs  naturally  in  GFP  (Tsien,  1998),  where  the  intrinsic  chro- 
mophore  is  positioned  in  the  core  of  a  (3-barrel  structure  (Ormo 
et  al.,  1996). 

In  addition,  either  multi-photon  events  or  the  absorption  of  a 
second  photon  by  a  molecule  already  in  an  excited  state  may  also 
be  involved  in  the  photobleaching  of  some  dyes.  Because  of  the  low 
absorption  cross-section  and  its  quadratic  dependence  on  the  inci¬ 
dent  power,  multi-photon  excitation  requires  an  excitation  intensity 
several  orders  of  magnitude  higher  than  that  needed  for  one-photon 
excitation.  Unfortunately,  this  increases  photobleaching  because  of 
the  high  probability  of  photochemical  degradation  in  the  long-lived 
triplet  state  and  the  interplay  of  multi-photon  ionization  processes 
(Dittrich  and  Schwille,  2001;  Michalet  et  al.,  2003). 

On  the  other  hand,  as  excitation  only  takes  place  in  a  tightly 
confined  volume  of  the  sample,  bleaching  away  from  the  plane  of 
focus  is  essentially  eliminated  and  the  overall  photobleaching  and 


photodamage  within  the  whole  sample  is  usually  less  (Bianco  and 
Diaspro,  1989;  Denk  et  al.,  1990;  Diaspro,  2004).  Moreover,  the 
tightly  controlled  excitation  volume  can  allow  very  precise  control 
of  intentional  photodamage  that  might  be  used  for  uncaging  or 
even  microsurgery  when  needed  (Diaspro  et  al,  2003;  Chapter  38, 
this  volume). 

Until  now,  the  photochemical  reactions  responsible  for  the 
photobleaching  have  been  investigated  in  greatest  detail  for  the 
dye  fluorescein.  Bleaching  can  be  described  in  terms  of  a  photo¬ 
bleaching  quantum  efficiency  Qb,  defined  as  the  ratio  between  the 
number  of  molecules  that  have  been  bleached  and  the  total  number 
of  photons  absorbed  during  the  exposure  period.  The  average 
number  of  photons  emitted  before  a  dye  molecule  bleaches  is  the 
ratio  between  the  fluorescence  quantum  yield  Qf  of  the  dye  and  Qb. 
At  excitation  intensities  of  1023  to  1024  photons  cm'2s_1,  approxi¬ 
mately  corresponding  to  1  mW  of  power  incident  on  a  spot  of 
0.25  pm  radius  at  488  nm  wavelength  (Schneider  and  Webb,  1981), 
fluorescein  bleaches  with  a  Qb  =  3  x  1 0  \  Therefore,  during  its 
useful  lifetime  the  average  fluorescein  molecule  dissolved  in  water 
emits  30,000  to  40,000  photons  before  being  permanently 
bleached  (Hirschfeld,  1976;  Mathies  and  Stryer,  1986;  Tsien  and 
Waggoner,  1995). 
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Both  the  experiments  and  the  mathematical  models  used  to 
investigate  the  behavior  of  free  and  bound  fluorescein  have  demon¬ 
strated  that,  in  fluorescence  microscopy,  its  photobleaching  cannot 
be  described  by  a  single-exponential  process.  However,  the 
single-exponential  process  is  appropriate  for  the  special  case  when 
the  photobleaching  reaction  is  primarily  between  the  triplet  state 
of  the  dye  and  molecular  oxygen  (Hirschfeld,  1976;  Song  et  al, 
1995). 

Although  quenching  of  the  triplet  excited  state  (that  forms  a 
semi-oxidized  radical  form  of  the  dye)  by  mercaptoethylamine 
restores  fluorescein  molecules  to  their  singlet  ground  state,  and 
reduces  photobleaching  (Song  et  al.,  1996),  there  is  no  general 
agreement  on  the  energy  states  involved  in  photobleaching.  In 
single-molecule  detection  conditions,  evidence  of  a  two-step  pho¬ 
tolysis  has  been  reported  for  several  coumarin  and  rhodamine 
derivatives  (Eggeling  et  al.,  1998).  These  authors  suggest  that 
excited  triplet  states  Tn,  higher  in  energy  than  T*,  can  occur  when 
two,  distinct,  successive  photon  absorptions  occur  at  high  irradi- 
ance.  Excitation  at  low  irradiance  yields  longer  survival  times 
because,  at  any  instant,  fewer  molecules  are  in  the  T*  state  to  be 
excited  to  Tn  (Eggeling  et  al,  1998;  Deschenes  and  van  den  Bout, 
2002). 

Fluorescence  correlation  spectroscopy  has  been  used  to  mea¬ 
sure  and  characterize  the  photobleaching  of  rhodamine  6G  and 
FITC  in  different  solutions  (Widengren  and  Rigler,  1996;  Delon 
et  al.,  2004). 

The  high  photon  flux  used  in  two-photon  excitation  micro¬ 
scopy  seems  to  lead  to  higher-order  photon  interactions.  The 
dependence  of  fluorescence  intensity  and  photobleaching  rate  on 
excitation  power  have  been  studied  for  Indo-1,  NADH,  and 
aminocoumarin  under  one-  and  two-photon  excitation  (Patterson 
and  Piston,  2000).  The  results  of  these  studies  suggest  that  higher- 
order  photobleaching  is  common  in  two-photon  excitation 
microscopy.  Therefore,  the  advantage  of  reduced  photobleaching 
associated  with  the  limited  excitation  volume,  characteristic  of 
TPEM  in  thick  samples,  is  offset  by  greater  in-focus  damage, 
making  the  technique  less  suitable  for  studying  thin  specimens.  A 
comparison  of  photobleaching  effects  under  single-  and  multi¬ 
photon  excitation  has  been  performed  in  plant  cells  (Kao  et  al., 
2002).  At  the  laser  intensities  used  to  study  chloroplasts  in  the  pro¬ 
toplasts  from  Arabidopsis  thaliana,  the  photobleaching  cross- 
section  under  two-photon  excitation  is  low,  requiring  the  high  peak 
power  levels  common  in  TPEM.  This  generates  greater  localized 
photobleaching  effects  than  using  conventional  excitation  (Kao 
et  al,  2002).  However,  reducing  specimen  exposure  allows  one  to 
obtain  significantly  more  signal  by  multi-photon  than  by  one- 
photon  excitation  in  high-resolution  optical  sectioning  of  thick 
samples  (Drummond  et  al.,  2002).  In  addition,  the  use  of  phase- 
optimized,  shaped  pulses  in  two-photon  excitation  microscopy  has 
been  demonstrated  to  attenuate  the  photobleaching  of  a  GFP 
variant  by  a  factor  of  4  (Kawano  et  al.,  2003). 

Intrinsically  fluorescent  proteins  such  as  the  phycobiliproteins 
(PBP)  generate  reactive  oxygen  species  (ROS)  in  living  cells. 
Photoactivation  of  R-phycoerytrin,  C-phycocyanin,  and  allophy- 
cocyanin  by  visible  light  results  in  the  generation  of  both  singlet 
oxygen  and  superoxide.  This  contributes  to  both  the  photobleach¬ 
ing  and  the  phototoxicity  of  PBPs  in  vivo  (He  et  al,  1997).  In 
addition,  flavins  and  flavoproteins  excited  to  a  triplet  state  by 
violet-blue  light  can  be  reduced  by  endogenous  cellular  reducing 
agents,  resulting  in  the  production  of  H202  in  cultured  mouse, 
monkey,  and  human  cells  (Hockberger  et  al.,  1999).  However, 
imaging  the  same  cells  by  TPEM  at  1047  nm  did  not  induce  mea¬ 
surable  production  of  H202  (Hockberger  et  al,  1999).  Although 


the  molecular  oxygen  present  in  the  surrounding  environment  does 
not  interact  directly  with  the  chromophore  of  GFP,  photoactivation 
is  thought  to  generate  endogenous  singlet  oxygen,  which  induces 
damage  to  this  chromophore,  as  demonstrated  in  transfected  COS7 
kidney  cells  and  E.  coli  bacteria  (Greenbaum  et  al,  2000).  In  BY- 
2  tobacco  cells  loaded  with  oxidation-sensitive  dyes,  excitation 
light  induces  the  production  of  ROS.  The  amount  of  ROS  increases 
if  the  cells  express  an  intrinsically  fluorescent  protein  that  absorbs 
the  excitation  light  (Dixit  and  Cyr,  2003).  This  process  results  in 
an  increase  in  mitotic  arrests  and  shows  a  nonlinear  relationship 
to  the  excitation  light  intensity  (Dixit  and  Cyr,  2003).  The  role  of 
molecular  oxygen  in  phototoxicity  exerted  by  extracellular  fluors 
has  been  demonstrated  in  animal  and  plant  cells  (Dobrucki,  2001). 
The  involvement  of  oxygen  in  photobleaching  processes  is 
supported  by  data,  which  demonstrate  that  anoxia  significantly 
reduces  photobleaching  rates  of  propidium  and  chromomycin  A3 
bound  to  DNA  (Bernas  et  al,  2004). 

Apart  from  fluorescent  proteins,  nonlinear  multi-photon  exci¬ 
tation  in  the  near-infrared  of  a  variety  of  endogenous  absorbers, 
such  as  water  and  NAD(P)H,  can  induce  the  formation  of  ROS 
resulting  in  oxidative  stress  (Konig  and  Tirlapur,  2002). 

Reducing  Photobleaching 

The  prevention  of  the  fading  of  fluorescence  emission  intensity  is 
very  important  not  only  for  quantitative  microscopy  but  also  for 
obtaining  high-quality  images.  Unfortunately,  reducing  photo¬ 
bleaching  by  decreasing  the  excitation  time  or  by  lowering  the 
excitation  intensity  leads  to  a  reduction  of  the  fluorescence  signal. 
As  lower  signal  increases  the  effect  of  Poisson  noise  and  obscures 
low-contrast  features,  deciding  on  the  optimal  excitation  level  is 
always  a  trade-off  in  which  ideally  one  reduces  the  light  dose  until 
one  can  no  longer  see  the  details  one  needs  to  understand.  Opti¬ 
mizing  the  excitation  intensity  can  be  beneficial  in  3D  imaging 
(Bernas  et  al.,  2004). 

A  good  strategy  starts  with  the  selection  of  a  fluorophore  with 
the  high  photostability.  Figure  39.3  (A.B)  shows  a  comparison  of 
the  results  of  photobleaching  of  four  fluorescent  probes  associated 
with  chromatin  in  different  preparations  of  fixed  HeLa  cells.  The 
probes  differ  in  molecular  structures  and  mechanisms  of  binding 
to  chromatin.  The  same  dose  of  light  delivered  at  similar  photon 
fluxes  resulted  in  dramatically  different  loss  of  fluorescence  signal 
[Fig.  39.3(A)]  and  a  different  number  of  total  photons  recorded 
(Fig.  39.3(B)].  The  choice  of  the  best  fluorophore  is  not  trivial, 
however.  In  this  example,  one  might  draw  the  conclusion  that  pro¬ 
pidium,  a  DNA  intercalator,  is  the  best  choice  for  imaging  chro¬ 
matin.  In  fact,  eGFP  fused  with  histone  H2B  gave  the  best  results 
in  this  experiment  because  eGFP  yielded  reasonable  images  even 
at  an  excitation  photon  flux  of  0.17  W/cm2,  where  photobleaching 
was  negligible.  Thus,  the  total  number  of  photons  emitted  by  GFP 
before  bleaching  was  the  greatest.  The  other  probes  tested  required 
higher  intensities  of  excitation  light,  bleached  at  greater  rates  and 
provided  less  total  photons  (Bernas  et  al,  2004). 

As  molecular  oxygen  is  one  of  the  main  actors  in  photo¬ 
bleaching,  deoxygenating  the  specimen  often  improves  the  sur¬ 
vival  period  of  both  the  dye  and  the  cell.  In  living  cells  or  tissues 
that  tolerate  anoxia,  or  even  hypoxia,  oxygen  may  be  removed  by 
bubbling  N2  through  the  media  or  by  using  biological  oxygen  scav¬ 
engers  (Bloom  and  Webb,  1984;  Dobrucki,  2001).  Chemicals 
capable  of  quenching  singlet  oxygen  can  also  be  employed  to 
reduce  the  effects  of  photobleaching.  Carotenoids  such  as  as  cro- 
cetin  and  etretinate  are  good  singlet  oxygen  quenchers  in  cell 
cultures  (Kohen  et  al,  1986;  Reyftmann  et  al.,  1986;  Manitto 
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FIGURE  39.3.  Loss  of  signal  of  four  fluorescent  probes  bound  to  chromatin  due  to  photobleaching  by  exciting  light.  Fraction  of  fluorescence  remaining  (A) 
and  the  total  number  of  photons  detected  per  one  pixel  (dwell  time  13.3  ps;  B)  after  delivering  increasing  doses  of  excitation  light  is  shown  for  propidium  (PI), 
Acridine  Orange  (AO,  green  fluorescence  of  intercalated  monomers),  chromomycin  A3  (CA3),  and  eGFP  fused  to  histone  H2B.  The  conditions  of  the  experi¬ 
ment  are  described  elsewhere  (Bemas  et  al .,  2004,  2005). 


et  al.,  1987).  Other  oxygen-reactive  protective  molecules  include 
vitamin  E  analogues,  vitamin  C,  reduced  glutathione,  imidazole, 
cysteamine,  and  the  amino  acid  histidine  (Tsien  and  Waggoner, 
1985).4 

Figure  39.4  compares  the  photobleaching  kinetics  of  propid¬ 
ium  bound  to  the  DNA  of  fixed  human  fibroblasts,  equilibrated 
with  air  or  argon,  as  a  function  of  the  excitation  dose  in  the  range 
from  0  to  50  kj/cnr  and  it  shows  that  argon  lessens  signal  loss 
resulting  from  photobleaching.  Figure  39.5  shows  the  fraction  of 
fluorescence  that  remains  unbleached  after  the  same  type  of  cells 
have  received  the  same  total  dose  of  light  at  different  photon  fluxes 
in  the  range  from  1  to  111  W/cm2,  respectively.  It  demonstrates 
how  a  given  dose  of  illumination  causes  less  bleaching  when  deliv¬ 
ered  at  a  low  photon  flux,  or  under  argon  (Bernas  et  al.,  2004). 
Figure  39.6  shows  the  potential  of  anoxic  imaging  in  protecting 
fluorescence  and  facilitating  collection  of  full  3D  datasets  (after 
Bernas  et  al.,  2004). 

Other  common  antifade  reagents  are  diazabicyclo-2, 2,2- 
octane  (DABCO),  n-propyl  gallate  and  /j-phenylenedi  amine.  2- 
mercaptoethylamine  is  used  mainly  when  observing  chromosome 
and  DNA  specimens  stained  with  propidium  iodide.  Acridine 
Orange,  or  chromomycin  A3. 

Although  the  anti-fade  agents,  whose  effects  are  diagrammed 
in  Figure  39.7,  are  highly  efficient  in  retarding  photobleaching, 
each  has  side  effects.  One  of  the  most  effective  antifade  reagents, 
p - p h e n y I e n e d i a m  i  n e  (PPD),  suffers  from  thermosensitivity,  auto- 


4  To  be  effective  as  radical  scavengers,  these  agents  must  be  used  at  concen¬ 
trations  high  enough  to  be  incompatible  with  live-cell  studies.  Even  cells  that 
do  seem  to  not  die  in  terms  of  popular  viability  assays  often  suffer  other  types 
of  damage  that  may  affect  their  physiology  sufficiently  to  preclude  acquisi¬ 
tion  of  relevant  information.  See  also  Chapter  38,  this  volume. 


fluorescence,  and  cell  toxicity,  making  it  unsuitable  for  in  vivo 
studies  (Bock  et  al.,  1985;  Krenik  et  al,  1989).  /V-propyl  gallate 
(NPG)  is  both  photo-  and  thermostable.  While  not  as  effective  as 
PPD,  it  can  be  used  for  in  vivo  studies.  However,  although  NPG 
is  very  effective  in  retarding  bleaching  in  FITC-stained  Reh6  cells, 
it  reduces  the  initial  value  of  fluorescence  intensity  both  in  con¬ 
ventional  epi-fluorescence  and  in  confocal  microscopy  (Souchier 
et  al.,  1993).  Comparing  NPG  with  other  anti-fading  agents,  these 
authors  conclude  that  the  choice  of  an  anti-fading  medium  depends 
on  the  desired  results:  a  slower  decay  of  fluorescence  combined 
with  initial  quenching  versus  a  faster  photobleaching  rate  but  start¬ 
ing  at  a  higher  initial  emission  intensity. 

An  extensive  comparison  among  different  homemade  and 
commercial  anti-fading  media  has  been  performed  by  CLSM  in 
NIH  3T3  cells  stained  with  FITC-phalloidin  (Ono  et  al.,  2001). 
These  authors  use  the  equation  EM,  =  ( EM0  -  B)e~'/A  +  B,  where 
EM,  is  the  emission  intensity  at  time  t,  B  the  background  intensity, 
and  A  the  anti-fading  factor,  that  is,  the  time  spent  until  fluores¬ 
cence  intensity  decays  to  He  of  EM0.  As  a  general  rule,  for  the 
media  examined  (SlowFade  and  ProLong,  Molecular  Probes,  Inc., 
Eugene,  OR;  PermaFluor,  Lipshaw/Immunon,  Pittsburgh,  PA; 
FluoroGuard,  Bio-Rad  Labs,  Hercules,  CA),  and  in  agreement 
with  Souchier  and  colleagues,  they  found  that  high  initial  emission 
intensities  are  accompanied  by  faster  decreases  of  fluorescence. 
With  bright  and  photostable  molecules,  such  as  the  Alexa  dyes 
(Landon,  1997),  media  that  include  an  anti-fading  agent  are  a 
good  choice,  even  if  they  have  a  lower  initial  fluorescence 
intensity. 

Very  recently,  a  new  family  of  synthetic  nanoparticle  fluo- 
rophores  has  been  introduced.  They  mainly  consist  of  a  core  of 
cadmium  selenide  coated  with  zinc  sulfide.  These  nanocrystals, 
named  Quantum  Dots  (Quantum  Dot  Corporation,  Hayward,  CA) 
have  a  large  Stokes  shift  and  are  very  bright  and  photostable  (Hines 


FIGURE  39.4.  Kinetics  of  photobleaching  of  propid- 
ium  bound  to  DNA,  in  air  or  argon.  Intensity  (0-250 
a.u.)  versus  dose  (0-50kJ/cm2).  Fixed  cells  were 
stained  with  propidium,  and  equatorial  sections 
scanned  with  the  laser  in  a  confocal  microscope  under 
air  (A)  or  argon  (B).  Twenty  scans  were  accumulated 
to  form  one  image,  and  the  average  intensity  (arbitrary 
units;  a.u.)  of  pixels  of  different  initial  brightness  in 
images  is  plotted  relative  to  the  accumulated  dose  of 
light.  The  flux  of  the  incident  light  is  indicated.  (A) 
Bright  pixels  (curve  1)  are  bleached  more  rapidly  than 
darker  ones  (curves  2,  3).  Curves  for  only  three  of  the 
25  brightness  classes  are  shown  for  clarity.  The  results 
are  the  means  of  five  experiments.  (B)  Argon  lessens 
photobleaching.  Curves  1  to  3  from  (A)  are  included 
for  comparison  with  their  counterparts  (i.e.,  11-13)  in 
argon  (after  Bemas  et  al.,  2004). 
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FIGURE  39.5.  Analysis  of  photobleaching  of  propidium  bound  to  DNA  in  air 
and  argon.  The  fraction  of  fluorescence  remaining  after  exposure  to  the  same 
total  dose  of  exciting  light,  but  delivered  by  laser  beams  of  different  photon 
fluxes.  Laser  intensity  was  adjusted  using  neutral  density  filters  and  the  total 
time  of  illumination  was  adjusted  to  deliver  the  same  total  light  dose  to  each 
sample.  Error  bars:  95%  confidence  intervals  (after  Bernas  et  al.,  2004). 
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FIGURE  39.6.  An  xz-projection  from  3D  images  collected  in  argon  (left)  or  in  air  (right)  of  chromatin  in  a  cell  in  mitosis,  with  condensed  chromosomes  stained 
with  propidium.  Sixty-nine  consecutive  images  were  collected  at  a  330 nm  z-spacing  using  an  average  excitation  flux  of  11  Wcm“2  in  the  focal  plane  and  start¬ 
ing  near  the  coverslip.  Maximum  intensity  projections  (right  and  left)  and  fluorescence  (center)  measured  along  the  white  vertical  bar.  The  intensity  drops  off 
more  rapidly  as  one  moves  away  from  the  coverslip  under  air  because  of  increased  photobleaching  (after  Bemas  et  al.,  2004). 
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Photobleaching  -  3t3  Fluorescein  Actin 
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FIGURE  39.7.  Loss  of  fluorescence  intensity  due  to  photobleaching  in  similar  samples  submerged  in  physiological  saline  or  in  mounting  media  supplemented 
with  various  commercial  anti-fade  agents.  The  efficiency  of  countering  photobleaching  and  a  loss  of  the  initial  signal  varied  widely  among  different  formula¬ 
tions.  Samples:  fixed  FoLu  lung  fibroblasts,  actin  stress  fibers  stained  with  phalloidin-fluorescein.  (Data  kindly  provided  by  M.W.  Davidson.) 


and  Guyot-Sionnest,  1996;  Mattoussi  et  al.,  2000;  Watson  et  al., 
2003;  Jaiswal  and  Simon,  2004).  A  surface  polymer  layer  enables 
conjugation  of  Quantum  Dots  to  antibodies  and  other  biomolecules 
(Wu  et  al,  2003),  allowing  the  development  of  new  immunofluo¬ 
rescence  protocols  that  take  advantage  of  their  fluorescent  proper¬ 
ties  (Ness  et  al.,  2003).  Quantum  Dots  allow  multicolor  imaging 
in  demanding  biological  environments  such  as  in  living  animals  or 
organs  and  within  cellular  cytoplasm  (Sokolov  et  al.,  2003;  Larson 
et  al.,  2003;  Jaiswal  and  Simon,  2004).  These  fluorescent  probes 
exhibit  two-photon  cross-sections  of  the  order  of  thousands  of 
Goeppert-Meyer  units  (GM),  (1GM  =  10_5° cm4 s/photon)  and  a 
wide  range  of  emission  wavelengths  (Diaspro  et  al.,  2005).  They 
should  be  good  candidates  for  deep,  multi-color  imaging  in  bio¬ 
logical  and  medical  applications  (Sokolov  et  al,  2003;  Voura  et 
al.,  2004).  On  the  other  hand,  some  problems  in  using  Quantum 
Dots  relate  to  their  relatively  large  size  and  mass,  their  intermit¬ 
tent  fluorescent  behavior,  their  high  propensity  for  aggregation, 
and  the  fact  that  their  high  cross-section  and  relatively  long  singlet- 
state  lifetime,  cause  them  to  reach  singlet-state  saturation  at  rela¬ 
tively  low  power  levels  in  scanned-beam  microscopy. 


PHOTOBLEACHING  AT  THE 
SINGLE-MOLECULE  LEVEL 

Any  single -molecule  spectroscopy  experiment  stops  when  the 
probe  molecule  photobleaches.  This  limitation  becomes  crucial 
when  the  time  scale  of  the  biological  process  under  examina¬ 
tion  is  comparable  to  the  survival  time  for  the  molecule  itself 
(Deschenes  and  van  den  Bout,  2002).  In  certain  matrices  and  at  low 
temperature,  photobleaching  can  be  delayed  almost  indefinitely 
(Basche,  1998).  However,  under  usual  laboratory  conditions  (room 
temperature,  in  atmosphere)  each  molecule  can  emit  a  finite,  though 


sometimes  very  large  (Deschenes  and  van  den  Bout,  2002),  number 
of  photons. 

Single-molecule  photobleaching  can  be  defined  as  a  permanent 
transition  to  a  dark  electronic  state  that  occurs  typically  after 
N  ~  106  to  10s  photon  absorption  events.  This  implies  a  few 
seconds  of  observation  at  an  average  emission  of  106  fluorescence 
photons/second,  and  leads  to  significant  signal-to-noise  challenges 
for  single-molecule  spectroscopists. 

In  fluorescence  laser  scanning  microscopy,  excitation  volumes 
range  from  0.1  to  1pm3.  For  molecular  concentrations  of  about 
100  nM  to  1  p M  (i.e.,  similar  to  that  of  most  of  the  protein  systems 
in  cells),  the  number  of  fluorescent  emitters  in  the  excitation 
volume  varies  from  10  to  1000.  This  number  lies  between  the 
single-molecule  and  the  bulk  level.  Fading  of  images  of  cells  and 
tissues  under  repeated  imaging  in  fluorescence  CLSM  or  TPEM, 
can  be  ascribed  to:  (1)  a  change  in  the  number  of  fluorophores 
present  (single-molecule  bleaching),  (2)  a  variation  of  the  fluores¬ 
cence  quantum  yield,  and  (3)  a  change  in  the  fluorescence  emis¬ 
sion  dynamics  (single-molecule  blinking). 

The  first  mechanism  leads  to  photobleaching  when  viewed 
at  the  bulk  level.  The  second  mechanism  can  be  due  to  chemical 
reactions  (e.g.,  photo-induced  protonation/deprotonation)  or  to 
photo-induced  transitions  to  other  electronic  states.  At  the  bulk 
level,  an  overall  decrease  in  the  fluorescence  quantum  yield  can 
also  be  due  to  an  increase  in  the  rate  of  reversible  transitions 
between  a  bright  and  a  dark  state  of  the  individual  molecules,  a 
phenomenon  known  as  blinking.  All  these  situations  may  lead 
to  a  decrease  in  overall  fluorescence  emission  under  imaging 
conditions. 

In  general,  blinking  characteristic  times  usually  occur  on  a 
time  scale  of  1  to  100  ms,  but  they  can  also  extend  to  much  longer 
intervals.  Fluorescence  emission  can  sometimes  recover  after 
minutes  to  hours  making  it  difficult  to  discriminate  between  per¬ 
manent  bleaching  and  blinking. 
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Photobleaching  of  Single  Molecules 

Not  much  data  and  few  systematic  studies  are  available  on  single¬ 
molecule  photobleaching,  a  process  that  is  usually  characterized, 
at  the  time  resolution  of  CLSM  or  TPEM,  by  a  steep  drop  of  the 
fluorescence  emission  to  the  background  level.  The  observation  of 
such  a  sharp  transition  is  indeed  used  as  a  fingerprint  of  the  single¬ 
molecule  emission  and  has  been  used  to  discriminate  single¬ 
molecule  spots  in  fluorescence  images  from  those  representing 
aggregates  (Chirico  et  al.,  2001.  2002;  Diaspro  et  al.,  2001;  Maher 
et  al.,  2002;  Cannone  et  al.,  2003).  Photobleaching  also  affects 
other  single-molecule  spectroscopic  approaches  such  as  surface- 
enhanced  resonant  Raman  spectroscopy,  a  process  recently  high¬ 
lighted  by  Maher  and  colleagues  (2002). 

The  photophysical  origin  of  single-molecule  photobleaching 
has  been  investigated  on  several  dyes  used  for  microscopy.  One  of 
the  first  attempts  at  a  systematic  study  dealt  with  a  series  of  four 
dyes,  pyrene,  rhodamine  6G  fluorescein  and  indo-1,  spread  by 
spin-coating  on  a  chemically  etched  glass  (see  details  in  Chirico 
et  al.,  2001,  2003).5  For  the  two-photon  excitation  used  in  the 
study,  the  authors  found  a  power  law  dependence  of  the  bleaching 
rate  on  the  excitation  power  with  an  exponent  =2.5.  This  finding 
proved  that  increasing  the  excitation  power  does  not  imply  a  gain 
in  the  total  fluorescence  photon  flux.  Because  the  fluorescence 
emission  depends  on  the  second  power  of  the  excitation  intensity, 
the  dependence  of  the  total  number  of  photons  emitted  versus  the 
excitation  intensity,  for  TPE,  scales  as  -I-0'5,  where  I  is  the  exci¬ 
tation  intensity.  Moreover  the  bleaching  rate  depends  on  the  duty 
cycle  of  the  excitation.  If  continuous  excitation  is  replaced  by 
pulsed  excitation  without  increasing  the  peak  brightness,  the 
bleaching  rate  diminishes  linearly  with  the  excitation  duty  cycle 
(the  ratio  of  the  time  the  molecules  are  irradiated  to  that  when  they 
are  not  irradiated).  A  marked  dependence  of  the  bleaching  rate  on 
the  substrate  temperature  was  also  reported. 

Recently,  Deschenes  and  van  den  Bout  (2002)  reported  a 
detailed  single-molecule  study  of  rhodamine  6G  molecules  em¬ 
bedded  in  a  film  of  poly(methylacrylate)  (PMA)  under  vacuum 
and  using  very  low  excitation  intensities,  between  2W/cm2  and 
1 0,000  W/cm2.  The  authors  reported  a  remarkable  decrease  of  the 
photobleaching  rate  at  low  excitation  intensities  and  an  increase  in 
the  total  number  of  photons  emitted  up  to  ~109  photons  per  mole¬ 
cule  before  bleaching.  They  propose  a  four-state  model,  sketched 
in  Figure  39.8,  that  accounts  for  the  observed  nonlinear  intensity 
dependence  of  the  photobleaching  rate.  This  model  seems  to  fit  most 
of  the  single-molecule  photobleaching  studies  published  up  to  now. 

Photobleaching  causes  the  fluorescent  output  from  a  group  of 
dye  molecules  to  decay  in  a  manner  described  by  either  a  single- 
or  a  multi-exponential  function  of  the  total  accumulated  exposure 
(Xie  and  Trautman,  1998;  Widengren  and  Riegler,  1996;  Eggeling 
et  al.,  1997,  1998;  Wennmalm  and  Riegler,  1999;  Ko,  2004). 


5  Solutions  of  various  dyes  were  prepared  as  follows:  (1)  lOOmM  rhodamine 
6G  in  dimethylsulfoxide;  (2)  lOOmM  fluorescein  in  TRIS  buffer;  (3)  lOOmM 
pyrene  powder  in  DMSO;  (4)  lOOmM  Indo-1  powder  in  MilliQ  water.  The 
rhodamine  and  fluoroscein  solutions  were  later  diluted  to  70  to  300  n M  in 
ethanol  before  use.  The  glass  slides  were  cleaned  by  being  soaked  in  a  1% 
sodium  dodecyl-sulfate  for  24  h,  then  in  a  saturated  NaOH-methanol  solution 
for  2h.  The  NaOH  is  removed  by  soaking  the  slides  in  0.1%  HC1  for  2h  and 
then  in  diluted  chromic  solution  (K2Cr04  in  concentrated  phosphoric  acid)  for 
2  h.  The  slides  were  then  stored  in  ethanol  and  finally  rinsed  thoroughly  with 
MilliQ  water  and  dried  with  nitrogen  flow  just  before  use.  A  drop  of  one  of 
these  work  solutions  was  placed  on  a  glass  slide  and  spin-coated  at  1000  rpm 
in  a  hood. 


FIGURE  39.8.  Perrin-Jablonski  diagram  of  the  relevant  states  for  rhodamine 
6G  photobleaching.  Absorption  (kabs)  from  S0  to  S  may  lead  to  non-radiative 
internal  conversion  (Aic),  fluorescence  (£§),  or  intersystem  crossing  (Aisc)  to  T,. 
From  T!,  the  molecule  may  relax  back  to  S0  (At,),  or  absorb  a  second  532nm 
photon  (ATi-,Tn)  and  go  to  Tn.  From  there,  the  molecule  may  either  relax  back 
to  T!  (ATn_,T1)  or  react  (Areact),  leading  to  irreversible  photobleaching.  (Derived 
from  Deschennes  and  van  den  Bout,  2002.) 


Multi-exponential  distributions  are  typically  found  for  ensemble 
measurements  while  single-exponential  bleaching  times  are  mea¬ 
sured  on  a  series  of  single,  entrapped  molecules. 

The  multi-exponential  distribution  of  photobleaching  times 
has  been  interpreted  either  as  being  caused  by  intrinsic  hetero¬ 
geneity  of  the  biological  sample  (Xie  and  Trautman,  1998),  or  as 
an  artifact  caused  by  the  use  of  non-uniform  excitation  profiles 
in  the  beams  used  to  produce  the  bleach.  Molski  (2001),  who 
calculated  bleaching  kinetics  in  single-molecule  fluorescence 
spectroscopy  using  renewal  theory,  suggested  that  non-single- 
exponential  distributions  of  bleaching  times  could  be  due  to  true 
heterogeneity  in  the  sample  (in-homogeneous  broadening).  He 
suggested  a  five-state  model  where  bleaching  occurs  from  the 
excited  triplet  states,  and  found  an  exact  formalism  to  calculate 
distributions  of  the  number  of  photon  counts  and  the  time  before 
photobleaching.  He  found  exponential  distributions  for  the  average 
bleaching  number  and  the  average  bleaching  time  and  related  these 
two  quantities  to  the  transition  rate  constants.  Berglund  (2004) 
has  extended  these  results  by  considering  theoretical  models  of 
the  decay,  via  photobleaching,  of  surface -immobilized  fluorescent 
molecules  excited  by  a  spatially  varying  laser  intensity  profile.  He 
shows  that  the  fluorescence  decay  measured  in  a  non-uniform  exci¬ 
tation  profile  is  always  non-exponential  because  molecules  near 
the  edge  of  the  spot,  where  the  light  is  dimmer,  bleach  more  slowly. 

These  results  suggest  that  the  observation  of  multiple  expo¬ 
nentials  in  the  distributions  of  single-molecule  photobleaching 
times  could  arise  at  least  partially  as  an  artifact  of  a  spatially 
varying  laser  profile  rather  than  indicating  any  heterogeneity  in  the 
bleaching  process.  Experimental  support  of  this  study  appeared  in 
an  article  by  Ko  (2004),  who  showed  that,  for  rhodamine  6G  in 
agarose  gel.  the  bulk  bleaching  decays  are  described  by  a  three- 
exponential  decay  while  the  single-molecule  bleaching  times  are 
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distributed  according  to  a  single-exponential  function.  It  is  encour¬ 
aging  that  the  average  bleaching  time  they  observed  in  bulk  agrees 
very  well  with  the  mean  of  the  single-molecule  distribution. 
Rhodamine  6G  entrapped  in  poly(vinyl  alcohol)  has  been  the 
object  of  another  recent  study  of  photobleaching  at  the  single¬ 
molecule  level  (Zondervan  et  al,  2004).  These  authors  have  inves¬ 
tigated  photobleaching  as  a  function  of  illumination  time,  excita¬ 
tion  intensity,  the  presence  of  oxygen,  and  temperature.  They 
observed  non-exponential  kinetics  related  to  primary  photo¬ 
bleaching  through  two  dark  states  —  the  triplet  state  and  a  radical 
anion  —  and  to  secondary  photobleaching  after  the  optical  excita¬ 
tion  of  those  dark  states.  The  important  claim  of  this  study  is  that 
the  presence  of  metastable  states  other  than  the  triplet  can  drasti¬ 
cally  affect  the  photobleaching  rate. 

Photobleaching  and  Photocycling  of  Single 
Fluorescent  Proteins 

Among  fluorescent  biomolecules,  single-molecule  photobleach¬ 
ing  studies  have  been  reported  on  several  variants  of  the  GFP, 
and  more  generally,  of  the  so-called  visible  fluorescent  proteins 
(VFPs).  Blue  mutants,  used  in  combination  with  yellow  GFP  were 
well  known  to  cellular  microscopists  as  a  good  donor-acceptor  pair 
for  measuring  intracellular  interactions  via  FRET.  However,  as  blue 
GFP  variants  are  generally  dim  (Rizzuto  et  al.,  1996;  Cubitt  et  al., 
1997)  and  tend  to  photobleach  readily  (Ellenberg  et  al,  1998), 
alternative  multicolor  pairs  were  developed.  Of  these,  the  most 
popular  currently  is  cyan  fluorescent  protein  (CFP)  (Heim  et  al., 
1994)  and  the  red-shifted  yellow  fluorescent  protein  (YFP)  (Ormo 
et  al.,  1996).  CFP  is  brighter  than  BFP  and  is  more  photostable 
under  imaging  (Ellenberg  et  al. ,  1998;  see  Chapter  45,  this  volume). 

Spontaneous  fluorescence  recovery  of  YFP  molecules  from 
seemingly  irreversible  photobleaching  using  405  nm  excitation 
was  reported  very  early  in  the  GFP  literature  (Dickson  et  al,  1997). 
The  authors  have  called  this  effect  optically  induced  switching.  In 
fact,  the  chromophores  of  GFP  occur  in  at  least  three  protonation 
states.  Two  of  them  correspond  to  the  cases  in  which  the  oxydril 
of  the  tyrosine  66  is  protonated  (neutral  state)  or  de-protonated 
(anionic  state).  A  third  state  corresponds  to  a  zwitterionic  case 
(Zimmer,  2002).  Most  of  the  observations  on  the  recovery  of  GFP 
mutants  after  bleaching  hint  at  the  possibility  that  some  of  the  pre¬ 
sumably  photobleached  proteins  were  actually  residing  in  a  dark 
state  caused  by  a  neutral  chromophore.  The  switching  effect  has 
an  important  impact  on  the  suitability  of  GFP  mutants  for  both  cel¬ 
lular  imaging  and  single  molecule  experiments.  However,  investi¬ 
gations  on  the  widely  used  EGFP  mutant  showed  no  optical 
switching  behavior  under  a  variety  of  experimental  conditions 
(Cinelli  et  al.,  2000). 

It  is  possible  to  detect  single-molecule  signals  with  sub¬ 
millisecond  time  resolution  using  an  avalanche  photodiode, 
provided  that  a  sufficiently  high  fluorescence  photon  flux  can  be 
established.  Early  confocal  microscopy  experiments  on  single 
molecules  of  the  GFP  mutant  E222  by  Jung  and  colleagues  (1998) 
increased  the  time  resolution  to  a  minimum  integration  time  of  10 
ms,  a  limit  more  recently  reduced  to  800 |is  (Garcia-Parajo  et  al., 
2000).  These  experiments  showed  that  spontaneous  recovery  of 
fluorescence  emission  from  single  GFP  molecules  can  be  observed 
as  long  as  1 80  min  after  an  apparent  photobleaching  event  and  that 
photoinduced  blinking  was  also  present.  The  blinking  on-times 
measured  for  this  mutant  were  dependent  on  the  excitation  power 
with  values  comparable  to  those  previously  reported  (Dickson  et 
al,  1997;  Moerner  et  al.,  1999;  Peterman  et  al.,  1999).  At  moder¬ 
ate  illumination  intensities  of  1.5kW/cm2,  half  of  the  molecules 


reside  in  a  dark  state  (Garcia-Parajo  et  al.,  2000).  It  is  worth  noting 
that  this  high  proportion  is  not  caused  by  the  comparatively  small 
transition  yield  of  5  - 10  6  into  the  dark  state  so  much,  as  by  the 
long  lifetime  of  this  state  toB=  1.6  s. 

An  enhanced  YFP  mutant,  termed  E2GFP,  has  been  obtained 
that  might  be  used  as  a  single-biomolecule  optical  switch  (Cinelli 
et  al,  2001).  As  with  most  GFP  molecules,  prolonged  or  intense 
excitation  results  in  photobleaching  (at  476  nm  in  the  case  of 
E2GFP).  However,  E2GFP  is  the  only  known  mutant  in  which  irra¬ 
diation  of  the  dark  photobleached  state  (at  350  nm)  produces  an 
excited  state  that  photoconverts  to  an  anionic  B  form,  which  is  also 
fluorescent. 

Recently,  a  TPE  investigation  (Chirico  etal.,  2004)  at  the  single¬ 
molecule  level  showed  that  single-molecule  photobleaching 
increases  with  a  power  law  of  2.4  and  that  the  fluorescence  recov¬ 
ery  after  bleaching  can  be  induced  also  by  near-infrared  light  at  720 
nm  (Fig.  39.9).  Several  characteristics  of  this  infrared  (IR)-induced 
recovery  makes  this  mutant  promising  both  for  prolonged  cellular 
imaging  and  for  opto-electronic  data  storage  applications.  Recov¬ 
ery  spectra  are  very  narrow  and  their  widths  do  not  change  appre¬ 
ciably  with  the  excitation  power,  as  shown  in  Figure  39.10. 
Maximum  recovery  efficiency  occurs  at  720  nm.  This  corresponds 
to  twice  the  peak  wavelength  of  the  spectrum  of  the  dark  state 
obtained  in  single-photon  experiments  (Nifosi  et  al.,  2003).  The 
fourth-power  dependence  of  the  recovery  shown  in  Figure  39. 10(B) 
indicates  that  two  two-photon  transitions  are  required  to  overcome 
the  energy  barrier  from  a  dark  state  to  the  optically  active  form 
coupled  to  it.  The  second  transition  is  likely  to  bring  the  system  from 
the  excited  dark  state  (or  its  relaxed  form)  to  a  higher  excited  level. 
Detuning  between  the  two  transitions  may  cause  the  observed  sharp 
recovery  spectrum.  Vibrational-electronic  couplings  could  also 
contribute  to  reduce  the  width  of  the  recovery  spectrum. 

Bleaching  and  Autofluorescence 

A  main  concern  about  photobleaching  at  the  single-molecule  level 
stems  from  the  effort  to  follow  a  single-protein  system  in  vivo.  The 
ability  to  do  this  would  allow  one  to  study  in  detail  basic  processes 
such  as  cellular  internalization  and  cell  trafficking  by  exploiting 
all  the  known  photophysical  properties  of  the  probes  being  used. 
Although,  within  the  nucleus,  autofluorescence  is  only  slightly 
above  the  level  encountered  in  common  gel  samples,  investiga¬ 
tions  in  the  cytoplasm  are  hampered  by  a  significant  level  of 
background  signal.  It  is  therefore  necessary  to  optimize  the 
experimental  conditions,  including  the  choice  of  a  fluorophore 
with  minimal  photobleaching,  before  single-molecule  detection  in 
living  cells  becomes  conceivable.  As  an  important  step  to  this  end, 
Kubitschek  and  co-workers  have  again  employed  YFP  proteins. 
The  authors  investigated  the  diffusion  of  the  S65G/S72A/T203F 
mutant  in  glycerol/water  mixtures  (Kubitschek  et  al.,  2000).  They 
were  able  to  track  single  YFP  molecules  with  a  time  resolution  of 
10  ms,  a  signal  of  8  ±  3  counts/pixel  above  background,  and  a 
signal-to-noise  ratio  of  4.  Photobleaching  was  still  a  limiting 
factor,  yielding,  on  average,  a  maximum  of  only  105  emitted 
photons  and  an  observation  time  of  100  ms. 

Other  Fluorescent  Proteins 

Besides  GFPs,  the  light  harvesting  protein  systems  of  photosyn¬ 
thetic  organisms  have  been  investigated  in  terms  of  their  single¬ 
molecule  photodynamics.  These  systems  consist  of  protein 
assemblies  in  which  various  chromophores  are  electronically 
coupled  so  that  the  summed  absorption  covers  a  wide  spectral 
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FIGURE  39.9.  Fluorescence  image  of  single  proteins  excited  in  the  A  band  (excitation  at  780 nm,  emission  at  440nm)  (upper  left).  (Right)  Same  held  of  view 
but  on  the  B  channel  (excitation  at  88()nm  emission  at  535nm).  (Lower  panel,  left)  Fluorescent  image  in  the  A  channel  after  irradiation  at  710nm  (2mW  and 
50ms).  (Lower  panel,  right)  Same  held  of  view  but  on  the  B  channel  and  after  irradiation  at  720nnt  (2mW  and  50ms).  Color  levels  of  left  panels  are  hve  times 
those  in  right  panels.  (After  Chirico  el  al. ,  2004.) 
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FIGURE  39.1 0.  (A)  Percentage  of  single-protein  fluorescence  recovery  versus 
the  illumination  time  at  720nm  for  different  excitation  powers:  3.34 mW  ( solid 
circles),  4.34 mW  (solid  squares ),  5mW  (open  squares),  and  5.7mW  (open 
circles).  The  solid  lines  are  best-ht  to  a  sigmoidal  function.  (B)  Recovery  effi¬ 
ciency  with  laser  excitation  at  720  nm  versus  the  product  of  the  illumination 
time  with  excitation  power:  3.34mW  (solid  circles ),  4.34mW  (solid  squares), 
5  mW  (open  squares),  and  5.7  mW  (open  circles).  The  solid  line  is  the  best-ht 
power  law  with  an  exponent  of  3.8  ±  0.2.  (C)  Recovery  efficiency  at  a  hxed 
illumination  time  of  16.5ms  versus  the  excitation  wavelength  at  5.7 mW  (open 
circles),  4.34mW  (filled  squares),  and  3.34mW  (filled  circles).  The  error  bars 
are  the  95%  statistical  errors  due  to  the  number  of  events  observed.  The  inset 
shows  the  photobleaching  rate  1/Tb  versus  excitation  power  for  the  two  states 
(A,  open  squares ;  B,  filled  squares).  (After  Chirico  el  al.,  2004.) 


range.  A  subtle  interplay  between  different  electronic  coupling 
mechanisms  is  responsible  for  an  efficient  transfer  of  the  absorbed 
energy  to  the  reaction  center.  Recently,  a  study  of  photobleaching 
and  energy  transfer  in  single  phycoerythrocyanin  (PEC)  monomer 
^  has  been  presented  by  Zehetmayer  and  colleagues  (2002).  The 
o  PEC  monomer  contains  two  different  chromophores,  phycoviolo- 
£  bilin  (PVB)  and  phycocyanobilin  (PCB)  (Zhao  and  Scheer,  1995). 
g  Zehetmayer  and  co-workers  (2002)  recorded  single-molecule 
g  images  of  phycoerythrocyanin  monomers.  Their  photobleaching 
behavior  was  studied  by  simultaneously  exciting  at  two  wave¬ 
lengths  according  to  a  method  earlier  applied  on  the  E222Q  GFP 
mutant  by  the  same  group  (Jung  et  al.,  2001).  The  PVB  chro- 
mophore  was  found  to  be  responsible  for  the  photobleaching  of 
PEC.  On  the  other  hand,  it  was  possible  to  ascertain  that  the  15E 
form  of  PVB  corresponds  to  one  of  the  short-lived  dark  states  of 
PEC.  This  form  does  not  induce  real  photobleaching  but  simply 
reduces  the  average  fluorescence  emission,  as  is  the  case  in  most 
of  the  GFP  mutants.  The  difficulty  in  discriminating  between  non- 
reversible  photobleaching  and  long-term  blinking  therefore  seems 
to  be  ubiquitous  in  single-protein  photophysics. 

CONCLUSION 

It  is  not  easy  to  indicate  conclusions  that  define  an  optimal  strategy 
for  either  reducing  or  exploiting  the  photobleaching  process.  We 
hope  that  the  facts  reported  in  this  chapter  could  be  useful  to  design 
new  experiments  or  to  revise  old  ones  with  the  aim  of  a  better  under¬ 
standing  of  the  delicate,  intricate,  and  complex  structure-function 
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relationship  that  is  the  basis  of  our  job  as  microscopists  or,  more 
generally,  as  biophysicists.  We  decided  to  attack  photobleaching  by 
considering  both  its  imaging  and  its  single-molecule  aspects 
because  we  think  that  feedback  between  these  related  aspects  of  the 
problem  can  greatly  improve  our  knowledge  of  photobleaching. 
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INTRODUCTION 

In  recent  years,  light  microscopy,  particularly  fluorescence 
microscopy,  has  been  extensively  useful  in  the  study  of  living  cells 
and  tissues.  Although  it  has  become  an  important  tool  in  biologi¬ 
cal  research,  both  single-  (Sheppard  and  Shotton,  1997)  or  two- 
photon  (Denk  et  al,  1990;  Cheng  et  al.,  1998,  2001)  excitation 
schemes  require  that  the  specimen  contain  either  intrinsic  or  extrin¬ 
sic  fluorescent  probes.  These  probes  include  fluorescent  dyes,  fluo¬ 
rescent  proteins,  and  quantum  dots  and  common  problems 
include  probe  penetration,  probe  toxicity,  and  photobleaching/ 
damage  (Konig,  1995;  Cheng  et  al.,  2001a).  To  be  useful,  the  flu¬ 
orescent  probes  must  usually  be  bound  to  specific  biostructures  or 
molecules,  either  by  partition  of  the  dye  into  various  compart¬ 
ments,  antigen-antibody  reactions,  affinity  or  site-specific  binding 
of  dye(s),  or  the  transgenic  expression  of  fluorescent  and  lumi¬ 
nescent  proteins.  In  addition,  the  probe  may  interact  with  the  ionic 
environment  to  give  a  specific  fluorescence  signature.  In  all  these 
cases,  the  fluorescence  signals  are  related  to  the  chemistry  of  the 
dye  and  the  chemistry  of  the  interaction  between  the  dye  and  the 
cell  or  tissue  or  the  genetic  expression  of  the  probe.  Therefore,  the 
term  “chemical”  and  “biochemical”  contrast  is  frequently  used  to 
describe  the  fluorescence  imaging  modality. 

On  the  other  hand,  nonlinear  imaging  modalities  exist  that 
create  contrast  based  on  the  organization  and  orientation  of  nanos¬ 
tructures  in  the  specimen,  and  these  can  often  provide  an  alterna¬ 
tive  tool  for  studying  the  dynamics  of  cellular  structures  and 
functions.  These  nonlinear  imaging  modalities  include  second  and 
third  harmonic  generation  (SHG  and  THG),  in  which  specific 
structural  features  of  the  optical  configuration  of  the  native  speci¬ 
men  generate  prompt  signals.  In  this  case,  signal  is  generated  by 
nonlinearity  in  the  physical  properties  of  the  specimen  (such  as 
short-period  modulations  or  discontinuities  in  its  refractive  index, 
RI)  particularly  those  that  occur  at  very  high  excitation  intensity, 
such  as  that  present  at  the  focus  point  of  an  objective  lens  illumi¬ 
nated  with  high-intensity,  femtosecond  pulsed  laser.  Unlike  fluo¬ 
rescence,  these  harmonic  signals  are  generated  with  no  time  delay 
at  all  and  emerge  at  wavelenghts  that  are  exact  integer  sub¬ 
multiples  of  the  excitation  wavelength  and  traveling  in  the  same 
direction  (forward). 

Recent  studies  on  man-made  nanoperiodic  structures  (e.g., 
super-lattices)  indicate  a  strong  enhancement  in  SHG  when 
high-intensity  light  is  incident  on  non-centri-symmetric  structures 
(Zhao  et  al.,  1999;  Kao  et  al.,  2000;  Sun  et  al.,  2000).  SHG 
occurs  because  the  structure  violates  the  condition  of  optical 
centro-symmetry.  This  sort  of  breakdown  occurs  in  the  inorganic 
crystals  used  for  doubling  the  frequency  of  laser  light  (see  Chapter 


5,  this  volume).  A  typical  application  is  the  frequency  doubling 
of  1064  nm  emission  from  a  semiconductor  laser  to  produce  the 
532  nm  light  in  the  green-laser  pointer.  More  generally,  SHG 
occurs  in  many  other  structured  samples,  some  of  which  are 
biological. 

A  super-lattice  structure  is  basically  a  one-dimensional  non¬ 
linear  %(2)  photonic  crystal  (where  /(2)  is  the  second-order  nonlinear 
susceptibility),  which  is  defined  as  a  material  with  a  periodicity  in 
its  second-order  nonlinear  dielectric  properties  (Berger,  1998). 
Strong  SHG-enhancement  has  also  been  reported  in  one-dimen¬ 
sional  and  two-dimensional  nonlinear  photonic  crystals  (Broder¬ 
ick  et  al.,  2000;  Dumeige  et  al.,  2001).  The  mechanism  is  similar 
to  the  bandgap-resonant  enhancement  that  occurs  in  common 
SHG-conversion  crystals  and,  although  SHG  conversion  efficiency 
is  always  highest  near  the  nonlinear  photonic  bandgap,  it  does  not 
vanish  even  when  the  illumination  (pump)  wavelength  is  far  away 
from  the  spatial  modulation  period  (the  bandgap).  Quasi-phase- 
matching  is  an  example  of  this,  as  the  operating  wavelength  of  the 
excitation  is  much  shorter  than  the  spatial  modulation  period  of  the 
nonlinear  coefficient  /<2).  Recently,  strong  SHG  has  been  reported 
in  several  types  of  biological  material,  mainly  in  orderly  packed 
biomolecules  or  macromolecular  structures  (Chu  et  al.,  2001; 
Cheng  et  al,  2002;  Sun  et  al.,  2003).  In  contrast,  no  SHG  occurs 
in  amorphous  materials,  such  as  the  almost  randomly  arranged 
macromolecules  and  other  nanostructures  that  make  up  a  cell. 

The  enhancement  of  SHG  by  the  nanophotonic  crystals  of  the 
membrane  protein  bacteriorhodopsin  (bR),  has  recently  been 
demonstrated  using  hyper-Rayleigh  scattering  (Clays  et  al.,  2001). 
The  bR  protein  forms  a  two-dimensional  (2D)  crystal  in  the  purple 
membrane  of  Halobacterium  salinarium  (Birge  et  al.,  1990).  This 
structure  has  an  alternating  change  in  second-order  nonlinearity 
with  a  ~5nm  period,  causing  it  to  act  as  a  nonlinear  photonic 
crystal  (Berger,  1998).  A  number  of  other  highly  organized  bio¬ 
logical  nanostructures  have  been  reported  that  appear  to  break 
optical  centro-symmetry  and  behave  as  SHG-active  nonlinear 
photonic  crystals.  Such  structures  include  stacked  membranes  such 
as  those  found  in  the  myelin  sheath,  the  endoplasmic  reticulum 
(ER),  the  Golgi  apparatus,  and  the  grana  in  the  chloroplast,  micro¬ 
tubules,  cellulosic  microfibrils,  collagen  fibers,  enamel  prisms, 
bone  matrix,  starch  granules,  and  mineral  deposits  in  plants  (Chu 
et  al.,  2001;  Cheng  et  al.,  2002;  Sun  et  al.,  2003,  2004).  Similar 
to  the  backward  SHG  and  THG  detected  in  a  waveguide,  Gu  and 
colleagues  (1999)  and  Sun  and  co-workers  (2005)  reported  strong 
backward  SHG  from  microtubules,  a  fact  that  allows  imaging  the 
cytoskeleton  and  the  mitotic  spindle  in  living  tissue. 

Collagen,  the  major  protein  of  the  extracellular  matrix,  is  one 
of  the  best-known  SHG  structures  in  biology.  The  collagen  mole- 
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cule  is  a  long,  stiff,  triple- stranded  helical  structure,  in  which  three 
collagen  polypeptide  chains,  called  a-chains,  are  wound  around 
one  another  in  a  ropelike  superhelix.  After  being  secreted  into  the 
extracellular  space,  these  collagen  molecules  assemble  into  high- 
order  polymers  called  collagen  fibrils  that  are  10  to  300  nm  in 
diameter  and  many  hundreds  of  micrometers  long.  Collagen  fibrils 
often  aggregate  into  larger,  cable-like  bundles,  several  microme¬ 
ters  in  diameter,  which  can  be  seen  in  the  light  microscope  as  col¬ 
lagen  fibers.  Under  electron  microscopy,  collagen  fibrils  have 
characteristic  cross-striations  every  67  nm,  reflecting  the  regularly 
staggered  packing  of  the  individual  collagen  molecules  in  the 
fibril.  This  regularly  staggered  packing  order  provides  the  needed 
structural  condition  for  efficient  SHG  (Williams  et  al.,  2005;  Sun 
et  al.,  2004). 

The  SHG  signal  is  created  in  proportion  to  the  square  of  the 
instantaneous  local  intensity  (Gannaway  and  Sheppard,  1978).  As 
a  result,  like  two-photon  fluorescence  (2PF)  microscopy  (Denk  et 
al.,  1990;  Cheng  et  al,  1998),  SHG  provides  intrinsic  axial  reso¬ 
lution.  SHG  microscopy  was  first  demonstrated  in  studies  of  SHG 
photonic  crystals  (Gannaway  and  Sheppard,  1978),  surfaces/inter¬ 
faces  (Shen,  1989),  electric  field  distribution  in  semiconductors 
(Sun  et  al.,  2001),  and  was  also  shown  to  be  present  in  studies  of 
tissue  polarity  (Freund  et  al.,  1986;  Guo  et  al.,  1997)  and  of  mem¬ 
brane  potentials  in  cells  (Campagnola  et  al,  1999;  Peleg  et  al., 
1999;  Moreaux  et  al.,  2000b). 

Because  most  biological  structures  are  not  highly  ordered,  they 
are  optically  isotropic  and  do  not  produce  any  SHG  signal.  Only 
those  few  biological  structures  that  are  ordered  or  that  involve 
some  spatial  organization  that  breaks  the  optical  centro-symmetry 
can  produce  harmonic  signals.  In  contrast  to  fluorescence 
processes  that  involve  the  excitation  of  the  fluorescent  molecule  to 
an  excited  state  having  a  finite  lifetime,  harmonically  generated 
signals  involve  only  virtual  states  that  involve  no  time  delay.  The 
marked  advantage  of  this  virtual  transition  is  the  lack  of  energy 
deposition  in  the  specimen.  As  a  result,  SHG  produces  no  photo¬ 
damage  or  bleaching,  and  like  backscattered  light,  can  be  regarded 
as  a  truly  “non-invasive”  imaging  modality.  However,  it  is  impor¬ 
tant  to  remember  that  the  very  high  instantaneous  power  levels 
required  to  produce  the  effect  may  cause  other  novel  and  damag¬ 
ing  nonlinear  damage  mechanisms  to  occur  (see  Chapter  38,  this 
volume).  As  both  harmonic  generation  and  nonlinear  absorption 
processes  can  occur  simultaneously,  it  is  also  possible  to  produce 
photodamage  due  to  absorption  processes  if  absorbers  are  present. 
In  fact,  as  some  small  amount  of  nonlinear  absorption  always 
occurs  in  biological  specimens,  the  main  advantages  of  SHG  are 
not  just  that  the  signal  generation  process  involves  no  energy  depo¬ 
sition,  but  that  it  provides  specific  structural  information.  In  con¬ 
trast  to  the  chemical  specificity  that  characterizes  fluorescence 
imaging,  harmonic  generation  (SHG  and  THG)  provides  an 
imaging  modality  specific  for  structural  configuration. 

Discontinuities  in  RI  or  the  optical  dispersion  properties  of 
biological  tissues  can  generate  third  harmonic  generation  (THG) 
signals  (Barad  et  al.,  1997;  Muller  et  al,  1998;  Moreaux  et  al., 
2000a, b;  Sun  et  al.,  2001)  and  the  intensity  of  these  signals  is  pro¬ 
portional  to  the  third  power  of  the  illumination  intensity.  Using  the 
same  illumination  wavelength,  THG  provides  even  better  optical- 
sectioning  resolution  than  SHG  or  2PF,  but  is  also  more  sensitive 
to  changes  in  the  intensity  of  the  light  in  the  focused  spot,  such  as 
those  caused  by  laser  instability  or  by  scattering  or  defocusing  of 
the  illumination.  The  THG  can  be  used  to  study  optical  interfaces 
in  the  cell  such  as  those  at  cell  membranes  or  organelle  surfaces. 
For  example,  the  surface  of  the  erythrocyte  can  generate  signifi¬ 
cant  THG  (Sun  et  al.,  2004). 


HARMONIC  GENERATION 

Shortly  after  the  first  demonstration  of  the  laser  by  Maiman  in 
1960,  the  next  year  Franken  and  co-workers  discovered  the  process 
of  SHG  in  man-made  materials.  This  work  is  often  taken  as  the 
beginning  of  the  field  of  nonlinear  optics.  Nonlinear  optical  phe¬ 
nomena  are  “nonlinear”  in  the  sense  that  they  occur  when  the 
response  of  a  specimen  depends  upon  the  strength  of  the  optical 
electric  field  in  a  nonlinear  manner.  For  example,  SHG  occurs  as 
a  result  of  that  part  of  the  atomic  response  to  the  oscillating  field 
of  the  light  that  depends  quadratically  on  the  strength  of  this  optical 
field.  Consequently,  the  intensity  of  the  signal  generated  by  the 
SHG  mechanism  increases  as  the  square  of  the  intensity  of  the  inci¬ 
dent  light.  By  the  same  token,  THG  signals  vary  with  the  cube  of 
the  intensity  of  the  incident  light. 

An  image  is  a  two-dimensional  recording  of  the  interaction 
between  light  and  a  specimen,  and  therefore  is  a  representation  of 
the  optical  properties  of  the  specimen.  In  conventional  optical 
imaging,  contrast  mechanisms  include  interactions  such  as  absorp¬ 
tion,  reflection,  scattering,  and  fluorescence,  and  the  response 
recorded  is  linearly  dependent  on  the  intensity  of  the  incident  light. 
In  the  nonlinear  optical  domain,  high-intensity  light  causes  a 
variety  of  “anomalous”  optical  responses,  and  therefore  the  image 
contrast  mechanism  represents  not  only  the  differences  in  optical 
properties  of  the  specimen,  but  also  the  result  of  the  modification 
of  those  optical  properties  that  depend  on  the  intensity  of  the  light 
in  a  nonlinear  manner.  In  the  case  of  conventional  (linear)  optics, 
a  linear  relationship  exists  between  the  electric  field  strength  of  the 
light  and  the  induced  polarization  of  the  object.  At  relatively  low 
incident  intensity,  the  optical  response  can  be  approximated  to  be 
the  first-order  response  as: 

P(t)  =  E0XmE(t)  (1) 

where  %(I)  is  the  linear  susceptibility,  P(t)  is  the  dipole  moment  per 
unit  volume,  or  polarization  of  a  material  system,  E(t)  is  the 
strength  of  an  applied  optical  field,  and  CO  is  the  frequency. 

Second  Harmonic  Generation 

In  contrast,  at  high  incident  intensity,  the  nonlinear  optical 
response  can  be  described  by: 

Pit)  =  £oXmE(t)  +  e0Xa)E(t)2  +  EoX  &E{tf  +  ... 

=  +  P2)(t)  +  P®(f)  +  . . .  (2) 

where  j((2)  is  the  second-order  nonlinear  susceptibility  and  /(3)  is 
the  third-order  nonlinear  susceptibility. 

In  the  above  equation,  because  the  fields  are  vectors,  the  non¬ 
linear  susceptibilities  are  tensors.  As  each  atom  acts  as  an  oscil¬ 
lating  dipole  that  radiates  in  a  dipole  radiation  pattern,  the  radiation 
phase  among  the  enormous  number  of  atoms  must  be  matched  to 
induce  constructive  interference  and  thus  nonlinear  generation  is 
allowed  under  phase-matching  conditions  (i.e.,  when  the  scattered 
light  is  in  phase).  This  leads  to  the  generation  of  radiation  at  the 
second  harmonic  frequency  (half  the  wavelength  of  the  illumina¬ 
tion).  However,  this  situation  does  not  lead  to  the  generation  of 
electromagnetic  radiation  because  its  second  time  derivative  van¬ 
ishes  and  a  static  electric  field  is  created  within  the  nonlinear 
crystal. 

Second  harmonic  generation  can  also  be  considered  as  an 
interaction  involving  the  exchange  of  photons  between  the  various 
frequency  components  of  the  field.  In  SHG,  two  photons  of  fre¬ 
quency  to  are  destroyed  and  one  photon  of  frequency  2(0  is  simul¬ 
taneously  created  in  a  single  quantum-mechanical  process.  The 
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FIGURE  40.1.  Energy  state  diagram.  Both  SHG  and  THG  involve  virtual  transitions  in  which  no  energy  is  absorbed  by  the  specimen.  In  contrast,  two-photon 
fluorescence  (2PF)  involves  the  absorption  of  energy  (real  transitions)  and  excitation  of  molecules.  The  wavelength  of  SHG  is  half  and  THG  is  one  third  that  of 
the  incident  wavelength,  while  2PF  has  an  emission  wavelength  more  than  half  of  the  incident  wavelength. 


solid  lines  in  Figure  40. 1  represent  the  atomic  ground  states,  and 
the  dashed  lines  represent  virtual  levels.  These  levels  are  not 
energy  eigenlevels  of  the  free  atom,  but  rather  represent  the  com¬ 
bined  energy  of  one  of  the  energy  eigenstates  of  the  atom  and  of 
one  or  more  photons  of  the  radiation  field. 

The  fact  that  SHG  vanishes  in  any  material  system  that  pos¬ 
sesses  centro-symmetry  (i.e.,  one  that  has  a  center  of  inversion) 
can  be  explained  by  changing  the  sign  of  the  applied  electric  field 
in  Eq.  2.  In  a  medium  possessing  inversion  symmetry,  the  sign  of 
the  induced  polarization  must  also  change.  Hence,  relation  2  is 
replaced  by 

-. Pa\t )  =  £o)C<2'[-E(/)]2  =  eo)C(2)£(02  (3) 

By  comparing  Eq.  3  with  Eq.  2,  Pit)  must  equal  - P(t ),  and  this 
can  occur  only  if  Pit )  vanishes  identically.  So  %<2)  is  equal  to  zero 
for  centro-symmetric  media  and  no  SHG  signal  is  expected. 
Because  the  molecules  making  up  most  biological  materials  are 
oriented  more  or  less  “at  random”  (compared  to  a  crystal,  for 
example),  they  generate  no  SHG  signals.  The  phase-matching 
condition  in  SHG  is  that  the  wave  vector  difference  between 
input  excitation  light  and  output  harmonic  light  is  zero.  Therefore, 
phase-matching  is  seriously  affected  by  polarization,  and  SHG  is 
sensitive  to  the  angle  between  the  polarization  of  the  incident  light 
and  the  symmetry  condition  of  the  material.  Pol-dependent  SHG 
can  provide  information  about  crystal  orientation  and  imperfec¬ 
tion,  macromolecular  structure  and  orientation,  and  regions  where 
the  centro-symmetry  either  breaks  down,  such  as  at  surfaces  and 
optical  interfaces  or  where  it  is  induced  by  organized,  submicron 
structures.  It  can  also  be  created  by  large  localized  residual 
electric  fields,  such  as  those  across  electrically  polarized  cell 
membranes. 

Third  Harmonic  Generation 

Third  harmonic  generation  involves  a  process  whereby  three 
photons  of  frequency  to  are  destroyed  and  one  photon  of  frequency 
3(0  is  created.  A  nonlinear  contribution  to  the  polarization  of  the 
medium  at  the  frequency  of  the  incident  field  leads  to  a  nonlinear 
contribution  to  the  refractive  index  experienced  by  a  wave  at  fre¬ 
quency  CO. 


Normally,  the  tightly  focused  condition  in  a  microscope  setup 
inhibits  the  production  of  third  harmonic  signals  (Barad  et  al., 
1997;  Berger,  1998),  and  a  positive  wave  vector  (A k  =  3 ka  -  fc3(0  > 
0)  mismatch  between  the  fundamental  and  harmonic  beam  is 
required  to  achieve  phase-matching.  Under  tight  focusing  condi¬ 
tions,  third  harmonic  production  is  allowed  if  an  optical  interface 
is  present  within  the  focal  volume,  no  mismatch  is  required,  and 
the  resulting  breakdown  in  focal  volume  symmetry  enables  effi¬ 
cient  THG  generation  for  A k  <  0  (normal  dispersion). 

When  one  calculates  the  third  harmonic  power  as  a  function 
of  the  interface  uniformity,  there  is  a  change  in  either  the  refrac¬ 
tive  index  or  in  the  third-order  nonlinear  susceptibility,  and  this 
has  the  result  that  the  third  harmonic  power  does  not  vanish.  Barad 
and  colleagues  (1997)  demonstrated  that,  as  a  result  of  this  inter¬ 
face  effect,  imaging  with  the  third  harmonic  is  possible  and  is  espe¬ 
cially  suitable  for  transparent  phase  specimens  with  low  intrinsic 
contrast  such  as  the  cell  boundary.  The  THG  signal  strength  is 
sensitive  to  changes  in  the  nonlinear  optical  properties  of  the 
specimen. 

Multi-Photon  Absorption  and  Fluorescence 

Because  in  2PF,  the  absorbed  power  is  proportional  to  the  square 
of  the  incident  light  intensity  (i.e.,  as  if  it  depended  on  the  square 
of  the  photon  concentration),  this  process  is  called  two-photon 
absorption.  The  absorbed  two-photon  energy  can  excite  a  mole¬ 
cule  to  its  excited  state  or  even  to  ionization  (two-photon  ioniza¬ 
tion).  In  the  process  of  two-photon  fluorescence,  an  atom/molecule 
makes  a  transition  from  its  ground  state  (S0)  to  a  real  excited  state 
(S2)  by  the  simultaneous  absorption  of  two  laser  photons  and 
quickly  transits  to  a  metastable  state  (Si)  and  then  emits  one  photon 
with  higher  frequency  (vem  but  less  than  2v„)  while  returning  to 
the  ground  state  (Si  — >  S0).  The  transition  between  the  first  excited 
state  and  the  metastable  state  can  produce  chemical  events  that 
lead  to  photobleaching,  a  process  that  removes  the  fluorescent 
species  from  the  fluorescing  pool  (see  Chapter  39,  this  volume). 
Moreover,  the  excess  absorbed  energy  that  is  lost  as  vibrational 
energy  during  the  transition,  can  result  in  photodamage,  and  this 
process  may  become  even  worse  as  the  energy  difference  between 
these  two  states  increases. 
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LIGHT  SOURCES  AND  DETECTORS  FOR 
SECOND  HARMONIC  GENERATION  AND 
THIRD  HARMONIC  GENERATION  IMAGING 

Because  SHG  and  THG  signal  strength  is  quadratically  or  third- 
power  dependent  on  the  intensity  of  the  illumination,  high  illumi¬ 
nation  intensity  is  required  for  the  efficient  generation  of  these 
signals.  As  continuous  illumination  (CW)  at  such  high  intensity 
would  result  in  unacceptably  high  energy  deposition,  one  instead 
uses  the  high  instantaneous  power  density  that  can  be  achieved 
using  a  mode-locked,  ultra-fast  pulsed  laser.  Four  such  laser 
systems  (Table  40.1)  are  currently  available  for  use  in  nonlinear 
microscopy:  the  titanium: sapphire  (Ti:Sa)  (various  suppliers),  the 
Cr:forsterite  laser  (Avista  Project,  Russia;  Ming-mei  Technology, 
Taiwan,  Republic  of  China),  the  Nd:glass  laser  (Time-Bandwidth 
Products,  Switzerland),  and  the  femtosecond  ytterbium  laser 
(Amplitude  System,  Talence,  France). 

The  Ti:Sa  has  a  tuning  range  of  about  700  to  980 nm  [near- 
infrared  (NIR)]  and  is  currently  used  extensively  in  two-photon 
fluorescence  microscopy.  The  SHG  wavelength  excited  by  a  Ti:Sa 
femtosecond  laser  operating  at  810nm  will  be  in  the  deep  blue 
at  405  nm,  while  2PF  will  be  in  the  visible  region  (>405  nm). 
However,  the  THG  signal  generated  by  a  810nm  primary  beam 
will  fall  at  270  nm,  in  the  deep  ultraviolet  (UV)  region,  while  three- 
photon  fluorescence  (3PF)  will  be  found  at  >270  nm.  Conse¬ 
quently,  the  THG  and  3PF  signals  will  suffer  from  the  high  UV 
absorption  of  most  biological  specimens  (Fig.  40.2;  Cheng  et  al., 
2001a;  Lin  et  al. ,  2001),  making  signal  detection  difficult.  In  addi¬ 
tion,  one  would  need  special  (and  expensive!)  UV  objectives, 
mirrors  and  photomultiplier  tubes  (PMT)  in  the  detecting  path. 

In  contrast,  the  Cr:forsterite  laser  operates  in  the  range  of  1230 
to  1270  nm  in  the  IR  spectrum,  making  it  an  excellent  light  source 
for  multi-modality  microscopy  (Bouma  et  al. ,  1996;  Chu  et  al., 
2002).  For  example,  operating  a  Cr:forsterite  laser  at  1230nm 
allows  SHG  (615 nm),  THG  (410nm),  2PF  (>615 nm),  and  3PF 
(>410nm),  all  to  fall  within  the  visible  spectrum.  In  addition,  the 
lowest  light  attenuation  in  biological  material  is  generally  found 


TABLE  40.1.  Pulsed  Lasers  Suitable  for  Generating  SHG 
and  THG  Signals 


Laser  System 

Wavelength 
Range  (nm) 

Pulse 

Width  (fs) 

Repetition 
Rate  (Hz) 

Ti:Sa 

700-980 

100 

76-1 00  MHz 

2 GHz,  Chu  et  al., 
2003b 

Nd:glass 

1053-1064 

150 

70-150MHz 

Cr:forsterite 

1230-1270 

65 

76-1 20  MHz 

Diode-pumped 

ytterbium 

1030 

200 

50  MHz 

in  the  1000  to  1300nm  range  (Fig.  40.2).  By  moving  the  excita¬ 
tion  wavelength  to  1230nm,  not  only  the  visible  but  also  the  NIR 
spectrum  is  open  for  signal  recording.  This  could  be  important  in 
imaging  botanical  specimens  that  have  a  high  autofluorescence 
output  over  nearly  the  entire  visible  spectrum  (see  Fig.  20.2,  this 
volume).  The  fact  that  autofluorescence  diminishes  at  wavelengths 
longer  than  the  fluorescence  emission  of  chlorophylls  makes  NIR 
fluorescent  probes,  such  as  AlexaFluor  750,  Cy5.5  and  Cy7  (Mol¬ 
ecular  Probes  Inc.,  Eugene,  OR),  very  attractive.  Development  of 
endogenous  fluorophore  mutants  (i.e.,  long-wavelength  fluorescent 
proteins)  having  emission  wavelengths  longer  than  DsRed  will  be 
helpful  for  the  study  of  botanical  specimens.  Liu  and  colleagues 
(2001)  and  Cheng  and  co-workers  (2001)  have  demonstrated  that 
a  number  of  commonly  used  fluorescent  probes  can  be  efficiently 
excited  at  1230nm  by  2PF  and  3PF. 

On  the  other  hand,  mode-locked  femtosecond  Nd:glass  and 
diode-pumped  ytterbium  lasers  provide  an  alternative  choice.  A 
typical  femtosecond  Nd:glass  system  offers  a  tuning  range  of  1053 
to  1064nm,  with  a  repetition  rate  of  70  to  150  MHz  and  a  150fs 
pulse  width  (Time-Bandwidth  Products,  Zurich).  This  laser  will 
produce  SHG  signal  at  526  to  532  nm,  which  is  at  the  second  atten¬ 
uation  minimum  of  green  botanical  specimens  (Fig.  40.2).  The 
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FIGURE  40.2.  Light  attenuation  spectra 
(absorption  and  scattering)  in  rice  leaf  (Oryza 
saliva )  and  chicken  dermis.  Note  the  operat¬ 
ing  ranges  of  Ti:Sa,  ytterbium,  Nd:glass,  and 
Cr:Forsterite  lasers.  The  light  green  lines 
indicate  a  typical  emission  wavelength  of 
Cr:Forsterite  (1230nm)  and  its  corresponding 
SHG  (615nm)  and  THG  (410  nm).  The  blue 
lines  represent  a  typical  Ti:Sa  emission  wave¬ 
length  (780  nm)  and  the  corresponding  SHG 
(390nm)  and  THG  (260nm)  wavelengths. 
The  purple  lines  indicate  the  emission  line  of 
Nd:glass  and  corresponding  SHG  (532nm) 
and  THG  (352  nm)  lines.  The  dark  green  line 
indicates  the  emission  line  of  ytterbium  laser 
(1030nm).  The  curves  labeled  S1336  and 
G8198  represent  the  sensitivity  of  typical 
visible  and  NIR  photodiode  detectors. 
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THG  is  situated  at  351  to  354  nm.  Because  both  the  excitation  and 
emission  wavelength  are  optimized  at  an  attenuation  minimum, 
using  Ndtglass  greatly  increases  the  imaging  depth  in  tissue,  par¬ 
ticularly  green  plant  tissue.  Using  this  laser  system,  one  can  obtain 
useful  3PF,  2PF,  and  SHG  simultaneously  and  it  has  the  added 
advantage  that  many  commercially  available  IR  objectives  have 
reasonable  transmittance  at  the  1064nm  wavelength.  On  the  other 
hand,  the  diode-pumped  ytterbium  laser  operates  at  1030nm,  with 
SFIG  and  THG  at  515nm  and  343  nm,  respectively. 

Figure  40.2  summarizes  the  operating  wavelengths  and  the 
SHG  and  THG  signals  of  the  four  mode-locked  ultra-fast  lasers  in 
reference  to  the  attenuation  spectra  of  biological  specimens  and 
detector  sensitivity.  In  the  near  UV  wavelength  region,  attenuation 
is  dominated  by  the  strong  absorption  of  common  tissue  con¬ 
stituents  such  as  proteins  and  carbohydrates.  For  IR  wavelengths 
longer  than  1400  nm,  however,  the  molecular  resonance  absorption 
of  water  starts  to  dominate  the  attenuation.  As  a  result,  biological 
specimens  have  a  relatively  transparent  window  between  350  to 
1300  nm. 

Although  the  Ti:Sa  laser,  operating  at  around  720-950 nm,  has 
been  widely  adopted  as  the  preferred  light  source  for  multi-photon 
microscopy,  one  can  easily  see  that  the  attenuation  minimum  is 
around  1000  to  1300 nm,  not  in  the  Ti:Sa  range.  In  fact,  this  should 
not  be  surprising  because  scattering  cross-sections  become  smaller 
as  the  wavelength  increases.  This  plot  is  also  in  good  agreement 
with  previous  measurements  of  the  attenuation  coefficients  of  bio¬ 
logical  materials  such  as  human  skin  (Anderson  and  Parrish, 
1981),  maize  stem  (Cheng  et  al.,  1998),  and  leaves  (Lin  et  al., 
2001). 

The  advantage  of  the  Ti:Sa  laser  is  its  large  tuning  range  com¬ 
pared  to  the  Cr:forsterite  and  Nd:glass  lasers.  In  addition,  while 
most  objective  lenses  work  well  in  the  NIR  spectrum  (up  to 
900  nm)  and  some  IR  objectives  have  reasonable  transmission  up 
to  -llOOnrn,  obtaining  reasonable  transmission  and  optical  cor¬ 
rection  in  the  1230  nm  range  requires  special  objective  lenses  and 
other  optical  components  (see  Chapter  7,  this  volume). 


How  do  the  signals  produced  by  these  four  lasers  match  the 
performance  of  the  available  detectors?  Because  the  Ti:Sa  wave¬ 
lengths  are  well  within  in  the  sensitivity  range  of  both  high 
quantum  efficiency  (QE)  silicon-based  detectors  and  most  PMT 
photocathodes,  any  scattered  excitation  illumination  will  be 
detected  by  these  detectors  and  recorded  as  background.  Although 
this  reduces  the  signal-to-noise  (S/N)  ratio  of  fluorescence  signals, 
it  also  affords  the  benefit  of  being  able  to  detect  the  backscattered 
light  (BSL)  signal  as  an  additional,  non-damaging  imaging 
modality. 

On  the  other  hand,  because  the  emission  wavelengths  of 
Nd:glass  and  Cr:forsterite  lasers  are  beyond  the  sensitivity  of  most 
of  the  commonly  used  PMT  and  silicon  detectors  (Table  40.2), 
detector  insensitivity  guarantees  a  low  background  level  from  scat¬ 
tered  light  but  sacrifices  availability  of  BSL  modality  unless  a 
special  NIR  detector  (e.g.,  InGaAs  photodiode,  Edmund  Scientific) 
is  installed.  Table  40.2  lists  the  sensitivity,  the  sensitivity  range, 
and  the  wavelength  of  maximum  sensitivity  for  each  of  the  pho¬ 
todetectors  commonly  used  in  microscopy  (see  also  Chapter  12, 
this  volume).  Because  the  peak  intensity  of  the  ultra-fast  laser  is 
so  high,  a  dichroic  beam-splitter  and  a  barrier  filter  may  not  be  suf¬ 
ficient  to  exclude  the  background  signal  caused  by  light  leakage. 
This  is  a  particular  problem  when  collecting  SHG  and  THG  signals 
in  the  transmission  mode  where  the  excitation  striking  the  emis¬ 
sion  filter  may  be  107X  brighter  than  the  SHG/THG  signal.  In  this 
case,  using  detectors  insensitive  to  the  illumination  wavelength  can 
further  limit  the  background  signal  level,  improving  the  S/N  of  the 
image. 

Using  a  Cr:forsterite  or  Nd:glass  laser  at  1230nm/1064nm  for 
nonlinear  microscopy  allows  one  to  fully  utilize  the  transparent 
window  in  most  biological  specimens.  At  1230nm,  Chu  and  col¬ 
leagues  found  that  the  SHG  and  2PF  signal  dropped  by  only  1 
order  of  magnitude,  when  generated  at  a  depth  of  360  |im  in  a 
maize  stem  fixed  in  70%  ethanol  (Chu  et  al.,  2001).  This  is  in  good 
agreement  with  previous  light  attenuation  measurements  of  maize 
stems  (Cheng  et  al.,  1998).  Comparing  the  attenuation  spectra  of 


TABLE  40.2.  Characteristics  of  Photodetectors  Used  for  SHG  and  THG  Microscopy 


Type 

Photocathode 

Composition 

Photoemission 
Threshold  (nm) 

Wavelength  at 
Maximum 
Sensitivity  (nm) 

Radiant  Sensitivity 
at  ^max  (mA/W) 

Quantum  Efficiency 
at  Kk  (%) 

SI 

AgOCs 

1100 

800 

2.3 

0.4 

S4 

SbCs3 

680 

400 

50 

16 

Sll 

SbCs3 

700 

440 

80 

22 

S13 

S20 

SbNa2KCs 

850 

420 

70 

20 

S20 

SbNa2KCs 

900 

550 

35 

8 

(extended-red 

multi-alkal) 

Bialkali 

SbKCs 

630 

400 

90 

28 

Bialkali  (green 

SbKCs 

700 

440 

100 

28 

extended) 

Solar  blind 

Silicon 

photodiode 

InGaAs 

photo-diode 

CsTe 

340 

Spectral  range 

320 

(190)-1100nm 
900-1700  nm 

235 

960  nm 

1550  nm 

20 

Photo  sensitivity 
(AAV) 

0.33  (A.  =  633  nm) 

0.9  (X=  1300 nm) 

0.95  (K=  1550 nm) 

10 

The  short  wavelength  limit  of  the  PMT  is  determined  by  the  window  material  (lime  glass,  300  nm;  borosilicate  glass,  250  nm;  fused  silica,  180nm.  The  short  wavelength 
sensitivity  cut-off  of  photodiode  is  also  limited  by  the  package  window  material;  number  in  parentheses  represents  fused  silica  window.  (Data  adopted  from  Photonics, 
Brive,  France  and  Hamamatsu  Inc.,  Japan.  QE  data  refer  to  the  photocathode  response  only.) 
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FIGURE  40.3.  (A)  Diagrammatic  represen¬ 
tation  of  a  nonlinear  laser-scanning  micro¬ 
scope.  The  xy-beam  scanner  can  be  removed 
(by  parking  the  xy- mirror  at  the  center  posi¬ 
tion)  from  the  system  to  allow  stage-scanning 
operation.  An  autocorrelator  can  be  intro¬ 
duced  into  the  beam  to  monitor  the  pulse 
width.  A  second  harmonic  generating  crystal 
can  also  be  introduced  into  the  beam  to  allow 
the  illumination  to  operate  at  X/2.  The  spec¬ 
trometer  can  also  be  connected  to  the  epi- 
illumination  path  to  detect  the  backward 
SHG  signal.  CCD,  charge-coupled  device;  *, 
optional. 


plant  tissue  with  that  of  a  chicken  skin  (Fig.  40.1),  over  the  emis¬ 
sion  range  of  the  Ti:Sa  laser,  one  can  expect  significantly  better 
depth  penetration  performance  from  Cr:forsterite/Nd:glass/ytter- 
bium  laser  sources  in  brain-slice  research.  This  superior  depth  per¬ 
formance  agrees  well  with  previous  studies  comparing  the 
penetration  depth  of  optical  coherence  tomography  (OCT) 
between  800  nm  and  1300nm  light  (see  Chapter  34,  this  volume; 
Bouma  et  al.,  1996). 

Moreover,  a  significant  reduction  in  photo-induced  damage 
was  observed  in  plant  materials  when  2PF  microscopy  was  con¬ 
ducted  using  a  femtosecond  Crlorsterite  laser  rather  than  a  fem¬ 
tosecond  Ti:Sa  laser,  under  similar  illumination  intensity  and 
exposure  conditions  (Chen  et  al .,  2002). 

NONLINEAR  OPTICAL  MICROSCOPY  SETUP 

Figure  40.3  shows  a  diagrammatic  representation  of  a  typical 
imaging  system  for  multi-modal,  nonlinear  microscope  using 
either  Cnforstcrite,  Nd:glass,  ytterbium,  or  Ti:Sa  lasers  (Cheng 
et  al,  2001b).  In  our  setup,  the  laser  beam  profile  is  first  shaped 
with  a  spatial  filter  and  collimated  by  a  beam  expander  to  fill  the 
back  aperture  of  the  objective  lens.  The  collimated  beam  was 
coupled  into  an  Olympus  1X71  microscope  through  the  confocal 
scanning  unit  with  a  45°  dichroic  beam-splitter  (Chroma 
970dcspxr)  to  reflect  the  IR  and  transmit  visible  light.  The  origi¬ 
nal  aluminum  coating  on  the  xy-scanning  mirrors  was  replaced 
with  a  silver  coating  to  enhance  IR  reflectivity.  A  second  beam  path 
was  constructed  to  bypass  the  scan  mirrors  for  fixed-beam,  stage¬ 
scanning  operation.  The  excitation  IR  light  is  focused  onto  the  bio¬ 
logical  sample  with  a  spot  size  close  to  diffraction  limit  using 
high-NA  objectives  [such  as  Olympus  ULWD  MIR  80x/0.75;  or 
UPlanApo  water-immersion  60x/1.20]  and  the  excited  photoemis¬ 
sion  spectrum  was  collected  using  an  opposing  high-NA  objective 


(of  the  same  type  as  the  illuminating  objective).  The  fundamental 
IR  beam  was  removed  with  an  infrared-blocking  filter  in  conjunc¬ 
tion  with  a  dichroic  beam-splitter  (Chroma  970dcspxr).  The  col¬ 
lected  light  (visible  and  NIR)  was  then  directed  into  a  spectrometer 
(SpectraPro-2300i,  Roper  Scientific)  and  recorded  by  a  Peltier- 
cooled  charge-coupled  device  (CCD)  detector  (DV42-0E,  Andor 
Technology).  Transmission  detection  is  used  because  most  SHG 
and  THG  is  emitted  in  the  forward  direction  (i.e.,  in  the  same  direc¬ 
tion  as  the  pump  beam;  Cheng  and  Lin,  1990). 1  In  addition,  a 
mechanical  scanning  stage  is  included  to  permit  stable  raster  scan¬ 
ning  so  that  xyzAAmages  can  be  obtained.  Is  these  images,  a 
detailed  nonlinear  spectrum  is  recorded  at  each  pixel  (xy-plane  is 
defined  as  the  plane  perpendicular  to  the  laser  propagation  direc¬ 
tion).  The  signal  spectrum  is  obtained  using  a  scanning  spectrom¬ 
eter  equipped  with  an  InGaAs  photodiode  (Edmund  Scientific). 
When  beam  scanning,  a  flat  mirror  is  placed  in  the  back-focal  plane 
of  the  collector  objective  to  send  the  SHG/THG  signal  back  up  the 
optical  path  so  that  it  can  be  de-scanned  and  detected  by  the  PMTs 
(the  same  setup  as  the  transmission  confocal  microscope  method 
discussed  in  Chapter  8,  this  volume).  To  obtain  an  image  with  a 
set  signal  wavelength,  we  used  a  simple  scanning-monochrometer 
method,  adopted  from  astronomy.  The  actual  system  is  shown  in 
Figure  40.4. 

Although  it  is  possible  to  select  harmonic  signals  using  narrow 
bandpass  interference  filters,  it  is  important  to  take  special  pre¬ 
cautions  if  the  bandpass  filter  is  used  to  isolate  the  SHG  signal 
from  nrulti-photon-excited  fluorescence.  In  certain  types  of  speci¬ 
men,  where  the  fluorescence  emission  wavelength  generated  by 


1  Recent  results  have  shown  that,  in  certain  biological  structures,  backward 
SHG  signals  can  be  even  stronger  than  forward  SHG  signals  (Sun  et  al., 
2005). 
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FIGURE  40.4.  The  nonlinear  microscope  setup  in  the  laboratory  of  one  of  the  authors  (PCC)  at  the  Department  of  Biological  Sciences,  National  University  of 
Singapore  (A,  B).  The  system  is  based  on  a  mode-locked,  ultra-fast  Crforsterite  laser  (65 fs;  120MHz;  tuning  range,  1230-1270nm)  (C)  and  an  Olympus 
Fluroview  FV300  confocal  scanning  unit  (D).  1,  transmission  detector  fiber  connection;  2,  Crforsterite  laser;  3,  SF1  generator;  4,  beam  scanner;  5,  autocorrela¬ 
tor;  6,  transmission  detector;  7,  pump  input  for  Crforsterite  laser. 


three-photon  excitation  coincides  with  the  SHG  wavelength,  spec¬ 
tral  contamination  can  be  serious  even  when  a  narrow-bandpass 
SHG  isolating  filter  is  used.  Figure  40.5(A)  shows  a  fluorescence 
spectrum  when  LysoTracker  Red  (Molecular  Probes,  Inc.,  L7528) 
is  excited  by  Crforsterite  laser  at  1240nm.  If  a  biological  speci¬ 
men  is  stained  with  the  dye  and  imaged  simultaneously  in  SHG 
and  fluorescence  mode,  the  expected  SHG  signal  is  at  620  nm  and 
THG  at  413  nm,  but  the  multi-photon-excited  fluorescence  (both 
2PF  and  3PF)  of  LysoTracker  Red  extends  from  -580 nm  to 
720  nm.  Therefore,  even  a  very  narrow  bandpass  filter  centered  at 
620  nm  will  allow  significant  fluorescence  contamination  in  the 
SHG  signals,  particularly  if  the  SHG  is  relatively  weak. 

The  best  way  to  obtain  a  “pure”  SHG  signal  is  to  use  a  mono¬ 
chrometer.  Figure  40.5(B)  shows  the  emission  spectrum  of  the  red 
leaf  of  Euphorbia  pulcherrima  (poinsettia)  excited  by  1234  nm  IR, 


and  the  fluorescence  emission  consisting  of  2PF  and  3PF.  Note  the 
position  of  the  SHG  at  617  nm.  No  trace  of  SHG  can  be  recognized 
in  this  spectrum  because,  in  this  case,  the  SHG  is  in  the  forward 
direction  with  respect  to  the  illumination  beam,  while  this  spectrum 
was  recorded  from  the  same  side  as  the  incident  illumination. 

Figure  40.5(C)  shows  the  emission  spectrum  of  the  leaf  of  Zea 
mays  (corn).  The  fluorescence  emission  is  in  the  red  region  and  a 
small  peak  at  617  nm  is  the  SHG  signal  scattered  back  from  the 
leaf  tissue.  Because  the  SHG  signal  propagates  in  the  same  direc¬ 
tion  as  the  illumination  beam,  placing  detector  on  the  far  side  of 
the  specimen  will  ensure  better  signal  strength.  Figure  40.6  shows 
nonlinear  absorption  by  a  methanol  extract  from  the  yellow  petals 
of  Canna.  Readers  are  also  referred  to  the  nonlinear  absorption 
spectrum  of  the  highly  efficient  APSS  fluorophore  [(Fig.  8.5(A)]. 
Sun  and  colleagues  (2003)  have  reported  that  using  a  longer 
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FIGURE  40.5.  Two-photon  fluorescence  spectra  of  three  different  samples. 
(Top)  LysoTracker  Red  (L-752,  Molecular  Probes),  excitation  at  1240nm; 
(middle)  red  leaf  of  Euphorbia  pulcherrima  (excitation,  1234nm);  and 
(Bottom)  leaf  of  Zea  mays  (excitation,  1234  nm),  note  the  small  SHG  peak  at 
617  nm.  The  top  two  spectra  show  overlap  between  the  2PF  emission  and  that 
from  SHG. 


wavelength  excitation  (1230  nm  vs.  800  nm)  can  minimize  the  two- 
photon  autofluorescence  signal,  hence  improve  the  S/N  in  har¬ 
monic  generation  images.  However,  it  is  not  clear  if  this  reduction 
in  autofluorescence  intensity  corresponds  to  a  lower  nonlinear 
absorbance  of  the  specimen  or  a  reduction  in  the  quantum  yield  of 
the  fluorophore. 


FIGURE  40.6.  Nonlinear  absorption  of  the  methanol  extract  of  the  yellow 
petal  of  Canna.  (Measurement  was  performed  by  Ms.  Mei-Hsin  Chen,  Grad¬ 
uate  Institute  of  Electro-Optical  Engineering  and  Department  of  Electrical 
Engineering,  National  Taiwan  University,  Taipei,  Taiwan.) 

OPTICALLY  ACTIVE  BIOLOGICAL  STRUCTURES 
Optically  Active  Structures  in  Plants 

The  cell  wall  of  higher  plants  consists  of  macrofibrils,  which  are 
orderly  bundles  of  cellulose  microfibrils  each  with  a  diameter 
about  10  nm.  Within  the  microfibrils  are  micelles,  representing 
another  degree  of  highly  ordered  crystalline  structure  (Peter  et  al. , 
1992).  These  photonic-crystal-like  structures  produce  the  optical 
anisotropy  that  gives  rise  to  SHG.  Figure  40.7(A)  shows  the 
nonlinear  emission  spectra  measured  from  the  cell  wall  of  a 
parenchyma  cell  in  maize  stem  ( Zea  mays  L.).  Symmetric  THG 
and  SHG  emission  peaks  are  visible,  centered  at  410  nm  and 
615  nm,  with  an  intensity  similar  to  or  stronger  than  the  two- 
photon  autofluorescence  centered  at  680  nm.  The  strong  THG 
signal  is  induced  by  the  optical  inhomogeneity  within,  and 
surrounding,  the  cell  wall,  while  the  SHG  signal  reflects  the  highly 
organized  crystalline  structures  in  the  wall  that  break  three- 
dimensional  (3D)  optical  centro-symmetry.  Figure  40.7(B-D) 
shows  the  THG,  SHG,  and  2PF  images  made  by  detecting  wave¬ 
lengths  corresponding  to  the  peaks  in  the  spectrum  shown  in 
Figure  40.7(A).  The  source  of  SHG  is  further  confirmed  by  the 
strong  SHG  signal  obtained  from  the  stone  cell  of  pear  ( Pyrus 
serotina  R.)  (Fig.  40.8).  The  extensive  secondary  wall  develop¬ 
ment  of  the  sclerenchyma  cell  generates  significant  SHG  signals. 

The  starch  granule,  a  highly  birefringent  structure,  consists  of 
crystalline  amylopectin  lamellae  organized  into  effectively  spher¬ 
ical  blocklets  and  large  concentric  growth  rings  (Gallant  et  al., 
1997)  (Fig.  40.9).  The  crystalline  lamellae  in  starch  granules  are 
believed  to  consist  of  the  ordered,  double-helical  amylopectin  side 
chains  and  are  interleaved  with  more  amorphous  lamellae  consist¬ 
ing  of  the  amylopectin  branching  regions.  The  amylopectin  side- 
chain  clusters  within  the  crystalline  lamellae  have  varying  sizes 
but,  on  average,  are  around  lOnm  wide  by  9  to  lOnm  long  (the 
length  represents  the  thickness  of  the  lamellae).  These  orderly 
nanolayers  form  the  biophotonic  structure,  breaking  the  centro- 
symmetry  and  producing  strong  SHG.  On  the  other  hand,  starch 
granules  contain  other,  larger  structures  made  up  of  crystalline 
hard  shells  and  semi-crystalline  soft  shells  having  dimensions  of 
hundreds  of  nanometers.  These  alternating  crystalline  and  semi- 
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FIGURE  40.7.  Nonlinear  emission  spectra  measured  from  the  cell  wall  of  a  parenchyma  cell  in  maize  stem  ( Zea  mays).  (A)  The  spectrum  present  a  point  in 
the  image  of  (B);  (B)  THG  image;  (C)  SHG;  and  (D)  2PF  images. 


crystalline  rings  have  spatially-modulated  nonlinear  properties  that 
could  behave  as  3D  nonlinear  photonic  bandgap  crystals  (Berger, 
1998)  and  may  be  responsible  for  the  strong  SHG  observed.  For 
example,  the  SHG  signal  from  a  potato  {Solatium  tuberosum  L.) 
starch  granule  is  so  strong  that  is  visible  to  the  naked  eye,  even 
under  ambient  room  light  (Fig.  40.10).  This  unexpectedly  strong 
SHG  activity  may  be  the  result  not  only  of  the  superhelical  amy- 
lopectin  nanostructure  but  also  suggests  that  the  specimen  may  be 
acting  as  a  collection  of  3D  photonic  bandgap  crystals,  the  recip¬ 
rocal  lattice  basis  vectors  of  which  would  be  capable  of  producing 
SHG  by  meeting  the  non-collinear  phase-matching  condition. 
Depending  on  the  illumination  wavelength  and  the  materials  of 
which  the  specimen  is  composed,  the  spatial  frequencies  present 
in  high-order  structures  from  200  nm  to  10  pm  in  size,  can  provide 
the  non-collinear  phase-matching  base  vector  needed  to  produce 
SHG.  The  potato  specimen  is  acting  as  an  array  of  nonlinear, 
biophotonic  bandgap  crystals. 

Figure  40.11(A-C)  shows  THG,  SHG,  and  2PF  images, 
respectively,  of  a  mesophyll  cell  of  Commelina  communis  L.; 
Figure  40.11(D)  is  the  corresponding  three-channel,  false-color 


image.  The  THG  image  shows  the  interface  signal  between  the 
chloroplast  and  the  surrounding  cytoplasm  while  the  SHG  reveals 
starch  granules  and  possibly  grana  and  thylakoid  membranes  in  the 
chloroplasts  and  the  2PF  results  from  chlorophyll  autofluores¬ 
cence.  Enlarged  images  of  individual  chloroplasts  are  shown  in 
Figure  40.11(E-H). 

The  distribution  of  chlorophyll  inside  the  chloroplasts  is  shown 
by  the  2PF  signals  while  THG  signal  provides  information  on 
various  sub-organelle  interfaces.  SHG  signal,  on  the  other  hand, 
indicates  the  presence  of  nano-organized  biophotonic  structures  in 
the  chloroplast.  By  matching  the  SHG  image  with  transmission 
electron  microscope  (TEM)  images  of  similar  specimens,  it  has 
been  possible  to  conclude  that  the  SHG  signals  are  produced  by 
stacked  thylakoid  membranes  present  in  the  grana  (crescent 
shaped)  and  by  highly  birefringent  starch  granules  (oval  or  round 
shaped)  in  the  chloroplasts  (Gunning  and  Steer,  1996).  The  stacked 
thylakoid  membranes  of  the  grana  and  the  orderly  deposits  of  amy- 
lopectin  in  the  starch  granules  provide  the  structural  requirement 
for  efficient  SHG,  again  resembling  the  behavior  of  photonic 
crystals. 
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FIGURE  40.8.  Spectrum  of  the  stone  cell  of  a  pear  fruit  ( Pyrus  serotina  R.) 
recorded  with  0.1 -s  integration  time. 
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FIGURE  40.9.  Conventional  polarization  microscopy  of  maize  starch  gran¬ 
ules.  The  image  was  taken  with  an  oblique-illuminated  microscope  equipped 
with  cross-polarizer.  Note  the  strong  birefringence  of  the  starch  granules. 
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FIGURE  40.10.  Photograph  showing  the  red  SHG  signal  generated  from  a  thin  slice  of  potato  tuber  {blue  arrow)  by  a  focused  laser  beam  at  a  wavelength  of 
1230nm  and  projected  onto  a  white  paper  {orange  arrow)  (A).  As  this  photograph  was  taken  with  a  conventional  camera,  which  is  insensitive  to  1230nm  light, 
the  small  red  spot  on  the  potato  slice  is  the  scattered  SHG  signal.  The  corresponding  setup  is  schematically  shown  in  (B).  The  emission  spectrum  is  shown  in  (C). 


FIGURE  40.1 1 .  Harmonic  and  fluorescence  images  of  a  mesophyll  cell  of  Commelina  communis  L.  (A)  THG  image;  (B)  SHG  image;  (C)  2PF  image;  (D)  false- 
color  image  of  the  combination  of  THG-SHG  and  2PF.  (E-H)  High  magnification  images  of  chloroplasts  showing  strong  SHG  generation  from  starch  granules 
(s)  and  possible  grana  (G).  Ex,  1230nm. 
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FIGURE  40.12.  Nonlinear  emission  spectra  from  the  cell  wall  of  (A)  an  epidermal  cell  in  rice  leaf  and  (B)  parenchyma  cells  in  maize  stem  with  three  differ¬ 
ent  sources:  UV  light  (dotted  line),  780nm  pulsed  signal  ( dashed  line),  and  1230nm  pulsed  laser  ( solid  line).  The  suppression  of  background  2PF  and  the  sym¬ 
metric  peaks  of  THG  and  SHG  in  1230nm  excitata  spectra  are  evident. 


The  tumbling  movement  of  chloroplasts  in  cytoplasm  results 
in  changes  in  the  apparent  periodicity  of  the  thylakoid  membrane 
stacks  along  the  direction  of  the  illuminating  beam.  This  tumbling 
therefore  changes  the  SHG  efficiency,  a  fact  that  can  be  used  to 
study  chloroplast  tumbling,  a  phenomenon  that  is  difficult  to  study 
by  other  means. 

Compared  with  the  emission  spectra  excited  by  a  femtosecond 
Ti:Sa  laser  in  these  plant  specimens,  the  intrinsic  2PF  signals  are 
generally  reduced  by  using  a  longer  wavelength  as  the  pumping 
source  (Chu  et  al.,  2001).  As  an  example.  Figure  40.12  shows  a 
quantitative  comparison  between  the  emission  spectra  excited  by 
a  150mW  1230nm  Criforsterite  femtosecond  laser  (120fs  pulse 
width,  solid  line),  a  50 mW  780 nm  Ti:Sa  femtosecond  laser 
(120fs  pulse  width,  dashed  line),  and  a  0.45  mW  365  nm  UV  light 
source  from  a  frequency-doubled  Ti:Sa  laser  (dotted  line).  The 
emission  spectra  from  the  cell  wall  of  rice  leaf  ( Oryza  sativa )  epi¬ 
dermis  and  of  the  parenchyma  cells  in  a  maize  stem  are  shown  in 
Figure  40.12(A,B),  respectively.  Broad  autofluorescence  covering 
the  whole  visible  and  NIR  region  is  evident  with  both  UV  (single¬ 
photon  fluorescence)  and  780  nm  excitation  (2PF);  whereas  with 
1230nm  excitation,  only  weak  residual  2PF  and  efficient  harmonic 
generation  were  observed.  With  background  autofluorescence  sup¬ 
pressed  by  using  a  longer  excitation  wavelength,  the  whole  visible 
and  NIR  region  is  available  for  detecting  signals  from  specially 
designed,  multi-photon  excited  fluorescence  dyes  that  can  be  used 
to  label  different  functional  molecules  (Cheng  et  al.,  1998).  Fur¬ 
thermore,  as  there  is  no  energy  deposition  at  all  during  harmonic 
generation  processes,  no  photodamage  effect  is  expected  with 
SHG  and  THG.  Thus,  with  efficient  SHG  and  THG  signals,  along 
with  the  reduced  but  still-measurable  intrinsic  2PF  signals,  the 
longer  wavelength  light  source  appears  to  be  a  better  choice  for 
intrinsic  harmonic  generation  imaging  because  it  leaves  a  wider 
spectrum  available  for  extrinsic  multi-photon  dye  labeling.  Figure 
40.13  compares  the  emission  spectra  of  maize  leaf  and  stem 
(epidermal  cells)  when  excited  by  1230nm,  780nm,  and 
400  nm. 

Figure  40.14  shows  the  specific  .ry^i-images  corresponding  to 
THG,  SHG,  and  2PF  in  the  parenchyma  cells  of  a  maize  stem, 
taken  at  a  depth  of  110pm  from  the  sample  surface.  The  total 
sample  thickness  was  around  500  pm.  With  an  average  power  of 


lOOmW  before  entering  the  sample  and  a  focused  spot  diameter 
of  ~1.3  pm,  the  intensity  at  focus  ranges  from  9  to  50  x  1010  W/cm,2 
depending  on  the  focal  depth  inside  the  sample.  As  expected, 
strong  SHG  and  THG  can  be  observed  in  the  cell  wall.  THG  shows 
the  longitudinal  cell  walls  in  the  center  of  the  image  as  well  as  the 
transverse  cell  walls  of  several  adjacent  cells  [Fig.  40.14(A)],  This 
demonstrates  the  ability  of  THG  to  pick  up  the  outline  of  the  whole 
cell.  SHG  may  show  the  relative  position  of  the  secondary  walls 
(arrows).  In  regions  with  extensive  secondary  wall  growth,  the  sep¬ 
aration  of  the  secondary  wall  can  be  clearly  observed  in  Figure 
40.14(B)  and  the  2PF  signal  indicates  the  distribution  of  fluores¬ 
cent  molecules  [Fig.  40.14(C)].  By  comparing  different  images 
made  using  different  modalities,  one  can  image  the  relation- 


FIGURE  40.13.  Normalized  emission  spectra  of  maize  tissue  under  different 
types  of  excitation.  1PF  at  400  nm  (blue).  Two-photon  excited  spectra  gener¬ 
ated  at  1234nm  from  a  leaf  (green)  lacks  the  hump  due  to  fluorescence  of  the 
cell  wall  found  in  stems  (magenta).  It  also  lacks  the  SF1G  peak  at  617nm,  prob¬ 
ably  generated  by  the  secondary  cell  walls  in  the  stem  sample.  Two-photon 
excitation  at  780nm  produces  fluorescence  (maroon)  that  peaks  at  a  shorter 
wavelength  than  the  fluorescence  excited  by  the  other  modes. 
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FIGURE  40.14.  Optical  section  obtained  near  the  longitudinal  wall  (LW)  of  a  parenchyma  cell  in  maize  stem.  Note  the  strong  THG  from  the  radial  wall  (RW) 
(A),  the  SHG  (B),  2PF  (C),  and  combination  of  THG-SHG-2PF  (D).  Arrows',  possible  secondary  wall  material.  The  image  was  obtained  110pm  below  the  surface 
of  the  specimen. 


ships  between  structural  (harmonic  generations)  and  chemical 
information  (two-  and  multi-photon  fluorescence)  in  biological 
tissues.  A  penetration  depth  of  more  than  500pm  has  been 
achieved  in  the  maize  stem  sample  with  1230nm  excitation  (Chu 
et  al.,  2001). 

Figure  40.15  shows  paradermal  optical  sections  of  the  adaxial 
surface  of  rice  (Oryza  sativa  L.)  leaf.  THG  provides  structural 
interfaces,  such  as  the  papillae  from  the  cuticular  layer  and  the  cell 
wall  of  bulliform  cells  [Fig.  40.15(B)],  As  expected,  SHG  reflects 
biophotonic  structures  including  the  cuticular  papillae  and  longi¬ 
tudinal  cell  walls  [Fig.  40.15(B)],  due  to  the  orderly  arrangement 
of  cutin,  waxes,  and  cellulose  microfibrils,  respectively.  2PF,  on 
the  other  hand,  picks  up  the  fluorescent  chromophores  [Fig. 
40.15(C)].  Figure  40.16  shows  a  through-focus  series  (5pm  step) 
of  an  area  on  the  surface  of  rice  leaf  where  silica  cells  can  be  found 
(Hodson  and  Sangster,  1989;  Cheng  1987,  Cheng  et  al,  1990). 
Note  the  outline  of  the  silica  cell  in  the  THG  images  and  the  high 
SHG  signals  from  the  silica  deposits.  A  strong  SHG  signal  can  also 
be  obtained  from  the  silica  wall  of  a  diatom. 

Optically  Active  Structures  in  Animal  Tissues 

There  are  many  structures  in  animal  tissue  that  are  also  good  can¬ 
didates  to  produce  a  strong  biophotonic  effect  (Table  40.3).  For 


example,  the  sarcomeres  in  skeletal  muscles  are  composed  of  crys¬ 
talline  myosin  and  actin  nanofilaments,  with  periods  of  40  and 
20  nm,  respectively,  that  fall  into  the  spatial  range  required  for 
strong  SHG  activity.  Figure  40.17  shows  longitudinally  sectioned 
xy-i  mages  obtained  from  the  somites  of  a  zebrafish  embryo.  The 
cardiac  muscle  fiber  produces  intense  SHG  signal  [green, 
Fig.40.17(A-D)]  and  the  surfaces  of  the  red  blood  cells  (RBC) 
produce  intense  THG  signal  [blue,  Fig.  40.17(C,D)]. 

Figure  40.18  shows  a  two-channel  harmonic  image  of  somites 
in  a  zebrafish  embryo.  At  low  magnification,  the  somites  are 
seen  to  be  separated  by  clefts  [Cl,  Fig.  40.18(A)].  At  higher  mag¬ 
nification  [Fig.  40.18(B,C)],  the  green  SHG  image  of  the 
sarcomeres  (s:  between  the  two  arrows)  indicate  that  the  strong 
blue  THG  signal  in  Figure  40.18(A)  is  probably  generated  by  the 
optical  discontinuity  occurring  between  the  somites.  The  SHG 
intensity  difference  in  individual  sarcomeres  [Fig  40.18(C), 
between  arrows]  reflects  differences  in  the  spatial  packing  of  F- 
actin  and  myosin  in  dark  bands  and  light  bands.  The  strong 
SHG  activity  from  the  actin/myosin  complex  can  make  har¬ 
monic  imaging  a  useful  tool  for  the  study  of  muscle  cell  dynam¬ 
ics  as  the  arrangement  changes  during  muscle  contraction  and 
relaxation. 

Microtubule  bundles  are  birefringent  structures  that  can  be 
studied  in  living  cells  using  polarization  microscopy.  Recent 


FIGURE  40.15.  Through-focus,  three-channel  harmonic  fluorescence  images  of  the  adaxial  surface  of  rice  leaf.  (A)  Near  surface,  (B)  epidermal  cells,  (C)  epi- 
dermal-mesophyll  interface,  (D)  mesophyll  cells.  Note  the  strong  2PF  emission  from  the  chlorophyll  of  the  mesophyll  cells.  White  arrows:  cuticular  papillae. 


Nonlinear  (Harmonic  Generation)  Optical  Microscopy  •  Chapter  40  715 


FIGURE  40.16.  Through-focus,  two-channel  im¬ 
ages  of  the  surface  of  rice  leaf,  showing  a  silica  cell. 
As  it  can  be  seen  in  the  focal  series  (A-D),  the  SHG 
signal  is  generated  by  the  bulk  of  the  phytolith  (Si), 
not  by  the  surface  of  the  silica  deposit. 


advances  in  dynamic  polarization  microscopy  utilizing  tunable 
liquid-crystal  polarizers  now  allow  one  to  study  cytoskeleton 
dynamics  and  spindle  behavior.  The  technique  not  only  provides 
information  on  the  retardance  of  the  birefringent  structure,  but  can 
also  provide  data  of  the  molecular  orientation  of  the  structures 
(s-axis)  (Oldenbourg  and  Mei,  1995). 

Figure  40.19  shows  a  typical  example  in  which  polarization 
microscopy  has  been  used  to  determine  the  orientation  of  the  col¬ 
lagen  fiber  scaffold  in  an  engineered  tissue.  Polarization 
microscopy  can  also  allow  one  to  directly  visualize  the  spindle 
during  cell  division  under  low  light  conditions.  Figure  40.20  shows 
the  spindle  of  a  fertilized  oocyte  during  a  cloning  operation.  Micro¬ 
tubules  have  also  been  shown  to  produce  strong  SHG  signal  in 
both  the  forward  and  the  backward  configurations  (Sun  et  al., 
2005).  Figure  40.21  demonstrates  the  specificity  of  the  SHG  signal 
from  the  spindle  apparatus  of  a  developing  zebrafish  embryo.  The 
interface  between  the  cell  membrane  and  the  surrounding  aqueous 
medium  produces  a  strong  THG  signal  (blue)  that  provides  the 


TABLE  40.3.  Biological  Structures  that  Produce 
Harmonic  Signals 


Structures 

Birefringence 

SHG 

Microtubules 

+ 

+ 

Microfilaments 

+ 

+ 

Spindle 

+ 

+ 

Collagen  fiber 

+ 

+ 

Elastic  fiber 

+ 

+ 

Cuticle 

+ 

? 

Cuticular  wax 

+ 

? 

Sarcomere 

+ 

+ 

Grana  and  thylakoid 

? 

+ 

Si02  deposit 

+ 

+ 

Starch  granules 

+ 

+ 

Cell  wall 

+ 

+ 

Bone  matrix 

+ 

+ 

Dentine 

+ 

+ 

Enamel 

+ 

+ 
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FIGURE  40.1 7.  Two-channel,  harmonic  images  of  developing  heart  in  a  zebrafish  embryo.  The  cardiac  muscle  fiber  produces  intense  SHG  signal  (green,  A-D). 
The  red  blood  cells  (RBC)  produce  intense  THG  signal  (blue,  C  and  D). 


FIGURE  40.18.  Two-channel  harmonic  image  of  somites  in  a  zebrafish  embryo.  (A)  Low  magnification  view  of  the  somites  separated  by  clefts  (Cl);  SHG, 
green;  THG,  blue.  (B)  SHG  image  showing  developing  myofibrils.  Note:  individual  sarcomeres  are  clearly  visible.  (C)  Higher  magnification,  SHG  image  of  the 
sarcomere  (s:  between  the  two  arrows)  indicating  that  the  strong  SHG  signal  is  likely  generated  from  the  dark  band  of  sarcomere. 
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FIGURE  40.19.  Polarization  microscopy  of  man-made  colla¬ 
gen  fiber.  The  image  demonstrates  the  orientation  of  the  colla¬ 
gen  fiber  imaged  by  using  a  Polscope.  The  color  wheel  indicates 
the  orientation  of  the  s-axis  of  the  collagen  fiber.  (Image  cour¬ 
tesy  of  Prof.  Hanry  Yu,  Department  of  Physiology,  National 
University  of  Singapore.) 


general  outline  of  the  embryo.  Figure  40.20  is  a  set  of  high  mag¬ 
nification,  polarization  microscope  images  showing  a  cell  in 
various  stages  of  cell  division. 

As  multi-harmonic  imaging  (SHG  and  THG)  allows  deep- 
tissue  imaging  of  biological  specimens,  it  is  suitable  in  develop¬ 
mental  biology  where  larger  specimens  are  commonly 
encountered.  The  use  of  a  high-repetition-rate  Ti:Sa  laser  offers  the 
possibility  of  real-time,  harmonic  imaging  of  biological  specimens 
(Chu  et  al.,  2003a;  Sun  et  al.,  2002). 

Collagen  fibers  are  abundant  in  animal  tissue,  and  are  both 
birefringent  and  capable  of  producing  SHG  signals.  SHG  is  now 
often  used  to  visualize  the  orientation-dependent  properties  of  con¬ 
nective  tissue  and  the  extracellular  matrix  (Roth  and  Freund,  1979, 
1981;  Freund  et  al.,  1986;  Guo  et  al.,  1997;  Stoller  et  al.,  2002a,b; 
Zipfel  et  al.,  2003;  Williams  et  al.,  2005). 

Polarization  Dependence  of  Second 
Harmonic  Generation 

SHG  signal  strength  from  biophotonic  structures  varies  according 
to  the  relative  orientation  between  the  beam  and  the  organized 


structure.  This  allows  one  to  study  the  orientation  of  a  structure 
using  SHG  with  controlled  illumination  polarization.  For  example, 
the  dumbbell-shaped  silica  deposits  on  rice  and  maize  leaves 
produce  intense  SHG  signals.  By  varying  the  incident  light 
polarization  (Fig.  40.23),  the  concentrically  deposited  silica 
layers  in  the  lumen  of  silica  cells  (Hodson  and  Sangster,  1989) 
produces  SHG  images  that  depend  on  the  orientation  of  the  illu¬ 
mination  polarization.  The  polarization  direction  of  the  excitation 
is  marked  in  the  LR  corner  of  the  left-hand  image  in  each  row.  The 
white  outlines  in  Figure  40.23(A)  demarcate  the  locations  of  two 
dumbbell-shaped  silica  cells.  In  contrast,  no  polarization  depen¬ 
dency  is  evident  in  THG  images.  It  is  possible  to  isolate  the 
epidermal  cuticle  with  the  attached  phytolith  (silica  deposits)  by 
ZnCIi — HC1  treatment.  The  fact  that  images  of  these  isolated, 
silica  deposits  reveal  little  polarization  dependency  in  the  SHG 
signal  (Fig.  40.24)  suggests  that  the  organic  matrix  on  which 
the  silica  is  deposited  is  the  source  of  the  photonic  activity  in  the 
phytolith.  Polarization-dependent  contrast  can  also  be  used  to 
study  the  orientation  of  collagen  fibers  in  tissue  engineering 
research  (Freund  et  al.,  1986;  Stroller  et  al.,  2002a,b;  Chu  et  al., 
2004). 


FIGURE  40.20.  The  spindle  apparatus  in  the  first  division  of  a  mouse  zygote  imaged  by  polarization  microscopy.  [The  image  was  obtained  by  Ms.  Gina  Chen 
of  Ming-mei  Technology,  Taipei,  using  a  Spindleview  dynamic  polarization  microscope  (Cambridge  Research  Instrument,  Cambridge,  MA).] 
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FIGURE  40.21.  Dual-channel,  harmonic  image  of  a  zebrafish  embryo  showing  the  cells  and  the  embryo  outlined  by  THG  (blue)  and  a  mitotic  spindle  (green, 
arrows )  in  SHG.  (A)  Overall  view  of  the  embryo;  (B)  part  of  the  embryo  showing  cells  in  telophase  {yellow  arrow)  and  anaphase  {green  arrow). 


FIGURE  40.22.  High  magnification  views  of  various  stages  in  the  formation  of  the  spindle  in  a  zebrafish  embryo.  (A)  Telophase,  (B)  anaphase,  (C)  late  anaphase, 
and  (D)  after  cytokinesis,  viewed  in  SHG  (green)  and  THG  (blue). 
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FIGURE  40.23.  Polarization  dependence  of  the  SHG  signal.  (A-C)  Ninety-degree  polarization;  (D-F)  45°  polarization;  and  (G-I)  60°  polarization.  Note  that 
the  change  of  illumination  polarization  has  little  effect  to  the  THG  image,  but  the  SHG  signal  does  show  polarization  dependence.  The  spectra  at  the  bottom  of 
this  figure  (J,  K)  were  obtained  at  the  points  labeled  in  (G). 


SUMMARY 

Highly  organized  nanoperiodic  structures  in  biological  samples 
exhibit  strong  SHG  activity  resembling  that  of  nonlinear  photonic 
crystals  and  thus  they  can  be  treated  as  nonlinear  biophotonic  crys¬ 
tals.  Many  biological  structures,  such  as  microfibrils  in  cell  walls, 
alternating  crystalline  lamellae  in  starch  granules,  cuticular  papil¬ 
lae  on  the  leaf  surface,  crystalline  myosin  and  actin  nanofilaments 
in  the  myofibrils  of  skeletal  and  cardiac  muscle,  thylakoid  mem¬ 
branes  and  grana  in  chloroplasts,  and  microtubules  in  both  the 
cytoskeleton  and  the  mitotic  apparatus,  all  exhibit  strong  biopho¬ 


tonic  effects.  Many  of  the  birefringent  structures  found  in  biolog¬ 
ical  specimens  also  exhibit  SHG  properties  (Table  40.3).  The  rel¬ 
ative  locations  and  orientation  of  these  biophotonic  structures  can 
be  examined  using  SHG  microscopy,  while  optical  interfaces  and 
the  functional  molecules  can  be  separately  located  by  THG  and 
2PF  contrast.  In  contrast  to  single-  and  multi-photon  absorption, 
harmonic  generation  involves  only  virtual  states  and  does  not 
involve  energy  deposition.  The  harmonic  signals  allow  3D  struc¬ 
tural  visualization  with  minimal  or  no  additional  preparation  of  the 
sample.  Meanwhile,  2PF  imaging  modes  can  be  added  to  monitor 
multiple  molecular  probes  in  living  cells  and  tissues,  such  as  those 
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FIGURE  40.24.  Isolated  phytoliths,  illuminated  using  different  polarization  directions.  Note  that  the  SHG  (green)  and  THG  (blue)  signals  vary  little  with  dif¬ 
ferent  illumination  conditions.  The  THG  patch  (arrow)  in  the  45°  image  may  be  the  result  of  focal  drift  bringing  the  underlying  cuticular  surface  into  focus. 


composed  of  transformed  cells  or  taken  from  transgenic  organ¬ 
isms.  Multi-modal  microscopy  can  provide  a  powerful  tool  for 
investigating  the  dynamics  of  structure-function  relationships  at 
the  molecular  and  subcellular  levels. 
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INTRODUCTION 

Brain  slices  are  convenient  preparations  to  study  synapses, 
neurons,  and  neural  circuits  because,  while  they  are  easily  accessed 
by  experimental  manipulations  such  as  drug  applications,  intra¬ 
cellular  recordings,  and  optical  imaging,  they  preserve  many  of  the 
essential  functional  properties  of  these  circuits.  In  this  chapter,  we 
describe  techniques  of  live  brain-slice  imaging  used  in  our  labo¬ 
ratory.  We  cover  in  detail  experimental  protocols  and  know-how 
acquired  over  the  years  about  preparing  neocortical  and  hip¬ 
pocampal  slices  and  slice  cultures,  loading  neurons  with  dyes  or 
using  biolistic  transfection  techniques,  two-photon  and  second 
harmonic  imaging,  morphological  reconstructions,  and  image 
processing  and  analysis.  These  techniques  are  used  to  study  the 
functional  or  morphological  dynamics  of  synaptic  structures, 
including  dendritic  spines  and  axon  terminals,  and  to  characterize 
circuit  connectivity  and  dynamics. 

The  importance  of  developing  methods  is  underestimated  in 
modern  biology.  The  education  of  biomedical  researchers  and  the 
federal  granting  agencies  are  dominated  by  the  ideology  that  good 
research  is  question-driven,  whereas  technique-driven  research  is 
of  lesser  quality.  We  disagree  with  this  exclusive  view  because  it 
seems  to  us  that  the  specific  technique  used  is  as  important  as  the 
question  addressed.  As  Sydney  Brenner  put  it:  “Progress  in  science 
depends  on  new  techniques,  new  discoveries,  and  new  ideas,  prob¬ 
ably  in  that  order.”  (Brenner,  2002).  As  an  example,  we  would 
argue  that  the  invention  of  high-affinity,  selective  calcium  indica¬ 
tors  have  revolutionized  many  fields  of  biology  (Grynkiewicz 
et  al.,  1985;  Tsien,  1989). 

We  feel  that  methods  are  essential,  not  only  for  performing  and 
validating  experiments,  but  as  exploratory  tools  that  generate  new 
ideas,  leading  into  new  fields.  Moreover,  in  our  experience,  the 
difference  between  a  difficult  experiment  working  or  not  often 
depends  on  minute  technical  details.  These  details  are  normally 
acquired  with  great  effort  by  the  investigator,  yet  generally  they 
must  be  left  out  of  publications.  To  help  compensate  for  this,  we 
present  in  this  chapter  a  detailed  account  of  current  methods  used 
in  our  laboratory  to  image  living  brain  slices.  The  general  goal  of 
our  work  is  to  use  brain  tissue  specimens  thin  enough  so  that  they 
can  be  successfully  imaged  optically.  As  explained  in  detail 
below,  we  use  different  types  of  brain  slices  and  keep  them  in  sub¬ 
merged  chambers,  where  we  seek  to  preserve  ideal  conditions  of 


temperature,  ionic  composition,  and  nutrients  to  enable  the  slices 
to  survive  as  long  as  possible.  Slices  are  imaged  normally  in 
upright  microscopes,  in  order  to  enable  electrical,  as  well  as 
optical,  access  to  the  surface  of  the  slice.  In  this  respect,  fixed  stage 
microscopes  are  ideal  because  they  enable  the  stable  positioning 
of  micromanipulators  and  mechanical  independence  of  the  focus¬ 
ing  of  the  objective.  Although  inverted  microscopes  enable  better 
optics,  they  are  very  difficult  to  use  for  electrophysiological  exper¬ 
iments  with  slices  because  the  electrical  approach  to  the  prepara¬ 
tion  comes  from  the  opposite  side  of  the  slice  than  the  optical 
approach. 

In  this  chapter  we  will  discuss  a  combination  of  methods  to 
image  brain  slices  that  are  used  in  our  laboratory.  We  will  first 
cover  in  detail  the  preparation  of  different  types  of  brain  slices, 
discuss  how  to  label  cells  in  slices  with  optical  probes,  and  then 
specifically  discuss  different  types  of  imaging  approaches  to  slices. 
We  finish  with  an  additional  section  of  useful  methods  to  mor¬ 
phologically  reconstruct  neurons  from  slices  for  histological  or 
ultrastructural  work  and  a  brief  discussion  of  different  image  pro¬ 
cessing  strategies  that  we  use.  We  hope  that  other  investigators  will 
profit  and  learn  from  our  experience  and  that  this  will  enable  more 
research  teams  to  enter  the  exciting  territory  of  imaging  slices. 


MAKING  BRAIN  SLICES 
Acute  Slices 

Acute  live  slices  prepared  from  the  brain  have  become  a  standard 
preparation  commonly  used  to  study  electrophysiological  proper¬ 
ties  of  neurons  in  circuits  (Alger  et  a].,  1984)  and,  more  recently, 
imaging  (Yuste,  2000b).  Most  of  our  work  is  carried  out  with  slices 
from  mouse  primary  visual  cortex  (Fig.  41.1).  The  relatively  high 
degree  of  preservation  of  neuronal  organization  after  slicing  and 
the  availability  of  a  variety  of  easy  experimental  manipulations 
make  acute  slices  an  attractive  experimental  preparation.  Gener¬ 
ally,  acute  slices  can  be  maintained  in  good  condition  for  up  to  12 
h.  At  the  same  time,  we  find  a  lot  of  variability  in  the  quality  of 
slices  from  day  to  day  and  even  from  slice  to  slice.  The  large 
number  of  variables  that  are  likely  to  be  important  in  the  preser¬ 
vation  of  the  health  of  the  slice  make  obtaining  good  slices  some¬ 
what  of  an  art  form.  Unfortunately,  systematic  studies  to  determine 
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FIGURE  41 .1 .  Brain  slices  of  mouse  visual  cortex.  Representative  coronal  slices  from  mouse  primary  visual  cortex.  Sections  cut  through  the  posterior  pole  of 
the  cerebral  hemisphere.  (A)  Photomicrograph  of  an  acetylcholinesterase-labeled  coronal  section  taken  from  the  Franklin  and  Paxinos  (1997)  atlas.  Note  the 
more  intense  staining  in  layer  4  due  to  the  high  density  of  small  granule  cells.  Violet  and  blue  shaded  areas  indicate  the  monocular  and  binocular  regions  of  the 
mouse  primary  visual  cortex,  respectively.  (B)  Nissl  stain  of  a  mouse  visual  cortex  slice  from  a  PI 8  animal.  Note  the  prominent  layer  4.  (C)  Cytochrome  oxidase 
staining  of  a  similar  section.  Note  the  intense  staining  near  layer  4,  which  is  an  indication  of  the  primary  visual  cortex.  Scale  bar  =  500  |im.  (Courtesy  of  Z. 
Peterlin  and  A.  Tsiola.) 


which  variables  are  important  to  make  healthy  brain  slices  have 
not  yet  been  done. 

Protocol  for  Acute  Neocortical  Slices 

Mice  are  anesthetized  with  120mg/kg  ketamine  and  lOmg/kg 
xylazine  (intraperitoneally)  and  decapitated  with  scissors.  Some 
investigators  in  our  laboratory  prefer  to  perfuse  the  mouse  with  a 
gravity-fed  cold  saline  solution  prior  to  decapitation.  The  skin  cov¬ 
ering  the  skull  is  severed  with  a  fresh  razor  blade  above  the  mid- 
saggital  line  of  the  skull.  The  skull  is  then  cut  along  this  line  and 
forceps  are  slid  under  the  skull  posteriorly,  such  that  an  air  bubble 
forms  anterior  to  the  forceps  tips.  This  air  bubble  provides  a  perfect 
pocket  for  the  forceps  to  slide  anteriorly,  with  care  taken  not  to 


touch  cortical  tissue  with  the  forceps.  Each  half  of  the  skull  may 
then  be  retracted  laterally.  The  brain  is  then  exposed  and  should 
be  immediately  placed  into  ice-cold  sucrose  artificial  cerebrospi¬ 
nal  fluid  (sucrose- ACSF;  222  m M  sucrose,  2.6  mM  KC1,  21  mM 
NaHC03,  1 .5  mM  NaH2P04,  2  mM  CaCl2,  2mMMgS04,  bubbled 
with  95%  02,  5%  C02).  After  approximately  3  min  in  ice-cold 
sucrose  ACSF,  the  brain  is  removed  and  situated  on  the  cutting 
block  such  that  the  cortex  faces  the  approaching  blade.  Slices  300 
to  400  pm  thick  are  cut  with  a  vibratome  (Leica  VT1000S;  Leica, 
Nussloch,  Germany;  high  vibration  and  slow  speed  setting)  and 
incubated  at  37°C  for  30  min  in  a  submerged  slice  chamber.  Slices 
are  then  incubated  at  room  temperature  for  up  to  12h,  until  used 
for  experiments. 
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Identification  of  Primary  Visual  Cortex 

The  primary  visual  cortex,  or  area  17,  of  the  mouse  is  located 
in  the  occipital  region  of  the  brain.  In  the  adult  animals  it  extends 
1.3  mm  anteriorly  from  the  posterior  end  of  the  cortex  (interaural 
line).  In  coronal  sections  it  extends  between  1  to  2  mm  laterally 
from  the  medial  line  at  its  anterior  border  and  between  2  to  3  mm 
in  the  most  posterior  (Franklin  and  Paxinos,  1997).  The  primary 
visual  cortex  of  the  mouse  is  divided  into  two  regions:  the  monoc¬ 
ular  region,  which  receives  input  from  the  contralateral  retina  and 
is  located  in  the  medial  part,  and  the  binocular  region,  which 
receives  input  from  both  retinas  and  is  placed  laterally  (Zilles  and 
Wree,  1985).  The  primary  visual  cortex  can  be  identified  in  coronal 
sections  by  the  densely  arranged  granule  cells  of  layer  4  (Fig.  41.1). 

Thalamocortical  Slice  Protocol 

Thalamocortical  slices  are  an  ideal  preparation  to  investigate  the 
effect  of  thalamic  inputs  onto  cortical  neurons  or  circuits  because  it 
preserves  both  structures  and  connections  between  the  ventrobasal 
nucleus  of  the  thalamus  and  the  somatosensory  cortex.  Preparation 
of  the  thalamocortical  slice  is  slightly  modified  from  Agmon  and 
Connors  (1991),  as  previously  described  (Beierlein  et  al.,  2002). 
Briefly,  C57BL/6  mice  postnatal  (P)  10  to  18  are  anesthetized  with 
120mg/kg  ketamine  and  lOmg/kg  xylazine  and  decapitated.  The 
brain  is  quickly  removed  and  placed  into  cold  artificial  CSF  (ASCF) 
containing  the  following  (in  millimolars):  126  NaCl,  3  KC1, 
1 .25  NaH2P04,  26  NaHC03, 10  dextrose,  1 .3  MgS04,  and  2.5  CaCl2 
(saturated  with  95%  02  and  5%  C02).  The  brain  is  midsagittally 
sectioned  into  the  left  and  right  hemispheres.  Each  hemisphere  is 
glued  (standard  cyanacrylate  “super”  glue)  to  a  plastic  block, 
midline  down,  anterior  end  pointing  toward  the  floor  and  ventral 
surfaces  facing  in,  toward  one  another.  The  hemispheres  are  rotated 
10°  from  center  line  of  the  block.  The  plastic  block  is  actually  a  right 
triangle  with  the  hypotenuse  55°  from  the  level  (floor).  Slices, 
400  pm  thick,  are  cut  with  a  vibratome  (VT1000S)  and  then  incu¬ 
bated  at  32°C  for  45  min.  Usually  two  or  three  viable  thalamocor¬ 
tical  slices  can  be  made  from  each  hemisphere. 

Cultured  Slices 

Because  acute  slices  cannot  be  maintained  in  good  condition  for 
more  than  12  h,  long-term  culture  is  required  for  manipulations 
involving  long-term  experiments,  such  as  those  requiring  the 
expression  of  genes.  Below,  we  describe  the  protocol  for  mouse 
hippocampal  slice  cultures,  which  we  have  used  extensively  to 
image  the  morphology  of  single  neurons  transfected  with  the  green 
fluorescent  protein  (GFP)  (Dunaevsky  et  al.,  1999;  Tashiro  et  al., 
2000). 

Protocol  for  Hippocampal  Cultured  Slices 

Neonatal  mice  (P0-P3)  are  cryoanesthetized  on  ice  for  1  min  and 
decapitated  with  scissors.  In  a  tissue  culture  hood,  skin  and  skull 
are  cut  with  scissors  and  separated  with  forceps.  The  brain  is  then 
gently  removed  and  placed  into  a  35  mm  tissue  culture  dish  filled 
with  cold  sucrose-ACSF  (see  above).  Under  a  dissecting  micro¬ 
scope,  the  two  hemispheres  are  separated  with  a  surgical  blade  and 
oriented  such  that  the  medial  surface  faces  down.  The  cerebellum 
and  the  mesencephalon  are  carefully  dissected  away  and  discarded. 
Then,  after  the  hemisphere  is  rotated  such  that  its  medial  side  faces 
up,  the  diencephalon  is  removed  with  a  surgical  blade  and  a  flat- 
ended  spatula.  The  remaining  piece  of  tissue,  representing  the 
cortex  and  the  hippocampus,  is  trimmed  into  a  rectangular  block 
along  the  anterior  edge  of  the  hippocampus  parallel  to  the  sep- 


totemporal  axis.  The  tissue  is  transferred  onto  a  tissue  chopper 
(TC-2  Tissue  Sectioner;  Smith  &  Farquhar)  with  two  spatulas.  The 
rectangular  block  of  tissue  is  positioned  such  that  the  chopping 
orientation  is  perpendicular  to  the  septotemporal  axis  of  the 
hippocampus.  Slices  300pm  thick  are  obtained.  With  two  flat- 
ended  spatulas,  the  slices  are  transferred  to  a  fresh  culture  dish 
containing  cold  sucrose-ACSF.  The  slices  are  separated  from  each 
other  with  a  surgical  blade  and  a  flat-ended  spatula  as  soon  as 
possible. 

Slices  are  cultured  in  culture  medium,  lOOmL  of  which  con¬ 
tains  50mL  Basal  Medium  Eagle  (catalog  #21010-046,  Invitro- 
gen),  25  mL  Hank’s  Balanced  Salt  Solution  (catalog  #24020-117, 
Invitrogen),  25  mL  heat-inactivated  horse  serum  (Hyclone),  0.65  g 
dextrose,  0.5  mL  L-glutamine  (catalog  #25030-149,  Invitrogen), 
l.OmL  HEPES  (catalog  #15630-106,  Invitrogen),  and  l.OmL 
100X  Pen-strep  (catalog  #15140-148,  Invitrogen).  Approximately 
1  mL  of  medium  is  poured  onto  and  under  the  culture  inserts 
(catalog  #PICM  030  50  or  PICM  ORG  50,  Millipore)  in  the  sterile 
hood,  so  that  the  membrane  of  the  inserts  is  completely  submerged 
in  culture  medium.  We  find  that  the  use  of  serum  from  Hyclone 
(Logan,  UT)  is  a  particularly  important  variable  because  cultures 
made  with  serum  from  other  sources  were  not  successful.  Indi¬ 
vidual  slices  are  then  transferred  onto  the  membrane  with  a  flat- 
ended  spatula.  Three  to  six  slices  are  cultivated  on  single  inserts. 
Most  of  the  medium  (but  not  all)  is  removed  from  the  inserts  with 
a  pipette,  and  the  slices  are  positioned  with  the  spatula  at  the  center 
of  the  insert,  but  separated  from  each  other  by  at  least  2  to  3  mm. 
Then,  all  remaining  medium  in  the  inserts  is  aspirated.  The  inserts 
are  transferred  into  6-well  culture  plates,  in  which  each  well  con¬ 
tains  1  mL  of  culture  medium.  The  culture  plates  are  kept  in  the 
incubator  (5%  C02,  37°C).  Every  other  day,  0.6  mL  of  culture 
medium  is  changed  with  fresh  medium.  During  the  first  few  days 
in  culture,  slices  spread  slightly  and  become  flattened  to  a  150  to 
250  |im  thickness. 


LABELING  CELLS 
Biolistic  Transfection 

To  image  neuronal  morphology,  we  transfect  GFP  using  biolistics 
(“gene  gun”)  (Arnold  et  al.,  1994;  Lo  et  al.,  1994).  The  principle 
of  this  method  is  that  metal  particles  coated  with  DNA  are  trans¬ 
ferred  physically  into  nucleus  by  pressured  gas.  Transferred  GFP 
genes  are  expressed  and  the  whole  neuronal  cytoplasm,  including 
their  axons  and  dendrites,  can  be  visualized  (Fig.  41.2). 

Biolistic  Protocol 

We  use  the  Helios  Gene  Gun  System  (Bio-Rad),  and  the  follow¬ 
ing  protocol  is  modified  from  the  procedure  described  in  the 
manual  for  the  system.  Plasmids  are  purified  on  a  Qiagen  Max- 
iprep  kit.  Weigh  12.5  mg  gold  (1  pm  diameter)  in  a  1.5  mL  tube  and 
add  100 pL  of  0.05M  Spermidine.  Sonicate  the  tube  for  5  s  to  dis¬ 
sociate  aggregated  gold  particles  (FS30,  Fisher  Scientific;  40kHz, 
130W).  Add  solution  containing  desired  amount  of  plasmid  [for 
eGFP-Cl  plasmid  (Clontech),  lOOpg]  and  precipitate  plasmid 
onto  gold  particles  with  100  pL  of  ImM  CaCL.  In  10  min,  gold 
particles  precipitate  to  the  bottom  of  the  tube.  Then,  remove 
supernatant  solution  without  disturbing  the  particles  and  wash 
particles  three  times  with  100%  EtOH.  The  particles  are  then 
suspended  in  100%  EtOH  and  transferred  into  a  new  15mL  tube 
to  a  final  volume  of  3  mL.  Transfer  particles  in  EtOH  into  Tefzel 
tube  (Bio-Rad).  In  approximately  90s,  gold  particles  precipitate  to 
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FIGURE  41.2.  Two-photon  imaging  of  living  neurons  in  slices.  Two-photon  micrographs  of  GFP-transfected  neurons  in  hippocampal  brain  slice  cultures.  (A) 
A  CA3  pyramidal  neuron  at  14  days  in  vitro.  (B)  Apical  dendrite  of  a  CA3  pyramidal  neuron  at  11  days  in  vitro.  (C)  Time-lapse  sequence  of  a  dendritic  spine 
in  (B).  Note  the  morphological  rearrangements  occurring  in  a  few  minutes.  (D)  Dentate  granule  cells  at  14  days  in  vitro.  Note  that  the  entire  dendritic  and  axonal 
processes  are  visualized.  Scale  bar:  (A)  50  pm,  (B)  2.5  pm,  (C)  1pm.  (D)  150  pm. 


the  bottom  of  the  tube.  Then,  EtOH  is  removed  from  the  tube 
slowly  without  disturbing  the  particles.  Tube  is  dried  with 
nitrogen  gas  until  the  inside  surface  of  the  tube  becomes  com¬ 
pletely  dry.  With  tubing  cutter  (Bio-Rad),  the  tube  is  cut  into  small 
pieces,  which  are  used  for  single  shots.  Tubing  sets  can  be  stored 
with  desiccant  at  4°C  for  up  to  a  month  and  at  -80°C  for  longer 
storage. 

To  drive  gold  particles  into  neuronal  nuclei,  we  use  high- 
pressure  helium  flow.  We  adjust  the  helium  pressure  to  100  to 
150psi  for  transfection  of  slice  cultures  and  acute  slices.  Two  to 
three  “preshots”  are  fired  with  an  empty  cartridge  holder  to  clean 
the  helium  pathway  and  make  sure  that  pressure  is  stable  after  each 
shot.  In  order  to  reduce  the  damage  to  slices  caused  by  high- 
pressure  flow,  the  tips  of  barrel  liners  are  covered  by  a  nylon  mesh 
(t|)90|im.  Small  Parts,  Inc).  For  cultured  slices,  the  cover  of  the 
culture  plate  is  removed,  and  the  gun  is  fired  perpendicular  to 
the  plate  with  a  distance  of  10  mm  between  the  tip  of  the  barrel 


liner  and  the  insert.  The  culture  plates  are  immediately  put  back 
into  the  incubator.  Slices  are  incubated  for  2  to  5  days  before 
imaging. 

Genetic  Manipulation  with  Dominant-Negative 
and  Constitutively  Active  Mutants 

One  of  the  advantages  of  biolistics  over  other  transfection  methods 
is  that  cotransfection  of  multiple  genes  is  quite  easy.  If  the  two 
genes  are  in  separate  mammalian  vectors,  they  can  be  cotrans¬ 
fected  with  high  cotransfection  efficiency  (>90%  in  our  hands)  by 
simultaneously  coating  gold  particles  with  these  two  vectors.  With 
cotransfection  of  GFP  and  dominant-negative  or  constitutively 
active  mutant  genes,  the  roles  of  specific  molecular  signaling  cas¬ 
cades  in  the  regulation  of  neuronal  morphology  can  be  examined. 
For  example,  we  have  been  studying  the  roles  of  Rho  GTPases  in 
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regulating  morphology  and  motility  of  dendritic  spines  (Tashiro 
et  al.,  2000;  Tashiro  and  Yuste,  2004). 


Diolistics  and  Calistics 

Conventionally,  lipophilic  dyes,  such  as  DiO  and  Dil,  are  used  to 
label  axonal  projections  between  different  regions  of  the  brain  by 
placing  a  crystal  of  these  dyes  in  a  defined  region  (Honig  and 
Hume,  1989).  However,  this  method  is  not  suitable  for  imaging  the 
morphology  of  single  neurons  because  the  region  near  the  crystal 
is  stained  too  densely  to  visualize  single  neurons.  To  label  single 
neurons,  Gan  and  colleagues  developed  diolistics,  a  variant  of 
particle-mediated  gene  transfer,  which  transfers  metal  particles 
coated  by  fluorescent  dyes  onto  cells  (Gan  et  al.,  2000).  Our  group 
has  extensively  used  this  method  to  visualize  neurons  in  fixed 
tissue  and  classified  cortical  pyramidal  neurons  in  mouse  VI  into 
different  morphological  categories  (Tsiola  and  Yuste,  2003). 

Calistic,  another  variant  of  biolistics,  serves  to  inject  Ca2+  indi¬ 
cators  such  as  calcium  green- 1  and  fura-2  into  neuronal  cytoplasm 
(Kettunen  et  al.,  2002).  With  this  method,  an  apparently  higher 
concentration  of  calcium  indicators  can  be  injected  into  neurons 
than  using  acetoxymethyl  ester  (AM)  loading,  and  a  large  number 
of  neurons  can  be  visualized.  Calistic  also  allows  the  simultane¬ 
ous  measurement  of  morphological  and  calcium  dynamics  of 
single  neurons  (Lohmann  et  al.,  2002). 

Dye  Injection  with  Whole-Cell  Patch  Clamp 

Whole-cell  patch  clamp  is  commonly  used  to  study  electrophysi- 
ological  properties  of  neurons  in  brain  slices  (Edwards  et  al., 
1989).  Using  electrodes  filled  with  fluorescent  dyes,  the  whole-cell 
configuration  of  patch  clamp  injects  the  dyes  into  neurons  by  dif¬ 
fusion  through  the  pipette  tip  into  the  neuron.  This  technique  has 
the  advantages  that  the  labeling  procedure  is  rapid  and  that  any 
neuron  in  the  slice  can  be  targeted  and  therefore  visualized.  When 
electrophysiological  measurements  are  combined  with  imaging,  a 
lower  concentration  of  dyes  is  used  and  the  whole-cell  patch  clamp 
is  maintained  during  an  experiment  (Yuste  and  Denk,  1995). 
However,  an  extended  period  of  patch  clamp  may  interfere  with 
cellular  functions  such  as  spine  motility  (Majewska  et  al.,  2000a), 
possibly  because  the  biochemistry  in  the  neuron  is  perturbed  by 
the  perfusion  of  intracellular  solution  or  the  diffusion  of  cytoplasm 
into  the  patch  electrode.  Because  of  this  problem,  we  routinely  fill 
the  electrode  with  higher  concentration  of  fluorescent  dyes  and 
pull  out  the  electrode  a  few  minutes  after  whole  cell  recording  is 
established  (bolus  technique,  see  below;  Helmchen  et  al.,  1996; 
Majewska  et  al.,  2000a). 

Bolus  Injection  Protocol 

Neurons  of  interest  are  identified  using  differential  interference 
contrast  (DIC)  optics,  and  then  patched  and  recorded  using  the 
whole-cell  patch  clamp  technique  in  current-clamp  configuration 
to  ensure  the  neurons  are  healthy.  Electrodes  are  filled  with  a  solu¬ 
tion  containing  (in  millimolars):  5  NaCl,  10  KC1.  10  HEPES,  135 
KMeSOj.  2.5  to  4  Mg- ATP,  0.3  Na-GTP,  and  100  to  500  pM  fluo¬ 
rescent  dye  such  as  Calcium  Green- 1  or  Alexa-488  (Molecular 
Probes,  Inc.,  Eugene,  OR).  Electrodes  are  then  pulled  out  1  to 
3  min  after  patching  to  prevent  dialysis  of  cytoplasm.  The  resis¬ 
tance  of  patch  electrodes  is  typically  7  to  14  MO.  Diffusion  of  dyes 
is  so  rapid  that  the  whole  dendritic  tree  is  visualized  in  a  few 
minutes. 


Slice  Loading  and  "Painting"  with  Acetoxymethyl 
Ester  Indicators 

In  our  past  work  we  have  pioneered  the  use  of  calcium  imaging  to 
characterize  the  activity  of  neuronal  populations  (Yuste  and  Katz, 
1989,  1991;  Smetters  et  al.,  1999;  Peterlin  et  al.,  2000).  The  bulk 
loading  method  for  double  incubation  of  cortical  slices  with  fura- 
2  AM  or  indo-1  AM  (Molecular  Probes)  calcium  indicators  has 
been  previously  described  (Yuste  and  Katz.  1989;  Yuste,  2000a). 
Briefly,  cortical  slices  are  initially  incubated  with  2  to  5  pL  of  a 
1  mM  fura-2  AM  or  indo-1  AM  in  100%  DMSO  solution  for  2  min, 
followed  by  a  second  incubation  in  3mL  of  lOpM  fura-2  AM  in 
ACSF  for  60  min.  However,  the  use  of  the  thalamocortical  slice 
preparation  (Beierlein  et  al.,  2002)  necessitated  the  development 
of  a  modified  bulk-loading  procedure  because  the  long-projecting 
thalamocortical  (TC)  axons  are  particularly  sensitive  to  the  double 
incubation  methodology,  even  though  local  connections  within  the 
cortex  remain  intact  (Kozloski  et  al.,  2001). 

We  have  been  able  to  circumvent  this  problem  by  applying  the 
fura-2  AM  or  indo-1  AM  solution  directly  to  the  region  of  interest 
in  the  cortical  slice  with  a  pipette,  tip  diameter  approximately 
30  pm,  filled  with  fura-2  AM.  The  region  of  interest  of  the  slice 
can  be  “painted”  with  fura-2  AM  or  indo-1  AM.  In  this  way  one 
is  able  to  achieve  good  loading  while  preserving  TC  connections 
up  to  postnatal  day  18  in  mouse  barrel  cortex  (Fig.  41.3).  The 
maintenance  of  intact  TC  projections  was  confirmed  using  thala¬ 
mic  stimulation  that  elicits  a  calcium  response  in  barrel  cortex 
indicative  of  intact  thalamocortical  axons  (Beierlein  et  al.,  2002). 

Protocol  for  Slice  AM  Painting 

1.  Deposit  TC  slice  carefully  onto  the  bottom  of  a  small  Petri  dish 
(35  x  10  mm)  filled  with  2mL  of  ACSF  aspirated  with  95%  02 
and  5%  C02  and  place  onto  microscope  stage. 

2.  Fill  a  fire-polished  pipette  (tip  diameter  ~30pm)  with  fura-2 
AM  from  a  previously  prepared  aliquot  of  50  pg  of  fura-2  AM 


FIGURE  41.3.  Two-photon  imaging  of  neuronal  ensembles.  Two-photon 
micrograph  of  an  acute  cortical  brain  slice,  loaded  with  indo-1  AM.  A  number 
of  neurons  are  loaded  with  the  calcium-sensitive  indicator,  indo-1.  Note  that 
dendritic  processes  are  also  visualized  in  many  neurons.  60  pixels  correspond 
to  20  pm.  Scale  bar:  50  pm. 
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dissolved  in  lOpL  of  DMSO  and  2\xL  of  pluronic  acid  (F127, 
Molecular  Probes). 

3.  Insert  the  filled  pipette  into  a  standard  patch  clamp  electrode 
holder,  with  tubing  attached,  and  using  a  micromanipulator, 
place  pipette  tip  100  to  200  pm  above  the  surface  of  the  slice. 
Apply  5  to  lOpsi  positive  pressure  to  the  pipette.  Slowly  move 
the  pipette  across  the  surface  of  the  slice  using  the  manipula¬ 
tor,  covering  the  area  of  interest  with  the  dissolved  fura-2  AM. 

4.  Incubate  the  slices  at  32°C  for  24  to  28  min  depending  on  the 
age  of  the  animal  from  which  the  slices  were  taken  (younger 
animals  require  shorter  incubation  times),  aspirated  with  95% 
02  and  5%  C02  throughout. 

5.  Finally  transfer  the  slices  to  oxygenated  ACSF  at  room  tem¬ 
perature  at  least  15  min  before  use  for  the  experiment. 

Green  Fluorescent  Protein  Transgenic  Mice 

Recently,  a  number  of  different  types  of  GFP  transgenic  mice  have 
become  available  commercially  or  from  independent  investigators. 
If  these  mice  express  GFP  in  neurons  of  interest  at  the  right  age, 
the  labeling  procedures  described  above  are  circumvented.  For 
example,  we  have  used  the  GFP-M  line  of  GFP  transgenic  mice 
developed  by  Feng  and  colleagues  (Feng  et  al.,  2000).  At  the 
second  postnatal  week,  this  line  of  mice  expresses  GFP  weakly  in 
VI,  but  strongly  in  pyramidal  neurons  in  pyriform  cortex.  In  addi¬ 
tion,  too  many  pyramidal  neurons  in  the  hippocampal  CA1  region 
are  labeled  by  GFP,  so  background  fluorescence  makes  it  difficult 
to  visualize  single  neurons. 


IMAGING  SLICES 
Two-Photon  Imaging  of  Slices 

For  imaging  brain  slices  we  almost  exclusively  use  upright  micro¬ 
scopes  (Olympus  BX50WI)  because  they  can  provide  easy 
combination  of  electrophysiological  techniques  (whole-cell  patch 
clamp  and  extracellular  stimulation)  with  simultaneous  imaging  of 
patched  and/or  stimulated  cells.  As  explained,  with  inverted  micro¬ 
scopes  it  is  difficult  to  position  the  patch/stimulating  electrode 
from  one  side  of  the  slice  and  image  from  the  same  side.  This 
requirement  is  satisfied  in  the  case  of  upright  microscope  and 
dipping-type  water-immersion  objectives  with  a  working  distance 
large  enough  to  enable  bringing  the  electrode  in  the  field  of  view 
from  the  same  side. 

Two-photon  imaging  is  carried  out  with  a  custom-built  two- 
photon  laser-scanning  microscope  (Majewska  et  al.,  2000b). 
A  more  recent  description  of  our  system  can  be  found  at 
www.twophoton.com  or  at  http://www.columbia.edu/cu/biology/ 
faculty/yuste/index.html.  The  microscope  consists  of  a  modified 
Fluoview  (Olympus,  Melville,  NY)  confocal  microscope  with 
a  titanium :  sapphire  (Ti:Sa)  laser  providing  ~130fs  pulses  at 
76  MHz  at  wavelengths  of  700  to  900  nm  (Mira,  Coherent,  Santa 
Clara,  CA)  pumped  by  a  solid-state  source  (Verdi,  Coherent). 
We  detect  the  fluorescence  with  a  non-descanned  detector  (see 
below). 

Imaging  is  done  at  low  excitation  intensities  (3-10mW  at 
sample).  Under  these  conditions  no  significant  photobleaching  or 
photodamage  is  observed,  allowing  us  to  image  for  long  periods 
of  time.  For  fast  time  resolution  we  can  record  continuous  movies 
(1000  frames  per  movie,  0.2-1.6s/frame),  acquiring  individual 
calcium  fluorescence  signals  from  hundreds  of  neurons  simulta¬ 


neously.  Alternatively,  in  a  time-lapse  mode  (1  frame/ 15  s),  we  can 
image  the  same  region  for  up  to  6h  without  appreciable  photo¬ 
damage.  The  major  limitation  of  the  use  of  two-photon  imaging  to 
monitor  the  activity  of  large  neuronal  populations  is  the  slow  time 
resolution  associated  with  laser-scanning  methods.  It  is  therefore 
best  suited  for  the  study  of  slow  events  rather  than  to  detect  single 
spike  correlations.  By  performing  online  analysis,  we  can  identify 
prominent  features  of  spontaneous  activity,  target  key  elements  of 
the  network,  perform  whole-cell  recordings  while  continuing 
to  image  the  slice,  and  characterize  electrophysiologically  the 
neurons  that  participate  in  these  events. 

Slice  Chamber  Protocol 

Acute  and  cultured  slices  are  continuously  perfused  with  standard 
ACSF  containing  (in  m M):  126  NaCl,  3  KC1,  2  CaCl2,  1  MgSOj, 
1.1  NaH3P04,  26  NaHC03,  and  10  dextrose  and  saturated  with 
95%02  and  5%C02.  To  hold  the  slices  on  the  microscope,  we  use 
a  temperature-controlled  chamber  (Series  20  imaging  chamber, 
Warner  Instrument).  Flow  is  gravity-driven  by  raising  a  container 
(60  ml  syringe,  for  example)  above  the  chamber  and  controlled  by 
a  flow  regulator.  ACSF  is  sucked  from  the  chamber  by  a  vacuum 
pump.  Sometimes  it  is  necessity  to  use  an  additional  flow  regula¬ 
tor  in  the  vacuum  line  to  stabilize  the  level  of  liquid  in  the  record¬ 
ing  chamber,  especially  when  a  powerful  vacuum  pump  (“Air 
Admiral”;  ColeParmer)  is  used.  Medium  flow  in  the  chamber  can 
cause  movement  of  the  whole  slice,  which  is  a  serious  problem 
in  time-lapse  imaging,  especially  with  small  structure  such  as 
presynaptic  and  postsynaptic  structures.  To  minimize  movement 
artifacts,  medium  flow  is  reduced  to  1  ml/min  and  the  slices  are 
stabilized  with  a  slice  anchor  (Warner  Instrument).  As  an  alterna¬ 
tive  to  using  a  weight  that  can  damage  the  slice,  we  also  use  the 
direct  adherence  of  the  slice  to  the  chamber.  To  do  so,  we  position 
the  slice  on  the  bottom  glass  of  the  chamber  and  drain  all  the 
ACSF.  After  a  few  seconds,  we  reperfuse  the  chamber  carefully  so 
as  not  to  lift  the  slice.  In  most  cases,  the  slice  has  adhered  to  the 
chamber  and  will  not  move  for  the  rest  of  the  experiment. 

ACSF  is  heated  before  flowing  into  the  chamber  by  an  in-line 
heater  (SH-27B,  Warner  Instrument),  and  the  base  of  the  chamber 
is  also  heated  by  a  platform  heater  (Series  20  platform,  Warner 
Instrument).  These  heaters  are  controlled  by  a  dual  channel  heater 
Controller  (TC-344B,  Warner  Instrument).  The  temperature  of  the 
in-line  heater  is  set  at  ~39°C  and  the  platform  heater  is  kept  at 
~39°C,  in  order  to  keep  the  liquid  in  the  chamber  at  -36-37C.  As 
an  independent  control  of  the  liquid  temperature,  we  use  an  addi¬ 
tional  thermosensor  (Warner)  or  a  thermocouple-based  handheld 
digital  thermometer  (TES  1300).  If  ACSF  is  saturated  with  02  and 
C02  at  room  temperature,  these  gases  come  out  of  solution  in  the 
heated  imaging  chamber  and  produce  a  number  of  small  bubbles. 
These  bubbles  degrade  image  quality  and  can  damage  the  slices. 
To  prevent  this,  we  keep  the  ACSF  container  in  a  hot  bath  and  sat¬ 
urate  the  ACSF  with  the  gases  at  37°C. 

Choice  of  Objectives 

We  use  40x  (0.8NA)  or  60x  (0.9NA)  dipping-type  water  immer¬ 
sion  objectives  (Olympus),  although  we  have  recently  started  to 
use  the  new  20x  0.95  NA  objective  (Olympus)  to  image  a  larger 
number  of  cortical  neurons.  With  this  low-magnification,  high- 
numerical  aperture  (NA)  objective,  we  can  simultaneously  monitor 
the  activity  of  large  neuronal  populations  (average  650  neurons, 
range  184-1396)  in  a  thin  optical  section  of  the  slice.  The  area 
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viewed  covers  up  to  5  different  layers  of  the  primary  visual  cortex 
(-400  X  700pm).  The  improved  depth  penetration  and  the  high 
sensitivity  of  two-photon  imaging  allows  us  to  image  at  depths  of 
>100  pm  from  the  slice  surface,  where  connectivity  is  less  affected 
by  the  slicing  procedure.  High-NA  objectives  further  increase 
the  fluorescence  collection  and  thus  allow  deeper  fluorescence 
measurements  with  good  resolution  than  conventional  objectives. 
Even  at  low  magnification,  we  can  resolve  individual  neurons 
and  some  of  their  dendritic  processes.  However,  a  major  dif¬ 
ficulty  associated  with  the  use  of  the  20x  objective  for  patching, 
stems  from  its  large  dimensions,  which  restricts  access  for 
electrodes. 

In  general,  it  is  also  necessary  to  minimize  the  working  depth 
in  the  tissue.  Deeper  imaging  leads  to  loss  of  excitation  light  and 
fluorescence  signal  because  of  light  scattering  by  the  tissue.  Also 
deeper  imaging  leads  to  lower  contrast  in  bright-field  and  makes 
patching  extremely  difficult.  That  is  why  one  of  the  major  require¬ 
ments  for  objectives  is  a  long  working  distance.  The  Olympus  20x, 
40x  and  60x  water-immersion  objectives  satisfy  this  requirement, 
with  working  distances  of  at  least  2  mm.  On  the  other  hand  it  is 
possible  to  use  objectives  with  shorter  working  distances  for 
studies  which  do  not  require  simultaneous  imaging  and  electro- 
physiological  recording  (morphological  studies  of  cells  labeled 
genetically  or  by  bolus  injection). 

The  efficiency  of  signal  collection  in  the  case  of  two-photon 
laser  scanning  microscopy  using  the  whole-area  detection  mode 
directly  depends  on  the  NA  of  the  objective  lens  used.  Also,  a 
higher  NA  decreases  the  diffraction-limited  size  of  the  excitation 
spot,  which  gives  better  spatial  resolution  and  provides  higher 
local  intensity  of  the  excitation  light,  thus  increasing  the  efficiency 
of  nonlinear  optical  effects  (multi-photon  absorption  or  second- 
harmonic  generation).  In  this  respect,  the  Olympus  20x  0.95  NA 
water-immersion  objective  was  crucial  and  allows  2P  Ca2+  imaging 
of  large  neuronal  populations  with  excellent  signal  to  noise  ratio 
(Fig.  41. 3 A). 

Choosing  the  right  magnification  for  the  objective  lens  is 
another  practical  issue.  Lower  magnification  is  good  for  imaging 
larger  structures;  although  low-magnification  objectives  are  not 
convenient  for  simultaneous  electrophysiological  recordings,  if 
cells  have  to  be  patched  while  visualizing  them  through  the  eye¬ 
pieces.  In  some  cases,  increasing  magnification  with  a  separate 
system  of  lenses  solves  this  problem.  As  an  example,  we  have  a 
“U-CA”  adaptor  from  Olympus  for  the  BX  50WI  microscope, 
which  works  as  a  magnifying  telescope  and  is  inserted  into  the 
optical  pathway  of  the  microscope  in  the  region  where  the  light  is 
collimated.  On  the  other  hand  it  is  not  ideal  to  use  magnifying 
adaptors  for  imaging,  because  they  lead  to  a  loss  of  fluorescence 
signal  in  collection  pathway.  Hence  this  adaptor  should  be  used 
only  for  convenience  when  observing  with  low-magnification 
objectives  (20x)  through  the  eyepieces. 

High  magnification  lenses  (40x,  60x,  100X)  with  higher 
numerical  apertures  provide  images  with  higher  spatial  resolution 
in  3D.  Using  the  "digital  zoom”  option  available  in  majority 
systems  for  laser  scanning  microscopy,  allows  one  to  set  the  pixel 
size  according  to  Nyquist.  This  kind  of  spatial  sampling  is  not 
always  necessary.  For  example,  when  imaging  neuronal  popula¬ 
tions,  we  are  interested  mostly  in  the  integrated  signal  from  indi¬ 
vidual  cells.  Nyquist  sampling,  or  even  oversampling,  is  important 
if  we  ask  questions  about  sub-resolution  movements  of  small  struc¬ 
tures  (i.e.,  quantitative  analysis  of  spine  movement).  In  this  case, 
one  should  not  consider  a  laser  scanning  microscope  as  an  imaging 
device  with  resolution  limit  defined  by  diffraction,  but  more  as  a 
position-measuring  instrument  that  measures  the  centroid  of  the 


distribution  of  fluorescent  molecules  and  the  dynamics  of  the  posi¬ 
tion  of  this  centroid. 

For  two-photon  laser  scanning  microscopy,  it  is  important  to 
make  sure  that  all  the  components  of  the  excitation  pathway  have 
good  transmission  in  the  near  IR.  Users  should  use  objective  lenses 
corrected  for  optical  distortions  and  made  to  be  transparent  in  the 
NIR.  In  many  cases,  additional  changes  are  needed  in  the  installed 
optics  to  make  them  IR-transparent  (pupil- transfer  lens  in  our  case; 
(Majewska  et  al.,  2000b). 

Objective  lenses  used  for  multi-photon  imaging  should  be  free 
of  geometrical  (spherical)  aberrations.  The  requirement  for  the 
absence  of  chromatic  aberrations  is  not  so  important  —  excitation 
light  is  practically  monochromatic  (the  spectrum  widening  caused 
by  the  finite  length  of  pulse  from  mode-locked  lasers  is  negligi¬ 
ble).  Also,  the  absence  of  a  confocal  aperture  in  front  of  the 
detector  and  the  general  architecture  of  the  collection  system, 
emphasizes  collecting  the  maximum  fraction  of  the  emitted  light, 
and  allows  one  the  freedom  of  using  collecting  optics  (objectives 
in  case  of  2P-fluorescence  and  condenser  lens  in  the  case  of  SHG) 
not  well-corrected  for  chromatic  aberration. 


Beam  Collimation  and  Pulse  Broadening 

The  majority  of  modern  microscopes  are  designed  for  infinity- 
corrected  objective  lenses,  so  it  is  important  to  provide  colli¬ 
mated  laser  light  to  the  back- aperture  of  the  objective  (Tsai  and 
Kleinfield,  2002).  Even  if  initially  the  microscope  system  and 
scanning  head  are  designed  to  provide  collimated  light  at  the  back- 
aperture  of  objectives,  custom  modifications  of  the  optical  pathway 
and  the  switch  to  NIR  excitation  can  distort  this  collimation.  This 
indeed  happened  in  our  custom-made  2P-microscope  and  we 
solved  the  problem  by  introducing  additional  optics  into  the  exci¬ 
tation  pathway  (Nikolenko  et  al.,  2003).  Specifically,  we  use  a 
simple  system  of  2  lenses  in  order  to  collimate  the  light  to  the 
objective  lens. 

Our  system  also  works  as  a  “beam  expander”  —  it  modifies 
the  laser  beam  in  such  a  way  that  the  excitation  beam  at  the  back 
aperture  of  objective  is  not  only  collimated,  but  also  is  large 
enough  to  slightly  overfill  the  objective  pupil.  One  of  the  major 
requirements  for  laser  scanning  microscopes  is  that  the  back 
aperture  should  be  overfilled  by  the  excitation  light  (Tsai  and 
Kleinfield,  2002).  This  minimizes  variation  of  excitation  power 
across  the  field  of  view  and  guarantees  that  the  full  numerical  aper¬ 
ture  of  the  objective  lens  is  used.  The  level  of  overfilling  should 
be  minimal  to  maximize  the  amount  of  excitation  power  deliveried 
to  the  sample. 

Another  important  factor  in  2P  microscopy  is  the  problem  of 
pulse  broadening.  Nonlinear  microscopy  requires  using  pulsed 
laser  light.  Mode-locked  lasers  generate  a  train  of  pulses  of  finite 
length  at  certain  repetition  rate.  Even  though  each  pulse  represents 
monochromatic  light,  the  finite  length  of  each  pulse  leads  to  a 
spectrum  with  certain  width  in  Fourier-space.  Linear  dispersion 
of  this  light  in  the  intermediate  optical  elements  disturbs  the 
phase  relations  between  the  different  spectral  components  of  the 
pulse,  which  in  practice  leads  to  the  pulse  being  broadened  in  time. 
This  decreases  the  peak  excitation  intensity,  and  hence  decreases 
the  efficiency  of  nonlinear  excitation.  In  order  to  avoid  this,  the 
optical  system  should  use  the  minimum  number  of  lenses  between 
the  laser  and  the  specimen.  Alternatively,  one  can  add  additional 
optics  with  negative  dispersion  in  order  to  compensate  for  the 
positive  dispersion  of  the  rest  of  the  optics  (Lechleiter  et  al., 
2002). 
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Image  Production,  Resolution,  and  z-Sectioning 

In  our  two-photon  microscope,  fluorescence  is  detected  with  pho¬ 
tomultiplier  tubes  (PMTs;  HC125-02,  Hamamatsu,  Japan)  used  in 
an  external,  whole-area  detection  mode,  and  images  are  acquired 
using  Fluoview  software  (Olympus).  Images  are  sometimes  taken 
at  the  highest  digital  zoom,  resulting  in  a  nominal  spatial  resolu¬ 
tion  of  20-30  pixels  per  |im  with  the  40x.  This  spatial  resolution 
is  suitable  for  imaging  very  small  structures  including  dendritic 
spines,  the  size  of  which  is  typically  on  the  order  of  pm. 

Since  brain  slices  are  three-dimensional,  we  collect  a  series  of 
images  (z-stack)  from  different  focal  planes  to  cover  the  whole  neu¬ 
ronal  structure  of  interest.  In  principle,  three-dimensional  struc¬ 
tures  can  be  reconstructed  from  the  z-stack.  However,  this  is  not 
practical  when  the  same  structure  is  imaged  repeatedly,  particularly 
in  time-lapse  imaging,  for  the  following  reasons:  (1)  perfusion 
causes  small  movements  of  the  slice  so  the  reconstructed  structures 
are  not  accurate  and  (2)  to  achieve  pixelation  in  the  z-direction  at 
a  similar  level  to  the  x-  and  y-direction,  many  focal  planes  have  to 
be  scanned.  This  is  impractical  because  it  compromises  fast  time- 
lapse  imaging  and  increases  the  possibility  of  photodamage. 

To  circumvent  these  problems,  we  scan  the  images  with  a 
1  pm  difference  between  focal  planes  (up  to  9  planes),  and  then 
project  the  z-stack  into  a  single,  two-dimensional  image.  Since  the 
point  spread  function  of  the  60x  objective  lens  in  our  microscope 
measures  approximately  0.4  X  0.4  X  1.3  pm  (Majewska  et  al., 
2000b),  the  images  with  1  pm  focal  distance  have  enough  overlap 
to  produce  a  good  projection. 

A  major  problem  associated  with  time-lapse  imaging  of  brain 
slices  is  slice  movement  in  the  x,  y  and  z-directions.  To  minimize 
movement  in  z,  we  routinely  scan  extra  focal  planes  at  the  top  and 
the  bottom  of  the  z-stack.  If  movement  in  the  structure  of  interest 
is  evident  in  these  extra  focal  planes,  we  move  the  whole  z-stack 
1  pm  up  or  down.  Thus,  structures  of  interest  are  not  lost  from  the 
z-stack.  Structures  of  interest  can  also  move  out  of  the  images  by 
moving  in  the  x  or  y  directions.  To  minimize  this,  we  try  to  make 
the  slice  adhere  to  the  bottom  glass  of  the  chamber.  In  addition, 
we  avoid  placing  the  structures  of  interest  near  the  edge  of  the 
image,  and  if  the  structures  move  near  the  edge,  we  reposition  the 
specimen  so  that  the  structures  move  toward  the  center  of  the 
imaged  area.  Although  this  prevents  the  loss  of  structures  of  inter¬ 
est  from  the  z-stack,  xy  movement  results  in  the  drift  of  the  whole 
image  in  the  time-lapse  movies.  In  these  movies,  it  is  extremely 
difficult  to  observe  and  analyze  changes  in  morphology  and  fluo¬ 
rescence  intensity.  Therefore,  in  the  analysis  we  compensate  the 
xy  movement  as  described  below. 

Choice  of  Indicators  for  Two-Photon  Imaging 
of  Calcium 

Two-photon  excitation  of  calcium  indicators  loaded  via  whole-cell 
recording  is  ideal  for  imaging  calcium  during  action  potential  gen¬ 
eration  or  during  synaptic  excitation  (Yuste  and  Denk,  1995).  We 
have  used  both  calcium  green- 1  and  fluo-4  successfully  and  find  that 
both  indicators  are  excited  well  by  a  mode-locked  laser  at  800  nm. 
However,  we  find  that  each  indicator  is  suited  for  different  condi¬ 
tions.  Because  calcium  green- 1  is  brighter  at  low  calcium  concen¬ 
trations,  it  is  ideally  suited  for  visualizing  fine  structures  such  as 
dendritic  spines.  However,  in  part  due  to  its  high  fluorescence  at 
rest,  its  increase  in  intensity  on  binding  calcium  is  compromised 
(Fca/Ffree=  ~  14),  such  that  it  is  imperfect  for  detecting  small  or 
heavily  buffered  signals.  On  the  other  hand,  although  fluo-4  is  dim 
at  rest  and  therefore  demands  higher  excitation  laser  power,  it 


undergoes  a  large  change  in  fluorescence  on  binding  calcium 
(^’max/^7min=  — 100).  Thus,  fluo-4  is  ideal  for  imaging  small  or  heavily 
buffered  signals,  such  as  in  cortical  interneurons,  which  have  high 
endogenous  buffering  capacities.  We  have  used  calcium  green- 1 
over  the  concentration  range  of  50  to  200  pM,  and  fluo-4  from  100 
to  400 p M.  Because  increased  indicator  concentrations  cause  larger 
distortions  of  the  true  calcium  signals,  the  smaller  the  concentra¬ 
tions  yield  more  physiologically-relevant  data;  however,  if  higher 
laser  intensities  are  necessary  to  observe  signals  in  environments 
with  low  concentration,  photodamage  may  be  accelerated. 

In  our  hands,  indo-1  is  the  best  choice  when  shorter  wave¬ 
lengths  (<750  nm  for  two-photon  excitation)  are  needed  to  com¬ 
bine  calcium  imaging  with  uncaging  techniques.  The  two-photon 
sensitivity  of  calcium  indicators  is  available  from  http:// 
cellscience.bio-rad.com/products/multiphoton/Radiance2 1 00MP/ 
mpspectra.htm. 

Photodamage 

The  Achilles’  heel  of  all  live  imaging  is  phototoxicity.  Using  our 
two-photon  microscope,  we  experience  two  types  of  photodamage, 
when  imaging  neurons  with  too-high  laser  power  (>20  mW  at  the 
sample).  First,  unstained  cells  that  show  strong  autofluorescence 
suddenly  collapse,  emitting  high-intensity  light,  like  an  explosion. 
This  often  masks  the  structure  of  interest.  We  recommend  not 
including  any  structures  with  high  autofluorescence  in  the  imaging 
area.  Second,  stable  structures  in  labeled  neurons  can  show  abnor¬ 
mal  morphological  rearrangements,  particularly  beading.  This  type 
of  photodamage  is  nonlinear  and,  in  some  cases,  can  start  to  occur 
even  minutes  after  the  illumination  has  stopped.  Moreover,  even 
though  previous  scans  can  be  normal,  the  same  intensity  of  exci¬ 
tation  can  suddenly  cause  photodamage,  particularly  if  the  imaged 
structures  are  near  the  surface  of  the  slice  and  if  the  concentration 
of  dye  is  high.  As  is  the  norm  in  all  microscopy,  we  recommend 
imaging  with  as  low  an  excitation  intensity  as  possible  and  adjust¬ 
ing  this  intensity  for  each  sample  depending  on  the  brightness  of 
the  image. 

In  some  experiments  where  photodamage  becomes  a  persis¬ 
tent  problem,  we  use  the  antioxidant  Trolox  (Sigma,  10- 
lOOpM),  added  to  the  ACSF  (Sheenen  et  al.,  1996).  We  have  not 
noticed  any  effect  of  Trolox  on  the  physiology  of  the  neurons, 
although  it  has  been  suggested  that  high  concentrations  of  it  can 
block  NMDAR  (A.  Konnerth,  personal  communication). 

Second  Harmonic  Imaging 

Second  harmonic  generation  (SHG)  is  a  nonlinear  optical  effect  in 
which  the  incident  light  is  coherently  scattered  by  the  specimen  at 
twice  the  optical  frequency  and  at  certain  angles  (Lewis  et  al., 
1999;  see  Chapter  40,  this  volume).  The  signal  can  be  produced  by 
endogenous  structures  or  from  inserted  chromophores.  Unlike  flu¬ 
orescence,  in  which  emitted  photons  are  best  detected  in  the  epi 
configuration,  SHG  photons  are  best  detected  in  the  transmission 
path  of  the  microscope.  The  SHG  photons,  generated  at  the  focal 
spot  of  the  laser  in  the  sample,  are  collected  by  a  condenser  lens 
which  has  to  have  the  same  NA  as  the  objective  lens  in  order  to 
collect  the  whole  cone  of  light.  This  is  important  because  the  SHG 
radiation  in  the  forward  direction  is  restricted  to  certain  off-axis 
angles.  It  is  best  to  have  a  spatially  filtered  laser  beam  for  SHG 
because  it  is  a  coherent  process  —  the  spatial  filter  acts  as  a  point 
source  and  restores  the  Gaussian  wavefront  and  phase.  Spatial  fil¬ 
tering  can  be  achieved  by  a  telescopic  system  of  two  positive 
lenses  and  a  pinhole  placed  between  them  such  that  the  pinhole- 


730  Chapter  41  •  A.  Tashiro  et  al. 


to-lens  distance  equals  the  focal  length,  respectively.  In  front  of 
the  photomultiplier  tube  (PMT),  which  is  placed  in  the  transmis¬ 
sion  path,  is  a  narrow  band  (~20nm)  filter  centered  at  half  the 
wavelength  of  the  laser.  The  amplification  of  the  signal  is  done  by 
standard  methods  as  in  fluorescence  detection.  Specific  instruc¬ 
tions  to  adapt  a  two-photon  microscope  for  SHG  imaging  can  be 
found  in  Nikolenko  and  colleagues  (2003). 

Silicon-Intensified  Target  Camera  Imaging 

While  two-photon  imaging  results  in  a  high  spatial  resolution 
imaging  with  the  least  photobleaching  and  most  depth  penetration, 
there  is  a  cost,  as  with  any  laser-scanning  microscopy,  in  terms  of 
temporal  resolution  when  many  neurons  are  simultaneously  imaged 
because  the  laser  beam  must  be  systematically  moved  over  a  large 
area.  While  future  scanning  modifications  may  alleviate  this 
problem,  at  present  single-photon  fluorescence  imaging  has  proved 
a  useful  technique  for  measuring  the  fluorescence  changes  in  several 
neurons  simultaneously  (Peterlin  etal.,  2000 ;  Kozloski  eta!.,  200 1 ) . 

For  example,  using  calcium  imaging  with  fura-2,  the  typical 
detectable  change  in  fluorescence  that  can  be  measured  in  a  neuron 
from  a  single  action  potential  has  a  fast  onset  (<100ms  rise  time) 
and  slow  decay  (>2s;  Smetters  et  al.,  1999).  This  allows  one  to 
detect  the  time  of  occurrence  of  action  potentials  in  identifiable 
neurons  with  100  ms  temporal  resolution,  provided  that  frames  are 
acquired  at  a  rate  of  50  ms  or  less.  We  have  achieved  this  using  a 
SIT  camera  (Dage),  BX50WI  microscope  (Olympus),  40x  0.8  NA 
water-immersion  lens,  an  LG-3  frame  grabber  (Scion  Corp.)  in  a 
Power  Macintosh  7600,  and  NIH  image  software.  With  this  equip¬ 
ment,  we  can  view  an  area  of  cortex  of  320  x  240  pm  with  a  spatial 
resolution  of  640  x  480  pixels,  and  we  can  capture  frames  at  a  rate 
of  30  frames/s.  Using  fura-2  AM  loaded  slices  (see  above  for 
loading  technique),  we  can  image  dozens  of  neuronal  somata  using 
a  mercury  source  (Olympus),  a  380 nm  excitation  filter  and  510nm 
emission  filter.  The  gating  of  the  light  source  is  accomplished  via 
a  triggered  shutter  (Uniblitz),  which  fully  opens  in  less  than  30  ms 
after  the  triggering  TTL  pulse.  The  acquisition  of  a  frame  or  a  set 
of  frames  (a  “movie”)  can  also  be  initiated  from  an  external  trigger. 


The  delay  from  triggering  the  movie  to  acquisition  of  fully 
illuminated  frames  is  from  100  to  150  ms  to  compensate  for  the 
100  to  150  ms  lag  of  the  camera.  Each  captured  frame  uses 
307.3  kB  of  memory.  Homemade  macros  have  been  written  using 
NIH  image,  controlling  the  shutter  through  the  modem  port  of  the 
computer.  These  macros  enable  the  acquisition  of  movies  time- 
locked  to  either  depolarizing  current  pulses  in  current-clamped 
neurons  (as  in  Kozloski  et  al.,  2001)  or  to  a  large  PSC  recorded  in 
a  voltage-clamped  neuron  via  a  window  discriminator  (WPI). 

MORPHOLOGICAL  PROCESSING 
AND  ANALYSIS 

As  our  most  reliable  method  for  morphological  reconstructions,  we 
use  biocytin  fills  and  processing  to  recover  the  morphologies  of 
the  neurons  imaged  (Fig.  41.4). 

Biocytin  Protocol 

Following  electrophysiological  recordings,  the  slices  are  immedi¬ 
ately  placed  in  4%  paraformaldehyde  in  0.12  M  phosphate  buffer 
(PB)  and  kept  at  4°C  overnight.  Slices  are  then  cryoprotected  in  20% 
sucrose  in  0. 12AZPB  for  2  to  8  h  and  frozen  on  dry  ice  in  tissue  freez¬ 
ing  medium  (catalog  #H-TFM,  Triangle  Biomedical  Sciences). 
Upon  defrosting,  slices  are  rinsed  in  0. 12MPB  three  times  and  pre¬ 
treated  with  1%  hydrogen  peroxide  in  0.12MPB  for  30min  under 
agitation  at  room  temperature.  The  tissue  is  then  rinsed  in  0.02 M 
potassium  phosphate  saline  (KPBS)  and  incubated  in  Avidin- 
Biotin-Peroxidase  Complex  (catalog  #PK-6100,  Vector  Laborato¬ 
ries,  Inc.)  overnight  under  agitation  at  room  temperature  (lOpL 
solutionAand  lOpL solutionBper  lmLof0.02MKPBSand0.3% 
Triton-X).  Slices  are  rinsed  in  0.02 M  KPBS  three  times  and  incu¬ 
bated  in  0.7  mg/mL  3,3'-diaminobenzidine,  0.2  mg/mL  urea  hydro¬ 
gen  peroxide,  0.06MTris  buffer  (catalog  #D-4293,  Sigma- Aldrich) 
in  0.02M  KPBS  for  5  to  15  min.  Upon  completed  DAB  reaction, 
the  slices  are  rinsed  in  0.02 M  KPBS  and  mounted  in  Vectashield 
mounting  medium  (catalog  #H-1000,  Vector  Laboratories,  Inc.). 


FIGURE  41 .4.  Histological  reconstruction  of  neurons  using  biocytin  and  Neurolucida.  (A)  Biocytin  staining.  A  pyramidal  cell  in  a  coronal  section  from  mouse 
visual  cortex  was  filled  intracellularly  with  biocytin  and  then  processed  for  visualization.  Intracellular  biocytin  staining  enables  a  large  signal/noise  and  allows 
a  fairly  accurate  reconstruction  of  the  dendritic  arbor  of  the  neuron.  Structures  as  small  as  spines  (-1pm  in  diameter)  can  be  visualized.  Arrowhead  indicates 
the  axon.  Scale  bar  =  50  pm.  (Courtesy  of  Z.  Peterlin  and  A.  Tsiola.)  (B)  Neurolucida  reconstruction.  (Left  panel)  Confocal  image  of  a  pyramidal  neuron  from 
a  P7  mouse  slice,  cultured  for  6  days.  The  cells  were  transfected  with  EGFP  using  a  gene  gun.  (Right  panel)  Reconstruction  of  the  neuron  using  Neurolucida. 
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Anatomy  with  a  Two-Photon/Neurolucida 
System 

We  have  experimented  with  direct  two-photon  reconstructions  of 
the  cells  in  the  brain  slices.  This  procedure  enables  the  investiga¬ 
tor  to  quickly  reconstruct  the  morphology  of  the  imaged  cell. 
Images  of  neurons  from  both  live  and  fixed  tissue  can  be  taken 
by  the  two-photon  microscope  and  the  stacks  of  images  can  be 
imported  into  a  computerized  reconstruction  and  measuring 
program,  such  as  Neurolucida  (Microbrightfield,  Brattleboro,  VT). 

Two-Photon/Neurolucida  Protocol 

When  examining  dendritic  morphology  alone,  z-stacks  of  the 
neuron  of  interest  can  be  captured  using  a  20x  or  40x  objective. 
When  spines  and  filopodia  are  also  of  interest,  a  60x  objective  with 
a  2.5x  digital  zoom  yields  good  images.  For  detailed  reconstruc¬ 
tions  of  the  protrusions  from  the  entire  neuron,  small  overlapping 
sections  of  the  neuron  are  imaged  using  the  60x  objective  and  2.5x 
digital  zoom.  After  capturing  the  z-stack  of  images,  they  should  be 
saved  as  a  Fluoview  Multi-Tiff  (*.tiff)  and  transferred  to  a  com¬ 
puter  running  the  Neurolucida  software.  The  images  can  either  be 
burned  onto  a  recordable  CD,  or  can  be  transferred  to  the  com¬ 
puter  over  a  local  area  network. 

The  neurons  can  then  be  reconstructed  using  Neurolucida 
software.  The  stack  can  be  opened  using  the  Image  Stack  Open 
command  under  the  File  icon  on  the  menu  bar.  Once  the  image  is 
opened,  the  brightness  and  contrast  of  the  image  can  be  adjusted 
by  selecting  Image  Effects  under  the  Video  icon.  Once  the  picture 
is  in  clear  focus,  the  neuron  is  ready  to  be  reconstructed.  The  image 
must  be  calibrated  and  a  reference  point  chosen.  The  PgUp  and 
PgDn  keys  on  the  keyboard  will  allow  for  scrolling  through  the 
stack  of  images.  The  mouse  is  used  to  trace  the  neuron  and  the 
type  of  tracing  can  be  set  by  right  clicking  on  the  image.  We  find 
that  rubber  line  tracing  is  very  effective.  The  thickness  of  the  line 
can  be  determined  by  the  scroll  feature  on  the  mouse  (the  wheel 
or  center  button).  In  the  tool  bar  we  are  able  to  select  the  section 
of  the  neuron  we  are  drawing,  for  example,  apical  dendrite  or  cell 
body.  Nodes  and  branches  are  added  by  right  clicking  on  the  image 
during  tracing.  After  the  contour  is  completed,  it  can  be  saved  and 
opened  in  the  Neuroexplorer  program.  This  program  will  allow  for 
easy  analysis  of  the  reconstruction. 

Correlated  Electron  Microscopy 

Although  we  can  image  neuronal  structures  in  live  brain  slices  at 
quite  high  resolution  with  two-photon  microscopy,  for  some  ques¬ 
tions,  such  as  confirming  the  existence  of  a  synapse,  we  find  it 
necessary  to  use  electron  microscopy.  We  have  pioneered  the 
combination  of  two-photon  live  imaging  with  serial  thin-section 
electron  microscopy  to  enable  us  to  examine  the  ultrastructure  of 
dendritic  spines  and  axonal  filopodia  (Dunaevsky  et  al.,  2001; 
Tashiro  et  al.,  2003). 

Protocol  for  Two-Photon/Electron  Microscopy 
Imaging  of  GFP-Labeled  Cells 

Neurons  are  transfected  with  eGFP  using  biolistics.  Slices  imaged 
with  two-photon  microscopy  are  fixed  with  5%  glutaraldehyde  in 
PB  for  1  h.  The  slices  are  then  embedded  in  3%  agar  and  resec¬ 
tioned  at  75  pm  with  a  vibratome  (Technical  Products  Interna¬ 
tional,  St.  Louis).  After  locating  the  imaged  neuronal  structure  in 
the  sections  using  a  fluorescence  microscope,  the  sections  are 
immunostained  with  anti-GFP  antibody  (Roche  Diagnostics  Corp.) 
overnight  at  4°C,  and  then  with  peroxidase-conjugated  goal  anti¬ 


rabbit  IgG  antibody  (Roche  Diagnostics)  for  1  h  at  room  tempera¬ 
ture.  After  developing  with  DAB,  the  sections  are  postfixed  in  1% 
osmium  tetroxide  in  PB,  dehydrated,  and  then  infiltrated  with  Epox 
812  resin  (Fullam),  placed  flat  in  resin  between  two  plastic  slides, 
and  polymerized  in  the  oven  at  60°C.  After  polymerization,  the 
plastic  slides  are  separated  and  the  imaged  areas  of  interest  were 
cut  out,  mounted  on  a  blank  block,  and  sectioned  to  10  pm  for 
examination  under  phase  contrast.  The  10  pm  sections  with  the 
imaged  areas  of  interest  were  then  remounted  on  a  blank  block, 
thin  sectioned,  and  examined  in  the  electron  microscope  (JEM 
1200EX).  Imaged  mossy  terminals  with  filopodia  are  recon¬ 
structed  from  serial  sections  (see  also  Chapter  49,  this  volume ). 

Protocol  for  Two-Photon/EM  Imaging  of 
Biocytin  Labeled  Cells 

Neurons  are  bolus  injected  with  electrode  containing  0.4%  bio¬ 
cytin  and  1  mM  Alexa-488  as  described  before.  Immediately  after 
imaging,  brain  slices  are  immersion  fixed  with  4%  paraformalde¬ 
hyde  and  0.5%  glutaraldehyde  in  0.1  MPB  overnight  at  4°C.  They 
are  washed  three  times  10  min  on  shaker  at  room  temperature  with 
0.1  MPB.  Then,  slices  are  incubated  in  1%  hydrogen  peroxide, 
50%  ethanol,  and  0.05 MPB  for  30 min  at  room  temperature  on 
shaker  for  eliminating  internal  peroxidase  activity.  After  washing 
again,  they  are  incubated  with  ABC  (Vectastain)  overnight  at  4°C 
and  1  h  at  room  temperature.  After  washing  again,  they  are  DAB 
reacted  using  fast  DAB%c  from  Sigma  for  about  3  min.  Then,  pic¬ 
tures  are  taken  from  the  imaged  region  to  facilitate  its  EM  recon¬ 
struction.  At  this  point,  slices  can  be  postfixed  with  glutaraldehyde. 
Finally,  slices  are  osmicated  by  1  %  osmium  tetroxide  together  with 
7%  glucose  and  0.005%  CaCl2.  If  there  is  a  problem  of  revealing 
the  imaged  region  with  biocytin.  slices  can  be  resectioned  after  fix¬ 
ation.  For  this,  slices  are  embedded  in  3%  agarose  and  resectioned 
using  a  vibratome.  Freeze-thawing  is  another  method  to  increase 
the  penetration  of  reactants  like  ABC.  For  this,  slices  are  kept  in 
30%  sucrose  until  the  slices  sink  to  the  bottom,  and  then  dipped 
(in  the  plastic  container  with  sucrose)  into  liquid  nitrogen. 

As  a  variant  on  this  protocol,  for  fixing  slices  for  EM,  prepare 
a  fix  solution  of  4%  paraformaldehyde,  0.05%  glutaraldehyde,  and 
15mL  of  saturated  picric  acid  in  100 mL  of  0.1  MPB.  Slice  should 
be  kept  in  this  solution  for  approximately  3h.  Thereafter,  slices 
may  be  stored  in  a  solution  of  0.05M  sodium  azide  in  0.1MPB. 
Then  follow  the  protocol  above. 

Morphological  Classification  of  Neurons  Using 
Cluster  Analysis 

One  of  the  problems  in  classifying  cortical  neurons  is  their  het¬ 
erogeneity  and  the  vast  number  of  parameters  that  can  be  used  for 
this  purpose.  These  parameters  usually  encompass  a  massive  array 
of  physiological,  morphological,  and,  most  recently,  gene  expres¬ 
sion  data.  A  rigorous  approach  to  classifying  cortical  neurons  must 
involve  a  thorough  analysis  of  the  structure  of  the  data  before  one 
attempts  to  assign  neurons  to  certain  clusters.  Principal  component 
analysis  (PCA)  and  cluster  analysis  (CA)  are  valuable  multi¬ 
variate  data  analysis  methods  that  can  be  used  jointly  with  CA  to 
address  these  issues  (Kozloski  et  al.,  2001). 

Protocol  for  PCA/CA 

As  a  first  step,  we  perform  a  PCA  analysis  using  Statistica  on  the 
variables  automatically  measured  by  the  Neurolucida  program.  In 
a  second  step,  we  perform  cluster  analysis  (Wards’s  methods),  also 
using  Statistica. 
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Let  X  denote  an  m  x  n  matrix  of  m  cells  (cortical  neurons)  with 
n  measured  or  computed  parameters.  The  elements  of  this  matrix 
denoted  by  xtj  represent  the  value  of  the  jth  parameter  for  the  ith 
neuron.  For  values  of  n  =S  3,  visualization  of  data  is  impossible 
and  thus  the  overall  distribution  of  cells  in  their  n-dimensional 
parameter  space  is  not  accessible.  In  the  case  of  cortical  neurons 
the  number  of  dimensions  can  easily  exceed  tens  or  even  hundreds 
with  the  inclusion  of  gene  expression  data.  The  major  advantage 
of  PCA  methods  is  to  help  reconstruct  a  new  parameter  space  with 
(optimally)  three  dimensions  in  such  a  way  that  it  still  faithfully 
represents  the  data  with  the  minimum  loss  of  information  during 
the  process  of  space  transformation.  This  transformation  involves 
computating  eigenvalues  (£)  and  eigenvectors  (n)  of  the  n  x  n 
correlation  matrix  (R)  of  the  original  data  matrix  X.  The  goal 
is  to  map  the  n-dimensional  vector  describing  the  parameters  of 
each  neuron  to  a  vector  with,  for  example,  three  dimensions,  so 
that  each  neuron  can  be  plotted  in  a  three-dimensional  (3D)  graph. 
The  desired  new  dimensions,  also  called  principal  factors  or 
components,  are  extracted  from  the  original  space  through  the 
eigenvalue  decomposition  of  the  correlation  matrix.  This  decom¬ 
position  will  provide  the  eigenvectors  (principal  components) 
and  its  related  eigenvalue  (total  variance  accounted  for  by  each 
component).  Principal  component  scores  computed  for  each  prin¬ 
cipal  component  give  the  coordinates  of  neurons  in  the  reduced 
space.  PCA  methods  also  provide  a  matrix  of  factor  or  principal 
component  loadings  that  show  the  correlation  value  of  each 
original  parameter  with  the  newly  computed  components.  These 
loading  values  show  which  parameters  contribute  significantly 
(highly  positive  or  negative  correlations)  to  the  derived  dimen¬ 
sions.  It  can  be  seen  that,  while  some  original  parameters  con¬ 
tribute  significantly  to  some  components,  they  contribute  very  little 
to  others.  Meanwhile,  some  original  variables  only  contribute  to 
principal  components  that  carry  a  relatively  low  proportion  of  the 
total  variance  (low  relative  eigenvalue).  These  latter  parameters 
are  thus  found  to  be  less  important  in  the  characterization  of  the 
data  structure. 

The  final  stage  of  classification  involves  application  of  CA  to 
the  new  coordinate  values  (principal  component  scores)  of  the 
neurons  in  the  reduced  space.  Appropriate  linkage  rules  for  CA  can 
be  chosen  based  on  the  apparent  shape  of  the  clusters  as  seen  in 
the  scatter  plot  of  neurons  in  the  principal  component  space.  Appli¬ 
cation  of  PCA  before  CA  allows  a  rational  choice  of  linkage  rules 
which  would  lead  to  a  better  segregation  of  clusters. 

IMAGE  PROCESSING 

Compensation  for  the  Drift  and  the  Vibration  of 
the  Slices 

As  described  above,  one  of  our  problems  is  the  movement  of  slices 
that  produces  the  drift  of  images  during  long  (>10min)  time-lapse 
movies.  This  drift  of  structures  of  interest  makes  it  particularly  dif¬ 
ficult  to  observe  and  analyze  changes  in  morphology  and  fluores¬ 
cence  intensity.  Indeed,  spine  motility  was  only  discovered  in  our 
laboratory  after  the  alignment  of  time-lapse  movies  was  performed 
(Dunaevsky  et  al.,  1999).  Therefore,  we  always  compensate  for 
the  movement  of  slices. 

Although  manual  alignment  of  time-lapse  movies  using  the 
structures  which  are  always  stable  as  references  works  well,  this 
is  time  consuming.  Instead,  we  have  been  using  three  automatic 
methods  of  alignment.  Two  of  them  are  custom-made  programs 
based  on  the  overlap  between  the  images  and  the  center  of  mass. 


respectively.  These  programs  are  written  in  NIH  image  and  ImageJ 
software,  respectively.  The  other  is  commercially  available  soft¬ 
ware  from  AutoQuant  Imaging,  Inc. 

Alignment  Based  on  the  Overlap 
Between  Images 

This  alignment  program  is  written  as  an  NIH  image  macro.  The 
principle  of  this  macro  is  quite  simple.  Take  two  images,  project 
these  two  images,  and  compare  the  average  pixel  values  of  the  pro¬ 
jected  image  and  the  original  image.  The  more  similar  the  two 
original  images  are  and  the  more  overlap  they  have,  the  closer  the 
two  average  pixel  values  will  be. 

Protocol  for  Overlap  Alignment 

Before  performing  automatic  alignment,  images  are  thresholded  to 
highlight  neuronal  structures.  Ideally,  the  original  images  should 
work  as  well.  However,  because  optical  noise  changes  during  time- 
lapse  sequences,  in  practice,  thresholded  images  work  better.  The 
macro  selects  two  images  and  performs  multiple  iterations  of  an 
alignment  procedure  using  the  above  principle.  Each  iteration 
comprises  shifting  the  second  original  image  by  1  to  a  certain 
number  of  pixels  in  four  directions  (up,  down,  left,  right),  com¬ 
paring  the  average  pixel  values  of  the  projected  images  and  the 
first  image,  determining  the  optimal  shift,  where  the  average  pixel 
values  of  the  projected  image  and  the  original  image  are  closest, 
and  moving  the  second  image  by  this  optimal  shift.  From  the 
second  iteration  on,  images  are  only  shifted  in  three  directions 
because  one  of  four  directions  is  toward  a  starting  point  to  the  pre¬ 
vious  iteration.  To  avoid  including  the  blank  peripheral  area  which 
arises  from  the  shift  of  the  second  image,  only  the  area  where  the 
first  original  image  and  the  shifted  second  image  overlap  are  used 
to  calculate  the  average  pixel  values  of  the  projected  image  and 
the  first  original  image.  These  iterations  are  repeated  until  an  iter¬ 
ation  finds  that  the  original  position  of  the  second  image  in  the  iter¬ 
ation  is  optimal,  or,  in  other  words,  the  average  pixel  values  of  the 
projected  image  and  the  original  image  are  closest  in  the  ending 
position  of  the  iteration. 

This  alignment  macro  works  very  well  for  most  time-lapse 
sequences  of  projected  images  when  the  images  have  more  area 
with  stable  structures  than  with  unstable  ones.  For  example,  in  the 
case  of  the  time-lapse  imaging  of  dendritic  spine  motility,  the  mor¬ 
phological  changes  in  spines  are  quite  small  compared  to  the  sta¬ 
bility  of  the  much  larger  dendritic  shafts.  Thus,  in  most  cases,  the 
alignment  macro  works  reasonably  well.  When,  in  rare  cases, 
alignment  does  not  work  completely,  we  align  the  movies  manu¬ 
ally  using  NIH  image  software. 

We  use  this  alignment  for  z-stacks  where  the  structure  of  inter¬ 
est  shifts  between  focal  planes  because  of  slice  movement.  Images 
in  z-stacks  are  not  aligned  as  well  as  time-lapse  movies  consisting 
of  projected  images  since  each  image  in  a  z-stack  is  different  from 
the  next  image  (with  small  overlapping).  Although  not  ideal,  the 
same  macro  helps  to  align  z-stacks.  We  check  all  aligned  stacks 
and  correct  them  manually  if  the  alignment  is  not  good. 

Alignment  Based  on  the  Center  of  Mass 

We  have  created  a  different  alignment  program  as  a  plug-in  for 
ImageJ  software  using  Java  programming  language.  This  program 
is  based  on  the  calculating  relative  positions  of  the  center  of  mass 
of  the  drifting  objects. 


Imaging  Brain  Slices  •  Chapter  41  733 


Protocol  for  Center  of  Mass  Alignment 

The  coordinates  of  center  of  mass  are  calculated  by  using  the  pixel 
value  as  a  mass.  For  meaningful  calculations,  a  cut-off  value  is 
used  in  order  to  prevent  including  background  pixels  in  the  calcu¬ 
lation.  Our  program  calculates  the  center  of  the  first  frame  in  the 
image  stack  and  uses  these  coordinates  as  the  reference  point  for 
aligning  the  rest  of  the  image  stack  (i.e.,  It  considers  the  first  frame 
as  not  drifted). 

The  program  then  calculates  coordinates  of  the  center  of 
mass  of  the  each  image  in  the  stack.  The  program  then  shifts 
each  image  in  order  to  align  images  in  such  a  way  that  centers  of 
mass  of  all  frames  have  the  same  coordinates  (they  overlap  each 
other). 

If  a  region  of  interest  (ROI)  is  chosen,  the  program  aligns  to 
the  center  of  mass  of  the  ROI.  The  ROI  can  be  any  shape  and 
allows  aligning  by  using  center  of  mass  of  a  selected  object,  not 
whole  image. 

This  algorithm  is  quite  simple,  fast,  and  works  well  if  the 
center  of  mass  is  always  calculated  from  the  same  structure. 
However,  drift  of  the  slices  can  make  a  new  structure  appear  or  a 
part  of  a  structure  disappear  from  the  edge  of  the  image.  Because 
this  edge  effect  can  make  the  center  of  mass  move  to  completely 
different  positions  in  different  images,  purely  automatic  alignment 
is  sometimes  unsuccessful.  On  the  other  hand,  choosing  a  stable 
structure  as  the  ROI  for  alignment  can  prevent  this  artifact. 


The  next  logical  step  in  the  development  of  this  type  of  algo¬ 
rithm  could  be  using  spatial  moments  of  higher  order  (center  mass 
coordinates  are  spatial  moments  of  the  first  order;  and  the  total 
mass  is  spatial  moment  of  zero  order).  For  example,  including  in 
the  alignment  algorithm,  spatial  moments  of  the  second  order  and 
third  order  will  allow  compensating  not  only  for  drift,  but  also  for 
image  rotation  and  squeezing. 

Online  Cell  Detection  of  Neurons 

Using  AM  loading,  we  can  simultaneously  image  over  3000 
neurons  (Fig.  41.3).  To  analyze  fluorescence  changes,  such  as 
those  indicating  Ca2+  concentration,  of  a  large  number  of  individ¬ 
ual  cells,  we  need  to  identify  and  select  all  the  neurons.  As  manual 
selection  of  this  many  cells  is  quite  tedious  and  cannot  be  per¬ 
formed  online,  automatic  cell  detection  algorithms  were  developed 
in  ImageJ  (NIH,  Bethesda,  MD)  and  Matlab  (Math Works,  Natick, 
MA)  [Fig.  41.5(A)], 

Protocol  for  Center  of  Mass  Alignment 

First,  time-lapse  movies  are  collapsed  in  time,  creating  a  single 
projected  image  by  averaging  the  fluorescence  of  each  pixel 
throughout  the  recording.  This  effectively  reduces  the  amount  of 
spatial  noise  in  the  image  and  reveals  smaller  elements,  such  as 
dendrites.  Due  to  the  slightly  unequal  loading  of  different  regions 


FIGURE  41.5.  Image  processing  algorithms.  (A)  Cell  detection  algorithm.  (Left)  An  original  image  of  a  fura-2  loaded  acute  slice.  Note  how  the  staining  is  not 
even  throughout  the  image.  (Right)  An  image  with  outlines  of  all  detected  cells.  The  contours  of  the  cells  are  drawn.  (B)  De-noising  algorithms.  (Upper  panels) 
Images  of  a  dendritic  growth  cone  from  a  cortical  pyramidal  neuron.  All  three  images  are  shown  with  the  same  brightness  and  contrast.  (Lower  panels)  Bina¬ 
rized  images.  All  three  images  are  thresholded  with  the  same  pixel  value.  (Left)  Original  images.  (Center)  Gaussian-filtered  images.  (Right)  Images  de-noised 
with  wavelet  transformation. 
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of  the  slice,  the  average  fluorescence  level  of  cells  in  the  same  slice 
often  vary,  making  automatic  detection  problematic.  To  account 
for  these  spatial  fluorescence  variations,  the  value  of  each  pixel 
is  normalized,  dividing  it  by  the  average  fluorescence  of  a  25  x 
25|im  square  centered  at  that  pixel.  In  the  resulting  image,  the 
fluorescence  level  is  almost  constant  for  all  cells  and  is  typically 
between  1.2  and  1.5.  Contours  can  be  plotted  at  a  manually  chosen 
value  in  this  range  using  the  Matlab  contour  drawing  algorithm. 
Every  closed  contour  corresponds  to  the  accurately  detected 
boundary  of  a  fluorescent  entity  in  the  imaged  field.  This  method 
identifies  practically  all  cells,  as  well  as  a  large  number  of  smaller 
non-cellular  elements  in  the  slice  [Fig.  41.3(A)].  By  measuring  the 
average  fluorescence  value  of  the  pixels  inside  each  contour  as  a 
function  of  time,  we  can  quickly  reconstruct  fluorescence  changes 
of  large  populations  with  single-cell  resolution.  In  total,  this  allows 
us  to  measure  calcium  fluorescence  signals  from  roughly  400  to 
1400  entities  per  slice. 

Image  De-Noising  Using  Wavelets 

The  external  PMT  in  our  two-photon  system  produces  uncorrelated 
dark  noise,  primarily  of  thermal  origin.  This  type  of  noise  has 
a  strong  dependence  on  applied  bias  voltage  (Majewska  et  al., 
2000b;  www.twophoton.com).  It  is  therefore  important  to  correctly 
choose  the  bias  voltage  in  order  to  balance  the  resulting  gain  of 
the  PMT  versus  noise. 

This  type  of  noise  is  intrinsically  random  and  does  not  have 
specific  spectral  components.  Therefore,  it  cannot  be  distinguished 
from  the  real  signal  by  classic  methods  of  linear  filtration.  For 
example,  the  widely  used  mean  and  Gaussian  windows  filters  are 
not  efficient  in  terms  of  removing  this  type  of  noise  (Fig.  41.5). 
Indeed,  background  noise  removal  is  very  important  for  quantita¬ 
tive  analysis  of  thresholded  images,  and  linear  filtration  of  noisy 
images  usually  gives  artifacts  [Fig.  41.5(B)]. 

As  an  alternative  approach,  we  use  wavelet  transformation  for 
the  purpose  of  image  de-noising  (Fio,  2003).  Wavelet  transforma¬ 
tion  is  widely  used  for  signal  compression  and  de-noising  and  rep¬ 
resents  further  development  of  classic  methods  of  analysis  such 
as  Fourier,  Gabor,  and  short-time  Fourier  transformation.  The 
wavelet  transformation  gives  full  representation  of  the  signal  [for 
one-dimensional  (ID)  time  signals,  a  correct  representation  in 
time  and  frequency  domains;  for  two-dimensional  (2D)  images,  a 
representation  of  spatial  frequencies  and  coordinates].  Whereas 
Fourier  transformation  presumes  the  infinite  dimensions  of  the 
image  space,  wavelet  transformation  is  inherently  local  and  gives 
a  better  representation  of  naturally  occurring  finite-size  objects  in 
image. 

Protocol  for  Wavelet  De-Noising 

The  general  de-noising  procedure  consists  of  the  following  steps: 

In  case  of  image  de-noising,  an  individual  image  is  represented 
as  2D  array  of  numbers  (pixel  values).  The  wavelet  transformation 
decomposes  this  2D  signal  into  wavelet  space  by  using  a  specified 
wavelet  family.  In  case  of  discrete  wavelet  transformations,  it 
computes  the  detail  coefficient  of  the  signal  up  to  the  certain  pre¬ 
defined  level.  For  the  purpose  of  signal  de-noising,  the  detail  coef¬ 
ficients  at  all  levels  of  decomposition  have  to  be  thresholded.  The 
numerical  value  of  the  threshold  can  be  chosen  based  on  the  noise 
model  used. 

There  are  several  methods  of  thresholding.  The  practical 
choice  of  the  method  depends  on  the  nature  of  the  signal  and  the 
chosen  model.  Reverse  wavelet  reconstruction  is  then  performed 
using  the  modified  detail  coefficients,  and  the  filtered  image  is 


regenerated  from  thresholded  detail  coefficients  by  using  the 
inverse  wavelet  transformation. 

The  latest  release  of  the  Wavelet  Toolbox  (version  2.2)  for 
Matlab  (The  MathWorks  Inc.,  Natick,  MA)  provides  a  variety  of 
ready-to-use  tools  for  wavelet  transformation  and  signal  de- 
noising.  There  is  an  interactive  graphical  user  interface  in  Wavelet 
Toolbox,  which  simplifies  the  task  of  choosing  the  parameters  for 
signal  de-noising  and  compression.  In  the  simplest  case,  practical 
de-noising  can  be  done  based  on  the  visual  perception  of  the  de- 
noising  quality,  but  also  using  different  recovery  criteria  (e.g., 
based  on  entropy  estimation). 

For  large-scale  image  processing,  we  created  a  custom  code 
in  Matlab,  which  performs  simple  image  de-noising  based  on  the 
chosen  model.  As  a  model,  we  use  wavelet  decomposition  to  level 
4  with  symlet-6  wavelets,  and  soft  thresholding  (see  more  theory 
of  de-noising  procedure  in  Wavelet  Toolbox  documentation).  The 
script  processes  raw  images  in  a  multi-TIFF  format,  gives  accept¬ 
able  de-noising,  and  does  not  change  the  quantitative  values  of 
intensity  in  the  principal  image  details  [see  Fig.  41 .3(B)].  Our  algo¬ 
rithm  processes  each  frame  individually,  therefore  processing  time 
depends  linearly  on  the  size  of  the  image  stack.  The  algorithm  is 
not  memory  demanding  and  turned  out  to  be  relatively  fast  —  it 
takes  approximately  5  s  to  process  a  800  x  600  pixel,  16-bit  image 
on  a  1.9 GHz  Pentium  4  PC. 

SUMMARY 

•  Brain  slices  are  convenient  preparations  because  they  permit 
the  easy  manipulation  of  their  environment,  access  for  imaging 
or  electrophysiological  equipment,  and  preservation  of  three- 
dimensional  organization  of  the  brain  region  studied. 

•  We  describe  the  techniques  of  live-slice  imaging  we  use  in 
our  laboratory,  including  slice  preparation  (acute  and  cultured 
slices),  cell  labeling  (biolistics,  diolistic,  calistics,  injection 
with  patch  electrodes,  and  AM  loading),  morphological  pro¬ 
cessing  and  analysis,  imaging  procedures  (two-photon,  second 
harmonic,  and  camera  imaging),  and  image  processing. 

•  Although  in  vivo  imaging  techniques  have  recently  developed 
in  many  species,  brain-slice  imaging  has  advantages  for  study¬ 
ing  many  questions  and  will  be  increasingly  important  for 
cortical  research. 
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INTRODUCTION 

It  is  likely  that  cell  behavior  is  regulated  by  the  summation  of  sub- 
cellular  transduction  events  and  these  are  generally  linked,  non- 
linearly,  to  subsequent  stages  of  signal  transduction.  Hence,  a  full 
understanding  of  signal  transduction  can  only  arise  from  the  analy¬ 
sis  of  subcellular  behavior  and  this  is  one  reason  why  microscopic 
imaging  has  become  so  important.  To  carry  this  idea  further,  since 
the  cell  is  a  three-dimensional  (3D)  structure,  methods  that  provide 
3D  imaging  (as  in  confocal  and  multi-photon  methods)  are  likely 
to  provide  better  insight  than  even  widefield/deconvolution 
imaging  which  is,  itself,  an  improvement  on  either  widefield  or 
simple  photometric  methods. 

To  analyze  cell  behavior,  we  need  to  measure  something,  and 
then  after  a  period  of  time,  we  need  to  measure  it  again.  This  often 
implies  the  use  of  probes  that  can  report  the  behavior  of  interest. 
In  the  case  of  intracellular  ions,  because  calcium  has  been  found  to 
be  a  ubiquitous  second  messenger  system,  it  is  not  surprising  that 
many  methods  for  measuring  it  have  been  developed,  each  with 
unique  advantages  and  disadvantages.  Here,  we  will  tend  to  focus 
on  calcium  and  pH  measurements  in  living  cells,  but  the  ideas  and 
problems  that  will  be  discussed  are  generally  applicable  to  all 
methods  that  employ  optical  probes  of  function;1  the  differences 
between  approaches  and  techniques  reside  in  the  degree  of  applic¬ 
ability  rather  than  the  presence  or  absence  of  any  single  problem. 

The  continuing  effort  to  improve  signal-to-noise  ratio,  speci¬ 
ficity,  and  resolution  (spatial  and  temporal)  of  these  techniques  has 
brought  wonderful  new  insight  into  our  understanding  of  cell  func¬ 
tion.  Each  major  new  advance  seems  to  have  taken  approximately 
a  decade  to  achieve  and  each  decade  has  improved  spatio- 
temporal  resolution  by  something  like  2  orders  of  magnitude  (for 
constant  signal-to-noise).  Thus,  from  the  development  and  appli¬ 
cation  of  aequorin  to  giant  nerve  and  muscle  cells  in  the  1960s 
(which  supplanted  tissue-level  chemical  methods),  we  are  now 
measuring  subcellular  events  on  the  scale  of  microns  with  fluo¬ 
rescent  probes.  Resolution  in  both  space  and  intensity  is  currently 
limited  by  the  quantal  nature  of  light  itself.  Further  major  advances 
in  resolution  may  be  confounded  by  the  problems  of  excessive 
buffering  (or  perturbation)  of  responses  by  the  probe  and  by  cell 
damage  caused  by  the  excitation  light.  This  pessimism  is  tempered 
by  the  fact  that  a  little  more  than  two  decades  ago,  it  was  thought 
unlikely  that  it  would  ever  be  possible  to  resolve  the  microscopic 
gradients  of  calcium  that  exist  inside  the  muscle  sarcomere  during 
activation  [which  at  that  time  were  only  predicted  on  the  basis  of 
computer  modeling  (Cannell  and  Allen,  1984)].  However,  the  early 


1  Functional  probes  also  exist  for  cAMP,  Na,  K,  Mg,  and  many  other  ions  (see 
Chapters  16  and  17,  this  volume). 


1990s  saw  the  potential  of  the  laser-scanning  confocal  microscope 
(LSCM)  applied  to  Roger  Tsien’s  dyes  (e.g..  for  reviews  see 
special  edition  of  Cell  Calcium  1 1  (2-3),  Williams  (1993).  The  dis¬ 
covery  of  calcium  sparks  (Cheng  et  al,  1993;  Cannell  et  al.,  1994; 
see  Fig.  42.1)  while  one  of  the  authors  (MBC)  was  developing 
methods  needed  to  look  for  some  spatial  non-unformity  in  the 
cardiac  Ca^  transient  (during  a  sabbatical  in  Dr.  Federer's  labora¬ 
tory)  was  completely  unexpected  and  would  probably  not  have 
occurred  without  the  LSCM.  Imagine  the  excitement  at  seeing 
these  events  rise  out  of  the  background  noise  displayed  on  a 
dimmed  computer  screen  in  a  blacked-out  laboratory. 

There  is  something  compelling  about  imaging  responses  under 
carefully-optimized  experimental  conditions,  perhaps  related  to 
the  immediacy  and  tangibility  of  the  results.  Seeing  is  believing 
and  imaging,  in  its  many  modalities,  allows  us  to  see  otherwise 
invisible  and  unexpected  cell  signaling  events  as  they  occur. 
Provided  artifacts  are  recognized  and  rejected,  new  insights 
develop  as  we  try  to  understand  the  cellular  basis  of  what  has  been 
observed.  Therein  lies  a  caveat:  Poor  science  inevitably  results 
from  not  appreciating  the  limitations  of  the  method.  Having  a  new 
instrument  or  method  does  not  remove  the  possibility  that  artifac- 
tual  results  may  be  obtained  from  excessive  perturbation  of  the 
experimental  system  and  measuring  something  about  a  system 
inevitably  involves  some  sort  of  perturbation.  Both  light  and 
probes  can  easily  damage  cells  or  modify  their  responses.  As  we 
seek  to  resolve  ever  smaller  and  faster  events,  the  perturbation  of 
natural  responses  becomes  larger. 

In  general,  experimental  problems  arise  from: 

1.  The  quantal  nature  of  light  (it  is  not  possible  to  directly 
measure  a  response  that  is  associated  with  a  change  of  signal 
of  less  than  a  single  photon/pixel). 

2.  Buffering  the  cell  response  (any  probe  must  interfere  with  the 
cell  processes  by  its  interaction  with  the  chemical  process  of 
interest). 

3.  Limited  time  resolution  (all  chemical  reactions  take  time 
before  the  products  become  available  for  analysis). 

4.  Damage  to  the  cell  either  by  the  toxicity  of  the  probe  or 
damage  by  the  light  irradiating  the  system. 

5.  Limited  spatial  resolution  associated  with  the  wavelength  of 
light  itself. 

In  this  chapter,  we  will  discuss  how  these  problems  relate  to 
the  measurement  of  calcium  levels  inside  living  cells,  although  the 
discussion  applies  to  all  other  optical  measurements  as  well. 

The  Limiting  Case 

If  we  confine  our  discussion  to  optical  resolution,  the  smallest  cell 
volume  that  can  be  observed  in  practical  light  microscopy  is  about 
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FIGURE  42.1.  Two  consecutive  confocal  images  of  a  fluo-3  loaded  cardiac 
myocyte.  The  areas  of  increased  fluorescence  are  spontaneous  Ca++  sparks. 
Note  the  scale  of  this  image.  The  lower  panel  shows  sequential  confocal  images 
of  spontaneous  Ca"  waves  propagating  and  colliding  within  a  cardiac  cell. 
Scale  bar  20  pm.  Frame  scan  time,  0.7  s. 


0.06  fl  (i.e.,  0.25  x  0.25  x  0.5  pm).  If  a  single  ion  resides  in  this 
volume  (high)  the  concentration  is  about  1  pM/L.  In  the  case  of 
calcium,  this  is  a  high  physiological  level.  However,  we  cannot 
directly  measure  an  ion  in  solution  inside  a  cell:  it  must  be  bound 
to  a  probe  whose  physical  properties  are  altered  by  that  binding. 
If  the  ion  is  always  bound  to  the  probe  then  we  must  detect  a  single 
probe  molecule  in  that  volume  (note  that  we  have  also  reduced  the 
level  of  free  calcium  to  zero  at  the  same  time  —  a  serious  problem 


that  will  be  discussed  later).  It  is  unlikely  that  we  could  detect  the 
bound  probe  by  a  change  in  absorbance  because  this  would  be  akin 
to  directly  imaging  a  single  molecule  in  transmission  light 
microscopy  and  Beers  Law  shows  the  futility  of  such  an  approach. 

Instead,  we  could  use  a  change  in  probe  fluorescence  and 
current  fluorochromes  have  a  fluorescence  lifetime  of  1  to  10ns. 
This  implies  that,  with  the  maximum  possible  excitation  rate,  we 
might  expect  to  generate  10s  photons/s/molecule.  This  large 
number  rapidly  declines  when  we  consider  the  detection  efficiency 
of  the  instrument.  A  good  water-immersion  dipping  objective  lens 
may  collect  about  10%  of  the  emitted  light  (as  limited  by  the 
numerical  aperture  of  the  lens  and  bathing  solution  refractive 
index)  and  after  accessory  optics  to  separate  the  excitation  light 
from  the  emitted  light,  focus  the  light  on  a  confocal  pinhole  and 
detector,  typically  <5%  of  the  light  collected  by  the  objective  will 
be  recorded  by  the  instrument.  This  implies  a  maximum  potential 
signal  rate  of  ~5  x  105  events/s.  If  the  probe  is  free  to  diffuse,  it 
will  remain  in  our  measurement  volume  for  ~0. 1  ms  so  that  we 
might  hope  to  detect  50  photons/s  residence.  If  the  probe  move¬ 
ment  is  detected  by  scanning,  then  the  dwell  time  per  pixel 
becomes  limiting  and  for  a  scanning  rate  of  lms/512  pixel  line, 
the  pixel  dwell  time  approaches  2ps.  In  this  case,  the  average 
signal  is  1  event/pixel.  However,  it  is  unusual  to  excite  probes  at 
such  a  high  rate  as  photodamage  becomes  a  serious  problem.  These 
rough  calculations  show  why  trying  to  perform  Ca^  imaging  at  the 
limits  of  optical  resolution  generally  produces  very  noisy  results. 
To  improve  the  signal-to-noise  ratio  we  must  either  increase  (1) 
the  concentration  of  the  probe  (assuming  there  are  enough  ions 
to  bind  to  it);  (2)  the  measurement  volume  and  sacrifice  spatial 
resolution;  or  (3)  the  measurement  period  and  sacrifice  temporal 
resolution. 

Put  another  way,  in  every  case  improvements  in  spatial  reso¬ 
lution,  temporal  resolution,  or  signal-to-noise  ratio  implies  a 
reduction  in  one  of  the  other  two  parameters,  a  problem  that  arises 
from  the  quantal  nature  of  both  light  and  molecules. 

As  noted  above,  optical  probes  of  ions  work  by  binding  to  the 
ion,  a  process  that  changes  the  characteristics  of  the  probe.  In  the 
case  of  aequorin  and  other  protein-based  bioluminescent  probes, 
ion  binding  triggers  a  conformational  change  in  the  protein  that 
allows  it  to  oxidize  a  chromophore  which  emits  light.  The  quantum 
efficiency  of  this  process  is  about  0.3  and  after  a  single  oxidation 
no  further  photons  are  emitted.  Thus,  while  the  specificity  of  the 
reaction  for  calcium  is  quite  good,  the  photon  yield  is  extremely 
low  and  does  not  usually  provide  sufficient  light  flux  for  imag¬ 
ing  —  except  in  large  cells  or  for  lower  time  resolution  studies 
(Blinks  et  al.,  1976). 

Because  aequorin  emits  light  spontaneously  rather  than  in 
response  to  exciting  radiation,  confocal  imaging  of  aequorin 
signals  is  impossible.  Only  a  widefield  detector  such  as  an  inten¬ 
sified  charge-coupled  device  (CCD)  or  electron-multiplier  CCD 
(EM-CCD)  has  any  chance  of  detecting  such  light,  and  even  then, 
the  fact  that  each  aequorin  molecule  emits,  on  average,  less  than 
a  single  photon  places  stringent  limits  on  the  statistical  accuracy 
of  signals.  As  this  book  emphasizes  microscopic  3D  techniques, 
we  therefore  do  not  consider  aequorin  as  a  viable  technique  for 
live-cell  3D  (confocal)  imaging. 

CHOICE  OF  INDICATOR 

In  many  cases,  several  different  indicators  may  be  available  for  the 
ion  of  interest.  Fluorescent  indicators  are  the  best  choice  for  con¬ 
focal  microscopy  as  they  are  generally  bright,  while  biolumines- 
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cent  probes  and  absorbance  probes  produce  very  small  signals  in 
single  cells.  Choice  should  be  guided  by  the  following  factors: 

1 .  Are  suitable  excitation  wavelengths  available  (note  that  multi¬ 
photon  methods  can  make  some  ultraviolet  (UV)-excited  indi¬ 
cators  work  well  and  avoid  chromatic  aberration  problems)? 

2.  Is  an  absolute  signal  calibration  important?  If  so,  ratiometric 
indictors  are  the  best  choice. 

3.  Can  the  cell  be  loaded  with  the  indicator?  All  dyes  can  be 
injected  but  chemical  loading  methods  do  not  work  well  in  all 
cells/tissues  (see  below). 

4.  Does  the  indicator  produce  large  signal  changes  for  ion  levels 
in  the  range  of  interest? 

5.  Does  the  indicator  respond  fast  enough? 

6.  How  well  does  the  cell  tolerate  the  indicator  and  the  illumina¬ 
tion  during  imaging? 

7.  Is  the  indicator  compartmentalized  within  the  cell  or  expelled 
from  it? 

8.  Is  the  perturbation  of  the  normal  response  by  the  indicator 
acceptable? 


INTRODUCING  THE  INDICATORS  INTO  CELLS 

The  most  reliable  method  of  indicator  loading  is  by  direct  injec¬ 
tion  through  a  microelectrode  or  patch  pipette  (e.g.,  Cannell  et  al., 
1987),  but  this  method  is  limited  by  the  increased  complexity  of 
cell  injection.  Once  introduced  into  cells,  most  fluorescent  indica¬ 
tors  generally  remain  there  for  a  reasonable  period  (typically  a  few 
hours),  allowing  stable  recordings  to  be  made.  Even  if  there  is 
some  loss  of  indicator  over  time  (either  by  cell  extrusion/ 
compartmentation  or  dye  bleaching),  the  effects  can  sometimes 
be  compensated  for  by  ratiometric  measurement  methods  (see 
below). 

However,  there  are  cases  in  which  loss  of  indicator  is  a 
problem,  and  this  effect  can  be  reduced  by  injecting  the  indicator 
in  a  dextran-linked  form.  Linkage  of  the  indicators  to  the  dextran 
polymer  does  not  in  general  seem  to  affect  their  properties  signif¬ 
icantly  but  does  inhibit  cellular  transport  of  the  indicator  [see  the 
Molecular  Probes,  Inc.  (Eugene,  OR)  catalogue  for  further  infor¬ 
mation  and  for  availability  of  these  forms] .  An  alternative  method 
is  to  use  a  drug  to  inhibit  the  cell  transport  of  the  dye.  Because 
most  dyes  are  organic  acids,  inhibitors  such  as  probenecid  may  be 
useful.  For  acute  experiments,  these  drugs  do  not  produce  obvious 
changes  in  cell  behavior  but  for  longer  experiments  they  may  be 
a  problem.  Finally,  lowering  the  temperature  can  also  reduce  the 
rate  of  loss  of  dye  (along  with  having  a  number  of  other  effects!). 

Many  indicators  can  be  introduced  into  cells  by  the  ester¬ 
loading  technique  (Tsien,  1981).  This  technique  has  extended  flu¬ 
orescence  Ca++  measurements  into  the  realm  of  very  small  cell 
fragments  such  as  blood  platelets,  that  cannot  be  microinjected. 
Because  most  fluorescent  ion  indicators  are  multiply  charged  at 
neutral  pH  with  many  of  the  charges  being  on  carboxyl  groups,  they 
are  not  very  lipid  soluble.  By  esterifying  these  carboxyl  groups,  an 
uncharged  derivative  of  the  indicator  can  be  produced.  While  this 
derivative  is  not  an  active  indicator,  it  is  sufficiently  lipophilic  to 
permeate  biological  membranes  and  thereby  enter  cells.  Inside 
cells,  the  derivative  is  converted  to  the  active  indicator  by  the  action 
of  intrinsic  esterase  enzymes  (see  Fig.  42.2).  Because  the  active 
indicator  is  relatively  membrane  impermeant,  it  is  now  trapped 
inside  the  cell  and  accumulates  under  the  continued  action  of  the 
esterase(s).  Incubation  conditions  need  to  be  adjusted  carefully  in 
order  to  achieve  satisfactory  loading  of  the  cells. 
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FIGURE  42.2.  Illustration  of  the  method  for  trapping  an  indicator  inside  the 
cell  with  acetoxymethyl  derivatization.  The  highly  polar  carboxyl  groups 
(which  form  the  Ca1 1  binding  region  of  the  BAPTA  region)  are  modified  as 
shown.  This  polar  molecule  then  enters  the  cell  and  is  modified  to  release  the 
active  indicator.  Clearly,  the  extent  of  trapping  is  esterase  dependent  and  partial 
de-esterification  is  also  a  possibility. 


It  must  also  be  noted  that  de-esterification  of  acetoxymethyl 
esters  liberates  formaldehyde  as  a  hydrolysis  product.  Although 
serious  toxicity  has  not  been  reported,  one  should  always  remem¬ 
ber  that  formaldehyde  may  be  affecting  your  results. 

The  major  problem  with  the  ester  (or  AM)  loading  technique 
is  that  the  experimenter  has  little  direct  control  over  where  the  indi¬ 
cator  ends  up  in  the  cell.  As  the  ester  will  enter  all  intracellular 
compartments,  the  active  indicator  concentration  in  each  compart¬ 
ment  will  depend  on  the  relative  esterase  activity.  Thus,  the  endo¬ 
plasmic  reticulum  and  mitochondria  will  also  contain  an  indicator 
that  may  confound  interpretation  of  cytoplasmic  signals.  An 
additional  problem  is  that  de-esterification  may  be  incomplete, 
creating  a  fluorescent  intermediate  that  is  not  ion  sensitive  (e.g., 
Highsmith  et  al.,  1986).  The  magnitude  of  these  effects  can  only 
be  ascertained  with  careful  control  experiments.  At  the  very  least, 
the  images  need  to  be  critically  examined  during  imaging  to  ascer¬ 
tain  where  the  fluorescent  signals  are  actually  coming  from. 

A  related  problem  is  that  some  types  of  cells  seem  to  be  very 
good  at  clearing  the  indicator  from  their  cytoplasm.  In  endothelial 
cells,  for  example,  after  about  an  hour  the  cell  fluorescence  appears 
punctate  and  mitochondria  are  clearly  visible  (unpublished  obser¬ 
vations).  Whether  this  represents  simply  the  removal  of  cytoplas¬ 
mic  indicator  or  accumulation  of  indicator  in  the  punctate  regions 
is  unclear.  In  any  case,  considerable  caution  should  be  applied  to 
the  interpretation  of  signals  from  the  cells  loaded  with  the  AM  ester 
form  of  the  indicator.  It  appears  that  cells  labeled  with  fluorescent 
probes  give  a  smaller  fluorescence  change  when  the  AM  indica¬ 
tor  is  used  compared  to  direct  injection  (unpublished  observation). 
In  summary,  good  results  can  be  obtained  with  the  ester  loading 
technique,  provided  adequate  control  experiments  are  performed 
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FIGURE  42.3.  A  comparison  of  different  incubation  temperatures  used  to  stain  rat  interossi  muscles  with  SNARF-1.  Cross-sections  through  the  two  muscles 
after  staining  reveals  that:  (A)  when  stained  at  room  temperature  (22°C  for  30min,  30°C  for  20mins)  nearly  all  the  SNARF-1  is  immobilized  in  the  outermost 
fibers.  (B)  Cold  loading  of  the  muscle  (5°C  for  60  min,  30°C  for  20  mins)  allowed  SNARF-1  to  diffuse  evenly  throughout  the  fibers.  Bar  =  250  pm.  (Bowser 
etal,  1999.) 


and  it  may  be  the  only  route  to  take  if  cells  cannot  be  loaded  by 
patch  pipette. 

An  additional  problem  for  tissues  is  that  as  cells  efficiently 
cleave  the  dye  into  its  non-membrane  permeant,  fluorescent  form, 
it  can  become  trapped  in  outer  cell  layers  (See  Fig.  42. 3A).  This 
can  be  ameliorated  by  using  cold  loading  techniques  (Cody  et  al., 
1993;  Williams  et  al .,  1993,  Bowser  et  al. ,  1999),  to  slow  down 
the  activity  of  the  intrinsic  cellular  esterases.  For  example,  incu¬ 
bation  of  mammalian  skeletal  muscle  for  1  hour  at  5°C  to  allow 
the  pH  indicator  SNARF-1  to  penetrate  the  deeper  fibers  and  an 
incubation  for  a  further  20  mins  at  30°C  to  facilitate  dye  cleavage, 
results  in  more  uniform  dye  distribution  throughout  the  prepa¬ 
ration  (Fig.  42. 3B).  Figure  42.4  shows  dual-channel  images  of 
SNARF-1  fluorescence  were  collected  before  the  fatigue  stimula¬ 
tion  (Fig.  42.4A  and  42.4B).  The  relative  uniformity  of  dye  loading 
is  apparent  and  as  ratio  images  show,  immediately  after  fatigue 
(Fig.  42.4C)  and  after  a  fifteen  minute  rest  period  (Fig.  42.4D) 
there  were  almost  no  changes  in  pFI  with  fatigue  in  either  the  EDL 
or  the  soleus  muscles  (Cody  et  al.,  1993). 

An  alternative  strategy  is  to  use  protein  expression  of  a  genet¬ 
ically  engineered  variant  of  a  fluorescent  protein  [most  often  green 
fluorescent  protein  (GFP)]  that  includes  an  ion-binding  region  in 
its  structure.  While  this  method  requires  time  for  the  expressed 
protein  to  be  produced,  it  offers  great  promise  as  the  indicator 
protein  can  be  targeted  to  organelles  [(as  was  first  demonstrated 
with  recombinant  aequorin  (Rizzuto  etal.,  1983)]  or  other  cell  loca¬ 
tions.  At  the  present  time,  these  methods  are  still  being  developed 
and  further  work  is  needed  to  make  the  probes  faster,  show  larger 
signal  changes,  and  be  more  specific  in  their  response.  Neverthe¬ 
less,  in  principle  the  method  could  be  used  to  make  a  probe  for 
almost  any  compound  and  many  probes  of  this  type  have  already 
been  produced.  Refer  to  Zhang  et  al.  (2002)  for  an  excellent 
review. 


CARE  OF  FLUORESCENT  PROBES 

A  few  words  on  the  handling  of  the  indicators  may  be  in  order. 
The  fluorescent  indicators  are  all  subject  to  oxidation  during 
storage  and  will  lose  activity  in  a  few  days  if  exposed  to  light  and 
air  at  room  temperature.  It  is  best  to  make  up  the  acetoxymethyl 
esters  in  dry  dimethylsulfoxide  (DMSO)  (solubility  may  require 
the  addition  of  a  non-ionic  detergent  as  well  such  as  Pluronic  [Mol¬ 
ecular  Probes])  as  a  stock  solution  that  will  be  diluted  more  than 
lOOx  when  used.  This  stock  should  then  be  split  into  a  number  of 
ampoules,  each  of  which  contains  enough  indicator  for  a  single 
experiment.  The  ampoules  can  then  be  frozen  (at  -80°C)  and  this 
will  avoid  repeated  freeze/thawing  cycles.  A  similar  procedure 
should  be  used  for  the  free-acid  form  of  the  indicator  except  that 
one  should  use  a  suitable  buffer  solution  (e.g.,  140  m M  KC1) 
instead  of  DMSO. 

The  free-acid  form  of  the  indicator  should  not  be  exposed  to 
metal  (such  as  stainless-steel  syringe  needles).  Prior  to  microin¬ 
jection  on  the  day  of  the  experiment,  the  aliquot  of  indicator  should 
be  dissolved  in  about  1  mL  of  intracellular  solution  and  placed  in  a 
1  mL  polyethylene  tuberculin  syringe,  the  end  of  which  has  been 
drawn  over  a  flame  into  a  fine  capillary  (to  allow  it  to  pass  down 
inside  the  electrode  barrel).  The  syringe  is  used  to  place  ~50pL  at 
the  shoulder  of  the  electrode,  allowing  a  reasonable  number  of  elec¬ 
trodes  and  cells  to  be  filled  before  the  syringe  needs  to  be  refilled. 
Finally,  positive  pressure  must  be  maintained  on  the  back  of  the 
electrode  at  all  times  before  forming  a  giga-seal  or  else  the  exper¬ 
imenter  will  find  that  the  indicator  will  be  seriously  diluted  at  the 
tip  of  the  electrode  (quite  apart  from  the  problem  of  contamination 
of  the  intracellular  solution  by  bathing  solution!).  After  breaking 
into  the  cell,  dialysis  of  the  cell  may  take  5  to  20  min,  but  the  speed 
of  dialysis  can  be  markedly  improved  by  applying  brief  pulses  of 
positive  pressure  to  the  back  of  the  electrode  to  inject  the  indicator 
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FIGURE  42.4.  A  rat  EDL  muscle,  stained  with  SNARF-1.  Ratio  images  are  quantitatively  calibrated  in  units  of  pH.  No  physiologically  relevant  change  in 
pH  was  detected.  (A)  Dual-emission  image.  (B)  Control.  (C)  Immediately  after  fatigue  stimulation.  (D)  After  a  15  minute  recovery  period.  Bar  =  100  pm.  (Cody 
et  al.,  previously  unpublished.) 


(considerable  care  and  experience  is  needed  for  this  technique!). 
The  extent  of  dialysis  can  be  determined  from  the  magnitude  of  the 
resting  fluorescence  signal,  and  experienced  patch  clamp  electro¬ 
physiologists  may  be  amazed  at  how  slow  dialysis  can  be. 

INTERPRETATION  OF  MEASUREMENTS 

Although  fluorescent  indicators  have  proved  to  be  extremely  pow¬ 
erful,  it  is  important  to  bear  in  mind  that  they  share  (as  with  all 
other  recording  techniques)  the  possibility  of  giving  inaccurate 
measurements  and/or  of  influencing  whatever  is  being  measured. 
In  particular,  it  should  be  appreciated  that  the  indicators  work  by 
reversibly  binding  to  the  target,  so  by  definition  they  have  a  buffer¬ 
ing  action.  The  extent  of  the  buffering  will  depend  on  the  concen¬ 
tration  of  the  indicator  relative  to  the  free  concentration  of  the 
target  ion  as  well  as  the  affinity  of  the  indicator,  so  it  will  be  most 
significant  for  those  ions  whose  free  concentrations  are  relatively 
low,  that  is,  Ca^  and  protons.  Although,  the  natural  cell  buffering 
capacity  for  these  ions  reduces  the  effect  of  indicator  buffering,  the 
amplitudes  and  time  courses  of  ion  changes  may  still  be  signifi¬ 
cantly  altered  by  the  increased  buffering.  To  gauge  this,  one  really 
needs  to  carry  out  experiments  over  a  range  of  indicator  concen¬ 
trations  to  demonstrate  that  the  results  are  reasonably  independent 
of  the  indicator  concentration.  The  binding  equation  is: 


F  =  \X}KKa  +  [AJ) 

where  F  is  the  fraction  bound,  and  Kd  the  dissociation  constant.  As 
an  example,  consider  an  experiment  using  a  calcium  ion  indicator 
with  a  Ka  of  300  nM  and  a  resting  Ca++  level  of  100  nM,  the  frac¬ 
tion  bound  =  0.4.  If  the  concentration  of  the  indicator  is  0. 1  m M  (a 
typical  value),  40  pM  Caw  is  bound  to  the  dye.  Thus,  the  buffer¬ 
ing  power  ( B )  =  bound/free  =  400.  This  value  can  be  compared  to 
the  endogenous  Ca' 1  buffering  powers  of  75  for  immobile  buffers 
and  20  for  mobile  buffers  (Neher  and  Augustine,  1992).  From  this, 
it  is  clear  that  high-affinity  indicators  can  dominate  the  intrinsic 
buffers  which  will  result  in  a  significant  reduction  in  the  amplitude 
of  physiological  Ca++  transients,  as  well  as  a  slowing  in  their  time 
course.  For  a  constant  signal-to-noise  ratio,  this  problem  cannot  be 
circumvented  by  using  a  lower  affinity  indicator,  as  the  signal 
depends  on  detecting  the  bound  form  whose  reduction  in  concen¬ 
tration  (due  to  a  reduction  in  affinity)  must  be  offset  by  an  equiv¬ 
alent  increase  in  overall  indicator  concentration.  However,  using  a 
lower  affinity  indicator  does  improve  the  response  speed.  Thus  the 
speed  of  response  may  be  increased  if  very  high  concentrations 
of  a  low  affinity  dye  are  used  so  that  most  of  the  released  ions 
become  bound  to  the  dye  (Pape  et  al.,  1993).  Of  course,  this  means 
that  other  physiological  responses  are  blocked,  but  the  speed  of 
response  can  be  increased  several-hundred-fold.  To  produce  such 
high  dye  concentrations,  some  injection  method  is  needed  as  the 
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esterase  technique  produces  dye  concentrations  of  typically  <1  mill 
(unpublished  observations). 

Finally,  buffering  of  ions  by  the  indicator  is  likely  to  alter  free 
diffusion  of  the  ion.  If  the  indicator  is  relatively  immobile,  free  dif¬ 
fusion  may  be  hindered  by  a  fraction  equal  to  the  fraction  bound. 
On  the  other  hand,  if  the  buffer  is  mobile,  diffusion  may  be 
enhanced  as  the  buffer  can  act  as  a  carrier  that  limits  intrinsic 
binding  by  cell  constituents  which  serve  to  reduce  the  diffusion 
coefficient. 


KINETICS 

With  most  ion  probes,  the  kinetics  of  the  fluorescence  change  does 
not  reflect  the  kinetics  of  the  underlying  ion  transient.  This  dis¬ 
parity  arises  from  (1)  saturation  effects  in  the  indicator  response 
and  (2)  the  kinetics  of  dye-ion  binding.  While  the  first  problem  can 
be  circumvented  by  converting  the  dye  signal  to  ion  concentrations 
(but  see  below),  workers  often  assume  that,  because  the  dye :  ion 
stoichiometry  is  1:1,  the  dye  is  a  linear  indicator  of  ion  concen¬ 
tration  and  that  the  time  course  of  fluorescence  change  should 
reflect  the  time  course  of  calcium  change.  However,  this  is  not  the 
case.  From  the  equation 

dF/dt  =  (d[X]/dt)  ( dF/dX) 

where  dF/d[X]  is  the  slope  of  the  relationship  between  F  and  [ X ], 
it  is  clear  that  we  must  consider  the  properties  of  the  relationship 
between  F  and  [X] .  Even  if  the  dye  kinetics  are  so  fast  that  the  dye 
reaction  is  at  equilibrium  at  all  times  (see  below)  and  the  binding 
is  1 : 1  with  no  buffering,  the  time  constant  of  the  dye  response  is 
quite  different  from  the  underlying  time  constant  of  ion  change. 
This  is  illustrated  in  Figure  42.5(A),  which  shows  the  relationship 
between  F  and  dF/d[X ]  as  a  function  of  [X\IKd  (the  ligand  con¬ 
centration  divided  by  the  affinity  of  the  indicator  for  the  ligand). 
It  is  clear  that  dF/d[X ]  is  considerably  less  than  unity  at  [X] 
>0.1  A'd.  As  this  is  the  range  within  which  most  indicators  are  used, 
the  dye  response  is  generally  quite  nonlinear.  Put  another  way,  if 
the  observed  signal  changes  with  a  time  constant  of  Y,  the  under¬ 
lying  ion  change  occurred  with  a  faster  time  constant  equal  to 
YdF/dX.  The  simplest  way  around  this  problem  is  to  convert  F  (or 
the  ratio  measurement)  to  [X]  with  a  calibration  curve,  if  one  is 
available  for  the  experimental  conditions. 

However,  dF/d\X ]  is  also  a  function  of  time  as  the  kinetics  of 
the  dye  are  limited.  For  the  bimolecular  reaction  D  +  X  =  DX  with 
a  forward  rate  of  kon  and  a  back  rate  of  koS,  if  the  ion  concentra¬ 
tion  undergoes  a  step  change  to  [Aj,  the  indicator  signal  will 
change  exponentially  to  a  new  level  with  a  rate  constant  of  (kon[X| 
+  kof f).  For  most  ion  indicators,  the  forward  rate  is  generally 
diffusion-limited  at  ~2.108/M/s  so  different  affinities  are  mostly 
due  to  changes  in  koS.  In  this  case,  we  can  examine  the  likely  kinet¬ 
ics  of  response  from  the  dye  Kd  and  the  ion  concentration  range. 
Figure  42.5(B)  shows  the  time  constant  of  response  for  a  range  of 
hypothetical  Ca^  dyes  with  Kd  in  the  range  100  to  lOOOnm  for  a 
jump  in  [Ca]  from  50  to  5000  nM.  From  this  double  logarithmic 
plot,  it  is  clear  that  the  response  is  in  the  range  of  2  to  40  ms  and 
highly  dependent  on  the  ion  level.  High  dye  concentrations  can 
alter  the  kinetics  of  the  response,  albeit  at  the  expense  of  buffer¬ 
ing  effects  (see  above). 

It  is  worth  noting  that,  as  most  LSCM  scan  lines  at  about 
2ms/line,  typical  dye  kinetic  changes  may  be  faithfully  recorded 
in  line-scanning  modes  but  not  when  full  frame  imaging  is  used. 
This  problem  may  be  reduced  with  disk-scanning  or  slit-scanning 
instruments  (see  Chapter  10,  this  volume ).  In  terms  of  microscopic 
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FIGURE  42.5.  The  relationship  between  fluorescence  and  ion  level  for  a  1 : 1 
stochiometry.  The  ion  concentration  is  expressed  in  units  of  Kd.  The  rate  of 
change  of  fluorescence  with  ion  levels  is  also  shown  and  this  is  considerably 
less  than  unity  for  ion  levels  >0. 1  Kd.  The  lower  panel  shows  the  rate  of  dye 
response  (assuming  a  diffusion  limited  on  rate)  for  different  Kd  and  ion  levels. 

subcellular  signals,  these  times  correspond  to  the  time  of  diffusion 
over  a  few  micrometers  so  that  it  is  possible  that  attempts  to  image 
the  evolution  of  small  spatial  changes  will  be  contaminated  by  the 
kinetics  of  dye  response. 

To  put  this  analysis  in  more  concrete  terms,  the  kinetics  of  the 
fura-2  reactions  with  calcium  have  been  investigated  in  stopped- 
flow  experiments  (Jackson  et  al.,  1987).  The  rate  of  calcium 
binding  by  fura-2  is  2.5  to  6.5  108/M/s  and  the  rate  of  dissociation 
is  84/s  at  20°C.  Thus,  at  a  level  of  200 n M  free  calcium,  the  fura- 
2  reaction  will  proceed  at  ~180/s.  While  this  response  speed  is 
clearly  superior  to  that  of  a  calcium-selective  microelectrode  and 
at  least  as  fast  as  aequorin,  it  is  not  so  fast  that  possible  kinetic  dis¬ 
tortion  of  the  calcium  signal  can  be  ignored.  It  is  notable  that 
Baylor  and  Hollingworth  (1988)  reported  that  the  fura-2  kinetics 
appeared  to  be  considerably  slower  in  single,  frog  muscle  fibers 
than  measurements  of  Jackson  and  colleagues  (1987)  would 
suggest.  Baylor  and  Hollingworth  (1988)  compared  the  antipyryl- 
azo  absorbance  signal  to  the  fura-2  signal  and  suggested  that  the 
fura-2  signal  might  be  explained  if  the  fura-2  off-rate  were  about 
25  s-1,  a  value  three  to  four  times  lower  than  that  expected  from  in 
vitro  experiments.  The  importance  of  this  observation  is  under¬ 
scored  by  the  fact  that  the  exact  effect  of  the  intracellular  envi- 
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ronment  on  the  properties  of  most  dyes  remains  largely  unknown. 
It  is  notable  that  Williams  and  colleagues  (1985)  and  Weir  and 
co-workers  (1987)  found  that  the  maximum  fluorescence  ratio 
obtained  from  the  intracellular  dye  appeared  to  be  slightly  less 
than  that  obtained  in  calibrating  solutions.  This  suggests  that  the 
intracellular  milieu  alters  the  properties  of  the  dye.  Baylor  and 
Hollingworth  also  observed  a  reduced  diffusion  coefficient  for 
fura-2  (1988),  an  effect  also  found  by  Timmerman  and  Ashley 
(1986)  in  barnacle  muscle. 

Fluorescence  anisotropy  measurements  suggest  that  some  dye 
is  immobilized  (Baylor  and  Hollingworth,  1988).  These  results 
may  all  be  due  to  the  dye  binding  to  some  intracellular  constituent, 
but  whether  such  binding  seriously  alters  the  calcium  sensitivity 
of  the  dye  remains  unclear.  On  the  other  hand,  both  Williams  and 
colleagues  (1985)  and  Baylor  and  Hollingworth  (1988)  reported 
that  the  fluorescence  spectrum  of  fura-2  inside  single  smooth 
muscle  cells  was  very  similar  to  the  spectrum  obtained  in  a  cuvette 
and  this  lends  confidence  regarding  the  reliability  of  these  dyes  in 
general.  Without  more  detailed  research  into  quantifying  these 
problems,  it  is  clear  that  caution  should  be  applied  to  quantitative 
interpretation  of  signals  from  cells.  Fortunately,  we  are  usually 
more  interested  in  relative  changes  in  [Ca”]  than  in  its  absolute 
value. 

Another  potential  difficulty  concerns  the  fact  that  any  indica¬ 
tor  will  report  concentration  accurately  only  over  a  fairly  narrow 
ion  concentration  range.  As  a  result,  attempts  to  use  one  indicator 
to  cover  all  experimental  situations  are  likely  to  give  inaccurate 
results.  For  example,  the  early  fluorescent  indicators  for  Ca”  ions 
(Grynkiewicz  et  al.,  1985)  were  devised  primarily  to  measure 
resting  ion  concentrations  in  small  cells.  Because  the  indicator  flu¬ 
orescence  will  be  midway  between  its  values  at  the  concentration 
extremes,  the  greatest  accuracy  for  such  measurements  is  obtained 
when  the  dissociation  constant  of  the  Ca-indicator  complex  is 
similar  to  the  free  Ca”  concentration.  Unfortunately,  this  is  not  the 
best  situation  for  measuring  transient  changes  in  concentration 
where  buffering  and  kinetic  responses  will  be  far  from  optimal  (see 
above).  In  addition,  as  the  Ca”  diffuses  away  from  the  release 
sites,  the  total  fluorescence  signal  will  increase,  even  though  the 
average  Ca”  concentration  may  remain  the  same,  because  the 
reduction  in  fluorescence  in  the  areas  near  the  release  sites  is  less 
than  the  increase  in  fluorescence  in  the  areas  further  away,  a 
problem  not  entirely  removed  at  the  limit  of  resolution  of  the  light 
microscope. 

In  summary,  for  accurate  transient  measurements,  one  should 
try  to  use  an  indicator  with  a  dissociation  constant  that  is  no  lower 
than  the  highest  concentration  transient  expected,  even  though  this 
may  lead  to  poor  signals  at  resting  ion  levels. 


CALIBRATION 

Although  ion  concentrations  can  be  estimated  from  fluorescence 
recordings  by  reference  to  in  vitro  calibration  solutions,  ideally, 
such  methods  should  be  supplemented  by  measurements  in  which 
known  ion  concentrations  are  measured.  For  example,  a  known 
Ca”  concentration,  near  to  the  resting  level  of  Ca”  can  be  imposed 
by  using  an  appropriate  buffer  solution  outside  the  cell,  and  then 
permeabilizing  the  membrane  with  a  Ca1 1  ionophore  such  as  ion- 
omycin.  For  fluorescence  measurements  made  at  a  single  wave¬ 
length,  the  free-ion  concentration  |X|  is  related  to  the  fluorescence 
F  by 

|X|  =  Kd(F  -  Fmm)l(Fmm  -  F) 


where  Fmin  and  Fmax  are  respectively  the  fluorescence  levels  at  zero 
and  saturating  ion  concentrations,  and  Kd  is  the  dissociation  of  the 
ion-indicator  complex. 

For  ratiometric  indicators  (where  the  response  to  ions  is  dif¬ 
ferent  at  each  of  two  wavelengths  (F1;F2),  a  ratio  R  =  Fl/F2  is 
obtained  leading  to 

[X]  =  KM  -  Rml„)/(Rm.a  -  R)(SF2/SB2) 

where  R^  is  the  ratio  of  fluorescence  at  zero  [X],  Fmax  is  the  ratio 
at  saturating  [X],  and  SF2/SB2  the  fluorescence  at  zero  [X]  divided 
by  the  fluorescence  at  saturating  [X]  at  F2.  This  method  removes 
the  concentration  dependence  of  the  absolute  fluorescence  level 
and  should  result  in  a  signal  that  depends  only  on  the  ion  concen¬ 
tration  (see  Fig.  42.6).  SF2/SB2  effectively  scales  the  KA  and  alters 
the  midpoint  of  the  response,  but  it  does  not  affect  the  amount 
bound  to  the  indicator  as  this  is  determined  by  [X]  and  the  Kd.  To 
alter  the  ion  levels  inside  the  cell,  one  can  use  an  ionophore  such 
as  ionomycin  (for  Ca’”'),  gramicidin  (for  Na”),  nigericin  (for  H+ 
and  K+)  or  valinomycin  (for  K+).  The  external  ion  levels  can  then 
be  manipulated  to  allow  one  to  measure  Fmin  and  Fmax  at  all  impor¬ 
tant  wavelengths. 

In  an  attempt  to  avoid  having  to  measure  both  Fmin  and  Fmax  to 
calibrate  signals  from  non-ratiometric  indicators,  Cannell  and  col¬ 
leagues  (1994)  suggested  that  if  one  used  a  Ca++  indicator  with  a 
negligible  Fmin  (such  as  fluo-3)  in  cells  whose  resting  ion  level  was 
relatively  constant  (see  Wier  et  al.,  1987),  Fmax  could  then  be  esti¬ 
mated  from  the  signal  at  the  resting  level  of  Ca”.  In  this  case,  Fmax 
=  Frest(Xd/[Ca”]rest  +  1).  Because,  in  the  confocal  microscope,  the 
volume  sampled  is  more  or  less  constant,  if  an  experimental  image 
is  divided  by  one  taken  when  the  cell  was  at  rest,  a  pseudo-ratio 
(R)  in  which  | Ca”]  =  Kd  ■  R\ (Xd/[Ca”  Ires,  +  1)  -  R].  This  method 
allows  comparisons  to  be  carried  between  different  experiments 
and,  if  the  resting  level  of  Ca”  is  known  to  reasonable  accuracy, 
estimates  of  [Ca”]  recorded  in  the  experimental  images  will  be 
equally  accurate. 

Using  this  method,  we  found  that  the  amplitude  of  the  evoked 
Ca”  transient  agreed  well  with  that  obtained  by  ratiometric  indi¬ 
cators  and  allowed  estimation  of  the  level  of  Ca”  produced  by  Ca” 
sparks  and  the  spark  Ca”  release  flux  (Cheng  et  al.,  1993;  Cannell 
et  al,,  1994).  Nevertheless,  it  is  important  not  to  lose  sight  of  the 
assumptions  underlying  this  method. 

Cell  autofluorescence  must  also  be  measured  and  subtracted 
from  all  measurements,  and  if  it  is  not  practical  to  do  this  before 
indicator  loading,  the  fluorescence  of  the  indicator  may  be 
quenched  with  Mn2+  at  the  end  of  the  experiment  (see  Fig.  42.7). 
These  measurements  can  then  be  referred  to  an  in  vitro  calibration 
curve  and,  if  all  is  well,  R^  and  Fmax  should  agree  with  this  curve. 

For  confocal  microscopes,  it  is  more  convenient  to  use  a  ratio- 
dye  that  shows  an  emission  shift  (such  as  indo-1  for  Ca”  and 
SNARF-1  for  pH)  rather  than  a  shift  in  excitation  wavelength, 
because  most  microscopes  have  multiple  detector  channels  but  few 
are  able  to  rapidly  modulate  the  laser  excitation  wavelength  (espe¬ 
cially  in  the  UV!).  The  spectral  properties  of  Indo-1  and  SNARF- 
1  allow  2-channel,  truly  simultaneous  acquisition  of  ratiometric 
images  on  a  standard  confocal  microscope  by  splitting  the  emitted 
light  to  two  detector  channels.  SNARF-1,  (5-(and-6)-carboxy 
seminaphthorhodafluor)  was  developed  by  Molecular  Probes 
(Whitaker  et  al.,  1991)  and  is  widely  used  in  micro-spectrofluo- 
rimetry  (e.g.,  Buckler  &  Vaughan-Jones,  1990;  Bassnett,  1990; 
Opitz  et  al,,  1994).  When  this  dye  is  combined  with  the  optical- 
sectioning  capability  of  the  confocal  microscope  (Cody  et  al., 
1993),  it  is  possible  to  map  intracellular  pH  in  three  dimensions 
with  a  very  high  spatial  resolution  (Fig.  42.8).  Furthermore, 
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FIGURE  42.6.  Use  of  a  ratiometric  indicator  in  mouse  egg  cells.  Indo-1  fluorescence  at  400  and  400nm  was  produced  by  two-photon  excitation  at  690nm. 
Note  the  difference  in  Ca*4  signals  from  the  sperm  and  egg  are  removed  in  their  ratio  so  the  overall  image  becomes  flat.  From  this  we  can  conclude  that  level 
of  Ca1 1  shortly  after  sperm  fusion  matches  that  of  the  egg  and  that  the  egg  cytoplasm  is  quite  uniform  (Jones  et  al.,  1998). 


SNARF-1  has  such  excellent  cellular  retention  that  the  manu¬ 
facturer  also  promotes  it  as  a  long-term,  cell-tracing  dye  — 
(Haugland,  2005).  Similarly,  Indo-1  can  be  used  for  confocal  or 
2-photon  ratiometric  Ca* 1  measurements  (e.g.,  Jones  et  al. ,  1998), 
although  there  are  significant  cell  retention  and  bleaching  problems 
associated  with  this  probe. 

No  matter  how  much  care  is  taken  in  performing  these  mea¬ 
surements,  the  values  calculated  using  the  calibration  equations 
listed  above  must  still  be  treated  with  some  caution.  Apart  from 
the  obvious  risk  that  KA  may  be  somewhat  different  in  the  cell  from 
in  the  calibration  solution,  there  is  also  the  possibility  that  the  flu¬ 
orescence  properties  of  the  indicator  may  be  different  in  the  cell, 


[Ca]  =  Kd  x  (R-R  J/(R-R  J  x  Sf2/Sb2 

FIGURE  42.7.  Illustration  of  the  method  for  calibrating  an  intracellular  Ca" 
indicator  by  imposing  known  ion  levels  and  quenching  fluorescence.  After 
external  Ca**  is  removed  (by  adding  EGTA),  ionomycin  (a  Ca  ionophore)  is 
added  to  the  bathing  solution  to  give  (after  subtraction  of  autofluores¬ 
cence).  Subsequent  addition  of  a  saturating  Ca  solution  (>IOOu.V/)  allows  deter¬ 
mination  of  Fm„.  Adding  M  n ' 1  quenches  the  dye  fluorescence,  allowing  a 
determination  of  cell  autofluorescence.  It  is  assumed  that  the  latter  was  not 
affected  by  Mn++  and  separate  experiments  should  be  carried  out  to  show 
whether  this  is  the  case. 


and  such  effects  have  been  observed  for  several  indicators  (see 
above). 

The  possible  errors  caused  by  not  using  a  ratiometric  approach 
are  illustrated  in  Figure  42.9  where  a  proximal  tubule  has  been 
labeled  with  pH  indicator  BCECF  and  imaged  at  a  single  wave¬ 
length.  The  low  intensity  fluorescence  evident  in  the  brush  border, 
could  be  caused  by  one  of  three  factors:  a  lower  pH  in  the  brush 
border,  a  lower  concentration  of  dye  in  this  region,  or  the  fact  that 
there  is  more  unstained,  extracellular  space  in  the  brush  border 
(path-length  error).  However,  when  a  similar  preparation  is  labeled 
with  the  ratiometric  SNARF-1  (Fig.  42.10),  the  ratio  image  shows 
the  true  pH  distribution  map  and  it  becomes  evident  that  apparent 
pH  differences  are  simply  dye-concentration  artifacts.  Similar  arti¬ 
facts  were  also  observed  in  a  single  wavelength  study  that  used 
Fura-2  in  heart  cell  nuclei  (Wier  et  al.,  1987). 

Application  of  the  calibration  equations  also  assumes  that  the 
indicator  is  sufficiently  selective  not  to  form  complexes  with  other 
ions  to  any  significant  extent.  Although  this  may  be  a  questionable 
assumption,  the  fluorescent  indicators  generally  perform  quite 
well  in  this  respect.  This  is  particularly  true  for  Ca++  indicators 
based  on  the  BAPTA  molecule,  for  example,  fura-2  and  indo-1 
(Grynkiewicz  et  al.,  1985),  which  are  more  selective  against  Mg++ 
and  protons  than  are  (say)  any  of  the  metallochromic  Ca*  *  indica¬ 
tors,  where  competing  effects  from  such  ions  pose  a  major  cali¬ 
bration  problem  (Thomas,  1991).  In  the  case  of  pH  indicators, 
selectivity  is  quite  good  so  intracellular  pH  may  be  simply  manip¬ 
ulated  by  adding  nigericin  and  a  C02  —  bicarbonate-based  buffer 
system.  For  example,  in  the  experiments  shown  in  Figure  42.4  a 
high  potassium  solution  was  used  (to  zero  the  membrane  poten¬ 
tial)  and  the  pH  of  the  superfusate  measured  with  a  pH  electrode. 
By  using  three,  premixed  gasses  (100%  02,  5%C02/95%  02  and 
100%  C02)  to  bubble  the  superperfusate  (which  included  18.5  mA/ 
NaHC03),  the  intracellular  pH  changes  could  be  calibrated  without 
needing  multiple  solution  changes.  This  increased  the  repro¬ 
ducibility  of  the  intracellular  calibrations  as  the  specimen  did  not 
need  to  be  manipulated  unnecessarily  (Fig.  42.11).  A  disadvantage 
with  this  method  is  that  Ca44'  must  be  removed  if  alkaline  pH 
calibrations  are  needed,  otherwise  the  Ca44  in  the  perfusate  will 
precipitate  with  the  bicarbonate. 

When  the  interference  effects  from  other  ions  are  significant, 
the  simplest  approach  is  to  calibrate  the  indicator  in  a  solution  that 
contains  appropriate  concentrations  of  those  ions.  Of  course,  such 
calibrations  only  apply  so  long  as  the  concentrations  of  the  inter- 


FIGURE  42.8.  Cultured  monolayer  of  rabbit  proximal  tubule  cells  stained  with  the  acetoxymethyl  ester  of  SNARF-1.  (A)  Wavelengths  greater  than  595  nm. 
(B)  Wavelengths  lower  than  595  nm.  C:  A  ratio  image  of  A  divided  by  B  (>595  nm/<595  nm)  creates  an  intracellular  pH  distribution  map  that  shows  the  nuclei 
have  a  higher  pH  (up  to  0.5 pH  units  higher)  that  the  cytosol.  Bar  =  25  pm.  (Dubbin  et  al.,  1993.) 


FIGURE  42.9.  Rabbit  kidney  proximal  tubule  stained  with  the  pH-sensitive  dye  BCECF:  single  excitation  (ex.  488 nm)  image  where  intensity  is  a  function  not 
only  of  pH  but  also  dye  concentration.  Arrow  indicates  the  brush  border.  Bar  =  5  pm.  (Cody  and  Williams,  1999.) 


FIGURE  42.10.  Rabbit  kidney  proximal  tubule  stained  with  SNARF-1.  (A)  Emission  wavelengths  less  than  595 nm.  (B)  Wavelengths  greater  than  595 nm. 
Arrow  indicates  the  brush  border,  with  lower  intensity  fluorescence.  (C)  Ratio  image  (<595  nm/>595  nm)  provides  a  pH  distribution  map  that  is  independent  to 
photobleaching,  dye  distribution  and  path  length  artifacts.  Bar  =  5  pm.  (Cody  and  Williams  1999.) 
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FIGURE  42.11.  A  typical  intracellular  calibration  curve  of  SNARF-1. 
Muscles  stained  with  SNARF-1  and  treated  with  the  hydrogen  ion  ionophore 
nigericin  (50  |lM).  The  pH  of  the  modified  Kreb's  solution  was  altered  by  bub¬ 
bling  with  different  concentrations  of  C02.  The  extracellular  pH  was  monitored 
with  a  pH  electrode,  and  dual  channel  ratiometric  images  of  intracellular 
SNARF-1  fluorescence  were  recorded.  (Cody  et  al.,  previously  unpublished.) 


fering  ions  in  the  cell  do  not  change  significantly  during  the  exper¬ 
imental  procedures  and  autofluorescence  remains  constant.  As 
accurate  calibration  remains  problematic  and  time  consuming  to 
do  properly  with  appropriate  controls,  it  may  be  better  to  design 
experiments  where  one  need  only  determine  the  direction  of 
change  rather  than  measure  the  absolute  ion  levels. 

CONCLUSION 

The  new  fluorescent  probes  coupled  with  the  LSCM  allow 
unprecedented  sensitivity  and  resolution  for  the  detection  of  ion 
levels  in  four  dimensions  within  living  cells.  The  current  range  of 
dyes  and  the  equipment  available  has  enabled  resolution  of  micro¬ 
scopic  signaling  on  the  scale  of  <  1  pm  and  1  ms  for  quite  long 
periods.  At  this  level,  we  are  measuring  such  small  numbers  of 
ions  and  molecules  that  the  system  can  become  severely  photon 
limited.  To  increase  resolution  much  further,  a  very  different 
approach  will  be  needed,  perhaps  one  where  we  no  longer  look  at 
the  second  messenger  per  se  but  force  the  cell  to  report  its  response 
by  generating  fluorescent  signals  from  engineered  target  proteins. 
Whatever  method  is  used,  we  should  not  lose  sight  of  the  fact  that 
measurement  of  responses  will  perturb  the  cell  response  we  are 
studying. 
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Confocal  and  Multi-Photon  Imaging  of  Living  Embryos 

Jeff  Hardin 


INTRODUCTION 

By  its  very  nature,  developmental  biology  requires  thinking  in  four 
dimensions.  Not  only  do  embryos  change  dramatically  over  time, 
as  the  seemingly  featureless  single-celled  zygote  is  transformed 
into  an  embryo  with  recognizable  body  axes  and  organ  systems, 
but  this  remarkable  transformation  occurs  in  three  spatial  dimen¬ 
sions.  The  coordinated  changes  that  occur  within  the  developing 
embryo  include  carefully  orchestrated  signaling  events,  changes  in 
gene  expression,  and  morphogenetic  movements,  that  is,  regulated 
cell  divisions  and  cell  movements  that  sculpt  the  basic  body  plan 
as  a  recognizable  organism  emerges. 

Non-confocal  imaging  modalities,  such  as  Nomarski 
microscopy  and  widefield  epifluorescence,  have  played  a  key 
role  in  the  analysis  of  whole,  living  embryos  in  several  important 
model  systems,  such  as  Caenorhabditis  elegans  (Thomas  et  al., 
1996;  Thomas  and  White,  2000)  and  zebrafish  and  other  teleosts 
(Concha  and  Adams,  1998).  In  less  transparent  embryos,  includ¬ 
ing  amphibians  such  as  Xenopus  (Wilson  and  Keller,  1991)  and 
avian  embryos  (Chapman  et  al.,  2001;  Czirok  et  al.,  2002),  impor¬ 
tant  work  has  been  done  to  analyze  overall  changes  in  shape  of 
embryos  or  in  microsurgically  explanted  tissues.  However,  it  is  no 
exaggeration  to  say  that  confocal  microcopy  has  been  a  revolu¬ 
tionary  technology  in  the  resurgence  of  microscopy  as  a  key  ana¬ 
lytical  tool  of  the  developmental  biologist.  Beginning  in  the  late 
1980s  and  continuing  throughout  the  decade  of  the  1990s  to 
the  present,  the  development  of  devices  that  could  capture  four¬ 
dimensional  (4D)  datasets  of  fluorescent  molecules  within 
embryos  with  increasing  temporal  and  spatial  resolution  has 
allowed  processes  progressively  deeper  within  embryos  to  be 
studied  for  the  first  time.  As  laser,  photomultiplier,  and  charge- 
coupled  device  (CCD)  technologies  have  improved,  so  has  our 
ability  to  image  large  embryos  in  situ ,  including  opaque  embryos, 
such  as  Xenopus  and  the  chick.  The  impact  of  confocal  microscopy 
has  not  been  restricted  to  the  specialist;  the  striking  images 
obtained  using  confocal  microscopy  have  also  transformed  how 
data  are  presented  in  the  teaching  laboratory  and  classroom. 

Despite  these  successes,  challenges  remain.  Some  of  these 
challenges  are  specific  to  particular  types  of  embryos,  but  others 
are  faced  by  all  developmental  biologists,  no  matter  which  model 
system  they  choose.  This  chapter  sets  out  several  of  these  common 
challenges,  and  how  developmental  biologists  are  overcoming 
them  as  they  attempt  to  bring  modern  confocal  and  multi-photon 
microscopy  to  bear  on  key  developmental  events. 


INTO  THE  DEPTHS:  EMBRYOS  ARE  THICK, 
REFRACTILE,  AND  SUSCEPTIBLE  TO 
PHOTODAMAGE 

Imaging  Embryos  Often  Requires 
"4D"  Imaging 

One  of  the  vicissitudes  of  imaging  living  embryos  is  that  they  are 
thick,  and  their  structure  is  constantly  changing.  As  a  result, 
embryos  must  often  be  imaged  throughout  their  thickness  and  over 
time.  Such  4D  imaging  entails  the  collection  of  a  stack  of  images 
made  at  different  focal  depths  along  the  imaging  axis  at  specified 
time  intervals.  Once  each  three-dimensional  (3D)  image  stack  has 
been  collected,  the  objective  is  repositioned  at  the  starting  position 
in  preparation  for  the  next  time  point,  at  which  time  this  cycle  is 
repeated.  For  typical  long-term  acquisition  experiments  involving 
fluorescent  probes  detected  in  living  embryos,  exposure  of 
embryos  to  the  excitation  beam  is  prevented  between  time  points 
by  a  shuttering  system,  thereby  minimizing  photodamage  to  the 
embryo.1  Although  the  details  differ,  modern  commercial  confo¬ 
cal,  multi-photon,  and  widefield/deconvolution  microscopes  all 
offer  acquisition  packages  that  allow  for  the  collection  of  such  4D 
datasets. 

Four-dimensional  datasets  of  fluorescent  specimens  acquired 
using  confocal,  multi-photon,  or  widefield  deconvolution  tech¬ 
niques  have  several  advantages  over  4D  datasets  acquired  using 
transmitted  light  optics,  such  as  Nomarski  microscopy  (Thomas  et 
al.,  1996;  Thomas  and  White,  2000).  First,  confocal  acquisition 
permits  much  more  refined  optical  sectioning  of  the  specimen  with 
little  contribution  by  out-of-focus  information.  For  large,  thick 
embryos,  this  means  that  events  at  significant  distances  from  the 
coverslip  can  be  imaged  crisply  without  contaminating  signal  from 
other  regions  of  the  embryo.  Second,  but  of  equal  importance,  it 
is  much  easier  to  understand  the  distribution  of  the  fluorescent 


1  In  this  chapter,  as  elsewhere  in  this  book,  we  assume,  that,  to  a  first  approx¬ 
imation,  photodamage  is  proportional  to  the  total  number  of  fluorescent  signal 
photons  actually  excited  and  that  one  scan  at  a  power  level  of  lOOtlW  will 
do  the  same  damage  as  10  scans  at  I  OpW.  This  means  that  one  must  use  much 
lower  levels  of  excitation  in  a  study  requiring  many  planes  per  --stack  and 
many  time  points  in  the  time  series.  This  proportionality  may  break  down  at 
higher  power  levels  that  approach  singlet-state  saturation  of  the  fluorophore 
where  the  chance  increases  that  each  excitation  will  produce  damage. 
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signal  from  a  sample  on  the  basis  of  a  3D  voxel  rendering  than  to 
derive  any  sort  of  “real”  data  from  a  3D  stack  of  differential  inter¬ 
ference  contrast  (DIC)  images  (for  an  attempt  at  the  latter,  see  Heid 
et  al.,  2002  and  Chapter  24,  this  volume). 

The  Quest  for  Better  Resolution:  Aberration  and 
the  Challenge  of  Imaging  Thick  Embryos 

It  is  common  for  beginning  students  of  developmental  biology  to 
become  preoccupied  by  the  dazzling  sophistication  of  modern 
computer  visualization  techniques.  While  they  certainly  have 
visual  appeal,  it  is  sometime  less  obvious  to  students  and 
researchers  that  optimizing  the  optical  system  to  get  better  data  can 
often  be  more  important  for  imaging  embryos  than  the  subsequent 
use  of  software  to  display  the  raw  image  data. 

Although  it  may  seem  obvious  to  anyone  who  has  tried  to 
image  events  during  embryogenesis,  it  is  worth  considering  the 
crucial  optical  challenge  developmental  biologists  face  as  they  try 
to  capture  and  analyze  dynamic  processes  during  embryogenesis: 
embryos  are  rarely  flat.  Indeed,  often  they  are  extremely  thick 
along  the  axis  of  the  light  path.  For  example,  Xenopus  embryos 
are  roughly  1  mm  in  diameter,  and  chick  embryos  are  larger  still 
at  late  stages  in  development.  These  distances  mean  that  even  in 
the  absence  of  significant  light  scattering  by  the  specimen,  spher¬ 
ical  aberration  can  be  an  important  constraint  on  obtaining  quality 
images  deep  within  a  specimen  (see  Chapters  2,  7,  and  20,  this 
volume ,  for  further  details  on  the  effects  of  spherical  aberration). 
For  thick  specimens,  aberration  introduced  due  to  refractive  index 
mismatch  between  oil-immersion  objectives  and  the  aqueous 
medium  is  significant.  For  thin  specimens  imaged  within  5  pm  of 
the  coverslip  (such  as  C.  elegans  embryos,  in  cases  where  surface 
features  are  of  interest,  or  when  an  explant  technique  has  been  used 
to  effectively  flatten  the  specimen),  a  high  numerical  aperture  (NA) 
lens  (NA=  1.4)  is  typically  the  best  choice. 

The  spatial  effects  of  these  aberrations  have  been  determined 
for  both  confocal  and  two-photon  microscopes,  and  they  can  be 
profound.  Mismatch  artifacts  can  affect  axial  resolution,  introduce 
focal  shifts  of  the  image  plane,  and  result  in  dramatic  decreases  in 
image  intensity  (Hell  et  al.,  1993;  DeGrauw  et  al.,  2002).  Such 
problems  may  be  solved  in  part  by  using  water-immersion  lenses 
for  specimens  more  than  ~10pm  in  thickness,  although  many 
developmental  biologists  assume  that  maximizing  numerical  aper¬ 
ture  by  using  oil-immersion  lenses  is  the  sine  qua  non  of  high- 
resolution  imaging.  For  aqueous  specimens  thicker  than  ~10pm, 
a  water-immersion  lens  with  a  lower  NA  outperforms  a  high-NA 
oil-immersion  lens. 

Embryos  Are  Highly  Scattering  and 
Retractile  Specimens 

Light  scattering  is  often  fairly  acute,  even  in  the  case  of  relatively 
small  embryos,  such  as  C.  elegans  and  Drosophila  ( — 25  pm  and 
200pm  along  their  short  axes,  respectively).  Within  the  cells  of 
such  embryos  particles  that  are  significantly  smaller  than  the  exci¬ 
tation  and  emission  wavelengths  contribute  to  Rayleigh  scattering 
(see  Chapter  21,  this  volume),  which  depends  inversely  on  the 
fourth  power  of  the  wavelength.  Consequently,  such  scattering  can 
be  lessened  by  using  longer  excitation  wavelengths;  for  strongly 
scattering  specimens,  two-  or  multi-photon  imaging  can  further 
reduce  scattering  of  excitation  photons  (see  below).  Unfortunately, 
there  are  other  major  sources  of  scattering  in  embryos.  The  cells 
of  early  embryos  typically  have  yolk  platelets  or  other  highly 


refractile  granules  within  the  cytoplasm  that  deflect  both  excited 
and  emitted  light.  Moreover,  most  embryos  are  spherical;  the  cur¬ 
vature  of  their  outer  surface  is  also  a  major  source  of  refraction. 

Imaging  Embryos  Involves  Inherent  Trade-Offs 

Imaging  thick  embryos  typically  means  a  trade-off  between  lateral 
and  axial  resolution  and  signal  strength  on  the  one  hand,  and 
working  distance  and  field  diameter  on  the  other.  Typical  epi- 
fluorescence  objectives  effectively  act  as  both  condenser  for  inci¬ 
dent  light  passing  through  the  excitation  filter  and  off  the  dichroic 
mirrors,  as  well  as  serving  as  the  objective  lens  that  collects  and 
focuses  the  photons  emitted  from  the  specimen.  As  magnification 
increases,  signal/area  in  the  image  decreases,  but  this  is  counter¬ 
balanced  by  the  higher  NA  of  the  objective.  For  high-resolution 
imaging,  developmental  biologists  typically  seek  to  maximize 
numerical  aperture  while  keeping  the  magnification  low  enough  to 
permit  them  to  see  the  whole  embryo  or  at  least  the  part  of  it  of 
most  interest. 

Although  the  discussion  of  contrast  is  complex,  it  is  clear  that 
for  fluorescent  specimens,  contrast  and  effective  resolution  of 
structures  labeled  with  fluorophores  are  linked  (for  further  discus¬ 
sion  of  the  issues  relating  the  contrast  transfer  function,  CTF,  see 
Chapters  2,  4,  8,  and  35,  this  volume).  At  a  simple  level,  contrast 
is  determined  by  the  ratio  of  local  intensity  differences  in  the  spec¬ 
imen  to  the  local  average  intensity  in  the  image.  A I/I.  To  a  first 
approximation,  the  local  emission  intensity  difference  for  typical 
applications  in  embryos  is  proportional  to  the  local  concentration 
of  fluorophores  above  background.  Moreover,  as  the  emitted  signal 
is  proportional  to  the  local  fluorophore  concentration,  develop¬ 
mental  biologists  usually  seek  to  increase  the  local  signal  as  much 
as  possible  by  increasing  staining  intensity  or  the  concentration  of 
green  fluorescent  protein  (GFP)-tagged  molecules  per  unit  volume. 
Here  again,  however,  there  are  often  trade-offs  because  excessive 
labeling  achieved  by  over-expression  of  a  fluorescent  protein  can 
interfere  with  cellular  function. 

Although  increasing  the  illumination  intensity  increases  the 
fluorescent  signal,  dynamic  imaging  of  fluorophores  within  living 
embryos  requires  multiple  exposures  to  the  excitation  light,  so 
developmental  biologists  can  rarely  afford  to  use  high  illumination 
intensity.  Indeed,  they  are  acutely  aware  that  this  simple  solution 
typically  results  in  dead  embryos!  Therefore,  a  balance  must  be 
sought  between  the  intensity  of  the  illumination  and  a  host  of  other 
factors  important  to  achieving  quality  images.  Because  embryos 
are  so  thick  and  their  cells  are  moving  in  four  dimensions,  multi¬ 
ple  focal  planes  must  be  acquired  at  each  successive  time  point. 
While  such  4D  information  is  powerful,  it  requires  exposing  the 
region  of  interest  to  more  laser  light  than  that  required  for  viewing 
flat  cells  in  culture.  Thus,  minimizing  exposure  to  the  excitation 
wavelength  is  a  key  factor  in  maintaining  the  viability  of  embryos 
during  4D  experiments. 

The  photobleaching  that  accompanies  repeated  observation  in 
4D  experiments  also  complicates  image  quantification.  If  photo- 
bleaching  is  not  too  severe,  it  is  sometimes  possible  to  normalize 
the  signal  from  bleached  specimens  using  a  reference  fluorescent 
signal,  but  this  only  works  if  the  specimen  and  the  reference  bleach 
at  the  same  rate.  In  general,  the  best  solution  to  this  problem  is 
reducing  the  power  of  the  excitation  beam  as  much  as  possible. 
Although  multi-photon  microscopy  has  the  advantage  that  fluo¬ 
rophore  is  excited  only  within  a  small  volume  near  the  focal  plane 
of  the  objective,  this  advantage  must  be  balanced  against  the  fact 
that  some  dyes  bleach  more  rapidly  per  photon  emitted  with  two- 
photon  than  with  single-photon  excitation.  Clearly,  multi-photon 
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excitation  is  the  imaging  modality  of  choice  if  bleaching  is  a 
serious  concern  and  the  stained  thickness  of  the  specimen  is  high. 

Common  Themes  in  Living  Embryo  Imaging 
Have  System-Specific  Solutions 

Although  developmental  biologists  have  developed  several  ways 
to  address  challenges  common  to  imaging  all  living  embryos, 
given  the  dramatic  differences  in  the  basic  cell  biology  between 
the  most  common  model  organisms,  solutions  are  typically  spe¬ 
cific  to  a  particular  model  system.  Rather  than  reviewing  the 
virtues  and  drawbacks  of  each  model  system  and  describing  highly 
detailed  techniques  for  solving  particular  imaging  issues  in  each 
one,  the  remainder  of  this  chapter  will  focus  on  examples  of  ways 
in  which  developmental  biologists  are  meeting  the  challenges  of 
long-term  imaging  of  fluorescent  molecules  deep  within  living 
embryos. 

Before  reviewing  such  examples,  several  generalizations  about 
the  apparatus  and  dyes  useful  in  imaging  living  embryos  are  in 
order.  First,  there  is  a  necessary  element  of  empiricism  in  deter¬ 
mining  which  type  of  microscope  is  necessary  for  a  particular  flu- 
orophore  in  a  particular  embryonic  context.  Currently,  standard 
laser-scanning  confocal  systems  still  dominate  over  other  types  of 
imaging  equipment,  and  thus  a  significant  amount  of  live  embryo 
imaging  is  being  attempted  —  and  successfully  performed  —  on 
such  equipment.  While  it  is  true  that  there  are  advantages  to  multi¬ 
photon  microscopy  in  terms  of  depth  of  penetration  into  a  speci¬ 
men  and,  in  some  cases,  viability,  laser  scanning  suffices  for  a  wide 
variety  of  imaging  situations.  Depending  on  the  density  of  fluo- 
rophore  labeling  and  the  nature  of  the  tagged  moiety,  the  majority 
of  experiments  have  been  performed  using  rather  unremarkable 
equipment,  such  as  a  Bio-Rad  1024  or  similar  laser-scanning  con- 
focal  apparatus.2  Although  such  experiments  sometimes  suffer 
from  limited  image  acquisition  speed,  live  imaging  of  embryos 
using  laser-scanning  confocal  microscopy  has  been  extremely 
successful. 

A  second  general  consideration  is  that  the  way  in  which  a  fluo- 
rophore  affects  viability  is  crucially  dependent  on  the  macromol¬ 
ecule  to  which  it  is  attached.  For  example,  in  our  laboratory, 
transcriptional  reporters  that  drive  the  expression  of  GFP  in 
the  cytosol  can  withstand  irradiation  at  10%  to  30%  power  of  the 
488  line  on  a  conventional  Bio-Rad  1024  microscope  with  a 
krypton/argon  (Kr/Ar)  laser  for  many  minutes  to  hours,  whereas 
imaging  embryos  expressing  a  GFP-tagged  protein  found  at 
epithelial  junctions  under  the  same  imaging  conditions  either  kills 
the  cells  or  renders  them  unable  to  develop  properly  (J.  Hardin, 
unpublished  observations). 

Third,  even  if  the  problems  of  working  with  thick,  refractile 
embryos  can  be  at  least  partially  solved  by  using  confocal  or  multi¬ 
photon  microscopy,  developmental  biologists  still  need  ways  to 
specifically  label  interesting  features  in  living  embryos.  In  some 
cases,  they  wish  to  highlight  multiple  structures  simultaneously 
within  the  same  specimen,  and  this  requires  delivering  multiple 
fluorophores  to  specific  sites  within  embryos.  Although  not  spe¬ 
cific  to  confocal  and  multi-photon  microscopy,  specific  labeling 


2  Interested  readers  are  urged  to  consult  reviews  describing  methods  of  confo¬ 
cal  and  multi-photon  imaging  in  specific  model  systems.  These  include 
methods  for  imaging  Xenopus  (Bement  et  al. ,  2003),  zebrafish  (Cooper  et  al., 
1999).  Drosophila  (Wood  and  Jacinto.  2004),  and  C.  elegans  (Mohler,  1999) 
embryos. 


strategies  are  nevertheless  a  key  adjunct  to  these  imaging  tech¬ 
nologies.  In  essence,  this  is  a  continuation  of  the  microscopist’s 
quest  to  increase  specimen  contrast,  no  matter  what  imaging 
modality  is  being  used. 

DEALING  WITH  DEPTH:  STRATEGIES  FOR 
IMAGING  THICK  SPECIMENS 

Avoiding  the  Thickness  Dilemma:  Going  Small 

Model  systems  in  developmental  biology  are  often  chosen  to  avoid 
the  thickness  dilemma.  For  example,  C.  elegans  embryos  are  only 
~25|im  thick  along  their  dorsoventral  axis,  commonly  used 
echinoderm  embryos  are  only  80  to  120  pm  in  diameter,  and 
Drosophila  embryos  are  ~200pm  along  their  dorsoventral  axis. 
Zebrafish  are  at  the  limits  of  penetration  via  multi-photon 
microscopy  (-600  pm  in  diameter),  but  because  most  relevant 
events  occur  near  the  surface,  they  too  are  effectively  fairly  flat. 
As  a  result,  there  are  many  examples  in  which  embryos  from  these 
model  systems  are  imaged  in  4D  using  confocal  and  multi-photon 
techniques.  These  include  the  imaging  of  GFP-tagged  proteins  in 
Drosophila  embryos  during  dorsal  closure  and  germ  band  exten¬ 
sion,  processes  that  take  several  hours  (Wood  and  Jacinto,  2004), 
cell  migration  in  the  epidermis  during  morphogenesis  of  C. 
elegans  embryos  (Mohler  et  al.,  1998;  Heid  et  al.,  2001),  events 
during  fertilization  and  subsequent  early  development  of  echino¬ 
derm  embryos  (Terasaki,  1998),  and  4D  analysis  of  morphogene¬ 
sis  in  zebrafish  embryos  (Cooper  et  al.,  1999). 

Grazing  the  Surface:  Superficial  Optical  Sections 
Are  Often  Sufficient 

While  many  model  systems  are  small  in  size,  some  are  at  the  limits 
of  standard  confocal  microscopy.  For  example,  Xenopus  embryos 
are  a  full  1  mm  in  diameter.  Moreover,  because  their  dense  yolk 
platelets  scatter  light,  imaging  deep  into  the  interior  of  amphibian 
embryos  is  an  acute  technical  challenge.  One  simple  strategy  is  to 
concede  that  imaging  events  deep  within  the  embryo  is  not  possi¬ 
ble,  and  to  focus  on  what  can  be  seen  near  the  surface  of  the  oocyte, 
fertilized  egg,  or  early  zygote.  Such  events  can  be  imaged 
extremely  well  using  a  single-beam  confocal  microscope  and  a 
variety  of  labeling  techniques,  including  fluorescent  dextrans, 
lipophilic  dyes,  fluorescently  labeled  proteins  or  antibodies  intro¬ 
duced  via  microinjection,  or  by  expressing  GFP-tagged  proteins 
following  injection  of  mRNA  (Bement  et  al.,  2003).  A  montage 
from  a  representative  dataset  from  a  Xenopus  oocyte  is  shown  in 
Figure  43.1(A),  which  shows  the  redistribution  of  active  Rho  and 
actin  during  the  closure  of  a  wound  in  a  Xenopus  oocyte.  Such 
imaging  experiments  often  involve  4D  imaging,  although  because 
of  the  high  speed  of  the  movements  involved,  only  a  few  optical 
sections  (<10)  can  typically  be  recorded  at  each  time  point 
(Bement  et  al.,  2003).  To  image  events  deeper  within  the  embryo, 
however,  additional  techniques  must  be  employed. 

Up  from  the  Deep:  Explants  Can  Reduce  the 
Thickness  of  Specimens  Dramatically 

In  some  cases,  imaging  the  surface  of  thick  embryos  is  simply  not 
sufficient.  For  example,  during  gastrulation,  cells  move  large  dis¬ 
tances  within  the  interior  of  the  embryo.  In  the  case  of  the  rela¬ 
tively  opaque  Xenopus  embryo,  it  is  simply  not  feasible  to  image 
such  tissues  directly,  because  they  may  be  several  hundred 
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FIGURE  43.1 .  Strategies  for  imaging  very  thick  embryos.  (A)  A  4D  study  of  events  on  the  surface  of  a  Xenopus  oocyte.  Shows  wound  closure  in  a  Xenopus 
oocyte  expressing  a  protein  that  binds  active  Rho  fused  to  eGFP  (eGFP-rGBD;  green)  and  Alexa-568  labeled  actin  (red)  imaged  using  a  Bio-Rad  1024  CLSM. 
Bar  =15  pm.  Times  indicated  are  in  seconds.  (B-D)  Imaging  events  in  an  explant.  (B)  Frames  from  a  fluorescence  confocal  movie  in  which  intercalating  Xenopus 
mesodermal  cells  are  visualized  at  two  levels:  the  surface  of  the  explant  (red  channel)  and  5  pm  deep  in  the  tissue  (green  channel).  Cell  membranes  are  labeled 
with  GAP-43 : :  GFP.  Bar  =  50  pm.  (C)  Frames  from  a  confocal  time-lapse  sequence  showing  migration  of  primordial  germ  cells  (PGCs)  in  a  slice  from  a  stage 
E9.5  embryo  expressing  Oct4APE : : GFP,  imaged  at  7min  intervals  using  a  Zeiss  LSM-510  CLSM.  PGCs  migrate  into  the  genital  ridges.  Data  taken  455min 
later.  Bar  =  50  pm.  (D)  Ventral  view  of  an  explanted  zebrafish  tail  rudiment  (14-somite  stage)  mounted  ventral  side  down  using  plasma  clot  immobilization  vitally 
stained  with  Bodipy  505/515  imaged  using  a  Bio-Rad  MRC600  CLSM.  The  notochord  and  a  neighboring  somite  (so,  arrowheads)  are  visible.  An  individual 
somitic  mesodermal  cell  (arrow)  is  in  mitotic  prophase.  Scale  bars  =  50  pm.  [(A)  is  from  Benink  and  Bement  (2005);  (B)  is  from  Keller  (2002);  (C)  is  from 
Molyneaux  and  colleagues  (2003);  (D)  is  from  Langenberg  and  colleagues  (2003),  used  by  permission.] 


micrometers  from  the  surface,  and  light  scattering  is  severe  at  that 
depth.  In  this  case,  a  better  approach  is  to  isolate  the  tissue  of  inter¬ 
est  using  microsurgery.  By  isolating  a  sheet  of  tissue  that  is  rela¬ 
tively  thin  along  the  imaging  axis,  the  embryo  has  in  effect  been 
flattened.  Such  an  embryo,  expressing  a  membrane-localized  GFP, 
is  shown  in  Figure  43.1(B).  Although  4D  datasets  must  still  be 
acquired  to  capture  the  relevant  aspects  of  cell  motility  in  this  spec¬ 
imen,  the  explant  is  sufficiently  transparent  that  one  can  acquire  z- 
stacks  that  encompass  its  entire  thickness  (Keller,  2002).  Other 
similar  techniques  have  been  used  to  render  tissues  in  other  large 
embryos  amenable  to  4D  analysis.  These  include  analysis  of  pri¬ 
mordial  germ  cell  migration  in  mouse  embryos  (Molyneaux  et  al., 
2003)  [Fig.  43.1(C)]  and  the  microsurgical  removal  of  the  yolk 
cells  from  post-gastrula  stage  zebrafish  embryos  (Langenberg  et 
al.,  2003)  [Fig.  43.1(D)). 


Multi-Photon  Microscopy  Can  Penetrate  More 
Deeply  into  Specimens 

Multi-photon  laser-scanning  microscopy  (MPLSM)  has  gained 
wide  acceptance  as  an  alternative  to  standard  laser-scanning  con¬ 
focal  microcopy  because  it  has  several  potential  advantages  over 
CLSM  for  live  imaging  of  embryos.  MPLSM  excites  fluorescence 
using  a  series  of  short,  high-energy  pulses  of  near-infrared  (NIR) 
photons  from  a  mode-locked  laser.  At  the  very  high  photon  densi¬ 
ties  that  result,  two  or  more  photons  can  be  absorbed  simultane¬ 
ously  to  excite  a  single  fluorophore,  which  then  returns  to  the 
unexcited  ground  state  by  a  standard  fluorescence  emission  event. 
In  the  case  of  two-photon  excitation,  the  excitation  wavelength  is 
set  to  roughly  twice  that  used  for  single-photon  (i.e.,  normal)  exci¬ 
tation  of  the  same  fluorophore  (see  Chapter  28,  this  volume,  for  a 
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detailed  description  of  multi-photon  microscopy).  Although  theo¬ 
retical  calculations  and  measurements  of  actual  point  spread  func¬ 
tions  indicate  that  the  axial  resolution  in  MPLSM  is  about  40% 
lower  than  in  CLSM  (Jonkman  and  Stelzer,  2002),  MPLSM  has 
two  key  advantages  for  live  embryo  imaging  experiments. 

As  the  two-photon  excitation  probability  is  proportional  to  the 
square  of  the  local  beam  intensity  (see  Chapter  28,  this  volume), 
given  the  conical  shape  of  the  standard  excitation  beam,  the  prob¬ 
ability  of  excitation  varies  as  the  inverse  fourth  power  of  the  dis¬ 
tance  from  the  focal  plane.  The  result  is  that  photons  are  absorbed 
only  in  a  very  small  volume  centered  on  the  plane  of  focus.  As 
long  as  no  regions  of  the  specimen  cause  single-photon  absorption 
of  light  at  this  wavelength,  it  is  essentially  transparent  to  the  inci¬ 
dent  laser  light.  This  eliminates  photobleaching  and  photodamage 
caused  by  excitation  of  fluorophores  above  and  below  the  plane  of 
focus. 

The  resulting  improvements  in  viability  can  be  quite  dramatic. 
For  example,  hamster  embryos  imaged  with  CLSM  using  the  mito¬ 
chondrial  marker  Mitotracker  Rosamine  did  not  develop  to  the 
blastocyst  stage,  whereas  similarly  labeled  embryos  imaged  using 
MPLSM  at  1047  nm  were  capable  of  late  fetal  development  and 
even  birth  (Squirrel  et  al.,  1999).  In  our  own  laboratory,  C.  elegans 
embryos  expressing  a  GFP-tagged  junctional  protein  survive  for 
30  to  90  min  when  imaged  using  a  Bio-Rad  1024  CLSM  at  low 
power  (10%;  J.  Hardin,  unpublished  observations)  but  the  same 
embryos  can  be  imaged  for  many  hours  using  MPLSM. 

Second,  the  longer  wavelength  of  the  MPLSM  excitation  beam 
scatters  less  and  is  not  absorbed  by  fluorophores  located  above  the 
focus  plane.  These  characteristics  permit  it  to  excite  fluorophores 
deep  inside  the  specimen  (Fig.  43.2),  in  some  transparent  embryos 


such  as  zebrafish,  up  to  -500  pm  from  the  surface  (Megason  and 
Fraser,  2003).  In  addition,  because  multi-photon  excitation  gener¬ 
ates  significant  signal  only  at  the  plane  of  focus,  no  pinhole  is 
required  to  detect  an  image  resolved  in  three  dimensions.  One  must 
simply  detect  light  that  was  emitted  at  a  specific  time.  As  a  result, 
one  can  used  non-descanned  detection,  a  process  in  which  even 
fluorescent  light  that  had  suffered  multiple  scattering  events  can 
still  contribute  to  the  image  as  long  as  it  reaches  a  photodetector. 
For  thick  specimens,  this  can  increase  signal  level  by  a  factor  of 
up  to  3x  (Centonze  and  White,  1998). 

Although  MPLSM  can  be  superior  to  CLSM  for  many  appli¬ 
cations,  there  are  several  issues  that  can  make  MPLSM  less  than 
optimal.  First,  the  typical  MPLSM  device  is  expensive,  placing  it 
out  of  reach  of  most  individual  laboratories.  In  contrast,  individ¬ 
ual  labs  can  often  afford  disk-scanning  confocal  microscopes,  a 
factor  that  is  particularly  important  for  live  embryo  studies,  which 
often  monopolize  microscope  time.  Second,  for  certain  fluo¬ 
rophores,  for  example,  those  that  emit  in  the  red  portion  of  the 
visible  spectrum,  the  wavelengths  needed  to  generate  a  two-photon 
event  are  longer  than  those  produced  by  the  titanium :  sapphire 
(Ti :  Sa)  lasers  commonly  used  in  commercial  MPLSM  devices. 
For  such  dyes,  Nd:YLF  lasers,  which  emit  at  1047  nm,  are  very 
effective  (Mohler  et  al.,  1998;  Squirrell  el  al.,  1999;  see  also  Fig. 
43.9),  but  may  not  be  readily  available.  Third,  some  pigments,  such 
as  those  found  in  living  Xenopus  embryos,  absorb  the  NIR  light 
used  to  excite  GFP  and  other  green-emitting  fluorophores  by  two- 
photon  events.  Such  absorption  usually  causes  severe  damage  (see 
Chapter  38,  this  volume ). 

Although  MPLSM  is  clearly  a  tremendous  tool  in  the  arsenal 
of  the  developmental  biologist,  in  light  of  these  caveats,  it  is 


FIGURE  43.2.  Multi-photon  microscopy  is  useful  for  penetrating  more  deeply  into  embryos.  (A)  A  bovine  embryo  stained  with  Mitotracker  rosamine  to  detect 
mitochondria,  imaged  via  confocal  microscopy  (left;  3.5W  of  532nm  light)  and  by  multi-photon  microscopy  (right;  YLF  laser,  direct  detection,  lOmW  of 
1047  nm  light),  using  a  Kalman  three-collection  average.  Mitochondria  are  roughly  uniformly  distributed  in  such  embryos.  Mitochondria  cannot  be  visualized 
using  the  CLSM,  but  are  clearly  visible  using  the  MPLSM.  Bar  =  20  gm.  (B)  Comparison  of  multi-photon  and  confocal  microscopy  for  imaging  morphogene¬ 
sis  in  the  Xenopus  embryo.  Images  of  GAP-43 ::  GFP  fusion  protein  in  developing  Xenopus  embryos  were  acquired  with  the  same  objective  lens  using  CLSM 
(Nikon  PCM  2000)  and  TPLSM  systems  (Bio-Rad  MRC  600  scan  head,  with  Ti :  Sa  laser  at  870  nm).  Less  detail  is  evident  deeper  in  the  specimen  with  CLSM. 
Bar  =  50pm.  [(A)  is  from  Centonze  and  White  (1998);  (B)  is  adapted  from  Periasamy  and  colleagues  (1999),  used  by  permission.] 
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usually  wise  to  try  standard  confocal  imaging  first  before  assum¬ 
ing  that  MPLSM  is  necessary  for  a  particular  specimen.  Our 
own  laboratory  has  found,  for  example,  that  GFP-tagged  proteins 
can  usually  be  imaged  in  living  C.  elegans  embryos  with  disk¬ 
scanning  or  standard  CLSM;  on  the  rare  occasions  that  the  image 
series  must  proceed  for  many  hours  and  many  focal  planes  must 
be  acquired,  MPLSM  remains  the  method  of  choice. 

Selective  Plane  Illumination  Can  Provide  Optical 
Sectioning  in  Very  Thick  Specimens 

Originally  developed  by  Voie  et  al,  (1993)  and  more  recently 
improved  by  Stelzer  and  colleagues  (Huisken  et  al.,  2004),  selec¬ 
tive  plane  illumination  microscopy  (SPIM;  Fig.  43.3),  involves  a 
strategy  akin  to  that  achieved  in  nonlinear  (two-  and  multi-photon) 
microscopy,  that  is,  fluorescence  is  only  excited  in  a  narrow  plane 
perpendicular  to  the  imaging  axis  (see  Chapter  37,  this  volume ).  A 
cylindrical  lens  is  used  to  focus  laser  light  into  a  light  sheet  that 
illuminates  only  a  thin  slice  of  material.  In  the  current  configura¬ 
tion,  a  specimen  is  embedded  in  an  agarose  capsule,  and  the  entire 
specimen  is  moved  within  a  bath  of  liquid.  Because  only  the  focal 
plane  being  observed  is  illuminated,  the  SPIM  has  similar  advan¬ 
tages  to  MPLSM:  far  fewer  excitations  are  produced  during  the 
acquisition  of  a  3D  dataset.  Unlike  MPLSM,  however,  the  signal 
is  recorded  on  a  high  quantum  efficiency  CCD  camera.  These  fea¬ 
tures  make  SPIM  well  suited  to  imaging  specimens  up  to  a  mil¬ 
limeter  in  size  (see  also  Chapter  37,  this  volume). 


FIGURE  43.3.  Selective-plane  illumination  (SPIM)  can  optically  section  huge 
embryos.  A  lateral  view  of  a  medaka  embryo  imaged  with  SPIM  by  two  dif¬ 
ferent  modes  of  illumination.  (A)  The  sample  was  illuminated  uniformly,  that 
is,  without  the  cylindrical  lens,  as  in  a  conventional  wide-held  microscope.  (B ) 
In  contrast,  selective  plane  illumination  results  in  optical  sectioning.  Bar  = 
500  pm.  [From  Huisken  and  colleagues  (2004),  used  by  permission.] 


In  analogy  to  the  pinhole  setting  on  a  standard  laser-scanning 
confocal  microscope,  the  axial  resolution  in  SPIM  depends  on  the 
thickness  of  the  light  sheet,  which  depends  in  turn  on  the  working 
aperture  of  the  cylindrical  lens  and  ranges  from  1  pm  to  6  pm.  This 
performance  can  be  improved  by  obtaining  additional  3D  stacks 
after  rotating  the  specimen,  a  process  that  can  provide  isotropic 
resolution  down  to  about  1  pm. 

Despite  these  impressive  results,  SPIM  is  still  an  emerging 
technology.  The  current  device  relies  on  embedding  specimens  in 
low-melting-point  agarose,  which  does  not  appear  to  affect  the 
development  of  fish  and  Drosophila  embryos  (Huisken  et  al., 
2004).  Our  laboratory  has  shown  that  C.  elegans  and  sea  urchin 
embryos  embedded  in  agarose  also  develop  surprisingly  well  (J. 
Hardin,  unpublished  observations),  suggesting  that  such  embed¬ 
ment  may  not  be  a  serious  limitation.  However,  as  agarose  embed¬ 
ding  imposes  a  mechanical  impediment  to  the  normal  development 
of  those  embryos  whose  overall  shape  changes  dramatically  during 
the  period  of  observation,  other  specimen  stages  may  have  to  be 
developed  to  accommodate  these  specimens.  In  addition,  SPIM 
cannot  currently  achieve  spatial  resolution  comparable  to  confocal 
or  multi-photon  imaging.  Nevertheless,  the  fact  that  SPIM  has  the 
resolution  to  identify  individual  cells  in  the  context  of  embryos  up 
to  1  mm  thick  makes  it  likely  that  it  will  become  a  useful  addition 
to  the  arsenal  of  imaging  techniques  at  the  disposal  of  the  devel¬ 
opment  biologist. 

Deconvolution  and  Other 
Post-Acquisition  Processing 

Widefield  deconvolution  has  typically  not  been  as  widely  used  for 
imaging  whole  embryos  as  have  confocal  and  multi-photon  tech¬ 
niques.  However,  widefield  acquisition,  followed  by  post¬ 
acquisition  deconvolution,  can  yield  images  comparable  in  quality 
to  confocal  or  multi-photon  datasets  (Paddy  et  al.,  1996)  [Fig. 
43.4(A,B)].  In  deconvolution  microscopy,  a  conventional  epi- 
fluorescence  microscope  with  attached  CCD  camera,  shuttering 
system,  and  focus  motor  is  used  to  capture  a  4D  dataset.  The 
dataset  is  subsequently  processed  to  remove  out-of-focus  contri¬ 
butions  to  the  image  at  each  focal  plane  (see  Chapters  23,  24,  and 
25,  this  volume,  for  a  detailed  discussion  of  the  merits  of  various 
deconvolution  algorithms  that  is  beyond  the  scope  of  this 
chapter). 

Deconvolution  has  been  used  less  frequently  for  imaging 
living  embryos  for  several  reasons.  First,  because  the  computations 
involved  are  extensive,  there  is  no  immediate  visual  feedback 
regarding  the  deconvolved  image.  This  immediacy  often  has  a  sig¬ 
nificant  psychological  effect  on  the  developmental  biologist  and 
may  help  her  to  guide  the  experimental  procedure.  Second,  the 
success  of  deconvolution  is  limited  by  the  noise  level  in  the 
recorded  image  data.  This  noise  is  primarily  Poisson  noise,  and 
the  Poisson  noise  associated  with  the  measurement  of  light  away 
from  the  focus  plane  is  particularly  troublesome.  Consequently, 
deconvolution  works  best  when  the  fluorescent  structures  are  con¬ 
fined  to  a  relatively  thin  layer  of  the  specimen.  Finally,  in  studies 
that  concentrate  on  imaging  only  a  single  plane  over  time,  con¬ 
focal  and  multi-photon  imaging  are  both  faster  and  less  damaging 
to  the  specimen. 

Despite  these  caveats,  widefield/deconvolution  microscopy 
has  several  advantages  for  imaging  embryos.  First,  excitation 
wavelengths  are  not  limited  by  available  laser  lines;  deconvolution 
microscopes  use  standard  fluorescence  filter  sets,  and  so  they  can 
easily  visualize  many  common  fluorophores,  such  as  the  cyan  and 
yellow  variants  of  GFP 
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FIGURE  43.4.  Deconvolution  microscopy  can  improve  optical  sectioning  of  embryos.  (A,  B)  Frames  from  a  time-lapse  movie  of  germ  band  retraction  in  a 
Drosophila  embryo  expressing  an  Engrailed::  GFP  reporter,  which  localizes  to  nuclei  in  specific  subdomains  within  each  of  14  segments  in  the  embryo,  obtained 
using  a  Delta  Vision  deconvolution  system.  Fine  details  of  the  expression  pattern  are  well  defined.  Bar  =  50  pm.  (C.  D)  Stereo-pair  images  of  immunostaining 
to  reveal  microtubule  distribution  in  a  syncytial  stage  Drosophila  embryo  imaged  using  a  Bio-Rad  MRC-600  LSCM  (C),  and  the  same  dataset  after  multiple 
rounds  of  deconvolution  (D).  Fine  details  within  the  mitotic  spindle  regions  are  visible  after  deconvolution  that  were  poorly  defined  prior  to  deconvolution.  Bar 
=  10pm.  [(A,  B)  Courtesy  of  I.  Davis,  University  of  Edinburgh.  (C,  D)  Courtesy  of  G.  Odell,  University  of  Washington.] 


Another  purported  advantage  of  deconvolution  microscopy  is 
its  claim  to  cause  less  photodamage  than  confocal  microscopy 
when  producing  an  image  of  a  given  quality.  If  true,  this  differ¬ 
ence  can  probably  be  traced  to  the  fact  that  the  arc  lamps  used  in 
widefield  epifluorescence  produce  lower  intensity  excitation  light, 
and  to  a  lesser  extent,  because  all  of  the  emitted  light  returned 
through  the  optical  path  reaches  the  camera  (Hammond  and  Glick, 
2000)  (see  also  Chapter  39,  this  volume). 

As  with  most  other  important  parameters  for  live  embryo 
imaging,  the  best  approach  is  probably  an  empirical  one.  Per¬ 
forming  a  “test  drive”  of  an  expensive  imaging  system  can  save 
time  and  significant  financial  resources.  Sometimes,  the  best  way 
to  take  such  a  test  drive  in  which  living  specimens  are  viewed  on 
a  variety  of  microscopes  over  a  relatively  prolonged  period  is  to 
attend  one  of  the  many  short  courses  offered  on  the  topic. 

Deconvolution  has  another  important  use:  it  can  almost  always 
improve  the  appearance  and  accuracy  of  4D  datasets  acquired 
using  confocal  or  multi-photon  imaging  [Fig.  43.4(C,D)].  Several 


studies  have  shown  that  postprocessing  of  confocal  and  multi¬ 
photon  stacks  using  deconvolution  can  significantly  improve  their 
z-resolution  (Periasamy  et  al.,  1999),  which  is  at  least  4x  lower 
than  their  jcv-resolution,  even  when  using  a  high-NA  lens 
(Jonkman  and  Stelzer,  2002). 3  Of  more  importance  for  studying 
living  embryos  is  the  fact,  that  because  Nyquist  sampling  spreads 
the  signal  from  a  single  point  over  about  100  voxels,  3D  decon¬ 
volution  has  the  effect  of  averaging  the  data  over  about  100  voxels. 
This  greatly  reduces  the  effects  of  Poisson  noise,  thereby  allowing 
one  to  use  much  lower  excitation  intensity  (see  Chapter  19,  this 
volume,  for  3D  data  collected  using  only  50  nW  of  laser  power, 
and  Chapter  25,  this  volume,  for  more  on  deconvolving  confocal 
and  multi-photon  data). 


3  Three  times  worse  for  oil,  NA  1.4  and  4x  for  water,  NA  1.2.  --resolution  is 
proportional  to  1/(NA)2. 
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Deconvolution  can  be  especially  useful  for  confocal  datasets 
of  living  embryos  as  these  are  often  collected  with  the  pinhole 
opened  somewhat  to  increase  signal.  While  this  compromise 
increases  the  total  signal  and  reduces  Poisson  noise,  it  also  reduces 
the  resolution  in  the  z-dimension  significantly,  and  so  postpro¬ 
cessing  may  provide  a  significant  improvement  in  z-resolution  as 
well  as  signal-to-noise  ratio  (see  Chapter  22,  this  volume).  Like¬ 
wise,  3D  datasets  from  disk-scanning  confocal  microscopes,  which 
have  inherently  less  resolution  along  the  imaging  axis  than  CLSM, 
can  typically  benefit  from  deconvolution.  Here  again,  however, 
whether  such  gains  are  worth  the  cost  in  computation  time  and 
effort  can  only  be  determined  empirically. 

STRIVING  FOR  SPEED:  STRATEGIES  FOR 
REDUCING  SPECIMEN  EXPOSURE 

Simple  Solutions:  Reducing  Image  Dimensions, 
Increasing  Slice  Spacing,  and  Scan  Speed 

Two  concerns  motivate  the  need  for  increasing  the  acquisition 
speed  when  imaging  embryos.  First,  rapid  cellular  events,  such  as 
cortical  granule  release,  transport  of  cellular  components  along 
microtubules,  and  responses  to  wounding,  occur  so  fast  that  there 
simply  is  not  time  to  acquire  finely  spaced  z-stacks  of  full-field 
images  at  maximal  resolution.  Second,  and  perhaps  more  impor¬ 
tantly,  as  exposure  to  the  scanning  beam  is  proportional  to  the 
number  of  images  acquired,  acquiring  more  images/second  means 
that  the  specimen  is  damaged  more  rapidly.  As  a  result,  the  speed 
with  which  images  can  and  should  be  collected  must  be  tailored 
to  each  new  situation  empirically. 

From  the  technical  point  of  view,  the  speed  at  which  one  can 
acquire  images  is  limited  by  the  rate  at  which  photons  are  emitted 
by  the  specimen:  if  more  frames  must  be  imaged,  more  photons 
are  needed.  The  rate  at  which  photons  are  produced  is  proportional 
to  the  concentration  of  fluorophore  in  the  focal  volume  and  the 
intensity  of  the  light  striking  this  volume.  Although  each  of  these 
parameters  can  be  increased,  neither  can  be  increased  arbitrarily. 
More  laser  power  will  produce  additional  signal  (and  additional 
photodamage!)  only  until  one  approaches  the  power  level  that  pro¬ 
duces  singlet-state  saturation.4  This  saturation  limit  can  only  be 
avoided  by  using  a  disk-  or  line-scanning  confocal  in  which  the 
excitation  light  is  distributed  either  among  an  array  of  many  indi¬ 
vidual  points  or  among  a  similar  number  of  points  arranged  to  form 
a  line  (see  Chapter  10,  this  volume). 

Of  course,  the  signal  must  not  only  be  produced,  it  must  also 
be  detected.  For  this  reason,  high-speed  imaging  is  also  con¬ 
strained  by  the  quantum  efficiency  and  read  noise  of  the  photode¬ 
tector  (a  factor  that  will  be  discussed  below  in  the  context  of  the 
development  of  a  new  photodetector,  the  electron-multiplier  CCD, 
or  EM-CCD). 

Another  instrumental  constraint  is  the  speed  of  the  scanning 
system.  Single-beam  systems  that  rely  on  galvanometers  to  scan 
the  beam  are  limited  to  at  most  8  k/line/s  or  about  16  frames/second 
if  the  raster  size  is  512  x  512,  while  those  employing  acousto- 
optical  scanners  move  about  5x  faster.  On  the  other  hand,  the 


4  In  the  small  spot  produced  by  a  high  NA  oil  lens,  this  occurs  for  most  dyes 
at  about  1  mW.  Because  a  10X  NA  0.5  objective  will  produce  a  spot  — 3x  larger 
in  diameter  and  lOx  larger  in  area,  the  same  dye  would  now  saturate  at 
lOmW.  In  addition  to  issues  regarding  saturation  itself,  there  is  reason  to 
believe  that  the  bleaching  rate/emitted  photon  increases  as  one  operates  closer 
to  saturation  (see  Chapters  38  and  39,  this  volume). 


maximum  scan  rate  of  disk  scanners,  which  have  over  a  thousand 
beams  in  the  field  of  view  at  any  one  time,  is  thousands  of 
frames/second,  and  is  limited  chiefly  by  the  vibration  produced  by 
the  rotating  disk.  Of  course,  to  collect  3D  data  one  must  also  scan 
in  the  z-direction,  and  whether  this  is  accomplished  by  moving  the 
specimen  or  the  objective,  maximum  z-scan  rates  are  often  limited 
by  the  deflection  of  the  specimen  slide  or  coverslip  by  the  forces 
transmitted  through  the  immersion  medium. 

The  final  limit  imposed  by  the  acquisition  hardware  is  the  rate 
at  which  the  signal  from  each  pixel  can  be  read  out  and  digitized. 
On  systems  utilizing  detectors  based  on  advanced  CCDs,  this  is 
currently  about  35  megapixel/s,  but  if  these  devices  do  not  employ 
EM-CCD  techniques,  the  read  noise  becomes  substantial  (~+50 
electrons/pixel).  Although  it  is  possible  to  digitize  the  output  of  a 
photomultiplier  tube  at  even  higher  clock  speeds,  as  noted  above, 
this  is  of  little  assistance  if  the  dye  in  the  voxel  being  interrogated 
is  in  singlet-state  saturation. 

A  final  limitation  to  consider  is  the  size  of  the  embryo  in  the 
x-  and  y-dimensions.  For  some  embryos,  such  as  later-stage  chick 
embryos,  the  region  of  the  embryo  that  must  be  imaged  constrains 
acquisition  speed.  If  the  linear  dimensions  of  the  embryo  in  the  xy- 
plane  are  large  (in  the  case  of  a  chick  embryo,  e.g.,  these  dimen¬ 
sions  can  easily  exceed  1  mm),  then  it  may  be  necessary  to  image 
multiple  xy-sectors.  These  sectors  would  then  have  to  be  joined 
into  a  seamless  montage.  While  semi-automated  methods  (Beck 
et  al.,  2000)  and  fully  automated  (Czirok  et  al.,  2002)  methods 
are  becoming  available  for  such  montage  construction,  the 
initial  acquisition  of  multiple  sectors  is  clearly  an  additional 
possible  constraint  on  speed  of  acquisition  and,  hence,  specimen 
exposure. 

Within  the  constraints  of  the  limitations  listed  above,  a  variety 
of  other  parameters  can  be  reduced  to  optimize  data  acquisition. 
These  include  ( 1 )  the  number  of  focal  planes  acquired  at  each  time 
point;  (2)  the  number  of  lines  imaged;  (3)  the  time  required  to  build 
up  an  acceptable  image  of  the  specified  xy-area  in  each  focal  plane; 
and  (4)  the  interval  between  the  acquisition  of  each  z-series.  The 
time  required  for  an  “acceptable”  image  will  depend  on  the  type 
of  apparatus.  In  the  case  of  laser-scanning  confocal  microscopes, 
the  time  required  per  focal  plane  will  depend  on  the  scanning  speed 
and  the  number  of  scans  integrated  to  build  up  the  final  signal. 
Here  again,  the  intensity  and  contrast  of  the  signal  in  the  specimen 
will  dictate  the  number  of  focal  planes  and  the  sampling  interval 
and  the  investigator  may  have  to  change  several  parameters  to 
achieve  a  satisfactory  result.  For  example,  when  imaging  C. 
elegans  embryos  in  typical  4D  experiments  in  our  laboratory  using 
a  Bio-Rad  1024  CLSM  or  a  Bio-Rad  1024  scan  head  for  de- 
scanned  multi-photon  imaging,  we  often  use  a  1024  x  1024  xy 
pixel  box  size  using  the  “slow”  scan  setting,  with  30  focal  planes 
0.5  |im  apart  collected  over  a  period  of  ~180s.  This  is  possible 
because  stacks  need  only  be  acquired  at  3  to  5  min  intervals.  In 
contrast,  to  image  actin  coat  assembly  around  exocytosed  cortical 
granules  in  Xenopus  oocytes,  15  focal  planes  must  be  acquired, 
and  sampling  intervals  need  to  be  5  to  10  s.  This  necessitates  a 
much  smaller  box  size  (256  x  256  or  128  x  128  pixels)  and  higher 
scan  speeds  (Bement  et  al.,  2003). 

When  speed  is  at  a  premium,  single  fast  scans  can  often  yield 
acceptable  results,  especially  if  a  Gaussian  filter  is  applied  to  the 
resulting  2D  images.  Acquiring  single  fast  scans  and  deconvolv¬ 
ing  the  resulting  3D  images  will  also  enhance  the  signal-to-noise 
ratio  substantially  (Jeff  Hardin,  unpublished  observations).  In 
the  case  of  widefield  deconvolution  or  disk-scanning  confocal 
microscopy,  the  time  needed  to  acquire  an  image  typically  repre¬ 
sents  the  time  required  to  integrate  sufficient  signal  on  the  CCD 
camera. 
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A  final  constraint  on  the  speed  of  acquisition  of  a  z-stack 
involves  the  eventual  display  of  the  collected  data  in  3D.  The 
process  of  calculating  and  displaying  renderings  of  3D  micro¬ 
scopical  datasets  is  complicated  by  the  fact  that  the  resolution  in 
the  z-direction  is  always  substantially  less  than  that  in  the  x-  and 
y-directions.  The  interplane  spacing  is  usually  3x  to  4x  larger  than 
the  xy-pixel  dimension,  and  this  causes  a  problem  for  3D  voxel¬ 
rendering  algorithms  that  assume  that  voxel  dimensions  are  the 
same  in  all  three  dimensions. 

There  are  three  solutions  to  this  problem:  duplication  of  slices, 
interpolation  of  slices,  and  collection  of  more  data.  The  simplest 
solution  involves  duplicating  the  raw  data  three  or  four  times  to  fill 
in  the  missing  planes.  This  process  is  fast  and  restores  the  proper 
proportions;  however,  it  makes  the  resulting  images  look  like  a 
stack  of  coins  and,  if  the  image  stack  is  retained  in  memory,  requires 
significantly  more  RAM.  The  second  solution  is  to  derive  the 
missing  planes  from  the  measured  planes  by  2D  or  3D  interpola¬ 
tion.  This  process  makes  the  rendering  less  blocky,  but  suffers  from 
similar  RAM  constraints,  and  also  requires  significant  computer 
processing  time.  Finally,  one  can  simply  collect  data  from  more 
planes  in  the  first  place,  perhaps  collecting  single-scan  images  at 
four  focus  positions  where,  to  satisfy  Nyquist  sampling,  one  would 
be  justified  in  collecting  the  Kalman  average  of  four  scans  over  a 
single  plane.  Because  it  avoids  additional  processing  steps,  this  last 
solution  is  the  one  we  most  often  employ,  even  thought  it  entails 
increased  exposure  of  the  embryo  to  the  excitation  beam. 

Disk-Scanning  Confocal  Microscopy  Allows 
High-Speed  Acquisition 

As  mentioned  above,  multi-point  confocal  microscopes  that 
employ  Nipkow  disk-scanning  technology  are  another  option  for 
increasing  data  acquisition  speed.  Because  the  Yokagawa  scan 
head  combines  laser  excitation  with  an  array  of  microlenses  to 
force  more  light  through  each  pinhole  in  the  disk,  a  high  rate  of 
signal  production  is  possible.  Because  the  intensity  in  each  spot  is 
far  from  singlet  saturation  and  because  the  effective  quantum  effi¬ 
ciency  of  the  CCD  is  higher  than  that  of  the  photomultiplier  tube, 
disk  scanners  seem  to  produce  little  photobleaching  while  record¬ 
ing  sufficient  signal  to  perform  high-quality  4D  experiments. 

The  literature  is  replete  with  examples  of  the  use  of  Yokagawa- 
based  systems  for  imaging  embryos  of  C.  elegans  (Oegema  et  al., 
2001;  Cockell  et  al.,  2004)  and  Drosophila  (Bloor  and  Kiehart, 
2002;  Grevengoed  et  al.,  2001)  in  particular.  Such  microscopes 
also  offer  the  advantage  that  the  confocal  image  can  be  viewed 
directly,  by  eye,  in  real  time  (see  Chapter  10,  this  volume,  for  an 
extensive  discussion  of  Yokagawa-based  systems  and  other  related 
technologies). 

For  the  developmental  biologist,  such  systems  are  an  inex¬ 
pensive  alternative  to  CLSM,  and  provide  many  of  the  benefits  of 
more  elaborate  technologies,  such  as  multi-photon  microscopy. 
Because  disk-scanning  systems  use  an  off-the-shelf  focus  motor, 
CCD,  filter  wheel,  and  shutter  components,  commercial  imaging 
packages  can  be  used  to  drive  data  acquisition.  For  some  labora¬ 
tories,  this  may  be  an  advantage  compared  to  the  proprietary  soft¬ 
ware  that  ships  with  some  of  these  units,  especially  if  they  are 
already  using  such  a  software  package  for  other  imaging  tasks. 
Although  on  thick  specimens,  disk-scanning  systems  are  signifi¬ 
cantly  inferior  to  CLSM  in  their  optical  sectioning  capabilities, 
post-acquisition  deconvolution  can  be  an  extremely  useful  adjunct 
for  improving  such  4D  datasets.  In  our  laboratory,  disk-scanning 
technology  has  largely  replaced  both  CLSM  and  multi-photon 
microscopy  for  routine  4D  data  acquisition  [Fig.  43.5(A-C)]. 


As  an  alternative  to  laser-based  disk-scanning  systems,  other 
systems,  such  as  the  Atto  CARV,  use  an  arc  lamp  illumination 
source.  When  such  systems  are  paired  with  sensitive  EM-CCD 
cameras,  they  can  often  detect  sufficient  signal  for  studies  on  living 
embryos  because,  in  this  case,  the  fact  that  they  send  less  excita¬ 
tion  light  to  the  specimen  than  the  laser/micro-lens  systems  can  be 
seen  as  an  advantage  [Fig.  43.5(D-F)]. 

Additional  Hardware  Improvements  Can 
Increase  Acquisition  Speed 

Additional  improvements  in  the  imaging  system  may  also  improve 
acquisition  speed.  An  often-overlooked  area  in  which  significant 
gains  have  been  made  in  recent  years  is  in  optical  coatings  and 
objective  lens  technology.  Recent  lenses  from  the  major  manufac¬ 
turers  have  greatly  improved  transmission,  and  this  has  resulted  in 
corresponding  improvements  in  the  ability  to  imaging  very  thin, 
small,  dim  structures  in  the  light  microscope. 

A  second  potential  improvement  involves  the  use  of  piezo¬ 
electric  focus  controls,  such  as  those  on  the  PerkinElmer  Ultra- 
View  RS,  which  can  move  the  objective  lens  rapidly  and  with  little 
hysteresis  in  a  step-wise  fashion.  Similar  focus  controls  have  been 
employed  to  move  the  objective  continuously  in  the  z-direction 
during  a  scan.  Although  this  produces  optical  sections  that  are 
slightly  tilted  relative  to  the  focal  plane,  one  avoids  the  vibration 
and  specimen  movement  that  occurs  when  an  immersion  objective 
moves  rapidly  in  a  stepped  manner  (Hammond  and  Glick,  2000). 

Finally,  significant  advances  in  camera  technology  allow  for 
unprecedented  speed  of  data  collection.  Back-thinned  CCD 
cameras,  such  as  those  produced  by  Roper  Scientific  and  Hama¬ 
matsu  Corporation,  and  EM-CCD  cameras,  such  as  those  produced 
by  Andor  Corporation  (Chong  et  al.,  2004),  are  having  a  profound 
influence  on  low-light  microscopy.  These  cameras  provide 
extremely  fast  readout  with  very  low  read  noise  and  effective 
quantum  efficiency,  2x  to  3x  higher  than  the  best  PMTs.  EM-CCD 
technology  in  particular  yields  dramatic  improvements  in  signal- 
to-noise  ratio  under  ultra  low-light  conditions  and  high-speed 
operation.  These  cameras  make  it  possible  to  capture  low-read- 
noise  images  much  more  quickly  for  use  in  either  deconvolution 
or  disk-scanning  confocal  microscopy. 

In  the  latter  case,  however,  because  the  current  generation  EM- 
CCD  cameras  rely  on  frame  transfer  rather  than  interline  transfer, 
the  integrated  charge  pattern  sweeps  across  the  image  field  as  it  is 
being  read  out  (i.e.,  the  “electronic  shutter”  does  not  work  so  well, 
especially  for  short  exposures).  This  creates  a  new  source  of  arti¬ 
fact:  if  the  position  of  the  scanning  disk  is  not  synchronized  with 
the  CCD  camera  exposure  and  the  vertical  transfer  axis  of  the  CCD 
is  not  oriented  properly  with  respect  to  the  motion  of  the  scanning 
disk.  Moire-type  banding  patterns  are  superimposed  onto  the 
image  [Fig.  43.5(G-I)],  particularly  when  one  uses  short  exposure 
times.  Although  the  time  scale  over  which  events  in  embryos  occur 
is  relatively  long  compared  to  the  CCD  exposure,  this  interference 
can  nevertheless  be  a  significant  problem  when  attempting  to 
minimize  CCD  exposure  time  during  4D  acquisition. 

Such  “banding”  can  be  greatly  reduced  or  eliminated  by  care¬ 
fully  matching  CCD  frame  transfer  rates  to  the  speed  of  Nipkow 
disk  rotation.  The  newer  CSU22  Yokagawa  units  permit  variable 
speed  disk  rotation,  which  makes  such  matching  easier  (Chong  et 
al.,  2004).  With  some  effort  one  can  also  match  the  fixed  scan 
speed  of  the  older  CSU10  scan  head,  with  the  exposure  times  of 
at  least  some  cameras,  such  as  the  iXon  (Andor  Corp.,  Belfast, 
UK).  As  this  moire  problem  is  unrelated  to  the  operation  of  the 
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FIGURE  43.5.  Disk-scanning  microscopes  can  acquire  4D  data  quickly.  (A-C)  Frames  from  a  4D  movie  of  ventral  enclosure  in  a  C.  elegans  embryo  express¬ 
ing  a  dlg-lp:  gfp  reporter  acquired  using  a  PerkinElmer  Ultra  View  LCI  system  equipped  with  a  Hamamatsu  Orca  II  ER  CCD  camera.  Elapsed  time  in  minutes 
is  shown,  z-stacks  were  acquired  at  50  s  intervals,  20  focal  planes/stack;  acquisition  time/image,  300  ms.  Fine  details  of  protrusions  are  visible  against  a  dark 
background  using  this  particular  transcriptional  reporter.  (D-F)  Frames  from  a  4D  movie  of  dorsal  intercalation  in  a  C.  elegans  embryo  expressing  AJM- 1 :  GFP, 
which  localizes  to  epithelial  junctions,  acquired  using  an  ATTO  Carv  system  and  a  Roper  Cascade  B  camera,  z-stacks  were  acquired  at  3  min  intervals,  20  focal 
planes/stack;  acquisition  time/image,  700  ms.  (G,  H)  A  30-focal-plane  z-stack  of  images  acquired  using  the  system  in  (A),  but  imaged  using  an  Andor  iXon  EM- 
CCD  camera.  Acquisition  time/image  15  ms.  Despite  >30x  shorter  exposure  time,  junctional  details  are  clearly  visible.  During  acquisition,  stop  frame  motion  of 
maturing  larvae  was  achieved  for  individual  focal  planes,  despite  their  rapid  muscle  contractions  (J.  Hardin,  data  not  shown).  In  (G),  all  50  frames  of  the  stack 
were  projected  using  a  maximum  intensity  procedure.  Moire  banding  is  clearly  visible  (arrow).  In  (H),  the  top  30  frames  were  projected.  No  banding  is  visible, 
because  the  moire  pattern  only  became  pronounced  as  the  bottom  half  of  the  embryo  was  imaged.  Bars  =  lOjim. 


electron  multiplier,  it  will  cease  to  exist  when  interline-transfer 
CCDs  with  EM  readouts  are  introduced. 

LOCALIZING  LABEL:  STRATEGIES  FOR 
INCREASING  EFFECTIVE  CONTRAST  IN 
THICK  SPECIMENS 

Methods  for  introducing  fluorescent  probes  into  living  embryos 
vary  from  one  model  system  to  another.  With  the  exception  of  flu¬ 
orescent  lipid  dyes,  fluorescent  probes,  including  dyes  and  mRNA 
encoding  various  fluorescent  proteins,  are  usually  introduced  into 
echinoderm,  Xenopus,  and  zebrafish  oocytes  and  embryos  by 
microinjection.  In  other  cases,  fluorescent  protein  probes  are  intro¬ 
duced  as  transgenes,  which  are  then  expressed  by  the  embryo  as 
fluorescently  tagged  proteins.  No  matter  what  the  method  of  intro¬ 
duction,  there  are  several  issues  that  must  be  addressed  by  devel¬ 
opmental  biologists  as  they  use  such  probes.  First,  any  probe  can 


be  susceptible  to  dominant-negative  effects  when  used  at  high  con¬ 
centrations;  fluorescent  protein-tagged  mRNAs  or  transgenes  are 
no  exception.  Thus,  developmental  biologists  must  often  balance 
the  need  for  higher  fluorophore  concentration  with  such  dominant¬ 
negative  effects.  Second,  such  effects  appear  to  be  system  specific. 
Probes  that  work  well  in  one  system  may  be  unacceptably  toxic  in 
another.  As  one  of  many  examples,  the  lipid  dye  FM1-43,  which 
has  been  used  in  several  studies  to  label  intracellular  vesicles  in 
living  embryos  (see  below),  causes  toxicity  to  Xenopus  oocytes, 
which  are  ordinarily  fairly  robust  (Bement  et  ai,  2003).  Thus,  as 
with  most  imaging  experiments  in  living  embryos,  trial  and  error 
will  be  unavoidable.  Third,  in  multi-channel  experiments,  the 
amounts  of  each  labeled  protein  may  need  to  be  adjusted  to  avoid 
additive  toxicity  or  artifactual  effects.  This  issue  is  particularly 
important  when  the  two  proteins  physically  interact.  Over¬ 
expression  of  such  interactors  can  lead  to  large,  artifactual 
aggregates  within  the  cells  expressing  them  (J.  Hardin  and  C. 
Lockwood,  unpublished  observations). 
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Addition  of  Labeled  Proteins  to  Embryos 

Direct  labeling  of  proteins  or  other  probes  is  a  tried  and  true 
method  for  introducing  fluorescently  labeled  proteins  into  living 
embryos.  This  approach  has  worked  particularly  well  for  studying 
very  early  embryos  or  oocytes  in  amphibians  (Bement  et  al.,  2003) 
and  echinoderms  (Jaffe  and  Terasaki,  2004;  Strickland  et  al., 
2004).  Although  falling  out  of  favor  in  deference  to  mRNA  expres¬ 
sion  for  production  of  transgenic  embryos,  direct  labeling  has  the 
advantage  that,  unlike  injection  of  mRNA  or  expression  of  fluo¬ 
rescently  tagged  transgenic  constructs,  there  is  no  lag  before 
expression  is  evident,  and  there  are  no  complications  involving 
efficient  maternal  expression  of  constructs.  This  virtue  can  also  be 
a  liability  for  studying  events  late  in  development:  turnover  of  the 
labeled  proteins  may  attenuate  the  concentration  of  labeled  protein 
so  severely  by  the  stage  of  interest  that  it  can  be  difficult  to  detect. 
However,  this  approach  has  worked  well  in  some  cases,  including 
the  study  of  embryonic  neurons  in  amphibian  embryos  (Gomez 
and  Spitzer,  1999).  On  the  other  hand,  production  and  purification 
of  fluorescently  labeled  proteins  is  typically  much  more  difficult 
than  injecting  mRNA  or  making  transgenics  (at  least  in  some 
organisms,  such  as  C.  elegans  or  Drosophila).  Moreover,  directly 
conjugated  fluorescent  proteins  must  typically  be  centrifuged  at 
high  speed  to  remove  unconjugated  label  (e.g.,  100,000 g).  Several 
companies  make  prelabeled  proteins  for  study  of  the  cytoskeleton 
that  ameliorate  these  difficulties,  including  fluorescently  labeled 
phalloidin  and  taxol  for  labeling  F-actin,  as  well  as  cytoskeletal 
monomers  labeled  with  fluorophores  (Bement  et  al.,  2003). 

Expressing  Green  Fluorescent  Protein  and  mRFP 
Constructs  in  Embryos  Allows  Dynamic  Analysis 
of  Embryos  at  Multiple  Wavelengths 

The  discovery  and  widespread  use  of  GFP  as  a  tag  for  visualizing 
gene  expression  and  protein  localization  within  living  organisms 
has  revolutionized  live  embryo  imaging  (Chalfie  et  al.,  1994).  GFP 
(especially  the  widely  used  multiply-mutated  version  known  as 
EGFP)  is  bright,  generally  non-toxic,  and  resistant  to  photo- 
bleaching.  The  advent  of  numerous  color  shifted  spectral  variants 
of  GFP  (Miyawaki  et  al.,  2003),  including  enhanced  cyan  and 
yellow  fluorescent  proteins  (ECFP  and  EYFP,  respectively)  has 
allowed  multiple  fluorescent  proteins  to  be  detected  simultane¬ 
ously  in  living  embryos  (Hadjantonakis  et  al.,  2003),  and  has  made 
possible  techniques  such  as  in  vivo  FRET  (see  below).  Recent 
identification  of  a  red  fluorescent  protein  from  coral  (DsRed) 
allows  the  fluorescent-protein  approach  to  be  extended  deep  into 
the  red  portion  of  the  visible  spectrum.  Although  earlier  versions 
of  DsRed  were  multimeric  and  hence  slow  to  assemble  into  a  func¬ 
tional  fluorophore,  multiple  mutations  have  now  resulted  in  a 
monomeric  form  of  the  protein,  mRFPl  (Campbell  et  al.,  2002), 
that  folds  much  more  rapidly  and  hence  serves  as  a  suitable  FP  for 
dynamic  events  in  early  embryos.  The  recent  advent  of  color- 
shifted  variants  of  DsRed  (Shaner  et  al.,  2004)  will  make  this 
family  of  fluorophores  even  more  useful.  We  (see  Fig.  43.7)  and 
others  (Benink  and  Bement,  2005)  have  begun  using  mRFPl  along 
with  EGFP  to  perform  dual-channel  imaging  of  fluorescence,  indi¬ 
cating  that  fluorescent  proteins  will  continue  to  be  the  method  of 
choice  in  genetic  systems  in  which  fluorescent-protein-tagged 
transgenes  can  be  introduced.  Ways  of  introducing  fluorescent- 
protein-tagged  constructs  into  embryos  involve  system-specific 
techniques  beyond  the  scope  of  this  chapter;  here  we  make  several 
generalizations  about  the  uses  of  various  strategies  for  imaging 
embryos  expressing  fluorescent-protein-tagged  constructs. 


Transciptional  Reporters  Allow  Analysis  of 
Dynamic  Processes  in  Embryos 

To  developmental  biologists,  fusing  the  coding  region  of  EGFP  to 
the  regulatory  DNA  associated  with  a  gene  of  interest  (i.e.,  GFP 
reporter  constructs)  is  often  used  to  assess  the  tissue-specific  and 
temporal  patterns  of  the  transcriptional  activation  of  a  gene.  Such 
data  provides  valuable  information  about  how  the  expression  of  a 
gene  is  regulated.  However,  such  transcriptional  reporters  can  also 
be  invaluable  for  live  embryo  imaging  for  several  reasons.  First, 
such  reporter  constructs  result  in  the  expression  of  GFP  in 
the  cytosol;  because  GFP  is  fairly  small,  these  reporters  are  cap¬ 
able  of  percolating  into  small  volumes  within  the  cytoplasm, 
including  the  fine  protrusions  extended  by  cells  as  they  migrate 
(Fig.  43.6).  Second,  the  highly  specific  expression  pattern  of  some 
genes  allows  either  many  or  a  very  small  number  of  cells  to  be 
visualized  against  a  dark  background,  dramatically  improving  the 
effective  contrast  of  the  specimen  being  imaged.  In  some  cases, 
such  effective  contrast  enhancement  can  be  aided  by  dual  labeling 
with  more  general  fluorescent  counterstains  (see  below).  Third, 
imaging  cytosolic  GFP  reporters  typically  does  not  cause  as  much 
photodamage  as  with  GFP  translational  fusions.  Thus,  in  some 
cases  where  a  membrane-localized  GFP  or  similar  translational 
fusion  construct  might  be  desirable,  a  cytosolic  GFP  may  be  suf¬ 
ficient  for  tracking  cell  trajectories  or  monitoring  protrusive 
activity. 

Translational  Fusions  Allow  Analysis  of 
the  Subcellular  Distribution  of 
Specific  Proteins 

Using  molecular  techniques  to  fuse  fluorescent  proteins  to  a 
peptide  or  protein  sequence  has  transformed  cell  and  develop¬ 
mental  biology  by  allowing  the  in  vivo  analysis  of  intracellular 
protein  dynamics  (Lippincott-Schwartz  and  Patterson,  2003).  In 
addition,  however,  fluorescent-protein-labeled  proteins  can  serve 
the  developmental  biologist  by  allowing  the  analysis  of  cell  and 
tissue  movement  in  4D.  For  live  embryo  work,  several  such  trans¬ 
lational  fusions  are  generally  useful.  Fluorescent-protein-tagged 
nuclear  proteins,  such  as  histone  H2B  (Megason  and  Fraser,  2003) 
can  be  an  effective  means  by  which  to  track  cell  positions,  in  addi¬ 
tion  to  their  obvious  use  in  studying  nuclear  dynamics  in  embryos. 
As  an  indirect  readout  of  cell  position  without  the  complexities  of 
fluorescent  cell  borders,  tracking  of  nuclei  can  be  a  useful  tool  for 
charting  overall  cell  movements.  Cell  membrane  localized  fusions, 
such  as  fusions  to  the  GAP-43  membrane-localization  signal  or  the 
Lck  membrane-localization  signal  to  drive  membrane  localization 
(Megason  and  Fraser,  2003)  can  be  used  to  outline  cell  membranes. 
For  studying  morphogenetic  movements  in  embryos,  junction- 
localized  fluorescent  proteins  are  extremely  useful.  Our  laboratory 
has  used  junction-localized  fluorescent  proteins  to  study  epithelial 
sheet  movement  in  C.  elegans  (Mohler  et  al.,  1998;  Koppen  et  al., 
2001).  Others  have  used  moesin ::  GFP  to  analyze  Drosophila  mor¬ 
phogenesis  in  a  similar  manner  (Edwards  et  al.,  1997).  Three 
examples  of  dual  labeling  with  EGFP  and  mRFPl  fusions  are 
shown  in  Figure  43.7. 

Translational  fusions  can,  of  course,  be  coupled  to  highly 
tissue-specific  promoters  to  yield  probes  that  are  both  subcellularly 
localized  and  expressed  in  a  restricted  group  of  cells.  As  one 
example  among  many,  this  approach  has  been  used  to  visualize 
actin-based  protrusions  in  leading  edge  cells  during  dorsal  closure 
in  Drosophila  by  driving  expression  of  constructs  using  the 
engrailed  promoter,  which  yields  well-defined  stripes  of  expres¬ 
sion  (Jacinto  et  al.,  2000). 
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FIGURE  43.6.  Cytosolic  markers  allow  imaging  of  cell  motility  and  cell  movement  in  embryos.  (A,  B)  Frames  from  4D  movies  of  dorsal  intercalation  in  C. 
elegans  embryos  expressing  Ibp-lp ::  gfp,  which  is  expressed  in  a  subset  of  dorsal  epidermal  cells.  (A)  An  embryo  imaged  using  MPLSM  (Heid  et  al.,  2001) 
using  a  Ti:Sa  laser  and  descanning  through  a  Bio-Rad  1024  scanhead.  z-stacks  were  subsequently  3D  projected  using  a  maximum  brightness  procedure.  (B)  A 
similar  embryo  imaged  using  the  disk-scanning  apparatus  described  in  Figure  43.5(A).  The  dataset  in  (B)  was  subsequently  subjected  to  surface  rendering  using 
Volocity  software.  Fine  protrusions  are  visible  in  both  cases.  In  (B),  it  is  clear  that  the  protrusions  are  wedge-shaped  in  the  z-dimension,  and  that  the  non-dorsal 
cell  {asterisk)  also  produces  fine  protrusions.  Bar  =  10  (im.  (C,  D)  Protrusive  activity  of  zebrafish  mesodermal  cells  imaged  in  wild-type  (C)  and  Wnt5a  mor- 
pholino  injected  embryos  (D)  expressing  a  combination  of  cell  surface  (GAP43 ::  GFP)  and  cytosolic  GFP.  Images  were  acquired  using  a  Bio-Rad  Radiance  2000 
MPLSM  equipped  with  a  Ti:  Sa  laser;  three  successive  stacks  of  120  focal  planes  each  were  acquired,  one  of  which  is  shown  here.  The  resulting  images  were 
surface  rendered  using  Volocity  software.  These  cells  display  oriented  protrusive  activity.  The  cell  from  the  morpholino  injected  embryo  displays  impaired  motil¬ 
ity.  Bar  =  10  (im.  (E,  F)  Quantum  dot  (QD)  labeling  of  cells  in  an  early  Xenopus  embryo.  QD  micelles  were  injected  into  an  individual  blastomere  during  very 
early  cleavage  stages.  Between  1.5  and  3nL  of  a  2.3  jxM  suspension  of  QDs  were  injected,  corresponding  to  2.1  to  4.2  x  109  injected  particles  per  cell.  (E)  Injec¬ 
tion  of  one  cell  in  an  eight-cell- stage  embryo  resulted  in  labeling  of  individual  blastomeres.  (F)  The  daughter  cells  of  the  injected  blastomere  are  labeled  at  the 
neurula  stage.  Embryos  were  imaged  with  Chroma  filter  set  41015  (wild-type  GFP  longpass  emission  with  a  50  nm  wide  bandpass  excitation  centered  at 
450 nm)  mounted  on  a  Zeiss  fluorescence  microscope.  Bar  =  500  |im.  [(A)  is  from  Heid  and  colleagues  (2001);  (B)  is  courtesy  R.  King  and  T.  Walston,  Uni¬ 
versity  of  Wisconsin;  (C,  D)  are  from  Kilian  and  colleagues  (2003);  (E,  F)  are  from  Dubertret  and  colleagues  (2002),  used  by  permission.] 


Using  Selective  Labeling  to  Reduce  the  Number 
of  Labeled  Structures 

Dextran  Labeling 

Injecting  fluorescent  dextrans  into  the  blastomeres  of  early 
embryos  has  a  long  history  in  developmental  biology  (e.g., 
Gimlich,  1991).  Focal  injections  allow  the  progeny  of  injected 
cells  to  be  visualized  against  an  unlabeled  background  during  later 
development,  a  process  that  permits  one  to  develop  fate  maps.  Live 
4D  imaging  using  confocal  or  multi-photon  microscopes  allows 


the  progeny  of  such  injected  cells  to  be  followed  with  remarkable 
clarity.  For  example,  the  cells  shown  in  Figure  43.10(D-F)  are  the 
progeny  of  blastomeres  of  a  zebrafish  embryo  injected  early  in 
development,  and  have  been  imaged  at  high  resolution  and  fol¬ 
lowed  in  4D  space. 

Quantum  Dots 

Fluorescent  semiconductor  nanocrystals,  or  quantum  dots  (QDs), 
have  the  potential  to  be  useful  for  long-term  observations  via  con¬ 
focal  and  (especially)  multi-photon  microscopy  as  an  alternative 
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FIGURE  43.7.  Simultaneous  imaging  of  GFP-  and  mRFP-tagged  proteins  in  living  oocytes  and  embryos.  Three  different  examples  of  using  dual-channel  imaging 
to  detect  GFP-  and  mRFP  translational  fusions.  (A)  Frames  from  a  movie  showing  that  RhoA  (detected  using  eGFP-rGBD;  green)  and  Cdc42  (detected  using  a 
fusion  between  mRFP  and  a  protein  fragment  that  binds  active  Cdc42;  red)  segregate  into  discrete  zones  during  wound  healing  in  a  Xenopus  oocyte  (time  in 
seconds).  Imaging  was  performed  as  in  Figure  43.1(A).  Bar  =  10pm.  (B,  C)  Frames  from  a  movie  of  elongation  of  a  C.  elegans  embryo  expressing  DLG-1 :: 
mRFP  and  HMP-l/a-catenin ::  GFP  (left)  DLG-1  ::mRFP;  (middle)  HMP-1  ::GFP  (right,  overlay).  Loss  of  junctional  material  occurs  at  sites  where  epidermal 
cells  are  fusing  (arrow).  (B,  C)  are  75 min  apart.  Imaging  was  performed  as  in  Figure  43.6(A).  Bar  =  10pm.  (D)  Dorsal-lateral  view  of  the  tail  of  24h  living 
zebrafish  embryo  injected  with  RNA  for  Histone  H2B-EGFP  and  membrane-localized  mRFPl.  (Left)  Both  channels  together.  (Center)  Histone  H2B-EGFP 
channel.  (Right)  Membrane  mRFPl  channel.  The  nuclei  can  be  discerned  even  in  areas  where  they  are  closely  packed  such  as  the  spinal  cord  (running  along 
middle  of  the  tail).  Bar  =  100pm.  [(A)  is  from  Benink  and  Bement  (2005);  (B,  C)  are  courtesy  C.  Lockwood,  University  of  Wisconsin;  (D)  is  from  Megason 
and  Fraser  (2003),  used  by  permission.] 
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FIGURE  43.8.  Using  fluorescence  recovery  after  photobleaching  (FRAP)  and  photoactivatiable  GFP  in  embryos.  (A-C)  FRAP  of  GFP-tagged  microtubules  in 
a  living  C.  elegans  zygote.  Time-lapse  images  of  embryo  expressing  p-tubulin ::  GFP  (A)  in  which  a  short  region  of  the  posterior  spindle  microtubules  was 
photobleached  during  anaphase  onset  (B,  bracket).  The  kymograph  below  (C)  follows  the  movement  of  the  photobleached  region  (indicated  by  gray  arrow¬ 
heads).  Frames  were  acquired  at  7  s  intervals  using  a  Zeiss  LSM  510  LCSM.  For  photobleaching,  a  selected  region  of  interest  was  photobleached  using  50  to 
150  Is  scans  of  100%  488  nm  laser  power.  Bar  =  10pm.  (D)  Photoactivation  of  PA-GFP  at  the  surface  of  a  Xenopus  oocyte  using  a  LCSM.  A  Xenopus  oocyte 
injected  with  rnRNA  encoding  mRFP-  (red)  and  PA-GFP  tagged  forms  of  a  synthetic  F-actin  binding  peptide  was  scanned  repeatedly  with  the  488  nm  line  from 
a  Bio-Rad  1024  LCSM  to  photoactivate  PA-GFP.  Bar  =  10pm.  [(A-C)  are  from  Labbe  and  colleagues  (2004);  (D)  is  courtesy  of  B.  Burkel  and  W.  Bement, 
University  of  Wisconsin.] 


to  fluorescent  dextrans  for  lineage  tracing.  Compared  with  organic 
dyes  and  fluorescent  proteins,  QDs  offer  several  advantages  (Gao 
et  al.,  2005;  Michalet  et  al.,  2005).  First,  the  manufacturing 
process  yields  QDs  with  highly  specific  composition-  and  size- 
dependent  emission  characteristics  that  are  tunable  throughout 
essentially  the  entire  visible  and  NIR  spectrum,  all  excitable  by  a 
single  wavelength.  Second,  QDs  have  a  very  high  absorption 
cross-section.  As  a  result,  a  very  small  number  of  QDs  are  neces¬ 
sary  to  achieve  a  detectable  signal;  in  some  cases  it  has  been 
possible  to  detect  the  fluorescence  of  single  or  small  numbers  of 
QDs.  Third,  QDs  have  a  large  Stokes  shift,  which  may  be  an 
advantage  in  situations  where  autofluorescence  of  the  tissue  would 
mask  the  emission  signal  of  fluorescent  dyes.  Finally,  the  high 
electron  density  of  QDs  allows  them  to  be  viewed  via  TEM, 
making  them  useful  for  correlative  studies  (see  Chapter  49,  this 
volume). 

Although  QDs  have  been  used  without  derivatization,  typically 
they  are  encapsulated  in  a  polymer  shell.  Dubertret  and  colleagues 
encapsulated  QDs  within  PEG-derivatized  phospholipid  micelles, 
and  injected  them  into  early  blastomeres  in  Xenopus  embryos 
(Dubertret  et  al.,  2002).  They  found  that  the  QDs  were  extremely 
stable,  and  when  injected  at  sufficiently  low  concentrations  caused 
no  toxicity  [Fig.  43.6(E,F)].  Although  this  technology  is  still 
emerging,  QDs  have  tremendous  promise  in  the  future  for  use  in 
living  embryos  under  prolonged  observation. 


Photobleaching 

GFP  can  be  used  very  successfully  in  studies  involving  fluores¬ 
cence  recovery  after  photobleaching  (FRAP)  and  related  tech¬ 
nologies  (Lippincott-Schwartz  et  al.,  2003),  which  is  easily  done 
via  repeated  scanning  of  a  selected  area  in  the  LSCM.  Although 
in  some  cases,  it  is  likely  that  photobleaching  will  result  in  unac¬ 
ceptable  photodamage  (see  Chapter  49,  this  volume),  it  has  been 
used  successfully  in  several  experiments  with  live  embryos  (e.g., 
Benink  et  al.,  2000;  Labbe  et  al.,  2004)  [Fig.  43.8(A-C)]. 

Photo-Activatable  Dyes  and 
Photo-Activable  GFP 

Another  technique  for  marking  small  groups  of  cells  involves 
the  use  of  photo-activatable,  or  caged,  fluorophores.  Such  fluo- 
rophores  only  fluoresce  after  the  caging  group  is  removed  by 
photocleavage  at  the  appropriate  wavelength,  typically  in  the  violet 
or  near  ultraviolet  (UV).  Caged  fluorescein  dextran  has  been  used 
for  fate  mapping  in  both  Drosophila  (Girdham  and  O’Farrell, 
1994)  and  zebrafish  (Kozlowski  et  al.,  1997).  Although  widefield 
imaging  was  used  to  track  the  resulting  fluorescein  fluorescence, 
confocal  and  multi-photon  imaging  are  also  well-suited  to  this 
approach. 

More  recently,  a  photoactivatable  variant  of  GFP  (PA-GFP) 
was  described  (Patterson  and  Lippincott-Schwartz,  2002).  PA-GFP 
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only  fluoresces  after  photo-activation  via  intense  irradiation  with 
413  nm  laser  light,  or  by  exposure  to  a  standard  mercury  arc.  The 
fluorescence  of  PA-GFP  has  been  reported  to  increase  at  least  60x 
after  activation  in  vivo.  Once  activated,  it  appears  to  be  as  stable 
as  other  GFP  variants  (Patterson  and  Lippincott-Schwartz,  2002). 
Recently,  PA-GFP  was  used  to  follow  local  movements  of,  and 
recruitment  of  actin  in,  wounded  Xenopus  oocytes  [Fig.  43.8(D)]; 
in  this  case,  PA-GFP  was  photoactivated  by  a  brief  pulse  of  laser 
light  from  the  standard  488  nm  line  of  a  Bio-Rad  1024  CLSM,  indi¬ 
cating  that  all  steps,  from  photo-activation  to  imaging,  can  be  per¬ 
formed  using  4D  CLSM  (B.  Burkel  and  W.  Bement,  personal 
communication).  The  use  of  PA-GFP  in  the  live  imaging  of 
embryos  will  undoubtedly  become  more  widespread  in  the  imme¬ 
diate  future. 

An  alternative  to  PA-GFP  is  the  use  of  kindling  proteins  (Chu- 
dakov  et  al.,  2003).  This  approach  relies  on  a  chromoprotein,  asCP, 
isolated  from  the  sea  anemone  Anemonia  sulcata.  asCP  is  not  flu¬ 
orescent  until  it  is  irradiated  ,  or  kindled,  using  intense  green  light; 
kindled  asCP  then  emits  at  595  nm.  A  mutant  form  of  asCP  ( KFP 1 ) 
has  been  produced  that  undergoes  irreversible  kindling  when  irra¬ 
diated  with  intense  green  light  (e.g.,  brief  irradiation  at  30%  power 
using  a  543  nm  laser  on  a  Zeiss  LSM  510  CLSM),  but  can  be 
reversibly  kindled  when  irradiated  at  lower  power  (e.g.,  5%  power 
on  the  same  apparatus).  Reversibly  kindled  KFP1  can  be  quenched 
by  irradiation  with  blue  light  (using  the  458  nm  line  of  the  same 
device  at  1%  power).  Like  original  forms  of  DsRed,  KFP1  cur¬ 
rently  requires  multimerization  for  function.  If  a  monomeric  form 
of  KFP1  can  be  produced,  it  may  have  properties  which  will  be 
more  generally  useful  for  imaging  living  embryos  beyond  current 
applications,  which  only  involve  fate  mapping  (Chudakov  et  al., 
2003). 

Directly  Conjugated  Antibodies  for  Visualizing 
the  Extracellular  Matrix 

Although  extracellular  matrix  (ECM)  components  have  been  suc¬ 
cessfully  visualized  by  making  GFP-tagged  translational  fusions, 
another  approach  for  localized  labeling  of  the  ECM  is  the  injec¬ 
tion  of  antibodies  specific  for  particular  components  directly  con¬ 
jugated  to  fluorophore.  This  approach  has  been  used  successfully 
to  visualize  fibrillin  in  the  chick  embryo  (Czirok  et  al.,  2004),  and 
fibronectin  in  Xenopus  embryos  (L.  Davidson,  R.  Keller,  and  D. 
Desimone,  personal  communication).  Using  this  approach,  labeled 
fibrils  can  be  imaged  for  long  periods  of  time  to  track  their  move¬ 
ment  during  morphogenesis  (Czirok  et  al.,  2004). 

Bulk  Vital  Labeling  Can  Enhance  Contrast 

While  the  use  of  GFR  mRFPl,  and  their  spectral  variants  has  led 
to  huge  advances  in  our  ability  to  image  embryos,  these  approaches 
have  some  limitations.  First,  producing  DNA  constructs  for 
expressing  GFP-tagged  versions  of  proteins  within  embryos 
requires  multiple  cloning  steps,  some  of  which  can  be  arduous, 
especially  for  large  genes.  Second,  when  introduced  as  mRNA  or 
as  transgenes,  these  proteins  are  often  over-expressed,  and  care 
must  be  taken  so  that  over-expression  artifacts  do  not  compromise 
the  final  results.  Third,  it  is  often  difficult  to  target  GFP-  or  mRFP- 
tagged  constructs  to  a  broad  spectrum  of  subcellular  compart¬ 
ments,  although  there  are  several  recent  examples  in  which  GFP 
has  been  targeted  to  membranous  organelles  (Poteryaev  et  al., 
2005).  Finally,  in  organisms  such  as  C.  elegans ,  it  is  difficult  to 
introduce  mRNAs  by  direct  injection.  This  can  make  it  hard  to 
achieve  high  levels  of  expression  in  the  germline,  with  the  result 
that  exogenous  promoters  must  sometimes  be  used  to  generate 


enough  expression  to  visualize  the  protein  of  interest  in  the  early 
embryo  (Strome  et  al.,  2001). 

One  solution  to  these  problems  is  to  use  fluorescent  vital  dyes. 
Such  dyes  have  the  virtue  that  they  can  label  many  cells  in  the 
embryo  simultaneously,  partitioning  preferentially  into  specific 
cellular  structures  based  on  their  chemical  properties.  Several  dyes 
have  proven  useful  for  this  purpose: 

1.  Unconjugated  BODIPY  and  BODIPY-ceramide  dyes: 
“BODIPY”  is  a  trade  name  for  a  class  of  photostable,  neutrally 
charged,  boron-containing  diazaindacene  fluorophores  that  span 
much  of  the  visible  spectrum  (Molecular  Probes,  2004).  Several 
unconjugated  BODIPY  fluorophores,  such  as  BODIPY  505/515, 
have  been  used  as  vital  fluorescent  stains  for  zebrafish  embryos, 
where  they  localize  to  intracellular  yolk  platelets  and  in  intercel¬ 
lular  spaces.  BODIPY-ceramide  has  been  used  to  label  the  plasma 
membrane  and  endomembranes  [Fig.  43.9(A-C)].  Embryos  can  be 
bathed  in  these  dyes,  allowing  most  cells  in  the  embryo  to  be 
stained  rapidly  (Cooper  et  al.,  1999). 

2.  FM4-64,  FM1-43,  and  other  lipophilic  membrane  dyes: 
FM4-64  is  a  red  fluorescent  membrane  dye  that  is  useful  for  out¬ 
lining  plasma  membranes.  Such  lipophilic  dyes  work  well  for  bulk 
labeling  of  plasma  membranes  in  a  variety  of  species.  Like  Dil, 
multi-photon  imaging  of  FM4-64  requires  the  use  of  a  Nd :  YLF 
laser.  Using  such  technology,  dynamic  membrane-associated 
events  have  been  followed  during  C.  elegans  development  using 
4D  imaging  (Mohler  et  al.,  1998)  [Fig.  43.9(D)].  An  alternative  is 
FM1-43,  a  water-soluble,  styrylpyridinium  dye  that  more  gener¬ 
ally  labels  membranous  structures,  likely  by  inserting  into  the 
outer  leaflet  of  the  plasma  membrane,  where  it  becomes  highly 
fluorescent.  FM1-43  has  been  used  to  study  exocytosis  during 
echinoderm  fertilization  (Terasaki  and  Jaffe,  2004)  and  to  study 
vesicular  trafficking  in  early  C.  elegans  embryos  (Skop  et  al., 
2001)  [Fig.  43.9(E)], 

3.  Dil  derivatives:  The  lipophilic  carbocyanines  Dil 
[DiIC18(3)],  DiO  [DiOC18(3)],  DiD  [DiIC18(5)],  and  DiR 
[DilC  1 8(7)],  are  weakly  fluorescent  in  water  but  highly  fluores¬ 
cent  and  very  photostable  when  incorporated  into  membranes.  Dil, 
DiO,  DiD,  and  DiR  exhibit  distinct  orange,  green,  red,  and  infrared 
fluorescence,  respectively,  thus  facilitating  multi-color  imaging 
(Molecular  Probes,  2004).  By  varying  the  number  of  carbons 
attached  to  the  fluorophore,  the  hydrophobicity  of  the  dye  can  be 
varied.  The  slightly  less  lipophilic  DilC  12(3),  DiIC16(3),  and 
DiOC16(3)  are  sometimes  easier  to  dissolve  and  load  into  mem¬ 
branes  than  their  C18  homologs  (Molecular  Probes,  2004),  but  can 
also  yield  more  non-specific  labeling  than  their  longer-chain 
counterparts  (J.  Hardin,  unpublished  observations).  Longer 
carbon  chains  result  in  poor  solubility,  even  in  the  presence  of  a 
carrier  such  as  DMSO.  The  result  is  that,  in  saline  media  or  sea¬ 
water,  the  dye  forms  tiny  crystals.  Cellular  structures  are  only 
labeled  when  a  crystal  physically  touches  a  membranous  structure. 
In  the  case  of  sea  urchin  embryos,  for  example,  incubation  of 
embryos  with  DiCi6  for  5  min  results  in  a  salt-and-pepper  pattern 
of  cell  labeling  that  allows  individual  cells  to  be  followed  against 
a  dark  background.  Multi-photon  imaging  of  labeled  cells  is 
possible  using  a  Nd:YLF  laser  operating  at  1047  nm  [Fig. 
43.9(F,G)]. 

4.  BODIPY  TR,  methyl  ester  dye:  BODIPY  TR, 

methyl  ester  dye,  readily  crosses  cell  membranes  and  localizes 
to  membrane-bound  organelles.  Because  the  dye  does  not  localize 
strongly  to  the  plasma  membrane,  BODIPY  TR,  methyl  ester  is 
useful  for  determining  the  position  of  the  nucleus  and  the  shapes 
of  cells,  and  for  outlining  whole  tissues.  The  emission  maximum 
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FIGURE  43.9.  Bulk  labeling  with  dyes  in  living  embryos.  Four  different  examples  of  the  use  of  bulk-dye  labeling  in  living  embryos.  (A-C)  Time-lapse  sequence 
showing  somitogenesis  in  a  one-somite  stage  zebrafish  embryo  (dorsal  view,  anterior  at  top).  Embryos  were  vitally  stained  with  the  fluorescent  lipid  Bodipy-C5 
ceramide  and  observed  using  a  Bio-Rad  MRC-600  CLSM.  The  elapsed  time  (min)  is  indicated  in  each  panel.  Individual  somitic  cells  are  colorized  for  clarity. 
As  somites  form,  intercalation  of  cells  forces  colorized  and  non-colorized  cells  to  rearrange,  changing  the  overall  shape  of  the  tissue.  Bar  =  20  pm.  (D)  Live 
MPLSM  imaging  of  epidermal  cell  fusions  in  the  C.  elegans  embryo.  Frames  are  5.5  min  apart.  Disappearance  of  lateral  membranes  ( arrowhead)  in  an  optical 
cross-section  of  an  embryo  labeled  with  FM4-64  indicates  cell-cell  fusion  is  occurring.  Images  were  acquired  using  a  1047  nm  Nd :  YLF  laser  and  a  Bio-Rad 
MRC-600  scan  head  with  no  pinhole  inserted  in  the  optical  path.  Bar  =  5  pm.  (E)  Plasma  membrane  dynamics  in  the  early  C.  elegans  embryo  studied  using 
FM1-43.  MPLSM  time  course  of  a  single  focal  plane  taken  from  an  embryo  labeled  with  FM1-43  during  cytokinesis.  Elapsed  time  is  shown  in  minutes.  Cyto- 
kinetic  furrow  formation  completed  at  3.0  min.  Membrane  accumulates  at  a  specific  focal  spot  (arrow,  4  min),  and  continues  to  accumulate  in  successive  time 
points.  Bar  =  10pm.  (F-G)  Cell  rearrangement  at  the  blastopore  of  a  sea  urchin  embryo  visualized  using  DiICi6.  (F)  Blastopore  view  of  a  DiICi6  normal  late- 
gastrula  embryo.  The  dark  area  in  the  center  is  the  blastopore  surrounded  by  DiICi6-labeled  cells  at  the  blastopore  lip;  the  inset  is  enlarged  in  (G,  H).  (G,  H) 
Frames  acquired  15  min  apart  showing  movements  of  cells.  Cell  a  has  produced  a  protrusion  (arrow)  and  made  contact  with  cell  b,  while  cells  c  and  d  have 
shifted  positions.  Bar  =  10pm.  [(A-C)  are  from  Henry  and  colleagues  (2000);  (D)  is  from  Mohler  and  colleagues  (1998),  used  by  permission.] 


of  BODIPY  TR,  methyl  ester  is  at  -625  nm,  making  it  an  excel¬ 
lent  counterstain  for  eGFP  (emission  maximum  -508  nm;  Cooper 
et  al.,  2005),  one  that  produces  little  spectral  bleed-through  of  GFP 
fluorescence  into  the  BODIPY  TR  channel.  The  excitation  curves 
of  the  two  fluorophores  are  well  separated  as  well,  making  them 
suitable  for  dual-channel  confocal  imaging  [Fig.  43.10(A-C)]. 

5.  Scatter  labeling  using  lineage  tracers:  When  injected  into 
blastomeres  that  generate  a  large  sector  of  the  embryo,  lineage 
tracers,  such  as  fluorescent  dextrans,  can  serve  as  an  effective 
counterstain.  This  technique  has  been  used  effectively  in  zebrafish 
embryos  [Fig.  43.10(D-G)]. 


SEEING  IN  SPACE:  STRATEGIES  FOR 
4D  VISUALIZATION 

To  one  accustomed  to  normal,  2D,  widefield  fluorescent 
microscopy,  the  images  presented  by  any  of  the  3D  microscopy 
methods  mentioned  above  can  seem  such  an  improvement  that 
many  are  tempted  simply  to  page  through  the  planar  images, 
feeling  perhaps  that  the  time  spent  to  make  a  proper  3D  or  4D  ren¬ 
dering  would  be  better  spent  obtaining  more  data.  However,  this 
temptation  should  be  avoided,  because  3D  reconstructions  provide 
wonderful  new  perceptual  insights  into  embryogenesis. 
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FIGURE  43.10.  Counterstains  can  improve  visualization  of  GFP-expressing  cells  in  embryos.  (A-C)  BODIPY  TR  methyl  ester  counterstaining  in  a  living 
zebrafish  embryo,  imaged  using  a  Bio-Rad  MRC-600  CLSM.  (A)  Tail  rudiment  of  a  tbx6-GFP  transgenic,  19-somite  embryo  (lateral  view).  GFP  is  expressed 
in  the  caudal  mass  stem  cells.  D,  dorsal;  P,  posterior.  (B)  BODIPY  TR  methyl  ester  dye  fluorescence.  Loosely  organized  mesenchymal  stem  cells  join  cells  in 
the  posterior  limit  of  the  presomitic  mesoderm  (psm).  Np,  neural  plate.  (C)  Merge.  Scale  bar  =  100  pm.  (D-G)  Counterstaining  using  fluorescent  dextran.  Zebrafish 
embryos  expressing  GFP  (green)  under  the  control  of  the  goosecoid  (gsc)  promoter  in  prechordal  plate  precursor  cells  were  scatter-labeled  with  rhodamine  (red) 
in  epiblast  cells  overlying  the  presumptive  prechordal  plate  and  followed  in  3D  over  time  by  dual-channel  confocal  microscopy  using  a  Bio-Rad  Radiance  2000 
CLSM.  Surface  renderings  were  performed  using  Volocity  software.  gscGFP  embryos  at  shield  stage  (D,  E;  E  is  a  rotated  view  of  D)  and  45  min  later  (F,  G).  In 
all  pictures,  anterior  is  to  the  top  and  posterior  to  the  bottom.  Bar  =  50  pm.  [A-C  are  from  Cooper  et  al.  (2005);  D-G  are  from  Ulrich  et  al.  (2003).  Used  by 
permission.] 


One  of  the  challenges  of  4D  live  imaging  of  embryos  is  visu¬ 
alizing  the  resulting  datasets  in  meaningful  ways.  Although  these 
challenges  are  not  unique  to  embryos,  the  widespread  use  of  4D 
imaging  in  developmental  biology  requires  efficient  ways  to  navi¬ 
gate  through  4D  datasets,  to  project  or  render  them  as  2D  +  time 
or  3D  +  time  datasets,  and  to  provide  compiled  datasets  to  other 
researchers  in  a  portable,  compact  format.5  As  an  example,  con- 


5  Older,  platform-specific  software  solutions  for  compiling  and  browsing  4D 
datasets  have  been  described  (Mohler,  1999).  More  recently,  several  platform- 
independent  solutions  have  emerged;  several  make  use  of  the  extensible 
plug-in  architecture  of  ImageJ  (http://rsb.info.nih.gov/ij/).  These  include  the 
Hyper  Volume  Browser  plug-in  by  R  Pirrotte  and  J.  Mutterer 
(http ://rsb .info . nih. gov/ij/plugins/hypervolume-bro wser.html),  which  allows 
a  folder  of  images  collected  as  a  4D  dataset  to  be  played  back  within  RAM. 
A  solution  that  is  much  less  RAM-intensive  is  to  compile  4D  datasets  into 
QuickTime  movies;  such  movies  have  the  tremendous  advantage  that  they  are 
portable,  and  can  be  played  back  on  rather  unremarkable  computer  hardware. 
I  have  written  a  Java-based  software  suite  that  fulfills  this  purpose  within 
ImageJ,  including  a  browser  to  preview  4D  datasets  before  compression 
(Browse4D),  a  compressor  (QT4D  Writer),  and  a  viewer  (QT4D  Player).  The 
suite  is  available  for  download  at  http://worms.zoology.wisc.edu/QT4D.html. 


sider  the  embryo  in  Figure  43.11,  which  expresses  a  GFP-tagged 
protein  that  localizes  to  epithelial  junctions,  and  has  been 
imaged  using  two-photon  microscopy  (Koppen  et  al .,  2001). 
Images  acquired  at  different  focal  depths  provide  simultaneous 
information  about  structures  throughout  the  thickness  of  the 
embryo  [Fig.  43.11(A,B)],  but  without  any  sort  of  projection  or 
reconstruction,  it  is  difficult  for  the  typical  student  or  researcher  to 
mentally  manipulate  information  from  each  of  the  focal  planes 
simultaneously. 

Depicting  Embryos  in  Time  and  Space: 

2D  +  Time  Versus  3D  +  Time 

While  4D  confocal  datasets  can  yield  powerful  views  of  single 
focal  planes  in  embryos  over  time,  examination  of  such  datasets 
immediately  presents  a  challenge:  single  optical  planes  contain  a 
limited  amount  of  spatial  information  along  the  imaging  axis. 
Thus,  a  key  challenge  in  4D  confocal  imaging  of  embryos  is  nav¬ 
igating  through  the  raw  4D  dataset  quickly  and  easily  (i.e.,  up  and 
down  within  a  single  time  point,  and  a  single  plane  across  time 
points).  The  challenge  extends  beyond  navigation  through  the  raw 
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FIGURE  43.11.  Using  projections  can  simplify  viewing  of  three-dimensional  embryos.  (A)  Single  C.  elegans  embryo  expressing  DLG-1  :GFP  (a  protein  that 
localizes  to  epithelial  junctions)  imaged  using  MPLSM.  Thirty  focal  planes  spaced  at  1  |tm  were  acquired.  (A)  A  focal  plane  deep  within  the  specimen  (focal 
plane  27),  showing  the  dorsal  epidermis.  (B)  A  focal  plane  near  the  coverslip  (focal  plane  5),  showing  the  ventral  epidermis  from  the  same  embryo.  (C)  A 
maximum  intensity,  2D  projection  of  all  30  focal  planes  allows  visualization  of  most  structures,  but  material  from  the  ventral  surface  obscures  dorsal  structures. 
(D)  A  maximum  intensity,  2D  projection  of  focal  planes  17  to  29,  effectively  visualizing  dorsal  structures.  (E)  A  corresponding  ventral-only  projection  (focal 
planes  3-10).  (F)  Focal  planes  3  to  10  projected  using  an  average  intensity  projection,  followed  by  contrast  enhancement.  (G)  A  3D  projection  of  focal  planes 
3  to  10  performed  using  ImageJ,  using  the  brightest  point  method,  with  0°  of  rotation.  (H)  Same  data  set  rotated  50°  around  the  y-axis.  I.  Same  dataset  visual¬ 
ized  as  a  voxel-based  projection  using  Volocity.  The  arrows  point  to  the  ventral  midline,  where  it  is  clear  that  epidermal  cells  lay  down  a  midline  septum  of 
some  thickness.  Bar  =  1 0  pm. 


4D  datasets,  however.  To  image  the  entire  embryo  or  a  spatial 
subset  of  the  4D  dataset,  the  embryo  must  be  rendered  at  each  time 
point  in  such  a  way  that  information  from  multiple  focal  planes  is 
displayed  over  time. 

There  are  two  common  strategies  for  displaying  typical  4D 
datasets.  The  multiple  optical  sections  from  an  image  stack  can  be 
projected  along  the  z-axis  to  form  a  2D  dataset,  thereby  flattening 
the  3D  data  (2D  +  time).  Alternatively,  the  voxels  of  each  original 
3D  volume  can  be  rendered  using  a  number  of  different  algorithms 
(3D  +  time)  and  then  these  renderings  can  be  linked  to  form  a 
movie.  In  the  2D  +  time  approach,  the  3D  spatial  information  in 
the  specimen  is  lost,  but  the  predominant  features  of  the  3D 
volume  are  represented  in  a  format  that  is  computationally 
less  demanding.  2D  +  time  strategies  are  particularly  effective  for 
displaying  information  from  a  volume  that  is  relatively  thin 
along  the  z-axis  [Fig.  43.11(D-F)].  If  most  cellular  movement 
occurs  in  a  plane  or  within  a  few  optical  sections,  this  approach  is 


often  the  best  for  qualitative  visualization  of  important  details  in 
the  dataset. 

Some  software  packages,  such  as  the  Ultra  View  software  pro¬ 
vided  with  PerkinElmer  disk-scanning  microscopes,  allow 
adjustable  selection  of  a  specific  subset  of  focal  planes  for  projec¬ 
tion  over  time.  This  provides  an  animated  movie  of  the  equivalent 
of  a  thick  optical  slab  over  a  specific  range  of  z-positions  within 
the  dataset,  but  with  an  important  difference:  unlike  an  actual 
optical  thick  section,  particular  features  within  the  slab  are  still 
seen  distinctly.  Many  other  software  packages,  including  ImageJ, 
allow  projection  of  single  z-series,  and  permit  the  writing  of 
plug-ins  or  macros  that  provide  semi-automated  production  of  pro¬ 
jected  2D  +  time  datasets. 

Various  algorithms  can  be  used  to  project  a  stack  of  optical 
sections  (see  Chapters  14  and  15,  this  volume).  Developmental 
biologists  typically  wish  to  view  the  readout  of  the  prominent  flu¬ 
orescent  details  within  a  volume  more  directly.  To  do  this,  two 
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sorts  of  projection  strategies  are  employed:  maximum-intensity 
and  average-intensity  projections.  The  maximum-intensity  algo¬ 
rithm  queries  the  intensity  of  a  pixel  (or  neighborhood  of  pixels) 
at  a  given  A,y-position  in  the  projected  image,  and  selects  the 
brightest  pixel  for  display  at  that  position  along  the  line  of  pro¬ 
jection  within  the  image  stack. 

Maximum-intensity  projections  [Fig.  43.11(C-E)]  are  typi¬ 
cally  the  favorite  of  developmental  biologists,  because  they 
emphasize  the  brightest  details  in  the  volume,  and  these  are  often 
the  most  interesting  (see  Chapter  14,  this  volume).  One  weakness 
of  maximum-intensity  projections  is  that  they  are  susceptible  to 
artifacts  from  noise  in  individual  optical  sections  within  the  stack. 
We  have  found  that  in  such  cases,  applying  standard  Gaussian 
filters  can  significantly  improve  the  starting  image.  In  general, 
however,  the  best  projections  begin  with  images  that  have  high 
intrinsic  signals.  Another  drawback  of  maximum-intensity  projec¬ 
tions  is  that,  because  they  deliberately  emphasize  small  but  bright 
fluorescent  structures,  they  are  not  really  suitable  for  quantifica¬ 
tion  and  they  also  tend  to  make  near  features  obscure  other  fea¬ 
tures  inside  the  data  volume. 

The  average-intensity  projection  algorithm  sums  all  of  the 
pixel  intensity  values  along  a  projection  through  the  stack,  calcu¬ 
lates  the  average  pixel  value,  and  displays  that  value  at  the  corre¬ 
sponding  position  in  the  projected  image.  Average  intensity 
projections  are  less  noisy,  and  are  typically  considered  to  produce 
a  more  quantitatively  accurate  representation  of  the  data.  Because 
calculating  an  average  along  a  projection  line  that  contains  many 
zero  intensity  pixels  reduces  the  visible  contrast  of  such  projec¬ 
tions,  the  results  should  be  viewed  using  a  high  contrast  look-up 
table  to  yield  results  that  are  similar  to  those  of  maximum  inten¬ 
sity  projections  [Fig.  43.11(F)]. 

Three-dimensional  representations  of  4D  live  embryo  data  (i.e., 
3D  +  time)  can  be  produced  in  one  of  two  ways.  In  the  first, 
less  computationally  demanding  method,  rotated  3D  projections 
of  image  stacks  are  first  produced  using  a  variety  of  algorithms 
[Fig.  43.11(G,H)].  Such  simple  renderings  can  be  extremely  useful 
for  identifying  the  3D  relationships  between  structures  within 
embryos.6  The  second  method,  real-time,  voxel-based  rendering,  is 
much  more  computationally  demanding;  for  large  datasets,  current 
computing  hardware  accessible  to  the  average  laboratory  does  not 
permit  on-demand,  true  real-time  manipulation  of  the  raw  4D 
dataset.  However,  once  datasets  are  rendered,  various  rendering 
software  suites  permit  rotation,  contrast  and  brightness  adjustment, 
etc.,  of  the  dataset,  thereby  permitting  some  direct  manipulation  of 
the  dataset  by  the  user  [Fig.  43.11(1)].  Rendered  datasets  can 
provide  striking  three-dimensional  views  of  structures  within 
embryos  (see  Chapter  14,  this  volume ,  for  further  details).7  Figures 
43.1(A),  43.6(B-D),  43.7(A),  43.8(A),  and  43.10(D-G)  all  used 


6  In  addition  to  the  built-in  functions  within  ImageJ,  which  perform  maximum¬ 
brightness,  nearest-point,  and  average  intensity-based  3D  projections,  several 
freeware  solutions  exist  for  rendering  single  time  points  as  rotational  projec¬ 
tions  within  ImageJ.  These  include  VolumeJ,  by  M.  Abramoff 
(http://bij.isi.uu.nl/vr.htm),  and  the  Volume  Browser  plug-in,  by  K.  U.  Barthel, 
which  allows  volumes  to  be  manipulated  in  3D  (http://rsb.info.nih.gov/ij/ 
plugins/volume- viewer.html).  I  have  written  a  series  of  ImageJ  plug-ins  for 
batch  rendering  of  rotational  projections  of  embryos,  available  at 
http://worms.zoology.wisc.edu/QT4D.html.  As  currently  written,  these 
plug-ins  support  various  kinds  of  4D  datasets,  including  .PIC,  .AVI  and  .TIF 
image  stacks,  and  output  rotated  datasets  as  sequential  TIFF  images  suitable 
for  compression  using  QT4DWriter. 

7  Several  commercial  solutions  exist  for  4D  visualization.  These  include  (1) 
Volocity  from  Improvision  (http://improvision.com);  (2)  Imaris,  from  Bit- 
Plane  AG  (http://www.bitplane.com/);  and  (3)  FluVR,  from  Scientific  Volume 


Volocity  (Improvision,  Coventry,  UK)  for  4D  rendering.  3D  +  time 
approaches  are  well  suited  to  situations  in  which  a  specimen  is  thick 
relative  to  its  other  dimensions.  If  the  process  being  studied 
involves  extensive  movements  of  cells  or  relevant  features  along 
the  ^-dimension,  3D  +  time  rendering  is  the  method  of  choice. 

The  decision  to  use  2D  +  time  versus  3D  +  time  strategies  to 
depict  living  embryos  depends  on  several  considerations.  If  the 
original  raw  4D  dataset  does  not  need  to  be  retained,  2D  +  time 
datasets  require  much  less  storage  space  than  true  4D  datasets, 
because  they  compress  the  data  to  1  In  its  original  size,  where  n  is 
the  number  of  focal  planes  in  each  stack.  As  DVD  optical  drives 
become  ubiquitous,  however,  saving  storage  space  is  becoming 
less  important.  A  second,  more  important  practical  consideration 
is  the  time  required  to  render  3D  +  time  datasets.  For  the  typical 
end  user  in  a  developmental  biology  laboratory,  the  processing 
power  require  for  ultrafast  rendering  of  voxel-based  data  is  finan¬ 
cially  difficult.  For  more  typical  hardware  configurations,  render¬ 
ing  of  volumes  from  4D  datasets  can  take  a  few  minutes  to  hours, 
depending  on  the  method  of  rendering,  the  software  used  and  the 
size  of  the  original  4D  dataset. 

OTHER  USES  FOR  CONFOCAL  AND 
MULTI-PHOTON  MICROSCOPY  IN  IMAGING 
AND  MANIPULATING  EMBRYOS 

Multi-Photon-Based  Ablation 

Galbraith  and  Terasaki  (2003)  showed  that  the  Ti :  Sa  laser  in 
typical  multi-photon  microscopes  can  be  used  to  cause  localized 
damage  within  unlabeled  cells  of  a  sea  urchin  embryo  via  a  multi¬ 
photon  process  (see  Chapter  38,  this  volume).  To  intentionally  gen¬ 
erate  damage,  laser  power  was  increased  10-fold,  and  pixel  dwell 
time  was  increased  70x,  that  is,  there  was  a  700x  increase  in  total 
exposure.  Because  the  multi-photon  absorption  events  are  so  local¬ 
ized,  and  because  the  longer  IR  wavelengths  involved  penetrate 
more  deeply  into  a  specimen,  they  can  produce  tissue  disruption 
at  depths  not  possible  with  standard  ablation  laser  systems.  Multi- 
photon-induced  damage  is  characterized  by  an  autofluorescent 
scar,  which  allows  monitoring  of  damage  and  identification  of  the 
wound  site  as  development  proceeds  (see  Chapter  38,  this  volume) 
[Fig.  43.12(A-D)]. 

Fluorescence  Resonance  Energy  Transfer 

The  use  of  fluorescence  resonance  energy  transfer  (FRET)  in 
embryos  has  not  been  reported  in  published  journal  articles,  but 
this  will  likely  change  in  the  near  future.  FRET  is  becoming 


Imaging  B.V.  (http://www.svi.nl/products/software/).  The  latter  offers  a  free 
program  for  single-time-point  surface  rendering,  FreeSFP,  which  can  yield 
excellent  rendered  views  of  certain  kinds  of  fluorescent  signals  within 
embryos.  Freeware  programs  include  (1)  Voxx  (Clendenon  et  al.,  2002; 
http://www.nephrology.iupui.edu/imaging/voxx/),  which  allows  freehand 
manipulation  of  rendered  3D  +  time  data,  manipulation  of  multi-channel  data, 
the  ability  to  render  data  as  anaglyph  stereo  projections,  and  export  of  ani¬ 
mations  to  several  file  formats  that  can  be  easily  converted  to  QuickTime 
movies.  (2)  VisBio  (Rueden  et  al .,  2004;  http://www.loci.wisc.edu/visbio/),  a 
Java-based  biological  visualization  tool  that  emphasizes  data  discovery  and 
analysis,  which  can  perform  slices  within  3D  datasets  and  can  perform  various 
measurements  on  the  data  as  well.  VisBio’s  visualization  capabilities  are  more 
limited  in  some  senses  than  those  of  Voxx,  but  VisBio  has  superior  slicing 
and  interactive  measurements  capabilities.  Because  both  load  datasets  into 
RAM,  their  memory  requirements  are  demanding. 
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FIGURE  43.12.  Multi-photon  microscopy  can  be  used  to  produce  controlled  damage  and  to  detect  protein  interactions.  (A-D)  Damage  to  a  mitotic  pole  in  a 
sea  urchin  embryo.  (A)  Multi-photon  damage  was  made  in  a  two-cell  stage  sea  urchin  embryo,  using  a  Zeiss  LSM  510  MPLSM  (Ti :  Sa  laser;  arrow);  subsequent 
development  was  observed  using  a  Bio-Rad  MRC-600  CLSM.  Scanning  transmission  images  (A,  C)  were  obtained  simultaneously  with  fluorescence  images  (B, 
D;  488  nm  excitation)  to  visualize  the  fluorescent  scar.  The  daughter  cell  that  inherited  the  damage  did  not  divide  (C),  whereas  the  undamaged  daughter  cell 
divided.  Bar  =  50  pm.  (E)  Two-photon  excitation  steady-state  intensity  (left,  middle)  and  FRET  efficiency  (E;  right)  images  of  dense  bodies  in  a  muscle  quad¬ 
rant  from  a  C.  elegans  embryo  expressing  PAT-4  :CFP  and  PAT-4  :YFP.  fchl,  fluorescence  intensity  in  the  CFP  channel;  fch2,  fluorescence  intensity  in  the  YFP 
channel.  FRET  efficiency  is  on  the  order  of  -30%,  indicating  that  PAT-4  protein  can  self-associate  in  muscle  dense  bodies.  For  details  of  the  procedure,  please 
consult.  Breusegem  (2002).  [A-D  are  from  Galbraith  and  Terasaki  (2003);  (E)  is  from  Breusegem  (2002),  Used  by  permission.] 


increasingly  accepted  as  a  method  for  determining  when  two  sub¬ 
units  of  the  same  proteins  interact  with  one  another  (intramolecu¬ 
lar  FRET)  or  where  and  when  two  different  proteins  physically 
interact  in  vivo  (intermolecular  FRET;  see  Chapter  45,  this 
volume).  Given  their  optical-sectioning  capabilities,  CLSM  and 
MPLSM  are  potentially  well  suited  to  performing  and  imaging  the 
results  of  FRET  in  embryos,  especially  because  currently,  the  most 
popular  donor/acceptor  pair  is  ECFP/EYFP  (Sekar  and  Periasamy, 
2003;  Wallrabe  and  Periasamy,  2005).  One  issue  common  to  both 
intra-  and  intermolecular  FRET  is  the  contribution  to  acceptor 
excitation  by  autofluorescence.  Embryos  often  display  significant 
amounts  of  autofluorescence  at  the  one-photon  excitation  wave¬ 
lengths  used  for  FRET,  creating  additional  non-FRET  signal 
that  can  complicate  the  analysis  of  acceptor  fluorophore  signals. 
This  problem  can  be  overcome  by  using  two-photon  excitation, 
which  has  the  added  advantages  of  increased  depth  of  penetra¬ 
tion  mentioned  previously.  Breusegem  (2002)  reported  that  exci¬ 
tation  at  850  to  870nm  reduces  autofluorescence  in  C.  elegans 
embryos. 

At  the  time  this  chapter  was  written,  no  published  studies  had 
reported  the  use  of  intramolecular  FRET  in  living  embryos. 
Flowever,  the  use  of  such  intramolecular  FRET  reporters,  which 
now  includes  a  variety  of  reporters  for  signaling  events,  such  as 
the  chameleons  for  assessing  calcium  dynamics  (Miyawaki  et  al., 
1999)  and  various  reporters  for  small  GTPase  activity  (Pertz  and 
Flahn,  2004),  will  likely  become  more  widespread  in  living 
embryos  as  the  technology  becomes  less  expensive  and  more 
widely  available.  One  published  example  of  intramolecular  FRET 


using  chameleons  measured  calcium  dynamics  in  living  C.  elegans 
adults  (Kerr  et  al.,  2000),  suggests  it  has  tremendous  promise. 
Because  chameleons  are  fusion  proteins  containing  CFP  and  YFP 
in  the  same  protein,  the  donor :  acceptor  ratio  is  always  1:1,  so 
many  of  the  complications  that  plague  intermolecular  FRET  are 
largely  obviated.  This  successful  use  of  an  intramolecular  FRET 
reporter  suggests  that  in  the  near  future  similar  experiments  will 
be  performed  on  living  embryos. 

Performing  intermolecular  FRET  in  living  embryos  presents 
several  additional  challenges  beyond  those  associated  with 
intramolecular  FRET.  In  the  simplest  cases,  intermolecular  FRET 
may  simply  be  used  qualitatively  to  show  that  there  is  an  inter¬ 
action  between  two  molecules  in  vivo,  after  suitable  correction  for 
spectral  bleed-through,  but  without  additional  computation 
(Wallrabe  and  Periasamy,  2005).  In  such  cases,  the  stoichiometry 
of  the  acceptor  and  donor  fluorophores  is  usually  known,  and 
FRET  is  reflected  in  changes  in  the  ratio  of  both  acceptor  and 
donor  emission. 

However,  the  power  of  intermolecular  FRET  lies  in  the  ability 
to  use  the  percent  energy  transfer  efficiency  ( E% )  as  a  measure  of 
intermolecular  distance,  that  is,  as  a  spectroscopic  ruler.  Quantita¬ 
tive  FRET  is  much  more  complicated  in  practice  than  qualitative 
FRET,  and  this  complication  is  exacerbated  in  living  embryos. 
First,  it  is  important  to  carefully  control  the  concentrations  of  the 
donor  and  acceptor  molecules  for  any  quantitative  assessment  of 
E%.  In  transgenic  animals  into  which  fluorescent-protein-tagged 
molecules  have  been  introduced,  careful  measurement  of  fluo¬ 
rophore  concentration  is  difficult  at  best.  However,  in  some  genetic 
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systems,  it  is  possible  to  replace  the  endogenous  gene  with  a 
fluorescent-protein-tagged  version,  ensuring  that  virtually  all  of 
the  protein  under  consideration  is  tagged.  If  the  replacement  gene 
is  integrated  as  a  single  copy,  it  will  typically  preserve  the  normal 
level  of  expression  of  the  labeled  protein.  A  second  problem  with 
intermolecular  FRET  is  that  photobleaching  of  the  donor  or  accep¬ 
tor  fluorophore  is  often  used  as  a  separate  demonstration  that 
FRET  has  occurred.  However,  photobleaching  may  not  be  suitable 
for  embryos,  many  of  which  are  especially  sensitive  to  photo¬ 
damage.  An  alternative  is  to  use  fluorescence  lifetime  imaging 
(FLIM)  to  measure  FRET  efficiency  (Clegg  et  al.,  2003;  see  also 
Chapter  27,  this  volume).  FLIM  has  the  advantage  that  it  can  be 
used  independent  of  a  measurement  of  fluorophore  concentration, 
because  the  donor  fluorescence  lifetime  decreases  in  the  presence 
of  FRET  irrespective  of  fluorophore  concentration.  This  solution 
introduces  a  third  complication,  however.  In  addition  to  the 
substantial  complexities  of  instrumentation,  quantitative  determi¬ 
nation  of  FRET  efficiency  using  FLIM  usually  requires  the  acqui¬ 
sition  of  several  images,  which  can  increase  the  likelihood  of 
photodamage  to  the  embryo.  For  thorough  quantitative  FRET 
analysis,  FLIM  is  often  performed  on  a  pixel-by-pixel  basis 
throughout  the  field  of  view,  increasing  this  risk. 

Despite  these  caveats,  recent  work  by  Breusegem  and  col¬ 
leagues  has  shown  that  quantitative  FRET  can  be  successfully 
adapted  for  use  in  C.  elegans  embryos  (Breusegem,  2002).  They 
showed  that  that  the  integrin-linked  kinase  (ILK),  PAT-4,  will 
engage  in  FRET  with  itself,  and  with  both  PAT-3/[i-integrin  and 
PAT-6/actopaxin  at  muscle  dense  bodies  [structures  associated  with 
the  attachment  of  muscles  to  the  epidermis  in  C.  elegans',  Fig. 
43.12(E)].  FRET  was  confirmed  in  several  different  ways,  includ¬ 
ing  sensitized  YFP  emission,  and  E%  was  measured  using  both  the 
acceptor  and  donor  photobleaching  techniques  and  by  performing 
FLIM  in  the  time  and  frequency  domains.  In  the  time  domain, 
photons  were  accumulated  by  repeated  imaging  of  the  same  field 
at  lOOps/pixel,  which  required  25  to  30  frames  to  collect  >100 
photons/pixel.  Although  such  photon-counting  methods  place 
severe  limits  on  acquisition  speed,  because  dense  bodies  are  stable 
structures  and,  hence,  essentially  motionless,  sampling  of  discrete 
points  is  an  adequate  substitute  for  full-field  sampling,  and  this 
reduces  the  likelihood  of  phototoxicity  (Breusegem,  2002). 
Although  the  use  of  FRET  in  embryos  is  in  its  infancy,  these 
results  suggest  that  in  the  near  future  this  imaging  tool,  the  use  of 
which  was  once  exclusive  to  cell  biologists  working  on  flat  cells  in 
tissue  culture,  will  soon  be  available  to  developmental  biologists. 

CONCLUSIONS:  A  BRIGHT  FUTURE  FOR  3D 
IMAGING  OF  LIVING  EMBRYOS 

Few  disciplines  within  biology  have  benefited  more  from  improve¬ 
ments  in  confocal.  multi-photon,  and  related  technologies  than 
developmental  biology.  As  more  exotic  probes  are  developed  for 
use  in  living  cells,  particularly  those  that  can  be  genetically 
encoded,  the  imaging  of  embryos  will  continue  to  become  more 
sophisticated.  As  each  new  probe  is  developed,  there  may  be  new 
challenges  for  developmental  biologists,  but  based  on  current 
imaging  modalities,  several  generalizations  will  probably  continue 
to  hold  true. 

First,  although  there  are  times  when  more  expensive  equip¬ 
ment,  such  as  multi-photon  microscopes,  4D  deconvolution  micro¬ 
scopes,  or  high-speed  imaging  approaches  are  necessary,  empirical 
tests  of  viability  using  off-the-shelf  confocal  equipment  should  be 
performed  first  before  such  equipment  is  assumed  to  be  necessary. 


While  similar  experiments  are  now  being  performed  using  multi¬ 
photon  microscopy,  numerous  published  experiments  indicate  that 
both  standard  single-beam  confocal  microscopes  and  disk-scan¬ 
ning  microscopes  can  often  be  used  to  image  living  embryos  in 
four  dimensions  without  the  need  for  additional  equipment. 

Second,  as  computer  processing  speed  continues  to  increase, 
deconvolution  and  3D  projection  of  4D  datasets  will  become  more 
routine.  This  can  only  improve  the  ability  of  the  developmental 
biologist  to  visualize  complex  processes  in  four  dimensions. 
The  more  routine  use  of  multi-wavelength  probes  will  likewise 
improve  the  ability  of  the  developmental  biologist  to  perceive  and 
comprehend  the  complex  beauty  of  embryogenesis. 

Finally,  although  there  has  typically  been  a  lag  between  the 
application  of  new  imaging  modalities  in  cultured  cells  and  their 
subsequent  use  in  embryos,  the  history  of  the  field  suggests  that 
eventually  many  of  the  approaches  first  worked  out  in  cultured 
cells  will  be  adapted  for  use  in  embryos.  Extension  of  the  suc¬ 
cessful  combination  of  microsurgical  methods  and  confocal  and 
multi-photon  methods  presented  in  this  chapter,  which  effectively 
make  the  embryo  flatter,  will  likely  allow  techniques  such  as  FRET 
to  be  used  routinely  in  embryos  in  the  near  future.  Ultimately,  4D 
live  imaging  of  fluorescent  probes  in  embryos  will  cease  to  be  the 
preserve  of  the  specialist,  and  will  become  a  part  of  the  standard 
repertoire  of  the  developmental  biologist. 
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INTRODUCTION 

The  history  of  imaging  plant  cells  is  intimately  related  to  the  very 
development  of  microscopes  and  microscopical  techniques.  Some 
of  the  early  microscopists  made  extensive  use  of  plant  specimens, 
and  Hooke’s  description  of  cork  microstructure  (Fig.  44.1)  will 
stand  in  the  imagination  of  many  as  the  structural  foundation  for 
the  cell  theory.  There  are  several  reasons  for  this  to  have  happened: 
in  many  respects  plant  tissues  are  easier  to  deal  with,  easier  to  slice 
and  peel  to  the  necessary  thickness  for  observation,  they  have  more 
water,  and  consequently  are  less  optically  dense  than  many  other 
tissues,  often  they  are  naturally  pigmented  and  the  cells  are  usually 
larger.  Above  all,  the  existence  of  a  skeletal  cell  wall  composed  of 
cellulose  and  other  molecules  makes  plant  cells  extraordinarily 
geometric  and  highly  regulated  in  their  structural  features.  In  many 
instances,  these  structural  components  of  the  plant  cell  form  the 
basis  of  its  function,  making  microscopical  analysis  a  recurrent 
method  for  cell  and  developmental  biology  research. 

Since  the  17th  century,  many  gifted  microscopists  have  given 
cumulative  accounts  of  the  different  levels  of  plant  cell  structure. 
Perhaps  the  first  truly  specialized  plant  microscopist  was  Grew 
(1673),  who  initiated  the  first  systematic  study  of  plant  micro¬ 
anatomy.  Robert  Brown,  Amici,  Schleiden,  and  Nawaschin  are 
among  the  many  notables  who  contributed  to  the  description  of 
some  of  the  fundamental  biological  features  of  plants,  such  as  their 
reproductive  cycles,  mostly  using  microscopes  as  tools.  More 
recently,  the  seminal  textbooks  of  Esau  (1977a,b)  and  Fahn  (1990) 
systematized  definitively  the  histological  and  cellular  features 
of  higher  plants.  Plant  cell  ultrastructure  has  subsequently  been 
described  and  systematized  in  detail  in  many  textbooks  and  atlases, 
such  as  the  ones  produced  by  Gunning  and  Steer  (1996).  This 
effort  was  recently  complemented  by  an  extensive  review  on 
CD-ROM  that  stands,  and  probably  will  stand  for  a  long  time,  as 
a  central  reference  for  anyone  seeking  information  on  micro¬ 
scopical  data  concerning  plant  cells  (Gunning,  2003;  http:// 
www.plantcellbiologyoncd.com). 

The  developments  in  technology  and  reagents  that  brought  flu¬ 
orescence-based  methods  to  microscopy  have  also  become  preva¬ 
lent  in  plant  cell  biology  (Lloyd,  1987).  However,  plant  cells  do 
present  a  challenge  to  fluorescence  microscopy  because  they  often 
contain  pigments  and  complex  excitable  molecules  in  subcellular 
structures  that  generate  copious  autofluorescence.  In  many  cir¬ 
cumstances  this  is  a  nuisance  in  terms  of  signal-to-noise  ratio:  the 
autofluorescence  can  swamp  signal  from  other  fluorescent  probes 


being  studied.  Furthermore,  without  the  optical  sectioning  of  the 
confocal  microscope,  autofluorescence  glaring  from  all  planes  may 
obscure  the  signal  from  any  in-focus  structural  information.  Nat¬ 
urally,  confocal  imaging  has  made  a  strong  impact  in  the  area 
of  plant-cell  imaging.  All  these  problems  have  been  addressed 
in  extensive  reviews  about  applications  of  confocal  microscopy  to 
plant  cell  biology  (Hepler  and  Gunning,  1998),  imaging  ions  and 
other  advanced  methods  (Blancaflor  and  Gilroy,  2000),  and  in 
hand-  and  textbooks  on  methods  and  applications  (Galbraith  et  al., 
1995;  Hawes  and  Satiat-Jeunemaitre,  2001). 

Green  fluorescent  protein  (GFP)  and  other  genetically  encoded 
fluorescent  probes  have  made  a  substantial  impact  on  the  field 
(Haseloff  and  Amos,  1995;  Haseloff  et  al.  1997),  and  extensive 
lists  of  references  of  different  applications,  spectral  conditions,  and 
transient  expression  systems  are  available  (Brandizzi  et  al.  2002). 
Detailed  comparisons  of  the  relative  merits  between  conventional 
and  widefield  (Shaw,  2001)  and  between  two-photon  excitation 
and  confocal  (Feijo  and  Moreno,  2004)  have  shown  that  there  are 
specific  niches  for  all  methods,  and  probably  none  should  be  con¬ 
sidered  universal,  irrespective  of  their  price  and  sophistication. 
Previous  reviews  also  included  detailed  protocols  for  image  acqui¬ 
sition  using  these  different  methods.  A  very  useful  collection  of 
practical  criteria  for  probe  choice,  empirical  methods,  and  many 
tricks  for  immobilization,  perfusion,  and  loading  protocols  are 
described  by  Fricker  and  colleagues  (2001). 

Specific  methods  for  some  of  the  most-used  cell  and  tissue 
types  have  also  been  described  (e.g.,  Kodama  and  Komamine, 
1995;  Raghavan,  1995;  Sheen,  1995).  Various  fixation  and  other 
histological  methods  specific  for  plant  cells  are  extensively 
described  in  many  references  (e.g.,  Spence,  2001),  including 
complex  and  sophisticated  freeze-fixation  and  freeze-substitution 
methods  (Galway  et  al.,  1995;  Parthasarathy,  1995). 

However,  recent  years  have  been  marked  by  the  introduction 
of  less  invasive  methods  and  vital  probes,  with  a  strong  emphasis 
on  those  that  are  genetically  encoded.  The  focus  now  is  on  keeping 
cells  and  tissues  intact  and  alive,  and  this  offers  the  added  value 
of  enabling  one  to  study  the  true  dynamics  of  vital  processes. 
Vesicle  trafficking  (Bolte  et  al.,  2004),  individual  gene  expression 
(Shav-tal  et  al.,  2004),  and  cytoplasmic  streaming  dynamics 
(Shimen  and  Yokota,  2004)  are  just  a  few  examples  of  the  living 
processes  now  reachable  using  state-of-the-art  microscopy  in  com¬ 
bination  with  genetic  and  molecular  techniques.  We  will  thus  focus 
this  chapter  on  recent  developments  that  might  affect  plant  bio¬ 
logical  research  beyond  the  topics  covered  in  the  earlier  reviews. 
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FIGURE  44.1.  Cork  now  and  then!  Comparison  of  Robert  Hooke’s  picture 
(A)  and  state-of-the-art  laser-scanning  microscopy  (B)  of  a  cork  slice,  having 
a  time  difference  of  almost  350  years.  It  was  in  the  book  Micrographia  pub¬ 
lished  in  1665  that  Hooke  used  the  term  “cells”  for  the  first  time  to  describe 
the  element  of  this  regularity.  Despite  the  obvious  differences  (one  drawn  by 
hand,  the  other  based  on  digital  acquisition  of  optical  slices  and  software 
enhancement  by  using  maximum-intensity  projection),  the  evidence  of  the 
highly  regular  structure  of  plant  cells  is  clearly  evident  in  Hooke’s  image,  and 
underlies  the  close  relationship  between  form  and  function  in  plant  cells. 
However,  the  confocal  image  also  shows  one  of  the  most  prevalent  features  of 
plant  cells:  the  almost  ubiquitous  existence  of  autofluorescence  in  various  sub- 
cellular  structures,  a  factor  that  can  be  both  informative  and  a  problem  to  deal 
with  when  imaging  probes  with  overlapping  fluorescence  spectra. 


THE  EVER  PRESENT  PROBLEM  OF 
AUTOFLUORESCENCE 

Light  is  potentially  damaging  to  cells,  yet  plants  live  in  the  fast 
lane.  In  order  for  photosynthesis  to  proceed,  leaves  and  other  aerial 
parts  are  often  exposed  to  high  levels  of  light  radiation,  and  evo¬ 
lution  has  resulted  in  a  number  of  mechanisms  for  filtering  photons 
before  they  reach  sensitive  parts  inside  the  cell.  In  addition,  many 
plant  cell  walls  accumulate  complex  hydrophobic  molecules  that 
regulate  apoplastic  water  loss  and  movement  (e.g.,  suberin  in  cork; 
Fig.  44. 1 ),  and  a  plethora  of  secondary  metabolites,  many  working 
as  pigments,  have  evolved  for  ecological  or  allelopathic  reasons. 
Many  of  these  molecules  use  varied  pathways  to  dissipate  photo¬ 
excitation,  namely,  non-radiative  decay  or  relaxation,  energy  trans¬ 
fer,  photosynthesis,  and  fluorescence.  Unfortunately,  the  latter  is 
common  and  autofluorescence  is  a  natural  feature  of  almost  every 
plant  cell  (see  Chapter  21,  this  volume,  for  detailed  spectra). 

This  autofluorescence  is  a  nightmare  for  many  fluorescence 
applications.  On  fixed,  sectioned,  and  stained  specimens,  protocols 
using  strong  oxidizing  agents  such  as  chlorine  bleach,  chloral 
hydrate,  or  sodium-borohydride  have  long  been  used  to  reduce 
autofluorescence  (Shaw,  2001;  Chapter  18,  this  volume).  Efficient 
as  they  are,  these  techniques  cannot  be  used  when  imaging  living 
cells,  and  the  molecular  degradation  they  produce  can  even  destroy 
the  specificity  of  immunostaining.  Many  fixation  reagents,  such  as 
glutaraldehyde,  also  add  to  the  problem  by  generating  autofluo- 


rescent  Schiff  bases  and  delocalized  electron  resonance  transfer 
when  they  react  with  cellular  components. 

The  problem  is  illustrated  in  Figures  44.1  and  44.2.  In  Figure 
44.1(A),  the  same  cork  that  Hooke  could  describe  because  of  its 
opacity  to  photon  transmission  is  also  shown  by  state-of-the-art 
confocal  microscopy  because  the  walls  of  dead  cells  are  impreg¬ 
nated  with  suberin,  a  complex  lipid  with  strong  autofluorescence. 
As  the  cells  are  empty,  after  extended  focus  stacking,  the  cell  walls 
can  be  seen  with  great  detail  and  the  confocal  capacity  of  reject¬ 
ing  out-of-focus  light  renders  images  with  great  visual  depth. 

Figure  44.2  shows  what  is  probably  the  most  common  source 
of  imaging  problems:  the  green  tissues.  Chlorophyll  accumulates 
inside  plastids  that  occur  in  great  numbers  in  green  tissue  (the 
round  red  organelles  in  this  image).  Although  in  the  shoot  meri- 
stem  region,  depicted  in  Figure  44.2,  the  cell  wall  is  still  relatively 
immature  and  thus  has  less  autofluorescence  (shown  in  blue),  these 
two  signals  render  the  observation  of  other  fluorophores  (in  this 
case  GFP  in  the  endoplasmic  reticulum  (ER),  shown  in  green) 
almost  impossible  unless  the  out-of-focus  light  can  be  rejected  and 
strong  spectral  separation  is  available. 

Cell  wall  autofluorescence  swamps  the  signal  from  added 
probes  and  the  absorption  and  scattering  of  both  the  excitation  and 
the  signal  severely  limits  the  distance  that  one  can  image  into  the 
tissue.  The  common  way  of  dealing  with  spectral  mixing  is  to  use 
selective  dichroic/emission  filters,  with  the  excitation  peak  as 
narrow  as  sensitivity  allows.  In  a  relatively  young  organ  such 
as  that  shown  in  Figure  44.2,  good  results  are  obtainable  with 
standard  confocal  settings  (in  this  case,  using  a  Zeiss  Pascal).  The 
importance  of  being  able  to  produce  stringent  optical  sections  is 
illustrated  in  Figure  44.2(B),  which  shows  a  two-photon  image  of 
the  GFP  in  the  central  section  of  the  same  plantlet.  The  image  is 
almost  clear  of  the  other  sources  of  signal. 

On  a  more  physical  basis,  absorption  in  green  tissues  usually 
involves  both  excitation  of  a  first  excited  singlet  state  (responsible 
for  red  absorption)  and  excitation  of  a  second  excited  singlet  state 
(responsible  for  blue  absorption).  In  both  cases,  emission  is  mainly 
in  the  near-infrared  because  the  blue-excited  state  relaxes  to  the 
first  singlet  [Malkin  and  Niyogi  2001;  Figure  44.2(C)].  This 
complex  response  can  be  either  a  drawback  or  a  bonus.  Chloro- 
plasts  emit  farther  into  the  red  [Fig.  44.2(D)  in  false  blue],  but 
other  emissions  in  the  visible  spectrum  can  superimpose  important 
information.  Overall,  they  usually  have  a  broad  emission  spectrum, 
and  it  is  not  a  trivial  matter  to  discriminate  it  from  labeling,  even 
in  a  spectral  microscope. 

Many  recent  confocal  microscopes  come  equipped  with  the 
capacity  for  spectral  analysis,  and  this  facility  is  becoming  a  useful 
tool  for  discriminating  against  autofluorescence.  There  are  two 
ways  of  using  the  new  spectral  tools.  In  the  first,  one  is  only  con¬ 
cerned  with  the  emission  from  a  single  dye,  and  chooses  the  nar¬ 
rowest,  most  selective  emission  window  for  the  dye  involved.  This 
is  a  one-step  method,  and  only  a  single  reference  spectrum  is 
needed  for  each  series  of  observations.  In  the  second  method,  one 
uses  linear  spectral  unmixing  to  separate  the  emission  of  different 
dyes.  Although  speed  is  usually  not  an  issue  with  the  first  method, 
it  may  be  with  the  second,  especially  if  the  signal  in  the  different 
channels  must  be  acquired  sequentially.  However,  parallel  acqui¬ 
sition  often  implies  channels  only  lOnm  wide  and,  when  an 
already  weak  signal  is  distributed  among  several  narrow  spectral 
windows,  there  is  even  less  signal  in  each  one.  As  a  result,  low 
signal  level  can  be  a  problem  and  no  commercial  system  is  devoid 
of  limitations  in  that  respect.  Although  low  signal  can  be  overcome 
to  some  extent  by  more  laser  power,  this  can  lead  to  other  sorts  of 
physical  limitations,  particularly  singlet-state  saturation  of  the  dye 
and  photodamage  to  the  specimen. 


Imaging  Plant  Cells  •  Chapter  44  771 


FIGURE  44.2.  Three-dimensional  reconstruction  of  an  Arabidopsis  hypocotyl/cotyledon  imaged  using  a  Zeiss  Pascal  with  three  emission  channels  and  three 
laser  lines  (405,  488,  and  633  nm)  using  a  63x  Plan  Apochromat  NA  1 .4  oil-immersion  objective.  This  is  an  enhancer  trap  line  with  GFP  linked  to  an  ER  domain 
protein,  which  in  this  case  was  activated  on  the  meristematic  primordial  cells.  Using  the  normal  dichroic/emission  filters,  GFP  (green)  is  well  resolved  from  the 
chloroplasts  (red)  because  their  emission  shows  up  mainly  in  the  near  infrared.  The  immature  (thus  dimmer  in  terms  of  autofluorescence)  cell  wall  appears  as 
shown  in  false  blue.  Insert  (B)  shows  a  different  way  of  resolving  the  GFP  signal,  by  two-photon  excitation  (TPE)  at  870nm.  Because  the  excitation  wavelength 
is  not  as  optimal  for  the  chlorophyll  as  it  is  for  GFP,  the  red  plastids  seen  in  (A)  are  much  dimmer  or  invisible.  (C)  Arabidopsis  .seedling  spectra  traced  in  situ 
with  a  Leica  SP2  AOBS.  After  making  a  lambda  scan  from  500  to  700 nm  with  a  spectral  gate  width  of  30nm,  using  a  488  laser  line  and  a  lOx  Plan  Apochro¬ 
mat  NA  0.4,  a  spectrum  from  each  part  of  the  tissue  was  traced  for  eGFP,  cell  wall  and  chloroplast  autofluorescence.  With  these  emission  spectra,  it  becomes 
possible  to  unmix  the  lambda  stack  (D).  This  software  tool  comes  with  all  the  spectral  confocal  systems  (Leica  SP2  AOBS  and  Zeiss  510  Meta)  and  works  in 
a  similar  way  to  spatial  deconvolution  but  in  this  case  the  bleed-through  is  not  caused  by  out-of-focus  signal,  but  from  overlapping  fluorescence  emission. 
Bar  =  200  pm. 


Figure  44.2  shows  an  Arabidopsis  seedling  with  GFP  expres¬ 
sion  in  the  ER.  Reference  spectra  were  generated  for  the  different 
pigments  [Figure  44.2(C)]  and  the  respective  signals  extracted 
from  the  raw  xyzA,-image  data  (a  so-called  lambda  stack),  and  then 
merged  again  with  false  colors  [Fig.  44.2(D)].  It  is  clear  that  the 
channels  have  been  sharply  separated.  While  the  principle  works, 
there  are  limitations,  and  when  several  fluorescent  proteins  are 


present,  it  can  be  more  difficult  to  separate  colors,  especially  if 
more  than  one  of  them  occurs  in  the  same  voxel.  In  any  case,  one 
needs  a  specimen  capable  of  producing  lots  of  signal. 

As  usual  there  are  several  engineering  solutions  to  the  problem 
of  spectral  detection.  Leica  uses  a  prism  and  a  set  of  moveable 
mirrors  to  break  the  spectrum  of  emitted  light  into  three  or  four 
segments,  each  going  to  a  separate  photomultiplier  tube  (PMT). 
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Zeiss  uses  a  grating  to  disperse  the  emission  spectrum  over  an 
array  of  32  mini-PMTs,  and  has  facilities  to  digitize  as  many  as  8 
channels  from  any  combination  of  the  32  outputs  (Nikon  and 
Olympus  are  also  launching  their  own  solutions).  There  are  pros 
and  cons  to  each  method:  generally  speaking  one  can  say  that  the 
latter  gains  in  terms  of  temporal  resolution  during  live  imaging, 
but  perhaps  at  the  cost  of  some  sensitivity,  especially  in  the  red 
extreme  of  the  spectrum. 

Once  the  spectral  data  have  been  obtained,  they  must  be  decon¬ 
volved  (or  unmixed)  using  programs  similar  to  those  used  to 
deconvolve  widefield  structural  data  (see  Chapters  23,  24,  and  25, 
this  volume),  except  that,  instead  of  starting  with  a  point-spread 
function,  one  starts  with  stored  spectra  of  each  of  the  dyes  present. 
Least-squares  algorithms  are  used  to  fit  the  spectral  data  measured 
in  each  voxel  or  region  to  a  linear  sum  of  the  spectra  of  the  dyes 
expected  to  be  present.  As  all  the  light  used  by  the  spectral  detec¬ 
tor  comes  through  a  single  pinhole,  one  cannot  use  pinhole  size  to 
balance  the  signal  intensities  from  different  dyes.  Consequently, 
the  procedure  works  best  when  the  signals  from  all  the  various 
dyes  are  approximately  equal  in  strength. 


SINGLE-PHOTON  CONFOCAL  MICROSCOPY 

Because  plant  tissue  generally  consists  of  deep  layers  of  highly 
refractile  cell  walls  and  aqueous  cytosol  and  contains  various  auto- 
fluorescent  and  light-scattering  components,  intact  tissue  proves  a 
difficult  subject  for  fluorescence  microscopy.  However,  direct 
imaging  of  living  tissue  is  possible  using  suitably  corrected  micro¬ 
scope  optics.  Plant  seedlings  or  excised  tissues  can  simply  be 
mounted  in  water  for  microscopy  and  examined  using  a  long- 
working-distance  water-immersion  objective  to  minimize  the 
effects  of  spherical  aberration  when  focusing  deep  into  an  aqueous 
sample.  Even  with  the  use  of  such  specialized  objectives,  using 
single-photon  excitation,  image  quality  degrades  rapidly  for 
optical  sections  deeper  than  60  to  80  p,M  within  the  tissue. 
However,  the  small  size  of  seedlings,  such  as  those  of  the  model 
plant  Arabidopsis  thaliana,  allows  very  useful  imaging  despite  this 
limitation.  For  example,  median  longitudinal  optical  sections  can 
be  obtained  from  intact  roots.  This  direct  approach  to  imaging 
plant  materials  has  been  reviewed  elsewhere  (Haseloff,  2003). 


FIGURE  44.3.  (A)  Optical  section  of  an  Arabidopsis  root  with  a  random  insertion  (Cuttler  et  al.,  2000)  of  Clontech-YFP  targeted  to  the  plasma  membrane  (con- 
focal  image  excited  at  525  nm  using  a  Leica  SP2  AOBS).  (B)  Seedling  shoot  epidermis  from  a  line  similar  to  that  shown  in  (A),  but  with  the  Clontech-GFP 
imaged  with  TPE  at  930  nm.  Despite  the  non-optimal  optical  response  to  this  wavelength,  the  beam  has  enough  power  to  produce  the  desired  excitation  and 
yields  outstanding  results  in  terms  of  tissue  penetration  and  signal-to-noise  ratio.  (C)  Dual-emission  confocal  imaging  from  a  stoma  in  which  the  ER  is  stained 
with  mGFP5  (enhancer  trap  line)  and  the  chloroplasts  are  emitting  auto-fluorescence.  Mixed  widefield  and  fluorescence  (D)  and  confocal  fluorescence  (E)  of 
mGFP5  enhancer-trap  transformed  Arabidopsis  root  hair.  GFP  is  fused  to  an  ER  motif,  and  the  dynamics  of  the  big  ER  inclusions  are  clearly  visible.  (F)  Double¬ 
emission  of  ER-targeted  mGFP5  (Siemering  et  al.,  1996)  fusion  product  with  the  SCARECROW  gene  (Wysocka-Diller  et  al.,  2000).  Despite  the  low  expres¬ 
sion  level,  the  ER  and  nuclear  envelope  are  clearly  resolved  in  the  endodermis,  four  cell  layers  deep  into  the  root.  The  cell  wall  is  stained  with  vital  PI.  The 
optical-sectioning  properties  of  TPE  are  well  illustrated  in  the  sharp  fading  pattern  of  the  cell  wall  tangential  section. 
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FIGURE  44.4.  Leaf  epidermal  cell  of  a  transgenic  Arabidopsis  with  a  micro- 
tubule-associated  protein  (MAP4)  GFP  fusion,  imaged  with  a  Zeiss  Pascal  con- 
focal.  z-stacks  were  acquired  (B )  and  3D  volume-rendered.  Red  organelles  are 
plastids.  The  subcortical  distribution  of  the  microtubules  is  clearly  visible  due 
to  the  depth  created  by  the  rendering  algorithm.  This  construct,  while  produc¬ 
ing  excellent  results  for  epidermal  microtubules,  seems  useless  for  labeling 
microtubules  in  other  cells.  (C,  D)  Leaf  trichome  of  a  transgenic  Arabidopsis 
with  the  actin-bundling  protein  talin  fused  with  GFP,  imaged  with  a  Zeiss 
Pascal  confocal  and  processed  as  in  (A).  Amplification  in  (D)  is  particularly 
noticeable  by  the  details  of  the  actin  bundles  inside  these  cells.  However,  great 
care  must  be  used  in  interpreting  this  result,  because  under  a  constitutive  strong 
promoter,  talin  is  prone  to  creating  “artificial’'  cables  of  actin,  which  are  not 
supported  by  other  means  of  visualization  (Ketelaar  et  al.,  2004). 


- ► 

Direct  visualization  of  GFP  fluorescence  in  living  tissues  is  not 
prone  to  fixation  or  staining  artifacts,  and  can  provide  images  of 
exceptional  clarity.  Moreover,  the  activities  of  living  cells,  such  as 
cytoplasmic  streaming,  are  clearly  evident  during  microscopy. 
Ordinarily,  movement  within  a  sample  is  a  nuisance,  placing  con¬ 
straints  on  the  use  of  sometimes  protracted  techniques  for  noise 
reduction  during  confocal  microscopy,  such  as  frame  averaging. 
However,  it  is  also  possible  to  monitor  dynamic  events  by  time- 
lapse  confocal  microscopy,  and  this  combination  of  a  vital  fluo¬ 
rescent  reporter  with  high-resolution  optical  techniques  has  proven 
valuable  in  cell  biological  and  physiological  experiments.  We  have 
also  found  that  autofluorescent  chloroplasts,  normally  present  in 
the  upper  parts  of  the  plant,  and  certain  red  fluorescent  dyes  can 
provide  useful  counterfluors  for  GFP.  For  example,  propidium 
iodide  can  be  applied  to  live  seedlings  in  water,  to  specifically  label 
root  cell  walls,  and  allow  accurate  identification  of  GFP  express¬ 
ing  cells  [Fig.  44.3(F)]. 

It  is  now  possible  to  genetically  mark  cells  or  subcellular  com¬ 
partments  within  a  living  organism  using  GFP  and  to  visualize 
these  directly  during  development.  A  number  of  collections  of 
transgenic  lines  have  been  developed  where  GFP  gene  expression 
has  been  targeted  to  particular  cell  types  or  where  GFP  protein 
fusions  have  been  used  to  decorate  cell  compartments  in  Ara¬ 
bidopsis.  For  example.  Cutler  and  colleagues  (2000)  have  pro¬ 
duced  a  library  of  transgenic  Arabidopsis  lines  that  express  random 
cDNA-GFP  fusions.  The  fluorescent  protein  is  targeted  to  various 
subcellular  compartments  in  these  lines,  and  they  provide  a  useful 
source  of  dynamic  markers  for  nuclei,  plastids,  different  mem¬ 
branes,  and  other  compartments  [Fig.  44.3(A,B)].  In  addition, 
enhancer  trap  strategies  have  been  used  to  direct  the  expression  of 
a  foreign  transcription  activator,  GAL4,  in  different  cell  types 
in  Arabidopsis  (Haseloff,  1999a, b).  The  GAL4  gene  was  inserted 
into  the  Arabidopsis  genome,  using  Agrobacterium  tumefaciens- 
mediated  transformation.  Expression  of  the  GAL4  gene  is  depen¬ 
dent  upon  the  presence  of  adjacent  genomic  enhancer  sequences, 
and  so  different  patterns  of  expression  were  generated.  The 
inserted  DNA  also  contains  a  GAL4-responsive  GFP  gene,  and 
patterns  of  GAL4  gene  expression  are  immediately  detectable, 
with  each  GAL4-expressing  cell  marked  by  green  fluorescence. 
These  lines  provide  a  valuable  set  of  markers,  where  particular 
cell  types  are  tagged  and  can  be  visualized  with  unprecedented 
ease  and  clarity  in  living  plants  [Figs.  44.2  and  44.3(C-E)].  The 
collections  of  cDNA  fusions  and  GAL4  enhancer  trap  lines  are 
available  through  the  Arabidopsis  stock  centers. 

A  particularly  exciting  field  has  also  emerged  with  the  suc¬ 
cessful  development  of  GFP-fusion  products  with  cytoskeleton 
associated  proteins,  which  enable  high  resolution  imaging  of 
both  microtubules  (Fig.  44.4)  and  actin  microfilaments 
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[Figs.  44.4(C,D)];  see  also  Chen  et  al.,  (2003);  and  Shaw  et  al, 
(2003).  Use  of  over-expressing  constructs  of  cytoskeleton  binding 
proteins  can,  however,  disrupt  the  delicate  balance  of  the  dynamic 
instability  that  usually  takes  place  during  the  polymerization  of  the 
cytoskeleton  under  physiological  conditions,  as  recently  demon¬ 
strated  for  talin  (Ketelaar  et  al,  2004).  Despite  their  obvious 
beauty  and  apparent  information,  these  images  and  their  spatial 
and  temporal  kinetics  should  be  interpreted  with  great  care. 

Staining  Plant  Tissues 

As  an  alternative  to  direct  imaging  of  live  specimens,  it  is  possible 
to  fix  and  stain  intact  plant  tissues  and  then  to  clear  the  material 
using  a  high-refractive-index  mounting  medium.  It  is  then  possi¬ 
ble  to  obtain  high-resolution  optical  sections  from  deep  within 
intact  tissues  using  objectives  corrected  for  oil  immersion. 

A  wide  variety  of  staining  techniques  have  been  adopted  for 
plant  specimens  over  the  last  150  years.  Perhaps  the  most  widely 
used  general  tissue  stains  are  Safranin  O  and  Haematoxylin.  These 
are  often  accompanied  by  the  use  of  counterstains  such  as  Fast 
Green,  Orange  G,  or  Alcian  Blue.  In  addition,  there  are  a  large 
variety  of  more  specific  staining  techniques  that  have  been  devel¬ 
oped  for  particular  plant  materials  and  organelles.  For  example, 
Feulgen  staining  has  been  used  for  specifically  labeling  DNA,  the 
periodic  acid-Schiff  reaction  can  be  used  to  label  carbohydrates, 
Aniline  Blue  for  labeling  callose,  Nile  Red  for  oil  bodies,  and 
Phloroglucinol  for  lignin.  A  multitude  of  published  protocols  are 
available.  An  excellent,  recently  published  source  of  procedures 
can  be  found  in  Ruzin  (1999). 

Interestingly,  many  of  the  synthetic  dyes  used  for  plant 
microtechnique  are  highly  fluorescent.  This  is  particularly  so  for 
the  red,  orange,  and  yellow  dyes  in  the  azine  (e.g.,  Safranin  O), 
acridine  (e.g.,  Acridine  Orange),  and  xanthene  (e.g.,  Rhodamine) 
families.  Thus,  many  classic  histological  techniques  unintention¬ 
ally  produce  specimens  that  are  intensely  fluorescent.  In  addition, 
aldehyde  fixation,  certain  mountants,  and  long-term  storage  of 
stained  preparations  can  result  in  tissue  fluorescence,  and  the  high 
concentration  of  stains  deposited  in  the  sections  can  lead  to 
metachromasia  (Mason,  2000).  In  our  hands,  it  is  rare  to  find 
stained  and  sectioned  botanical  material  that  is  not  highly  fluores¬ 
cent.  Current  confocal  microscopes  can  sometimes  allow  the  clean 
separation  of  different  fluorescent  emission  signals  and  the  bal¬ 
ancing  of  signal  levels  in  different  channels.  Thus,  fluorescent 
images  of  exceptional  clarity  and  vivid  color  can  be  easily  obtained 
(Fig.  44.5).  In  addition,  the  optical  sectioning  properties  of  the  con- 
focal  microscope  can  be  used  to  collect  clear  images  from  within 
thick  sections  and  whole  mounts. 

Clearing  Intact  Plant  Material 

The  three-dimensional  (3D)  anatomical  arrangements  of  plant  cells 
have  conventionally  been  observed  using  microtomy  techniques. 
However,  the  laborious  nature  of  thin  sectioning,  the  problem  of 
obtaining  the  desired  plane  of  section,  and  difficulty  of  obtaining 
a  complete  series  of  sections  has  limited  its  use  to  the  skilled  and 
patient.  Optical  sectioning  has  many  advantages  from  the  point  of 
view  of  speed  and  simplicity,  and  it  allows  software  reconstruc¬ 
tion  of  whole  mount  specimens,  assembled  from  a  series  of  z-axis 
images.  However,  it  is  difficult  to  observe  cellular  details  deep  in 
living  plant  tissue.  Any  light  penetrating  the  tissue  must  pass 
through  many  layers  of  cytoplasm,  watery  vacuoles,  and  highly 
refractile  cell  walls.  The  different  refractive  indices  of  the  mater¬ 
ial  contribute  to  spherical  aberration,  and  particulate  subcellular 


FIGURE  44.5.  Confocal  microscopy  of  Equisetum  arvense  spores.  Equisetum 
sporangiophore  tissue  was  fixed,  cleared  in  xylene,  embedded  in  paraffin,  sec¬ 
tioned  using  a  microtome,  and  stained  with  Safranin  O  and  Fast  Green  FCF.  A 
z-series  of  three-channel  color  images  were  collected  using  a  Leica  SP  confo¬ 
cal  microscope  with  laser  excitation  at  488  nm,  568  nm,  and  633  nm.  These 
were  visualized  using  a  maximum  intensity  projection  algorithm.  Spores  can 
be  seen  within  the  sporangiophore.  The  spores  are  surrounded  by  exosporial 
elaters,  and  lignified  cells  involved  in  sporangial  dehiscence  are  seen  (top 
right). 

matter  also  causes  light  scattering.  Various  techniques  have  been 
developed  in  order  to  produce  samples  with  glass-like  optical  prop¬ 
erties  and  to  maximize  image  quality.  Clearing  agents,  such  as 
xylene,  clove  oil,  cedar  oil,  and  chloral  hydrate  have  been  adopted 
and  combined  with  compatible  mountants  such  as  Canada  balsam 
and  Hoyer’s  solution.  All  of  these  reagents  have  a  high  refractive 
index  similar  to  that  of  glass  (-1.55).  As  a  result  of  these  clearing 
treatments,  tissue  sections  become  more  transparent,  greatly  reduc¬ 
ing  problems  with  light  scattering  and  spherical  aberration  as  long 
as  oil-immersion  lenses  are  used.  This  allows  high-resolution 
imaging  of  thin  sections.  When  the  same  techniques  are  applied  to 
thick  sections  or  whole  mounts,  the  results  of  clearing  are  even 
more  startling.  However,  the  stain  or  fluorochrome  is  generally  dis¬ 
tributed  throughout  the  cleared  tissue  and  details  deep  in  the 
sample  are  still  obscured  by  overlying  signal.  Here  the  confocal 
laser-scanning  microscope  proves  its  worth.  High-resolution 
optical  sections  can  be  collected  to  distances  of  greater  than 
200  pin  in  such  cleared  samples.  The  depth  of  image  collection  is 
limited  mainly  by  the  working  distance  of  available  high 
numerical-aperture  objectives.  The  application  of  classic,  highly 
fluorescent  stains  and  clearing  techniques  creates  a  new  field  of 
opportunities  for  modern  confocal  microscopy  and  computerized 
display  methods. 

Recently,  classic  botanical  methods  have  been  modified  to 
allow  intense  and  specific  staining  of  plant  cells  and  clearing  for 
3D  microscopy.  For  example.  Aniline  Blue  has  been  used  as  a  stain 
for  the  cell  contents  of  Arabidopsis  embryos  subjected  to  3D 
imaging  (Bougourd  et  al.,  2000).  More  recently,  plant  carbohy¬ 
drates  have  been  labeled  by  treatment  with  periodic  acid  to 
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FIGURE  44.6.  (A)  Deep  optical  sectioning 
of  cleared  Arabidopsis  thaliana  root  tissues. 
Mature  Arabidopsis  embryos  were  treated 
with  periodic  acid  to  produce  aldehyde 
groups  within  carbohydrates,  and  stained 
using  a  pseudo-Schiff  reaction  to  specifi¬ 
cally  label  the  cell  walls.  The  tissues  were 
mounted  in  a  chloral-hydrate-based  clearing 
agent  for  microscopy.  The  combination  of 
clearing  and  intense  staining  allows  deep 
optical  sectioning  of  an  entire  embryo 
cotyledon.  A  series  of  736  optical  sections 
were  obtained  to  span  the  cotyledon,  pro¬ 
ducing  a  dataset  with  a  depth  of  147  \iM.  (B) 
3D  segmentation  of  plant  cells.  A  series  of 
confocal  optical  sections,  corresponding  to  a 
segment  of  cotyledon  from  an  Arabidopsis 
embryo,  is  visualized  using  the  AMIRA 
orthogonal  slicing  routines.  The  AMIRA  3D 
segmentation  editor  was  used  to  seed  and 
label  particular  voxels  that  correspond  to 
chosen  plant  cells  within  the  confocal 
dataset.  The  use  of  a  specific  cell  wall  stain 
allows  easy  selection  of  the  internal  volumes 
of  individual  cells.  A  closed,  triangulated 
surface  could  then  be  formed  over  the 
selected  group  of  voxels,  using  a  marching- 
cubes  algorithm.  Rendered  cells  are  dis¬ 
played  at  the  correct  position  and  scale 
within  the  dataset,  to  build  an  accurate  rep¬ 
resentation  of  the  shapes,  arrangement,  and 
connectivity  of  cells  within  the  tissue. 


produce  aldehyde  groups  that  are  reacted  with  fluorescent  pseudo- 
Schiff  reagents.  If  fixed  plant  tissue  is  treated  in  this  way,  cell  walls 
(and  starch-containing  plastids,  if  present)  become  intensely  and 
covalently  labeled  with  the  fluorophore.  The  labeling  of  wall  mate¬ 
rial  produces  a  complete  outline  of  each  cell.  The  tissue  can  then 
be  directly  cleared  in  a  high-refractive-index  agent  containing 
chloral  hydrate,  and  mounted  in  Hoyer’s  solution  for  microscopy 
[Haseloff  and  Bougourd,  unpublished  results;  Fig.  44.6(A)].  The 
combination  of  high  levels  of  fluorescence  and  high  refractive 
index  mountant  allows  the  collection  of  extended  z-series  images 
at  very  fine  resolution  (0. 1-0.5  pM  steps),  using  a  confocal  aper¬ 
ture  close  to  1  Airy  unit,  and  without  fear  of  photobleaching  or 


signal  and  resolution  loss  due  to  spherical  aberration.  Imaging  to 
a  depth  of  around  200  pm  allows  simple  optical  sectioning 
throughout  an  entire  Arabidopsis  root  at  high  resolution.  In  fact, 
every  cell  within  a  mature  Arabidopsis  embryo  can  be  clearly 
visualized  (Fig.  44.6). 

3D  Reconstruction 

The  basic  features  of  a  plant’s  body  plan  are  established  during 
embryogenesis,  but  its  final  form  results  from  the  continued 
growth  of  meristems  and  the  formation  of  organs  throughout  its 
life,  often  in  a  modular  and  indeterminate  fashion.  Because  plant 
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cells  are  constrained  by  rigid  cell  walls  and  are  generally  non- 
motile,  there  is  the  clear  possibility  that  cell  fates  within  a  meri- 
stem  are  determined  by  lineage.  However,  evidence  from  plant 
chimera  and  wounding  studies  have  demonstrated  a  more  impor¬ 
tant  role  for  cell-cell  interactions  during  fate  determination.  It  is 
likely  that,  during  plant  development,  positional  information  is 
exchanged  between  cells,  and  that  the  fate  of  cells  within  a  devel¬ 
oping  tissue  is  determined  by  a  network  of  local  cellular  interac¬ 
tions.  In  order  to  dissect  such  a  network,  it  is  crucial  that  we  can 
clearly  map  individual  cells  and  their  neighbors  inside  intact  meri- 
stems  —  and  have  means  to  manipulate  them.  Thus,  the  cellular 
anatomy  of  plants  is  of  particular  relevance  to  the  understanding 
of  development  and  morphogenesis. 

Three-dimensional  visualization  techniques  similar  to  those 
used  in  medical  imaging  can  be  applied  to  confocal  datasets.  This 
involves  the  selection  and  labeling  of  particular  voxels  that  corre¬ 
spond  to  a  3D  object  of  interest.  Various  techniques  are  available 
for  selecting  volumetric  objects,  which  range  from  the  fully 
manual  to  automatic  tools  that  detect  volume  boundaries  through 
differences  in  local  intensity  or  texture  (see  Chapters  14  and  15, 
this  volume).  The  use  of  specific  staining  techniques  can  aid  the 
labeling  process.  For  example,  cell  wall  staining  produces  an 
outline  of  every  cell  in  the  tissue.  This  is  very  helpful  as  it  allows 
the  use  of  automatic  segmentation  tools  to  determine  the  interior 
volume  of  a  chosen  cell.  We  routinely  use  AMIRA,  a  general- 
purpose  physical  modeling  and  data  visualization  program 
(Mercury  Computer  Systems,  www.tgs.com),  for  our  3D  visual¬ 
ization  and  segmentation  [Fig.  44.6(B)].  The  software  provides 
an  interface  for  visualizing  large  multi-dimensional  confocal 


microscopy  datasets.  AMIRA  provides  a  very  useful  set  of 
input/output,  data  handling  and  visualization  modules,  and  allows 
software  routines  to  be  combined  in  a  modular  fashion.  This  allows 
specialized  3D  reconstruction  and  visualization  techniques  to  be 
applied  in  a  flexible  way  to  different  types  of  data,  including  con¬ 
focal  datasets.  In  addition,  a  developer’s  version,  AMIRADev, 
allows  the  incorporation  of  custom  visualization  techniques.  The 
program  provides  a  simple  interface,  sophisticated,  fast  visualiza¬ 
tion  routines,  is  affordable  and  robust,  and  is  suitable  for  both  high- 
end  PC  and  UNIX  hardware. 

3D  Segmentation 

High-resolution  confocal  datasets  can  be  rapidly  segmented  to 
allow  direct  visualization  of  cell  arrangements  within  intact  plant 
embryos  and  meristems.  The  large  data  files,  up  to  250MB  in  size, 
can  be  directly  rendered  to  allow  excavation  of  the  data,  produc¬ 
tion  of  sections  in  arbitrary  planes,  and  rendering  of  surface  fea¬ 
tures.  In  practice,  cells  are  generally  chosen  by  placing  a  “seed 
point”  manually  within  the  center  of  a  cell  in  the  confocal  dataset. 
This  seed  is  then  inflated  in  three  dimensions  to  find  the  cell  bound¬ 
aries  that  are  marked  by  a  high  staining  intensity  (Fig.  44.7).  The 
program  provides  a  segmentation  editor  for  this  purpose  [Fig. 
44.6(B)].  The  exterior  geometry  of  a  segmented  cell  can  then  be 
described  using  a  “marching  cubes”  algorithm,  which  if  needed 
can  be  converted  to  a  solid  geometry  for  the  computer-assisted 
milling  of  3D  models  or  for  finite  element  analysis  [Fig  44.8].  We 
can  now  routinely  reconstruct  the  cellular  structure  of  entire  meri¬ 
stems  for  various  experiments. 


FIGURE  44.7.  Balloon  model-based  segmentation  of  plant  cells.  A  deformable-mesh  algorithm  for  segmentation  was  implemented  in  C++  as  a  software  module 
inside  AMIRA.  The  module  allowed  the  interactive  placement  of  a  seed  mesh  within  the  dataset  (top  left  panel).  The  surface  was  then  inflated  using  a  discrete¬ 
time  physical  model.  Expansion  was  accompanied  by  adaptive  subdivision  of  the  surface  during  inflation,  and  ultimately  was  constrained  by  a  “force”  based  on 
the  image  intensity  gradient  vector.  Vertices  on  the  surface  are  attracted  to  intensity  maxima  in  the  data,  which  correspond  to  the  stained  boundaries  of  the  cell. 
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FIGURE  44.8.  (A)  Watershed  algorithm  for  segmentation. 
Code  from  the  National  Library  of  Medicine  Insight  Toolkit 
(www.itk.org)  was  adapted  to  provide  a  watershed  algorithm  for  the 
AMIRA  visualization  platform.  The  algorithm  was  applied  to  a  3D 
dataset  derived  from  the  root  primordium  of  an  Arabidopsis 
thaliana  embryo.  A  single,  root-cap  protodermal  initial  cell  from  the 
segmented  dataset  is  displayed,  and  its  different  shared  walls  are 
shown  color-coded.  Each  colored  segment  corresponds  to  a  con¬ 
nection  with  a  different  neighboring  cell  within  the  meristem.  (B) 
Graph  of  3D  connections  between  cells.  Using  the  dataset  shown  in 
(A),  an  adjacency  network  was  computed  and  cell-cell  contact  mea¬ 
sured.  This  data  was  represented  using  graph-generation  software 
(AT&T  Graphvis).  The  16  cells  making  contact  with  the  root 
cap-protodermal  initial  (RCP_initial)  are  shown  on  the  diagram, 
with  measurements  of  the  area  of  each  cell-cell  contact. 


There  are  many  opportunities  for  improved  analysis  of  this 
kind  of  3D  dataset.  For  example,  plant  cells  are  generally  convex 
and  simple  in  shape,  and  this  allows  the  use  of  a  more  robust 
model-based  segmentation  approach:  a  deformable  mesh  can  be 
placed  within  a  3D  dataset  at  chosen  seed  points  and  inflated 
(Mclnemey  and  Terzopoulous,  1996).  The  mesh  simulates  an 
elastic  surface  expanding  from  the  interior  of  a  cell.  The  surface 
evolves  through  a  discrete-time  physical  model  and  adaptively 
subdivides  to  fit  the  object  boundary  (Fig.  44.7;  Rudge  and 
Haseloff,  unpublished  results).  This  type  of  model-based  segmen¬ 
tation  is  much  less  sensitive  to  noise  in  the  experimental  data  and 
produces  a  compact  description  of  plant  cell  shapes  directly.  Other 
techniques  produce  an  intermediate  segmented  volume,  from 
which  a  surface  must  be  generated  and  smoothed. 

In  addition,  it  is  possible  to  automatically  obtain  a  measure¬ 
ment  of  the  number  and  area  of  shared  walls  between  cells  and 
their  neighbors.  These  values  are  highly  relevant  biologically  as 
they  correspond  to  shared  walls  that  contain  plasmodesmata. 


provide  conduits  for  informational  molecules  that  regulate  cell 
behavior,  and  are  an  important  parameter  for  modeling  approaches. 
The  watershed  algorithm  is  based  on  the  metaphor  of  water  catch¬ 
ment  basins  in  a  landscape.  First,  an  initial  classification  of  all 
points  into  catchment-basin  regions  is  done  by  tracing  each  point 
down  its  path  of  steepest  descent  to  a  local  minima.  The  confocal 
image  intensity  is  used  as  the  landscape  height.  This  process  gives 
ridges  between  cells  and  shows  slightly  uneven  terrain  in  the  cell 
interiors.  Next,  neighboring  regions  and  the  boundaries  between 
them  are  analyzed  according  to  minimum-boundary  height  to 
produce  a  tree  of  merges  among  adjacent  regions.  By  changing  the 
“flood  level,”  we  can  interactively  traverse  the  merge  tree  and  thus 
finely  tune  the  segmentation.  The  technique  produces  a  single 
boundary  between  cells  with  no  empty,  unclassified  intervening 
space.  We  can  use  this  property  to  compute  an  adjacency  network 
for  the  cells  and  measure  cell-cell  contact  areas.  This  can  then  be 
represented  diagrammatically  using  graph-generation  software 
(Fig.  44.8;  Rudge  and  Haseloff,  unpublished  results). 
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FIGURE  44.9.  Three-dimensional  reconstruction  of 
cell  arrangements  within  an  Arabidopsis  thaliana 
embryo.  (A)  An  8-cell  Arabidopsis  embryo  was 
treated  with  periodic  acid  and  stained,  using  propid- 
ium  iodide  as  a  pseudo-Schiff  reagent.  The  sample 
was  cleared  with  chloral  hydrate  and  mounted  in 
Hoyer’s  Solution.  A  z-series  of  confocal  optical  sec¬ 
tions  was  collected,  and  a  single  optical  section  from 
near  the  middle  of  the  3D  dataset  is  shown.  (B)  The 
3D  image  was  then  processed  using  a  deconvolution 
algorithm  (Huygens  Essential,  Scientific  Volume 
Imaging).  A  single  optical  section  is  shown  from 
the  processed  volume.  Deconvolution  results  in 
improved  image  detail  and  S/N.  (C)  Visualization  of 
the  deconvolved  3D  dataset,  using  the  Voltex  module 
in  AMIRA,  provides  direct  volume  rendering.  The 
stalk-like  embryo  and  some  surrounding  maternal 
tissues  can  be  seen.  (D)  Cells  in  the  lower  tier  of 
the  embryo  (yellow)  and  upper  cell  of  the  sus- 
pensor  (green)  have  been  segmented,  converted 
to  new  geometrical  surfaces,  and  visualized  within 
the  dataset.  The  cells  are  shown  within  a  semi¬ 
transparent  outline  of  the  embryo.  Extracted  cell 
geometries  can  be  used  to  define  regions  of  interest, 
or  co-visualized  with  the  original  microscopy  data. 


These  computer  visualization  methods,  which  are  adapted 
from  the  medical  imaging  field,  reduce  large  datasets  to  a  much 
more  compact  and  simple  description  of  the  3D  shapes  and  rela¬ 
tive  arrangement  of  cells  in  a  meristem  or  embryo  (Fig.  44.9). 
Because  cell-cell  signaling  plays  such  an  important  role  in  plant 
development,  these  techniques  show  much  promise  for  the  analy¬ 
sis  of  genetically  perturbed  plants,  and  as  a  basis  for  modeling  the 
cellular  interactions  themselves. 

TWO-PHOTON  EXCITATION:  ARE  TWO  BETTER 
THAN  ONE? 

Recently,  two-photon  excitation  (TPE)  fluorescence  microscopy 
has  become  a  common  tool  in  many  advanced  cell  biology  labo¬ 
ratories.  A  growing  body  of  literature  points  to  several  advantages 
of  TPE  over  other  fluorescence  imaging  methods,  with  improved 


signal-to-noise  ratios,  deep  penetration,  and  benefits  for  living-cell 
imaging  (Feijo  and  Moreno,  2004).  TPE  microscopy  is  still  in  an 
early  stage  of  development  and  reproducible  protocols,  probes,  and 
applications  remain  relatively  scarce.  However,  botanical  tech¬ 
niques  already  in  use  provide  clear  advantages.  In  particular,  TPE 
is  relatively  immune  to  the  presence  of  out-of-focus  absorbing 
structures.  This  section  covers  some  of  the  aspects  of  imaging  plant 
cells  using  TPE. 

Improved  Signal-to-Noise  Ratio  and 
Dynamic  Range 

Of  all  the  possible  advantages  of  TPE  microscopy,  the  one  we  feel 
most  confident  to  stress  is  the  high  signal-to-noise  ratio  (S/N)  and 
dynamic  range  of  the  final  signal.  Generally  speaking,  once  con¬ 
ditions  have  been  sufficiently  optimized  to  produce  a  good  image, 
TPE  provides  better  contrast,  crispness,  and  image  quality.  Numer- 
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ous  examples  of  this  rule  have  been  shown  elsewhere  (Feijo  and 
Moreno,  2004,  and  references  therein;  see  also  Chapter  21.  this 
volume).  Comparison  of  images  collected  using  single-  and  two- 
photon  excitation  showed  much  better  detail  and  signal-to-noise 
ratio  for  the  TPE  data,  especially  for  tangential  sections  at  the 
extremes  of  a  deep  z-stack.  On  such  sections,  TPE  resolves  without 
loss  of  dynamic  range,  whereas  confocal  microscopy  gives  blurry 
and  ill-defined  images  (Fig.  44.10).  Consequences  of  this  increase 
in  S/N  ratio  and  dynamic  range  become  even  more  evident  on 
deep/thick  or  whole-organ  imaging.  In  a  quantitative  comparison 
with  confocal  and  widefield  epi-fluorescence,  TPE  was  recently 
shown  to  outperform  the  other  methods  in  terms  of  effective 
resolution  (Cox  and  Sheppard,  2004). 

Imaging  Thick/Opaque  Specimens 

The  second  generally  accepted  TPE  advantage  is  its  capacity  to 
image  deep  into  tissues  that  usually  allow  poor  or  no  penetration 
in  confocal  microscopy.  This  trend  is  well  reflected  in  the  litera¬ 
ture  and  in  our  own  studies  on  whole-organ  imaging  of  Ara- 
bidopsis  (Feijo  and  Moreno,  2004).  Flowever,  it  should  be  pointed 
out  that  the  ability  to  image  whole  organs  or  thick  tissues  varies 
tremendously,  especially  in  plants.  Although  people  have  been 
able  to  image  >500  pm  into  living  brain  (M.  Vaz  Afonso  and  T. 
Bonhoeffer,  personal  communication),  brain  tissue  is  far  less 
opaque  than  most  plant  tissues.  As  a  result,  penetration  values  this 
large  are  very  uncommon  for  botanical  specimens  unless  they  have 
first  been  fixed  and  cleared. 

On  some  tissues,  we  could  not  penetrate  through  more  than  the 
epidermis  (e.g.,  Arabidopsis  living  styles),  apparently  because  it  is 
covered  with  an  outer  cuticle  that  is  so  opaque  that  deeper  pene¬ 
tration  requires  laser  power  levels  that  damage  the  tissue  (they  can 
literally  boil  the  epidermal  cells).  Other  tissues  (e.g.,  immature 
anthers  of  Agapanthus)  are  reasonably  transparent  and  allow  live 
imaging  up  to  the  full  working-distance  of  a  high-resolution  oil- 
immersion  lens  (ca.  200pm).  Roots  usually  allow  much  deeper 
imaging  than  leaves  or  other  green  tissues  [Fig.  44.10(A,B)].  Add 
a  low  level  of  stain  to  these  absorption  problems,  with  the  frame 
averaging  and  the  extra  excitation/fading  this  implies,  and  one  can 
see  that  any  estimate  on  how  deep  one  can  go  must  be  assessed  on 
a  case-by-case  basis. 

Several  direct  comparisons  of  confocal  and  TPE  microscopy 
are  offered  in  the  literature.  Vroom  and  colleagues  (1999),  working 
on  microbial  biofilms,  made  a  quantitative  comparison  of  signal 
intensity,  and  concluded  that,  compared  to  confocal,  they  were  able 
to  record  images  four  times  deeper  and  that  these  deep  images  did 
not  lose  contrast.  In  fixed  material  embedded  in  Nanoplast  resin, 
TPE  showed  improvements  both  in  penetration,  contrast,  and  fade 
resistance  (Decho  and  Kawaguchi.  1999).  A  practical  example  in 
which  this  improvement  led  to  novel  information  was  documented 
by  Meyer  and  Fricker  (2000).  While  studying  glutathione  distri¬ 
bution  in  different  tissues,  TPE  provided  more  detail  from  optical 
sections  deep  in  the  tissue  with  less  signal  attenuation,  and  this 
ability  was  pivotal  in  being  able  to  distinguish  vacuoles  from 
cytosol  and  to  get  a  better  separation  of  the  sequestered  signal. 

On  a  more  technical  level.  Sun  and  colleagues  (2001)  showed 
that  the  attenuation  of  the  excitation  signal  in  plant  tissues  is 
reduced  with  TPE.  More  significantly,  Cheng  and  colleagues 
(2001b)  made  probably  the  most  thorough  analysis  of  signal  atten¬ 
uation  as  a  function  of  the  excitation  wavelength  (see  Chapter  21, 
this  volume).  They  determined  that  in  mesophyll  cells  and  whole 
leaf,  while  major  attenuation  of  the  signal  occurs  only  below 
700  nm  (the  major  peak  of  attenuation  at  ca.  690  nm.  attributable 


to  chlorophyll  absorption),  the  attenuation  decreases  continuously 
up  until  about  1000  nm,  which  makes  it  clear  that  the  longer  the 
wavelength,  the  less  appreciable  will  be  the  attenuation  of  the 
excitation  signal. 

Better  dynamic  resolution  and  improved  penetration  are  easy 
to  demonstrate  in  a  number  of  Arabidopsis  organs.  Figure  44.10 
(A,B)  shows  an  application  to  whole,  living  roots,  in  which  the  cell 
walls  have  been  stained  with  propidium  iodide  (PI)  at  a  vital  con¬ 
centration.  Comparison  of  Figure  44.10(A)  (confocal)  with  Figure 
44.10(B)  (TPE)  makes  it  clear:  TPE  provides  high  contrast  and 
resolution  to  the  mid-sagittal  level  of  the  root,  in  both  yz-  and  xz- 
projections,  and  up  to  the  10th  cell  layer  (the  red  lines  indicate  the 
relative  position  of  the  zz'-projections).  This  comparison  is  espe¬ 
cially  noteworthy  as  exactly  the  same  specimen,  optics,  and  acqui¬ 
sition  protocol  were  used  to  obtain  both  images,  and  the  only 
changes  were  the  excitation  source  and  the  pinhole  aperture  ( 1  Airy 
unit  for  confocal  and  wide  open  for  TPE).  Because  roots  are  cylin¬ 
drical,  this  level  of  performance  makes  it  reasonable  to  speak  of 
the  high-resolution  characterization  of  an  entire  living  root.  In  con¬ 
trast,  under  the  very  same  imaging  conditions,  confocal  resolves 
no  better  than  five  cell  layers  (note  that  the  red  line  indicating  the 
projection  plane  is  not  even  over  the  central  stellar  tissues),  and 
the  contrast  and  definition  shown  on  the  negative-contrast  projec¬ 
tion  shows  a  clear  deterioration  of  the  signal  as  the  image  plane 
goes  deeper. 

In  other  organs,  such  as  living  leaves  of  Arabidopsis,  penetra¬ 
tion  is  much  more  limited  (Feijo  and  Moreno,  2004).  While  one  can 
appreciate  improved  dynamic  range  and  better  structural  accuracy 
(specially  with  external,  non-descanned  PMTs),  compared  to  con¬ 
focal,  TPE  penetration  was  still  barely  more  than  50  pm  (see  also 
Chapter  21,  this  volume).  The  conspicuous  presence  of  abundant 
chloroplasts  explains  this  result.  Not  only  is  penetration  impaired, 
but  specimen  damage  also  becomes  an  issue.  Complex  interactions 
among  all  the  pigments  present  can  generate  a  number  of  potentially 
harmful  products  that  may  make  TPE  even  more  damaging  than 
confocal  in  green  tissues  (see  next  section). 

We  have  imaged  living  anthers  at  power  levels  consistent  with 
structural  recording  of  dynamic  processes  (Feijo  and  Cox,  2001). 
Both  DNA  stained  with  4'-6-diamidino-2-pheny  I  indole  (DAPI) 
and  autofluorescence  from  the  cell  walls  contribute  to  the  observed 
fluorescence  at  780 nm  [Fig.  44.10(C,D)].  Under  TPE,  DAPI  fades 
very  little  and  produces  a  strong  signal  over  an  excitation  band¬ 
width  of  more  than  100 nm  (720-850nm).  In  confocal  microscopy, 
the  autofluorescence  of  the  inner  cell  layers  makes  it  difficult  to 
image  deeper  than  two  to  three  cells,  leaving  the  important  tapetum 
and  the  sporogenic/meiotic  cells  occluded.  However,  under  TPE, 
anthers  show  more  transparency,  allowing  sections  close  to  200 
pm  [Fig.  44.10(B);  6-7  cell  layers  deep],  which  is  the  working  dis¬ 
tance  limit  of  a  high-NA  immersion  lens  (in  this  case,  a  Nikon 
PlanApo,  60x,  NA  1.4,  oil).  Remarkably,  the  same  level  of  laser 
excitation  was  used  to  acquire  the  entire  z-stack  from  the  surface 
epidermis  to  the  central  sporogenic  tissue  [Fig.  44.10(A.B)].  With 
these  improvements,  an  exquisite  degree  of  contrast  and  detail  is 
obtained  in  images  of  the  pit  fields  and  wall  stress  fibers  in  tan¬ 
gential/oblique  optical  sections  of  intermediary  cell  walls  [Fig. 
44.10(A),  arrow].  The  ability  to  image  the  mid-sagittal  section  of 
an  anther  allows  direct  viewing  of  the  activities  of  meiotic  cells 
(Feijo  and  Cox,  2001). 

Fading,  Vital  Imaging,  and  Cell  Viability 

Two-photon  excitation  is  effectively  confined  to  a  sub-femtoliter 
volume  at  the  point  of  focus  within  the  sample,  and  although  it  is 
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generally  agreed  that  TPE  can  minimize  phototoxicity  and  fading 
during  microscopy  (Potter,  1996),  a  word  of  caution  is  due  in  many 
situations.  As  both  damage  and  signal  is  proportional  to  (at  least) 
the  square  of  the  power,  power  control  is  often  a  critical  issue. 
Fragile  tissues  or  those  that  dissipate  heat  poorly  may  pose  a 
problem.  For  example,  thin,  fixed,  sectioned  materials  (e.g.,  spec¬ 
imens  used  for  immunolabeling)  are  sometimes  much  more  diffi¬ 
cult  to  image  with  two-photon  than  using  widefield  or  confocal 
microscopy,  because,  in  the  focus  plane,  fading  occurs  more 
strongly  with  TPE.  Clearly,  TPE  is  not  a  cure-all. 

It  takes  some  effort  to  determine  the  best  experimental  condi¬ 
tions  for  vital  TPE  imaging,  and  it  is  particularly  important  to  find 
the  best  balance  between  power,  excitation  wavelength,  and  probe 
concentration  (when  controllable).  It  usually  implies  a  careful 
search  for  the  best  wavelength  (one  that  maximizes  S/N  while  min¬ 
imizing  photodamage)  and  then  setting  the  power  level  so  that 
there  is  just  enough  signal  to  form  an  image.  The  tuneability  of 
titanium :  sapphire  (Ti :  Sa)  laser  sources  allows  one  to  search  for 
the  optimum  excitation  wavelength,  and  this  often  leads  to  the 
choice  of  a  wavelength  away  from  the  peak  in  the  excitation  spec¬ 
trum.  Continuous  wave  laser  sources  used  for  confocal  microscopy 
are  generally  restricted  to  a  small  number  of  discrete  excitation 
wavelengths,  which  makes  this  sort  of  optimization  impossible.  As 
a  rule  of  thumb,  the  longer  the  wavelength  the  less  the  damage. 
Even  at  low  power,  using  wavelengths  less  than  800  nm  often 
causes  much  more  damage  in  the  form  of  fading,  arrest  of  stream¬ 
ing  or  even  slowing  growth.  A  second  rule  of  thumb  is  that  wave¬ 
lengths  longer  than  870  nm  are  almost  always  more  suitable  for 
imaging  plant  cells.  Finally,  dye  molecules  that  are  more  asym¬ 
metrical  (e.g.,  DAPI)  tend  to  show  better  results  than  symmetrical 
ones  [e.g.,  fluoroscein  isothiocyanate  (FITC)].  In  symmetrical 
molecules,  usually  more  than  one  excitation  peak  is  obtained,  and 
using  the  longer  wavelength  tends  to  produce  much  better  results 
in  terms  of  less  fading,  even  though  it  may  also  show  less 
emission. 

An  exhaustive  analysis  and  comparison  of  metabolic  imaging 
led  Fricker  and  Meyer  (2001)  to  conclude  that  TPE  might  be  the 
best  means  to  study  primary  metabolism  in  vivo,  if  and  when  the 
relevant  probes  are  produced  and  optimized.  Because  pollen  tubes 


show  easily  observable  signs  of  vitality  in  the  form  of  streaming 
and  growth,  we  have  used  them  to  assay  comparative  phototoxic¬ 
ity.  Being  relatively  resistant  to  radiation,  they  can  also  be  used  to 
studying  fading  (Feijo  and  Moreno,  2004).  When  viewing  sensitive 
features  such  as  actin  dynamics,  streaming,  growth  rate,  pump 
activity,  etc.,  TPE  always  seems  to  produce  superior  results  when 
operated  above  870  nm.  For  example,  expression  of  an  ADF :  GFP 
construct  that  labels  actin  microfilaments  in  pollen  tubes,  com¬ 
bined  with  TPE  allowed  the  collection  of  up  to  600  sequential 
frames  without  interval,  without  producing  noticeable  fading,  and 
with  minimal  effect  on  cell  growth  rates. 

Figure  44.10(E-N)  illustrates  another  example  of  a  result  dif¬ 
ficult  to  achieve  using  confocal  microscopy.  Phil  Benfey’s  labora¬ 
tory  produced  lines  of  Arabidopsis  with  GFP  (von  Arnim  et  al, 
1998)  fused  to  the  putative  transcription  factor  Shortroot  ( SHR ) 
under  the  control  of  its  own  promoter  and  these  were  then  imaged 
using  TPE  in  our  laboratory.  Again,  better  contrast,  dynamic  range, 
penetration,  and  sensitivity  were  obtained,  and  the  whole  root  was 
imaged  down  to  the  distal  side  of  the  central  stele  using  TPE  at 
920  nm.  Despite  the  weak  signal,  a  diffuse  cytosol  labeling  was 
discernible  in  the  central  stele  cells  and  labeling  in  the  nuclei  of 
the  endodermis  shows  a  perfectly  resolved  and  exclusive  perinu¬ 
clear  location  of  SHR.  Many  of  these  features  are  not  easily 
resolved  in  the  equivalent  confocal  images.  The  TPE  images 
helped  to  support  the  exciting  finding  that  transcription  factors  may 
move  between  cell  layers  (Hake,  2001;  Nakajima  et  al.,  2001),  and 
SHR  was  assigned  a  role  in  defining  cell  fate  in  the  root  (Naka¬ 
jima  et  al.,  2001).  The  sequence  in  Figure  44.10(E-N)  shows 
frames  of  a  division  of  a  root  meristematic  cell  (arrow),  in  which 
the  putative  transcription  factor  diffuses  out  during  nuclear  enve¬ 
lope  breakdown  and  later  re-aggregates  in  the  two  daughter  nuclei. 
Using  standard  confocal,  the  signal  faded  before  all  the  data 
could  be  collected  because  the  extremely  low  fluorescence  levels 
required  a  number  of  frames  to  be  averaged  to  produce  each  high- 
definition  optical  section  (G.  Senna,  2002,  personal  communica¬ 
tion).  In  the  TPE  results  shown  here,  not  only  was  fading  reduced 
to  a  manageable  level,  but  the  radiation  seems  not  to  have 
affected  either  the  division  cycle  or  root  growth  to  any  measurable 
extent. 
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FIGURE  44.1 0.  Imaging  of  a  whole,  living  root  of  Arabidopsis,  vitally  stained  with  PI  (  lOpg/mL;  wild-type  root  at  ca.  8-12  days  after  germination)  with  con¬ 
focal  (A)  and  TPE  (B)  microscopy.  In  healthy  cells,  the  membrane  is  impermeable  to  PI,  rendering  the  cell  wall  fluorescent  and  clearly  defining  the  cell  bound¬ 
aries.  (A)  The  central  confocal  image  shows  an  optical  section  30  |lm  deep.  The  right  and  bottom  panels  show  negative  images  of  xz  and  yz  reconstructions  of 
the  whole  zz-stack  acquired  (confocal  excitation  using  the  three  visible-light  lines  of  the  Kr:  Ar-ion  laser,  emission  filter  HQ  598/40;  the  stack  was  acquired  with 
0.5  |Jm  steps).  The  lines  mark  the  relative  position  of  the  reconstructions  on  the  central  image,  or  the  depth  of  the  specific  plane  shown  on  the  side  panels.  Wall 
boundaries,  although  visible,  are  diffuse,  despite  generous  use  of  Kalman  averaging.  Three-dimensional  projections  accentuate  that  feature,  and  the  xz-image 
shows  little  contrast  and  resolution  beyond  4  to  5  cell  layers.  Thus,  it  is  only  possible  to  image  the  half-diameter  of  the  whole  root  up  to  about  100  pm  from  the 
tip  and  the  important  context  of  the  central  stele  is  lost.  Bar  =  50  pm.  Plan  Fluo  40x,  oil-immersion  objective  (NA  1.3).  However,  under  TPE  (B),  anthers  show 
more  transparency,  allowing  sections  close  to  200pm  [Fig.  44.5(B);  6-7  cell  layers  deep],  the  working  distance  limit  of  a  high-NA  immersion  lens.  (Reproduced 
from  Moreno  and  Feijo,  2004,  with  permission  from  Springer- Wien.)  (C)  Image  of  a  living  Agapanthus  umbelatus  anther,  stained  with  DAPI.  This  organ  is  very 
translucent,  permitting  imaging  up  to  the  working-distance  limit  of  a  high-NA  objective  (ca.  200  pm,  montage  from  surface  to  ca.  200  pm  is  shown  from  a  to  f). 
Nuclei  (C,  double  arrow),  cell  walls  (arrow),  and  plastids  (arrowhead)  all  show  some  autofluorescence.  (D)  A  tangential  optical  section  through  the  cell  wall  of 
the  third  to  the  fourth  cell  layer  reveals  pit  fields  and  cellulose  stress  directions  with  exquisite  detail.  Detailed  nuclear  structure  is  clearly  visible  up  to  the  seventh 
cell  layer,  eventually  reaching  the  central  meiocyte  tissue  (B,  arrow).  Imaged  with  a  Plan  Apo  60x,  oil-immersion  objective  (NA  1.4);  excitation,  780  nm;  no 
barrier  emission  filter,  internal  PMT.  Bars:  (C)  20pm;  (D)  50pm.  (Reproduced  from  Feijo  and  Cox,  2001,  with  permission  from  Elsevier.)  (E-N)  Time-lapse 
sequence  of  the  division  of  a  meristematic  cell  in  the  apex  of  a  lateral  root  of  A.  thaliana  that  is  expressing  a  fusion  protein  constructed  of  SHR  and  GFP.  The 
montage  was  extracted  from  a  sequence  acquired  with  a  rate  of  1  frame/5  min  over  almost  3  h.  During  that  period  the  root  grew  unaffected,  and  several  meri¬ 
stematic  and  endodermal  cells  divided.  In  this  sequence,  a  meristematic  cell  is  shown  throughout  the  cell  cycle:  the  SHR: GFP  fusion  protein  is  first  located  at 
the  periphery  of  the  nucleus  (E,  arrow),  but  diffuses  to  the  cytosol  as  soon  as  the  nuclear  membrane  disappears  (G).  At  this  stage,  the  fluorescence  seems  con¬ 
tinuous  with  the  central  stele  cells  (G-I).  Following  division  and  by  the  onset  of  cytokinesis  (J-L)  the  nuclei  are  again  discrete  and  SHR :  GFP  is  again  visible 
in  the  two  daughter  nuclei  (K-N).  (Excitation,  920nm;  emission,  530nm;  external  PMT.)  Bar  =  10pm.  (Reproduced  from  Feijo  and  Moreno,  2004,  with  per¬ 
mission  from  Springer- Wien.) 
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Again,  a  word  of  caution  should  be  issued.  Potential  pitfalls 
will  remain  until  the  right  conditions  to  image  specific  probes  in 
particular  tissues  are  better  defined.  Until  then,  any  study  should 
be  undertaken  systematically,  with  time  allowed  for  long  sessions 
in  which  the  procedures  can  be  optimized  for  a  particular  experi¬ 
mental  condition. 

There  are  also  cases  where  confocal  provides  the  faster  and 
more  reliable  path.  In  a  recent  case  reported  by  Reddy  and  col¬ 
leagues  (2004),  the  shoot  apical  meristem  (SAM)  was  imaged  in 
an  attempt  to  resolve  a  number  of  cell  and  organ-fate  issues 
(Running,  et  al.  1995).  A  number  of  transgenic  lines  with  markers 
of  cell  division  and  cell  fate  were  created,  and  methods  were  devel¬ 
oped  for  imaging  the  shoot  growth  on  the  same  confocal  plane  for 
over  a  week.  The  workers  were  able  to  image  deep  stacks  encom¬ 
passing  the  whole  meristem  down  to  the  first  primordia  every  4h 
(Reddy  et  al.,  2004).  Using  Zeiss  optics  and  confocal  scanner,  they 
made  extensive  comparisons  with  TPE  and  concluded  that  confo¬ 
cal  significantly  outperformed  TPE  in  terms  of  keeping  the  SAM 
growing  and  dividing  (Reddy,  personal  communication).  Being  a 
green  tissue,  these  results  come  as  no  great  surprise  because,  as  we 
discussed  in  the  autofluorescence  section,  the  complex  interaction 
between  pigments  and  the  high  levels  of  near-infrared  radiation 
makes  it  predictable  that  green  tissues  will  always  pose  a  problem 
for  TPE  imaging.  This  example  should  definitely  be  kept  in  mind 
when  deciding  on  the  amount  of  time  and  effort  one  might  need 
to  develop  a  viable  protocol  with  a  new  method. 

Two-Photon  Imaging  of  Plant  Cells 
and  Organelles 

To  date  there  is  a  growing  body  of  literature  showing  the  applica¬ 
tion  to  TPE  of  practically  all  of  the  common  dyes  and  labels  used 
for  visible,  one-photon  fluorescence.  Here  we  restrict  ourselves  to 
plant  applications,  plus  our  own  experience  managing  a  multi-user 
facility  (see  also  Table  II  in  Feijo  and  Moreno,  2004;  Chapter  17, 
this  volume).  Again,  most  information  regarding  the  use  of  differ¬ 
ent  dyes  with  TPE  should  be  considered  preliminary  and  depen¬ 
dent  on  the  specific  experimental  context  (dye,  vital/fixed  samples, 
microscope,  etc.).  The  available  information  should  be  used  as 
a  starting  point  for  the  process  of  fine-tuning.  Implicit  in  this 
statement  is  the  recognition  that  it  is  difficult  to  obtain  clear 
reproducible  spectra  for  TPE  excitation.  In  contrast  to  confocal 
microscopes  and  spectrofluorimeters,  in  which  power  can  be  kept 
within  close  tolerances  and  the  average  level  has  little  variation, 
TPE  depends  critically  on  the  peak  power,  which  changes  dra¬ 
matically  (i)  along  the  tuning  range  of  most  ultrafast  lasers,  (ii) 
with  position  inside  the  focal  volume,  and  even  more  so  (iii)  with 
the  pulse  width.  Generally  speaking,  longer  wavelengths  provide 
shorter  pulses,  hence  higher  peak  powers.  To  complicate  matters, 
the  average  output  power  of  a  Ti :  Sa  laser  peaks  at  around  800  nm, 
but  falls  off  rapidly  at  the  extremes  of  the  tuning  range.  As  most 
biology  laboratories  do  not  possess  (or  want  to!)  the  very  complex 
analytical  instruments  necessary  to  measure  TPE  cross-sections 
properly,  wavelength-optimization  routines  are  more  likely  to  be 
based  on  somewhat  crude  empirical  experiments.  Data  are  often 
hard  to  reproduce  directly  on  another  system  with  different  optics 
and  design,  but  the  experimental  parameters  are  often  close  enough 
to  indicate  a  range  for  optimization.  One  important  reference  that 
should  be  kept  in  mind  is  the  classic  work  of  Xu  and  Webb  (1996) 
listing  spectra  for  most  of  the  commonly  used  dyes.  [For  updates, 
check  the  largest  library  of  TPE  spectra  at  www.drbio.cornell.edu; 
a  small  list  of  commonly  used  dyes  is  also  given  in  Diaspro  and 
Robello  (2000).] 


The  ultraviolet  (UV)-excited  DNA  dyes  DAPI  and  Hoechst 
33342  are  TPE  dyes  par  excellence  and  it  is  of  no  surprise  that  one 
or  the  other  has  been  used  in  every  paper  published  so  far  to  illus¬ 
trate  the  TPE  principle  (e.g.,  Denk  et  al.,  1990;  Konig,  2000; 
Tirlapur  and  Konig,  2002).  In  plant  tissues  other  than  the  classic 
onion  root  system,  these  dyes  have  been  used  to  image  live  meiotic 
nuclei  in  anthers  (Feijo  and  Cox,  2001),  nuclear  distribution  during 
arbuscular  mycorhization  of  Aspergillus  nidans  and  tomato  (Bago 
et  al.,  1999)  and  deep  imaging  in  nanoplast-embedded  tissues 
(Decho  and  Kawaguchi,  1999).  The  popularity  of  these  dyes  is  due 
to  the  fact  that  they  are  extremely  bright  and  fade  resistant  under 
TPE,  and  that  they  can  be  excited  over  a  wide  range  of  wave¬ 
lengths  (720-850  nm  for  DAPI  in  our  system),  a  feature  that  facil¬ 
itates  double-  and  triple-labeling  as  well  as  searches  for  the  best 
vital  imaging  conditions. 

TPE  of  Nile  Red  has  been  used  to  follow  the  mobilization  of 
lipid  globules  in  fungal  hyphae  (Bago  et  al.,  2002).  Autofluores¬ 
cence  was  used  for  high-resolution  dynamic  analysis  of  chloroplast 
division  (Tirlapur  and  Konig,  2001)  and  to  study  the  cell  wall 
(Konig,  2000). 

Perhaps  one  of  the  most  interesting  and  informative  trends 
relates  to  TPE  visualization  of  glutathione  (GSH).  The  method  was 
introduced  to  plants  by  Fricker  and  colleagues  (2000),  and  it  uses 
the  dyes  monochlorobimane  (MCB)  and  monobromobinane 
(MBB),  which  complex  with  GSH  to  produce  glutathione-S-bimane 
(GSB).  Under  normal  confocal,  these  workers  determined  the  con¬ 
centration  of  GSH  to  be  2  to  3  m M  in  most  cell  types.  However, 
using  TPE,  they  were  able  to  obtain  more  detail  with  less  signal 
attenuation  deep  in  the  tissue  and  this  was  pivotal  in  distinguishing 
vacuoles  from  cytosol  to  get  a  better  separation  of  the  sequestered 
signal  (Meyer  and  Fricker,  2000).  Only  under  TPE  was  it  possible 
to  measure  that  GSH  concentration  in  trichoblasts  (2.7  ±  0.5  mM) 
was  significantly  different  from  that  in  atrichoblasts  (5.5  ±  0.8  mM). 
More  recently,  trichomes  were  shown  to  have  300x  more  GSH  than 
cells  from  the  basement  layer  and  the  epidermis  (Gutierrez- Alcala 
et  al,  2000).  These,  and  other  results,  support  the  conclusion  that 
TPE  is  currently  the  best  approach  to  studying  primary  metabolism 
in  vivo  (Fricker  and  Meyer,  2001;  Meyer  et  al,  2001). 

Two-Photon  Excitation  Imaging  of  Green 
Fluorescent  Protein 

TPE  is  well  suited  for  viewing  GFP  in  plants  (Xia  et  al.,  1999, 
Volkmer  et  al.,  2000),  and  indeed  GFP  is  easily  excitable  at  a  wide 
range  of  wavelengths  and  in  combination  with  other  dyes,  such  as 
propidium  iodide  [Fig.  44.3(F)].  Special  manipulation  of  the  GFP 
levels  have  been  used  to  raise  signals  above  the  wild-type  chloro¬ 
phyll  autofluorescence  by  using  “super”  promoters,  such  as 
35S35SAMV,  driving  especially  bright  GFPs,  such  as  S65T,  a 
combination  successfully  used  to  visualize  mitochondria  using 
TPE  with  good  S/N  even  in  green  tissues  (Kohler  et  al.,  1997). 
This  example  points  to  the  need  to  consider  the  different  kinds  of 
GFP  currently  available. 

The  first  one  introduced  was  the  wild-type  GFP  which,  while 
excitable  at  about  800  nm.  faded  rapidly,  perhaps  even  more  so 
than  with  confocal.  This  and  other  problems  led  to  the  develop¬ 
ment  of  several  enhanced  forms  of  GFP.  Here  we  offer  data  about 
some  of  the  commonly  used  forms:  Clontech’s  EGFP,  Jim 
Haseloff’s  mGFP  forms  (Siemering  et  al.  1996;  Haseloff  et  al., 
1997)  and  also  the  forms  engineered  by  Chiu  and  colleagues 
(1996)  and  by  von  Arnim  and  co-workers  (1998). 

Clontech’s  version  is  perhaps  currently  the  most  popular.  It 
shows  a  relatively  broad  TPE  excitation  peak  from  920  to  940  nm, 
but  can  hardly  be  visualized  below  900.  unless  under  very  strong 
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FIGURE  44.11.  Fast  dynamics  in  plant  cells  is  illustrated  by  the  movement  of  the  protein  inclusion  on  the  ER  of  the  hypocotyl  epidermis  on  enhancer  trap 
lines  with  GFP  fused  to  an  ER  motif.  In  this  specific  sequence,  TPE  at  870nm  was  used  to  stretch  the  limits  of  viability  of  this  kind  of  imaging.  The  images 
shown  were  extracted  from  a  time  series  of  a  consecutive  (no  interval)  sequence  of  acquisitions,  which  was  extended  up  to  650  frames.  Despite  the  amount  of 
exposure,  we  observed  no  effect  on  fading  or  streaming  rates. 


promoters  or  when  fused  to  abundant  and  stable  proteins.  Although 
these  wavelengths  are  near  the  edge  of  the  tuning  range  of  the 
Ti :  Sa  laser  and  produce  some  dye  fading,  they  still  produce 
better  images  than  with  confocal  microscopy  [Fig.  44.3(B)],  and 
even  very  faint  markers  such  as  membrane  transporters  can  be 
imaged  in  living  cells.  The  same  result  was  obtained  for  the  S65T 
mutation  described  by  von  Arnim  and  colleagues  (1998)  [Fig. 
44.10(E-N)]. 

Nevertheless,  we  have  found  that  the  two  other  enhanced-GFP 
versions  (Chiu  et  al.,  1996;  Siemering  et  al.,  1996)  are  even  better 
for  our  purposes.  First,  they  seem  to  behave  best  at  870  to  890  nm, 
a  much  more  convenient  wavelength  range  for  the  Ti :  Sa  laser,  and 
one  that  any  user  can  mode-lock.  Second,  if  reasonably  expressed, 
they  seem  to  experience  no  quantifiable  fading,  even  when  sequen¬ 
tially  imaged  (Fig.  44.11).  They  provide  high-resolution  signals, 
and  the  mGFP5-ER,  expressed  in  enhancer  trap  lines,  provides 
superb  material  for  structural  characterization,  either  alone  [Fig. 
44.3(D,E)]  or  in  combination  with  PI  [Fig.  44.3(F)].  Fine  details 
of  the  nuclear  envelope  and  of  the  ER-derived  system  are  resolved, 
and  the  exclusive  location  of  the  tag  in  the  endodermis  is  evident 
from  the  cell-edge  fading  that  can  be  seen  in  tangential  optical  sec¬ 
tions  of  the  root.  In  streaming  movements  of  the  cortical  ER,  large 
particles  have  been  followed  in  mGFP5-ER  enhancer  trap  lines 
during  many  hundred  consecutive  frames,  without  any  visible 
fading  or  quantifiable  effect  on  the  streaming  rates  and  patterns 
(Hawes  et  al.,  2001;  Fig.  44.11).  These  two  engineered  forms 
of  GFP  have  proven  to  be  exceptional  tools  for  TPE  in  plants, 
and  can  be  expected  to  become  the  source  of  many  important 
advances  in  our  understanding  of  dynamic  cell  and  developmen¬ 
tal  processes. 

DYNAMIC  IMAGING 

While  cell  mobility  in  plants  is  strongly  limited  by  the  presence  of 
a  semi-rigid  extracellular  matrix,  plant  cells  can  display  impres¬ 
sive  spatial  dynamics  at  the  subcellular  level.  Generally  speaking, 
cytoplasmic  streaming  is  much  faster  than  mechanical  movements 
in  animal  cells,  with  organelles  moving  up  to  2 pm/s  [e.g.,  ER 
inclusions  on  Arabidopsis  hypocotyl  epidermis  (Fig.  44.11)  or 
mitochondria  in  pollen  tubes  [(Fig.  44.12)].  The  easiest  way  to 


FIGURE  44.12.  Comparison  of  spinning-disk  and  TPE  imaging  effects  on 
pollen  tube  growth,  Rhodamine  123-labeled  mitochondria  in  Lilium  longiflo- 
rum  pollen  tubes  after  600  consecutive  acquisitions.  Frames  were  taken  con¬ 
secutively  without  any  interval  (1  frame/2  s)  in  the  case  of  the  TPE  (820  nm 
excitation  with  an  emission  at  598/40  nm)  at  a  speed  of  5/s  in  a  scanning  disk 
(488  nm  excitation  with  a  triple  dichroic).  Unlike  laser-scanning  confocal  (Feijo 
and  Moreno,  2004),  both  systems  show  a  very  low  phototoxicity.  Flowever,  in 
the  case  of  the  TPE  we  should  point  out  the  low  noise  picture,  and.  in  the  case 
of  the  spinning-disk,  the  fast  image  acquisition.  With  TPE,  the  growth  rate 
decreases  by  30%  compared  to  10%  with  the  NSDC. 
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image  such  a  fast  process  is  to  use  a  widefield  microscope  with  a 
scientific-grade  charge-coupled  device  (CCD)  camera,  and  apply 
a  cost-effective  algorithm  to  the  two-dimensional  (2D)  data,  such 
as  no-neighbors  2D  filtering  (Monck  et  al.,  1992;  McNally  et  al., 
1999).  This  method  is  a  similar  to  one  developed  by  Castelman 
(1979),  in  which  three  planes  are  acquired  and  the  information  of 
the  outer  planes  subtracted  to  produce  an  improved  image  of  struc¬ 
tures  present  in  the  central  plane. 

However,  more  stringent  applications  do  need  optical  section¬ 
ing.  In  many  cases,  deconvolution  is  not  fast  enough  to  provide 
enough  temporal  resolution  on  a  moving  sample,  and  other 
methods,  such  as  single-beam  laser-scanning  microscope  (LSM), 
TPE,  or  Nipkow  spinning-disk  confocal  (NSDC),  or  even  video¬ 
rate  confocal  (VRC)  must  be  used. 

Frame  size  is  the  next  critical  choice.  In  a  LSM/TPE,  one  can 
go  up  to  a  few  frames  per  second  by  choosing  a  box  of  only  ~10 
kilopixels,  usually  at  the  cost  of  proper  sampling  and,  therefore, 
resolution.  For  many  applications,  as  long  as  one  does  not  need  to 
go  faster  than  2  to  3  frames  per  second  (fps),  it  is  possible  to  find 
a  compromise  between  speed  and  resolution.  On  the  other  hand, 
if  speed  and  resolution  are  needed  simultaneously,  a  different 
approach  is  needed. 

NSDC  and  VRC  use  detectors  that  need  time  to  integrate:  thus, 
the  dimmer  the  sample,  the  more  time  needed  to  record  a  decent 
picture.  As  most  spinning-disk  confocals  use  cooled  CCD  cameras 
[or  the  new  intensified  CCDs  or  even  an  electron  multiplier  (EM)- 
CCDs],  one  can  adjust  the  exposure  time  and  the  binning  indepen¬ 
dently.  The  first  parameter  affects  acquisition  speed  and  the  second, 
resolution.  Sensitivity  is  an  issue,  and  exposures  as  long  as  several 
seconds  are  not  unusual  for  lightly  stained  specimens.  Binning 
2  x  2  in  a  megapixel,  cooled  CCD  improves  this  parameter  dra¬ 
matically.  Although  it  will  bring  the  effective  pixel  size  close  to  the 
Nyquist  sampling  criteria,  either  the  exposure  time  can  be  reduced 
to  roughly  one  quarter  or  the  S/N  increased  by  a  factor  of  2. 

Though  common  sense  suggests  that  plant  cells  should  be  able 
to  handle  lots  of  light,  this  is  not  always  true.  Because  their  growth 
rate  is  quite  sensitive  to  light,  pollen  tubes  can  be  used  to  quantify 
phototoxicity.  As  previously  mentioned,  as  long  as  a  reasonable 
amount  of  signal  is  present,  TPE  above  870  nm  produces  almost 
no  effects  on  the  growth  and  streaming  rates  of  pollen  tubes  [Fig. 
44.12(A)].  However,  even  when  imaging  continuously,  without 
interval,  good  resolution  still  implies  a  2s  frame-scan  time.  At  this 
speed,  much  of  the  dynamics  of  fast-moving  organelles,  such  as 
ER  (Fig.  44.11)  or  especially  streaming  mitochondria  (Fig.  44.12) 
will  be  lost. 

Provided  that  there  is  enough  signal  to  allow  use  of  a  fast  CCD 
(the  case  in  Fig.  44. 12,  stained  with  the  very  bright  Rhodamine  123), 
Nipkow-disk-based  systems,  such  as  the  PerkinElmer/Yokogawa, 
definitely  show  a  much  more  informative  view  [Fig.  44.12(B)]. 
In  this  case,  it  was  easy  to  achieve  an  increase  of  one  order  of  mag¬ 
nitude  in  the  time  resolution  (2  s/frame  on  TPE  and  200ms/frame 
with  the  NSDC)  and  this  allows  one  to  clearly  visualize  different 
cytoplasmic  domains  in  which  mitochondria  move  faster  or  slower. 
We  were  unable  to  record  similar  data  with  any  single-beam  scan¬ 
ning  method.  As  can  be  appreciated  by  comparing  Figure 
44.12(A,B),  although  the  optical  thickness  is  lower,  the  more  accu¬ 
rate  dynamic  picture  clearly  outweighs  the  information  lost  by 
lower  ^-resolution.  The  ability  to  record  a  z-stack  of  a  large,  intact 
specimen  in  a  fraction  of  the  time  also  makes  the  technique  suitable 
for  time-lapse  studies  of  fast-moving  or  growing  organs. 

We  expect  that  NSDC  microscopes  will  be  crucial  to  being 
able  to  resolve  fine  temporal  dynamics  with  minimal  fading.  The 
sensitivity  issue  does  exist,  and  lightly  stained  samples  such  as 


many  GFP-fusion  lines  could  either  not  be  imaged  at  all,  or 
required  an  integration  time  so  long  that  the  system  performance 
was  less  than  that  of  TPE.  Intensified  CCDs  or  EM-CCDs  may 
soon  change  this  picture  quite  dramatically,  and  bring  sensitivity 
to  the  level  of  TPE  and  LSM.  Although  the  new  generation  of 
VRCs  use  high-speed  computer  interfaces  with  a  bandwidth  of  100 
MB/s  that  permit  high  frame  rates,  problems  with  singlet-state  sat¬ 
uration  prevents  single-beam  confocal  scanners  from  providing 
useful  information  about  biological  specimens  except  at  low 
spatial  resolution.  On  the  other  hand,  high-speed  line-scanners 
such  as  the  Zeiss  5-Live  avoid  this  trap  by  scanning  many  points 
at  one  time  (see  Chapter  10,  this  volume). 

DECONVOLUTION 

In  the  last  two  decades  there  has  been  a  boom  in  the  use  of  3D 
microscopy  in  biology,  and  botanical  samples  are  no  exception 
(Hepler  and  Gunning,  1998).  Although  this  trend  has  been  driven 
largely  by  the  emergence  of  the  confocal  microscope,  widefield 
microscopes  can  also  generate  raw  3D  data,  and  software  that  can 
run  on  any  modern  personal  computer  is  now  available  to  extract 
sharp,  3D  reconstructions  from  these  data. 

If  one  considers  a  microscope  to  be  a  “convolution  machine,” 
its  transfer  function  can  be  determined  by  measuring  its  response 
to  a  subresolution  fluorescent  bead:  namely,  its  point  spread  func¬ 
tion  (PSF).  To  the  extent  that  one  knows  the  PSF  and  records  the 
widefield  3D  data  stack  accurately,1  one  should  be  able  to  invert 
the  convolution  process  (deconvolution)  to  obtain  a  3D  image  of 
the  object  imaged.  The  details  of  both  convolution  and  deconvo¬ 
lution  are  covered  in  detail  in  Chapters  23,  24,  and  25. 

Because  of  the  difficulties  encountered  when  trying  to  measure 
the  PSF,  we  usually  make  an  average  PSF  from  the  3D  images  of 
several  beads  (Holmes  et  al,  1995).  Although  the  biggest  draw¬ 
back  of  this  approach  is  the  requirement  that  the  sample  must  not 
move  or  change  during  data  acquisition  (restricting  the  technique 
to  fixed  cells  or  structures  moving  very  slowly),  even  minimal 
problems  with  the  optics,  such  as  misalignment,  spherical  aberra¬ 
tion,  and  dirtiness,  will  affect  the  final  image  to  a  much  greater 
extent  than  in  confocal  and  TPE.  To  some  degree,  it  may  be  pos¬ 
sible  to  compensate  for  the  degradation  of  the  PSF  brought  about 
by  spherical  aberration  (Boutet  de  Monvel  et  al.,  2001)  or  by  the 
heterogeneous  refraction  index  of  a  living  cell  (Kam  et  al.,  1998). 
As  these  techniques  improve,  they  may  be  important  when  decon¬ 
volution  techniques  are  applied  to  images  of  plant  cells  in  which, 
for  example,  spheroidal  central  vacuoles  can  act  as  miniature 
lenses.  Despite  these  limitations,  deconvolution  retains  some 
substantial  advantages  for  living-cell  microscopy:  phototoxicity, 
photobleaching,  and  price  are  all  often  lower. 

It  is  also  important  to  note  that  deconvolution  protocols  not 
only  can,  but  should,  be  applied  to  all  3D  confocal  and  TPE 
datasets.  Doing  so  not  only  substantially  reduces  the  “single-pixel 
noise  features”  produced  by  the  statistical  uncertainty  attendant  on 
measuring  small  numbers  of  photons,  but  it  also  substantially 
improves  the  statistical  accuracy  of  the  image  data  by  effectively 
averaging  the  signal  over  many  voxels  (see  Chapter  25,  this 
volume). 


1  The  main  practical  limitation  to  doing  this  is  the  presence  of  both  Poisson  and 
read  noise  in  the  final  data. 
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FIGURE  44.13.  The  effect  of  deconvolution  on  a  widefield 
stack.  (A)  A  plane  from  a  stack  of  a  unicellular  green  algae.  (B) 
A  deconvolved  version  of  the  same  plane  using  a  measured  PSF 
and  a  glycerol  immersion  objective  (PlanApo  63x  NA  1.3).  The 
whole  projection  was  about  30  pm  deep  (autofluorescence). 
The  improvement  in  contrast  is  obvious,  and  is  highlighted  in 
the  line  scan.  The  PSF  was  generated  using  orange  beads  from 
Molecular  probes  (PS-Speck)  with  170nm  diameter. 


Figure  44.13(A)  shows  an  autofluorescence  image  of  a  uni¬ 
cellular  green  alga.  In  this  specific  sample,  several  attempts  with 
both  confocal  and  multi-photon  microscopes  failed  to  produce  a 
sharp  image  over  extended  focus  stacks  of  the  entire  depth  of  the 
cell.  The  main  problem  was  that  the  high  intensity  illumination 
caused  the  signal  to  get  dimmer  and  the  autofluorescence  to  shift 
from  red  to  green  (Cheng  et  al.,  2001).  Deconvolution  of  data 
stacks  collected  using  a  much  lower  intensity  of  excitation  light 
made  a  considerable  improvement  [Fig.  44.13(B)]. 

CONCLUSION 

Ever  since  the  time  of  Hooke,  plant  cells  have  been  the  founda¬ 
tion  of  many  of  the  fundamental  discoveries  that  have  shaped  cell 
biology.  Underlying  all  these  findings  were  significant  advances  in 
microscopy  that  have  helped  to  push  forward  our  conceptual  think¬ 
ing,  for  example,  supporting  the  acceptance  of  the  cell  theory.  In 
this  chapter  we  have  highlighted  what  is  now  the  leading  edge  of 
this  technological  effort. 

Imaging  is  now  more  than  lenses  and  microscopes.  Comput¬ 
ers  are  also  essential,  both  to  accumulate  and  display  research 
images,  and  to  extract  and  analyze  enormous  amounts  of  quanti¬ 
tative  information  from  them. 

Recent  advances  range  from  major  hardware  (e.g.,  two-photon 
equipment)  to  the  development  of  computer  software  that  now 
enables  us  to  derive  fresh  insights  from  old  histological  techniques. 

We  have  tried  to  emphasize  that  all  these  techniques  are  only 
of  value  when  they  enable  us  to  describe  previously-unknown  bio¬ 
logical  features,  and  indeed  many  of  the  most  important  technical 
developments  were  developed  only  as  part  of  specific  research 
projects. 

The  ever-growing  field  of  genetically  encoded  probes,  such  as 
GFP,  seems  likely  to  trigger  more  new  and  important  technical  adap¬ 
tations  that  will  enable  us  to  obtain  more  and  better  dynamic  infor¬ 
mation  from  living  systems.  Plant  cell  biology  continues  to  bloom, 
and  much  of  this  growth  is  supported  by  modern  imaging  methods. 
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INTRODUCTION 

How  to  make  a  good  science 

La  luna  densa  The  moon  is  dense, 

Ogni  densa  e  grave  Everything  dense  is  substantial. 

Come  sta  la  luna  ?  How,  moon,  do  you  exist? 

Leonardo  da  Vinci  ( 1452-1519)} 

For  the  purpose  of  this  chapter,  a  free  paraphrasing  of  Leonardo’s 
point  might  be: 

Living  cells  are  moving. 

Movement  reflects  their  functional  activity. 

How,  cells,  do  you  function? 

After  formulating  this  philosophic  question  in  a  poetic  form, 
Leonardo  the  Scientist,  provides  us  with  a  real  experimental 
(optical)  setup.  “As  I  propose  to  treat  the  nature  of  the  moon,  it  is 
necessary  that  I  first  describe  the  perspective  of  mirrors,  whether 
plane,  concave,  or  convex,”  (B.M.94r  —  Arundel  MS  in  British 
Museum).  Next,  in  the  pages  of  Codex  Atlanticus  (C.A.190r), 
Leonardo  invites  us  to  “Construct  the  glasses  to  see  the  moon  mag¬ 
nified”  and  half  a  millennium  later  we  are  still  following  him  for, 
as  Bulgakov  famously  said,  “Manuscripts  do  not  burn!” 

As  a  master  of  light  and  shadow,  he  knew  how  the  uneven 
surface  of  the  ocean  reflects  light  in  all  directions,  breaking  it  into 
myriad  beams.  Analyzing  the  brightness  of  the  light  reflected  from 
the  moon,  Leonardo  suggests  that  its  outer  surface  might  be 
covered  with  a  rippled  liquid. 

As  a  scientist  he  was  able  to  understand  how  the  images  of  the 
moon’s  phases  would  appear  to  an  observer  on  the  Earth  [Fig. 
45.2(A),  arrow].  In  its  monthly  cycle  the  moon  always  exposes  the 
same  side  towards  the  Earth.  Only  when  it  is  opposite  to  the  Sun 
(with  respect  to  an  Earth  observer)  is  the  moon  fully  illuminated 
by  sunlight.  Leonardo  follows  his  interest  further  to  conclude  that 
when  the  moon  is  on  the  same  side  of  the  Earth  as  the  Sun,  it  will 
be  illuminated  by  brilliant  daylight  reflected  from  the  watery 
surface  of  the  Earth.  This  situation,  known  in  astronomy  as  lumen 
cinerium  or  “new  moon  in  the  old  moon’s  arms,”  was  described 
and  drawn  by  Leonardo  (see  Fig.  45.1).  Perhaps  it  was  this  under¬ 


1  Leonardo  —  Art  and  Science,  2000,  Crispino,  ed.,  Giunti  Editore  S.p.A.  a, 
Florence  Milan. 


standing  that  later  led  him  to  question  whether  the  Sun  really  does 
circle  around  the  Earth.  Much  later,  Johannes  Kepler  reached 
similar  conclusions  in  “Astronomiae  pars  optica.” 

The  scientific  prediction  of  Leonardo  (based  on  optics)  may 
have  met  “real  life”  when  Amerigo  Vespucci  described  the  New 
World  to  him,  possibly  during  the  painting  of  a  portrait  by 
Leonardo  that  is  mentioned  in  Lives  of  Artists  by  Giorgio  Vasari: 
“sketch  of  Amerigo  Vespucci  shows  head  of  a  very  handsome  old 
man  drawn  in  charcoal.” 

For  us  here,  what  Leonardo  discovered  is  less  important  than 
the  approach  he  used  to  experience  nature  and  to  obtain  new 
knowledge.  His  further  research  in  optics  led  him  to  conclude  that 
perspective  images  would  be  closer  to  reality  if  they  are  projected 
not  onto  a  flat  paper  but  onto  a  concave  spherical  surface,  as 
happens  in  the  human  eye.  In  Figure  45.2(B)  we  see  a  painter,  that 
might  be  Leonardo  himself,  working  with  perspectrograph  (from 
Codex  Atlanticus,  folie  5r).  Next,  Leonardo  concludes,  that  “the 
gradation  of  light  but  not  of  the  colors  is  what  defines  the  depth  in 
space”  (Manuscript  G,  8r).  Today,  in  modern  microscopy  we  still 
use  these  ideas:  DIC  images  code  shape  and  brightness,  color 
information  is  used  to  draw  attention  to  the  locations  of  transfected 
fluorescent  proteins  and  our  best  3D  rendering  algorithms  are 


FIGURE  45.1.  Leonardo  da  Vinci,  image  of  a  moon  phase  from  the  Codex 
Leicester:  35v,  2r  (2A). 
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FIGURE  45.2.  Leonardo  da  Vinci.  (A)  Schematic  drawing  of  the  light  reflected  from  the  moon  that  can  be  analyzed  by  an  observer  on  the  Earth:  Codex  Leices¬ 
ter  [drawn  between  1508-1510,  35v,  2r  (2A)].  (B)  Painter  working  with  perspectrograph,  Codex  Atlandcus,  folie  5r  (fragment).  (C)  Schematic  drawing  of  the 
laser  light  path  used  to  illuminate  the  object  in  an  inverted  microscope.  (Right)  Fluorescence  image  of  a  living  cell  obtained  by  this  setup,  inserted  DIC  image 
(for  details,  see  text).  Both  arrows  indicate  positions  of  the  observer’s  eye. 


designed  to  code  objects  that  are  farther  away  so  that  they  appear 
less  bright,  less  distinct  and  sometimes  even  bluish  [Fig.  45.2(C)]. 

Although  Leonardo  had  never  seen  a  microscope,  his  mind 
worked  with  the  precision  of  this  useful  device:  starting  from  a 
general  overview,  dissecting  and  moving  to  higher  magnifications 
where  he  captured  the  smallest  details  necessary  to  finally  under¬ 
stand  the  essence  of  the  entire  object  under  investigation. 

In  a  sense,  Leonardo’s  habit  of  portraying  biological  structures 
in  3D  foreshadows  our  modern  interest  in  3D  microscopy.  Not  yet 
trapped  in  the  2D  world  of  the  standard  light  microscope,  his  view 
was  inherently  three-dimensional.  He  knew  instinctively  that  3D 
insight  was  necessary  to  understand  how  biological  surfaces  must 


interdigitate  to  maximize  the  surface  area  of  the  adjoining  regions. 
Figure  45.3(A),  shows  the  interface  between  contacting  surfaces 
in  a  womb  wall  (RL  manuscript,  19102r,  detail).  His  3D  image  of 
these  surfaces  is  impressively  correct  and  in  agreement  with  our 
modern  understanding  of  cellular  contacts  on  a  much  smaller  scale: 
that  of  synapses  [Fig.45.3(B)]  and  cell-cell  contacts  [Fig.  45.3(C)], 
resolved  by  modem  microscopes  almost  500  years  later. 

His  way  of  following  “functionality”  and  performing  “prelim¬ 
inary  studies”  (that  might  take  him  many  years)  was  so  correct  that, 
even  in  the  21st  century,  his  approach  remains  as  important  for  us 
as  it  was  for  him.  Leonardo  was  a  great  philosopher  and  the  first 
one  to  integrate  the  four  most  productive  aspects  of  scientific 


FIGURE  45.3.  (A)  Taken  from  an  anatomical  study  of  Leonardo  da  Vinci  1508-15 10  (Manuscript  RL,  19102r,  Windsor  Library,  UK).  The  left  panel  shows  the 
interface  between  dissected  tissues  (wall  of  a  womb )  where  Leonardo  used  the  tools  available  to  him  to  analyze  how  two  biological  surfaces  create  an  adhesive 
junctional  interface.  (B)  An  insert  representing  junctional  folds  of  a  synaptic  terminal  that  creates  integrity  of  the  structure  (reproduced  with  permission  from 
Cohen-Cory,  2002).  (C)  Junctional  cell-cell  inerface  between  two  cultured  primary  astrocytes  revealed  with  antibodies  against  a  gap  junction  protein,  Connexin- 
43  (Butkevich  et  al.,  2004). 
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thinking:  (1)  Follow  your  naive  interest;  (2)  Open  your  mind,  get 
new  ideas  and  try  to  understand  how  things  work;  (3)  Have  the 
patience  to  develop  new  and  versatile  experimental  approaches  to 
get  new  information;  (4)  Test  the  new  knowledge  against  your 
wider  experience. 

We  too  are  at  the  beginning  of  a  new  era  in  biology  and 
biotechnology,  with  neurobiology  and  stem  cell  research  leading 
to  prominence.  Although  our  tools  are  now  molecular  imaging,  3D 
fluorescence  microscopy,  proteomics,  genetic  screens,  spec¬ 
troscopy,  nanobiophysics,  animal  models,  and  cellular  physiology, 
the  overall  process  remains  much  the  same. 

Beauty,  Functionality,  Cell  Cycle,  and 
Living-Cell  Imaging 

Rule  1:  Get  To  Know  Your  Cells 

Before  moving  on  to  the  methods  of  fluorescence  resonance  energy 
transfer  (FRET)  analyses,  it  is  important  to  include  a  reminder 
about  the  main  object  of  our  investigations:  living  cells.  Living 
cells  imaged  in  the  microscope  allow  one  to  obtain  insight  into  the 
complexity  of  dynamic  processes  in  real  time,  whereas  fixed  cells 
maintain  only  the  last  steady  state  distribution  (at  the  moment  of 
death)  for  the  molecules  of  interest.  For  live-cell  FRET  analyses, 
it  is  crucial  to  keep  cells  in  good  condition  during  the  time  course 
of  an  experiment.  This  includes  temperature  and  C02  control  of 
the  medium,  proper  transfection  of  the  cells  with  fluorescent  pro¬ 
teins  tested  for  functional  activity,  minimal  level  of  laser  illumi¬ 
nation  throughout  the  experiment,  etc. 

The  beauty  of  transfected  cells  may  often  serve  as  an  indicator 
of  their  biological  functionality,  and  thus  can  be  used  as  an  (almost) 
scientific  criterion.  In  a  sense,  beauty  is  a  biological  functionality. 
For  example,  in  cells  transfected  with  yellow  fluorescent  protein 
(YFP)-tubulin  to  highlight  the  microtubule  network,  microtubules 
are  impressively  beautiful  when  the  cells  are  taken  straight  from 
the  incubator  and  recorded  at  37°C.  Cells  forgotten  on  the  table 
without  temperature  control  or  adequate  C02  will  show  partially 
depolymerized,  ugly  microtubules.  In  such  cells,  transport  of  pro¬ 
teins  in  the  secretory  and  endocytic  pathways  is  impaired,  and  the 
cells  are  dying.  Investing  hard  work  into  badly  transfected  cells  or 
attempting  functional  studies  on  dying  cells  is  a  wasted  effort 
(except,  of  course,  when  studying  apoptosis).  In  contrast,  beauti¬ 
fully  transfected  cells  are  a  source  of  a  great  happiness  to  the 
researcher,  stimulating  him/her  to  acquire  the  best  possible  images. 

Figure  45.4  shows  two  examples  of  cells  after  transfection: 
Figure  45.4(A)  shows  a  cell  expressing  a  physiological  level  of  the 
transmembrane  KDEL  receptor  (ERD2)  that  is  recognized  by 
cholera  toxin  to  enable  toxin  transport  through  the  Golgi  complex 
(Majoul  et  al.,  1996,  2001).  Figure  45.4(B)  shows  a  cell  imaged 
at  longer  times  after  transfection  and  that  is  now  over-expressing 
KDEL  receptor,  which  aggregates  around  the  Golgi.  Cholera  toxin 
recognizes  and  binds  to  functional  KDEL  receptor  molecules  [Fig. 
45.4(A)],  but  does  not  enter  the  over-expressing  structures  [Fig. 
45.4(B)].  Instead,  cholera  toxin  recognizes  native,  still  functional 
receptors  between  the  aggregates  and  thus  enters  the  Golgi  [Fig. 
45.4(B),  arrow].  Clearly,  the  cells  in  Figure  45.4(B)  should  not  be 
used  for  experiments.  The  transfection  setup,  transfection  reagents, 
cells,  and  possibly  even  the  chimeric  proteins  must  be  further  opti¬ 
mized  to  prepare  a  better  experiment. 

Because  many  biological  processes  are  cell  cycle-dependent, 
the  cell  cycle  stage  should  be  considered  for  FRET  experiments. 
Figure  45.4(C)  shows  different  degrees  of  surface  expression 
for  a  glycosylphosphatidylinositol  (GPI).  GPI-anchored  protein  is 


detected  with  antibodies  that  make  FRET  under  the  plasma  mem¬ 
brane.  Acceptor  bleach  revealed  different  degrees  of  FRET, 
showing  that  the  level  of  GPI-anchored  protein  expression  varies 
with  the  stage  of  the  cell  cycle. 

We  have  also  shown  that  the  appearance  of  lipid  receptors  (gly- 
cosphingolipids)  on  the  cell  surface,  and  thus  the  lipid  composi¬ 
tion  of  the  plasma  membrane  as  well  as  the  distribution  of  certain 
lipid-binding  proteins  under  the  plasma  membrane,  is  strongly  cell 
cycle-dependent  (Majoul  et  al.,  2002a).  Surface  expression  of 
lipid  receptors  can  be  mapped  to  a  specific  cell  cycle  stage  using 
incorporation  of  BrDU  to  mark  S  phase  [Fig.  45.4(D)]  or  by  mon¬ 
itoring  the  expression  of  cyclins,  for  example,  cyclin  B  (mitotic 
marker).  To  map  different  lipid  microdomains  on  the  plasma  mem¬ 
brane,  we  used  fluorescently  labeled  AB5  toxins  (cholera  toxin  and 
shiga  toxin)  and  found  a  strong  cell  cycle-dependent  heterogene¬ 
ity  of  binding  to  cell  surfaces  [Fig.  45.4(E)].  The  molecular  basis 
of  this  mechanism  is  an  intracellular  lipid  modification  that 
involves  modification  of  a  common  lipid  precursor  (e.g.,  lactosyl 
ceramide)  with  sugar  moieties  by  galactosyl-  and  glucosyltrans- 
ferases  and  the  expression  of  these  enzymes  turned  out  to  be  cell 
cycle-dependent  [Fig.  45.4(F);  Majoul  et  al.,  2002a].  Thus,  the 
behavior  of  many  analyzed  proteins  will  be  different  in  the  Gl,  S, 
G2,  or  M  phases  of  the  cell  cycle.  The  take-home  message  here  is: 
if  a  strong  FRET  heterogeneity  is  observed  in  a  cell  population,  it 
may  be  useful  to  synchronize  cells  and  to  test  directly  whether  this 
heterogeneity  is  cell  cycle-dependent. 

FLUORESCENCE  RESONANCE  ENERGY 
TRANSFER  THEORY 

More  than  50  years  ago,  the  German  scientist  Forster  discovered 
that  close  proximity  of  two  chromophores  changes  their  spectral 
properties  in  predictable  ways  (Forster,  1948a).  If  two  biological 
molecules  in  a  living  cell  interact  functionally,  the  interaction  will 
bring  into  close  proximity  any  chromophores  attached  to  these 
molecules.  After  collecting  spectral  data  and  deciphering  this 
information  based  on  Forster’s  theory,  we  can  understand  changes 
that  occurred  in  the  near-vicinity  microenvironment  of  two  chro¬ 
mophores  and  thus  draw  biologically  meaningful  conclusions 
about  the  specificity  of  protein-protein  interactions. 

The  idea  of  non-radiative  dipole-dipole  interactions  was  first 
formulated  by  Forster  in  1948  (Forster,  1948a, b).  Further  devel¬ 
opment  of  this  idea,  biological  application,  and  introduction  of 
spectrally  distinct  fluorescent  molecules  came  later  (Stryer  and 
Haugland,  1967;  Clegg,  1992;  Tsien,  1998;  Lacowicz,  1999). 
Forster’s  theory  explains  FRET  as  a  dipole-dipole  interaction 
between  neighboring  molecules  (Fig.  45.5).  Forster  showed  that 
the  presence  of  an  appropriate  acceptor  in  FRET  proximity  will 
decrease  the  time  a  donor  spends  in  the  excited  state  due  to  non- 
radiative  energy  transfer  to  the  acceptor  in  close  proximity: 

K 

D  +  A  +  hvE=i>D*+A^>D  +  A* 

D  A  (1) 

Here,  kt  is  the  rate  of  energy  transfer  from  the  donor  to  the  accep¬ 
tor,  kD  =  kD f  +  kDi  and  kA  =  kA  +  kAi,  kD  and  kA  correspond,  respec¬ 
tively,  to  the  rate  of  the  radiative  decay  of  the  donor  and  the 
acceptor  (for  us  here,  fluorescence  emission),  and  kDi  and  kAi  to 
the  rates  of  the  non-radiative  decay  constant  of  the  donor  (D)  and 
the  acceptor  (A).  The  ratio  between  the  number  of  photons  emitted 
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FIGURE  45.4.  (A)  Cells  transfected  with  a  transmembrane  Golgi  receptor  (ERD2-CFP/ERD2-YFP).  At  near-physiological  expression  levels,  it  is  localized  in 
the  Golgi  where  it  binds  to  cholera  toxin  (red).  (B)  When  over-expressed,  the  receptor  aggregates  and  is  excluded  from  the  Golgi.  In  this  case,  cholera  toxin  still 
recognizes  a  portion  of  the  native  receptor  ( shown  by  arrows)  and  can  thus  enter  the  Golgi  compartment.  (C)  FRET  demonstrated  by  acceptor  bleach  on  the 
surface  of  a  cell  monolayer  labeled  with  antibodies  against  GPI-anchored  proteins  and  5'  NT.  The  cell-to-cell  variation  in  FRET  (Eeff)  distribution  of  5'  NT  mol¬ 
ecules  on  the  cell  surface  may  have  deep  biological  roots.  (Taken  from  Kenworthy  and  Edidin,  1998.)  (D)  Incorporation  of  BrDU  can  be  used  to  determine  the 
stage  of  the  cell  cycle  in  individual  cells  in  a  non-synchronized  cell  monolayer.  Only  cells  carrying  out  DNA  synthesis  (S  phase)  will  be  labeled.  (E)  Overlay 
of  the  same  monolayer  with  labeled  CTX  (cholera  toxin,  used  as  a  lipid  raft  marker)  and  ST  (Shiga  toxin)  revealed  that  cells  in  S  phase  have  weak  binding  of 
both  toxins.  Cells  in  G1  phase  express  more  GM1  on  the  cell  surface  and  bind  CTX  more  strongly  (shown  in  green).  Gb3-bound  ST  is  shown  in  red.  (F)  Cartoon 
representing  cell  cycle-specific  modifications  of  a  common  lipid  precursor  lactosyl  ceramide  that  can  be  modified  by  glycosyl  and  galactosyltransferases  to 
produce  more  branched  GM1  (G1  phase)  or  Gb3  (around  M  phase).  Thus,  differential  expression  of  lipid  receptors  on  the  plasma  membrane  can  be  a  marker 
for  the  phase  of  the  cell  cycle. 
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FIGURE  45.5.  FRET  theory.  (Upper  panel)  Proximity  and  spatial  orientation 
between  the  donor  (D)  and  the  acceptor  (A)  pair.  An  increase  in  the  Forster 
radius  between  donor  and  acceptor  (RD A)  will  lead  to  a  decrease  in  efficiency 
of  energy  transfer.  Mutual  orientation  (angles  0D,  0A,  0T)  between  the  donor 
and  acceptor  molecules  influence  the  transfer  efficiency  EtS.  (Lower  panel) 
Jablonski  diagram  for  single-  and  two-photon  excitations  of  the  donor  may 
result  either  in  the  emission  of  the  donor  (CFP  =  474  nm)  or  (in  case  of  a 
close  proximity)  to  the  non-radiative  dipole-dipole  transfer  of  energy  ( kt )  to 
the  acceptor  molecule  followed  by  the  Stokes  shift  and  the  emission  of  fluo¬ 
rescence  by  the  acceptor  molecule  at  a  longer  wavelength  (YFP,  =  526  nm). 


to  the  number  of  photons  absorbed  by  the  donor  or  the  acceptor  is 
known  as  quantum  yield  of  the  donor  (2d)  or  of  the  acceptor  ( QA). 
The  lower  the  quantum  yield  of  the  donor  (2d),  the  more  photon 
energy  will  undergo  either  non-radiative  decay  kDi  compared  to  the 
radiative  kD  fluorescence  emission  of  the  donor  or  possible  FRET 
(with  kt,  the  rate  of  FRET  transfer).  The  quantum  yield  of  the  donor 
in  the  absence  of  an  acceptor  is  represented  as: 


2d  —  kol{kD  +  ka)  (2) 

Thus,  the  quantum  yield  of  the  donor  in  the  presence  of  the  accep¬ 
tor  will  be: 


2da  —  hJ(kt  +  kD  +  £Di)  (3) 

The  fraction  of  the  photon  energy  absorbed  by  the  donor  that  can 
be  transferred  without  radiation  to  an  acceptor  represents  the  effi¬ 
ciency  of  energy  transfer,  Eefs.  The  relationship  between  EcB  and 
the  quantum  yield  of  the  donor  in  the  presence  and  absence  of 
acceptor  can  be  described  as: 

2da/  2d  +  Fell  =  1  (4) 

where  2da/2d  is  the  efficiency  of  the  radiative  process  that,  added 
to  the  efficiency  of  the  non-radiative  energy  transfer  (FRET) 
equals  1.  EeS  can  be  also  described  in  terms  of  k„  kD,  kDi  by  sub¬ 
stituting  of  Eqs.  2  and  3  into  Eq.  4: 

Et ff  =  1  -  2da/2d  =  1  ~  fc  +  kDi)/(kt  +  kD  +  kDt)  (5) 
kt/kt  +  kD  +  kDi  (5a) 

To  perform  practical  FRET  measurements,  one  needs  to  keep 
in  mind  that  the  efficiency  of  energy  transfer  (EeS)  to  the  acceptor 
will  be  strongly  decreased  by  an  increase  in  both  radiative  ( kD )  and 
non-radiative  (kDi)  decay  of  the  donor.  FRET  efficiency  is  a  direct 
measure  of  the  photon  energy  absorbed  by  the  donor  and  trans¬ 
ferred  to  an  acceptor  (Figs.  45.5  and  45.6).  EeS  depends  strongly 
on  the  actual  proximity  R  between  the  donor  (D)  and  the  acceptor 
(A)  molecules  and  on  their  mutual  orientation  (Fig.  45.5).  For  effi¬ 
cient  FRET  to  occur,  the  distance  ( R )  between  the  excited  donor 
and  the  fluorescent  acceptor  needs  to  be  typically  2  to  7  nm 
(Forster,  1948a,b;  Stryer,  1978;  Lakowicz,  1999;  Patterson  et  al., 
2000).  The  dependence  of  Edf  on  R  was  described  by  Forster  as: 

Ecs=  1/[1  +  (R/R0)6]  (6) 

The  distance  between  the  donor  and  the  acceptor  for  which  E eff 
=  0.5  is  called  Ra  and  reflects  the  properties  of  a  particular  D/A 
pair.  It  is  a  function  of  the  quantum  yield  of  the  donor  (2d),  inte¬ 
gral  spectral  overlap  between  the  donor  emission  and  the  acceptor 
excitation,  JDA  and  a  factor  reflecting  their  relative  spatial  orienta¬ 
tion  (k2)  (Figs.  45.5  and  45.6). 

If  K  is  known  and  fixed,  then  the  FRET  efficiency  between  two 
chromophores  can  be  taken  as  a  direct  measure  of  R,  using 
Forster’s  conversion  formulae  (Eq.  6).  Quantitative  FRET  analy¬ 
ses  of  different  subcellular  compartments  in  a  living  cell  requires 
that  we  know  the  real  value  of  the  spatial  orientation  factor  (k2). 
Figure  45.5  describes  the  common  arbitrary  case  and  two  extreme 
cases:  maximal  FRET  to  no  FRET  based  on  orientation  factor  (k2). 
A  Jablonsky  diagram  for  single-  and  two-photon  excitation  in 
FRET  [cyan  fluorescent  protein  (CFP)  donor,  YFP  acceptor]  is 
shown  below  (Fig.  45.5).  Our  preliminary  data  revealed  that  the 
same  chromophores,  CFP  and  YFP,  measured  with  the  identical 
microscope  will  display  different  levels  of  FRET  when  attached  to 
proteins  from  the  different  subcellular  compartments.  Two  trans¬ 
membrane  Golgi  donor  and  acceptor  proteins  (i.e.,  donor  and 
acceptor  that  can  oligomerize  only  by  diffusing  laterally  in  the 
plane  of  the  Golgi  membrane)  display  less  FRET  than  the  combi¬ 
nation  of  the  transmembrane  acceptor- YFP  and  a  free  cytosolic 
donor-CFP  that  interact  with  each  other  (as  shown  in  Fig.  45.8).  It 
is  clear  that  the  rotational  mobility,  and  thus  relative  spatial  orien¬ 
tation  factor  (K2),  can  be  higher  for  the  cytosolic  proteins  than  for 
the  laterally  fixed  transmembrane  fluorescent  proteins. 

Because  in  most  experiments  the  actual  orientation  is  difficult 
to  determine,  it  is  common  to  assume  a  random  orientation  of  the 
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FIGURE  45.6.  Spectra  illustrating  the  FRET  effect.  Spectro- 
fluorimetry  allows  one  to  follow  the  time-dependent  appear¬ 
ance  of  FRET  (left).  The  decrease  in  donor  fluorescence  ( long 
arrow )  correlates  with  the  increase  in  sensitized  emission  of 
the  acceptor  {short  arrows ).  Spectral  overlap  (/DA)  between  the 
emission  spectrum  of  the  donor  (CFP)  and  the  excitation  spec¬ 
trum  of  the  acceptor  (YFP)  ensures  that  FRET  can  potentially 
occur  between  a  selected  pair  of  biological  molecules  (e.g., 
proteins  X  and  Y  of  interest),  if  they  functionally  interact  in 
living  cells  (upper  right  comer).  Formulae  used  for  R0  calcula¬ 
tions  ( R0  corresponds  to  EeS  =  50%  of  a  maximal).  For  a 
detailed  description  see  the  FRET  Theory  section  in  the  text. 
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green  fluorescent  proteins  (GFPs)  even  when  they  are  fused  to 
macromolecules.  This  uncertainty,  plus  the  uncertainty  in  chro- 
mophore  location  within  the  entire  chimeric  protein,  means  that 
FRET  with  GFP  variants  is  more  suited  to  detecting  changes  in 
conformation  or  percentages  of  association  than  to  quantifying  the 
absolute  value  of  R. 

The  dominant  uncertainty  causing  R0  or  isFRET  fluctuations  is 
the  relative  orientation  of  dipoles,  as  is  described  for  three  partic¬ 
ular  cases  in  Figure  45.5.  When  the  labeled  proteins  are  not  known 
to  be  attached  to  a  larger  structure,  it  is  common  to  assume  that 
the  two  dyes  are  freely  rotating  on  a  time  scale  comparable  to,  or 
faster,  than  the  fluorescence  lifetime  and  that  K2  can  be  dynami¬ 
cally  averaged  to  a  fixed  value  of  K2  =  2/3.  To  test  whether  this 
assumption  holds  true  for  new  constructs  under  investigation, 
ensemble  polarization  anisotropy  measurements  can  be  performed 
on  selected  constructs  (Lacowicz,  1999;  Mattheyses  et  al.,  2004). 
For  free  cytosolic  proteins,  or  a  combination  of  a  cytosolic  and  a 
membrane-attached  protein,  it  has  been  found  that  both  free  and 
attached  GFP  have  large  anisotropy  values  (around  r  =  0.20),  indi¬ 
cating  obstructed  rotational  diffusion. 

In  this  chapter,  calculated  values  of  Ra  are  based  on  the 
assumption  that  K2  =  2/3,  even  though  we  recognize  that  it  is  pos¬ 
sible  that  in  some  subcellular  compartments  K2  can  be  as  high  as 
2  or  even  4  (Fig.  45.5).  If  the  wavelength  is  measured  in  centime¬ 
ters  and  the  molar  absorption  in  AT1  cm-1,  and  the  JDA  value  is 
given  in  AT1  cm-3,  then  the  following  formula  results: 

Ro  =  8.79  x  10-25  {QD  tew?"4  /DA)  cm6  (7) 

However,  if  the  wavelength  is  depicted  in  nanometers  and 
the  molar  absorption  in  AT1  cm-1,  and  the  JDA  value  is  given  in 
AT'cnT'nnf4,  the  Eq.  is  the  following  (see  also  Fig.  45.6): 

Ro  =  8.79  x  10“5  {QD  k2  ■  nJ  ,/l)A}  A6  (in  Angstroms)  (8) 

A  number  of  experimental  data  have  confirmed  the  linear 
dependence  of  EeS  on  the  value  of  J  in  the  range  of  distances  used 
to  measure  FRET.  It  is  important  to  mention  here  that,  as  the  GFP 
variants  exhibit  relatively  small  Stokes  shifts,  Ra  values  calculated 


between  selected  GFP  variants  can  be  used  for  FRET  calculations 
(Patterson  et  al.,  2000). 

As  the  energy  absorbed  by  the  single  dipole  will  induce  this 
dipole  (e.g.,  donor)  to  oscillate  along  x-,  y-,  and  z-oriented  direc¬ 
tions  (Fig.  45.5),  this  energy  will  be  in  the  reverse  proportion  to 
the  radius  R1.  With  the  appearance  of  the  second  oscillating  dipole 
in  resonance  proximity  (acceptor),  their  common  energy  will  be 
calculated  in  reverse  proportion  to  the  R6.  Eeft  is  highly  and 
inversely  dependent  on  the  distance  (R)  between  the  donor  and  the 
acceptor  chromophores,  and  can  be  obtained  in  the  experiment. 
The  Forster  equation  can  in  principle  be  used  to  calculate  the 
intermolecular  distances  on  a  nanometer  scale  (i.e.,  FRET  = 
nanobioscopy): 

EeS  =  R60/(R6o  +  R6)  (9) 

The  actual  distance  between  a  particular  pair  of  donor  and 
acceptor  molecules  can  be  calculated  if  their  Forster  radius  (Ra)  is 
known  from  previous  experiments.  Calculations  of  Ra,  based  on 
the  spectral  properties  of  some  donor-acceptor  pairs  of  fluorescent 
proteins  are  given  in  Table  45.1.  From  Eq.  9,  it  follows  that  an 
increase  in  the  distance  between  acceptor  and  donor  from  R  =  R0 
to  R  =  2 Ra  will  decrease  EeS  from  50%  to  1.5%.  Ra  is  determined 
experimentally  as  the  distance  in  Angstroms  (1  nm=  10  A).  EtSS  can 
be  obtained  directly  from  the  experiment: 

Eeff  =  1  -  Fda/Fd  (10) 


TABLE  45.1.  Calculated  Donor-Acceptor  Distance  (Ro)  in 
Nanometers  (Patterson  et  al.,  2000) 


Donor 

Acceptor  (GFP  Variant  or  DsRed) 

Green 

Yellow 

Red 

Cyan 

4.82 

4.92 

4.17 

Green 

4.65 

5.64 

4.73 

Yellow 

3.25 

5.11 

4.94 
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where  F  is  the  fluorescence  intensity  measured  in  the  experiment. 
Fda  is  the  fluorescence  intensity  of  the  donor  in  the  presence  of 
acceptor  and  FD  is  the  fluorescence  intensity  of  the  donor  alone. 
FRET  efficiency  EeS  can  also  be  measured  as  the  relative  fluores¬ 
cence  of  the  donor  in  the  presence  (Dbb)  and  absence  (Dab)  of  the 
acceptor  (i.e.,  after  photobleaching  of  the  acceptor).  As  it  is 
usually  difficult  to  achieve  complete  photoinactivation  of  the 
acceptor  in  living  cells,  an  image  of  the  acceptor  is  acquired  both 
before  bleach  (Abb)  and  after  bleach  (Aab).  The  ratio  of  Aab/Abb 
then  reflects  the  percentage  of  non-bleached  acceptor  that  needs  to 
be  included  in  calculations  of  FRET  efficiencies.  Acceptor  photo- 
bleaching  allows  one  to  obtain  the  experimental  parameters  Dbb 
and  Dab,  used  for  FRET  calculations. 

£eff  =  1  -  Dbb/Dab  (11) 

Anisotropy  analysis  of  chimeric  fusion  proteins  can  also 
provide  a  solid  basis  for  structural  analysis  (Velez  and  Axelrod, 
1988).  Different  experimental  variations  of  FRET  measurements 
have  been  developed  for  biological  applications  within  the  last  two 
decades  and  continue  to  be  improved  in  a  growing  number  of  lab¬ 
oratories  around  the  world.  Nevertheless,  it  is  still  useful  to 
perform  spectral  analyses  for  any  new  chromophores  selected  for 
FRET  to  control  for  their  exact  properties  once  they  are  inside  the 
cell  line  selected  for  experiments.  Alternatively,  one  can  calculate 
FRET  efficiency  from  fluorescence  lifetime  data  [x  from  a  fluo¬ 
rescence  lifetime  imaging  microscope  (FLIM)  (Lakowicz,  1999; 
and  Chapter  27,  this  volume ].  Using  the  previous  equations,  we 
can  describe  lifetime  as: 

T=  Q/k 

where  Q  is  the  quantum  yield  of  the  fluorophore  and  kf  the  number 
of  photons  emitted.  The  lifetime  of  the  donor  in  the  presence  of 
the  acceptor  is  described  as: 

Ida  =  Gda/^d  =  1  l(k,  +  kD  +  kDi  )  or 

kj  +  kD  +  kDi  =  1/tda  (12) 

The  lifetime  of  the  donor  in  the  absence  of  the  acceptor  will 
be 

Td  =  Qolko  =  1/(£d  +  km) 

kD  +  kDi  =  1/Td  (13) 

Substituting  Eq.  12  and  Eq.  13  into  Eq.  5,  FRET  efficiency  can 
be  calculated  directly  from  the  experimental  measurements  of  the 
lifetime  of  the  donor  in  the  presence  (xDA)  and  in  the  absence  of 
the  acceptor  (xD): 

Esg  =  1  —  Xda/Xd  (14) 


FLUORESCENT  PROTEINS  AND  FLUORESCENCE 
RESONANCE  ENERGY  TRANSFER 

Considering  all  the  non-fluorescent  proteins  in  living  organisms, 
the  discovery  and,  ultimately,  cloning  of  a  naturally  fluorescent 
one,  GFP,  from  the  marine  organism  Aequoria  victoria  came  as  a 
lucky  surprise  (Ward  et  al.,  1982;  Prasher  et  al.,  1992;  Chalfle  et 
al.,  1994;  Chalfle,  1995;  Tsien,  1998).  What  drove  evolution  to 
create  a  fluorescent  protein?  One  reasonable  theory  suggests  that 
GFP  molecules  appeared  as  a  byproduct  in  the  development  of 
antioxidant  systems  that  evolved  into  the  pocket  of  an  excitable 
chromophore  (Heim  et  al.,  1994;  Labas  et  al.,  2002).  The  story  is 
made  more  interesting  by  the  fact  that  the  native  GFP  chromophore 
is  excitable  with  blue  light  (at  a  peak  of  395  nm)  that  is  internally 


transferred  by  a  FRET  mechanism,  resulting  in  emission  at  508  nm 
(Heim  and  Tsien,  1996).  Thus,  FRET  already  exists  as  an  intrinsic 
property  of  native  GFP  (Ward  et  al.,  1982;  Heim  et  al.,  1995). 

It  took  about  20  years  after  the  discovery  of  GFP  before  it  was 
put  to  practical  use  in  the  field  of  cell  biology,  but  then  it  produced 
a  scientific  revolution  (Ward  et  al.,  1982;  Prasher  et  al.,  1992; 
Chalfle  et  al.,  1994;  Tsien,  1998,  2004;  Lippincott-Schwartz  and 
Patterson,  2003).  GFP-based  methodologies  and  live-cell  imaging, 
combined  with  FRET  and  other  biophysical  approaches,  yielded  a 
vast  amount  of  information,  allowing  one  to  study  and  dissect  bio¬ 
chemical  pathways  at  the  single  cell  level  (Hirose  et  al,  1999; 
Miyawaki  et  al.,  1999;  Janetopoulos  et  al.,  2001;  Lippincott- 
Schwartz  and  Patterson,  2003;  Jones  et  al,  2004;  Kusumi  et  al., 
2005). 

GFP,  and  its  spectrally  shifted  variants,  CFP  and  YFP  (Heim 
and  Tsien,  1996;  Tsien,  1998;  Patterson  et  al.,  2001)  are  particu¬ 
larly  useful,  either  for  colocalization  studies  or  as  partners  for 
FRET-proximity  imaging  experiments  (Mochizuki  et  al.,  2001). 
CFP  was  created  by  introduction  of  the  S65A,  Y66W,  S72A, 
N146I,  M153T,  and  V163A  mutations  into  GFP,  resulting  in  an 
excitation  peak  of  434  nm  and  an  emission  maximum  at  474  nm. 
YFP  differs  from  GFP  by  the  S65G,  V68L,  S72A,  and  T203Y 
mutations  and  has  an  excitation  peak  of  514nm  and  an  emission 
maximum  at  527 nm  (Heim  and  Tsien,  1996;  Tsien,  1998).  Good 
spectral  overlap  between  CFP  and  YFP  currently  favors  these  two 
chromophores  as  the  best  pair  for  analyzing  protein-protein 
interactions. 

GFP  provides  the  great  advantage  of  allowing  us  to  follow  in 
a  living  cell  any  cellular  protein  for  which  the  coding  sequence  is 
known.  GFP-based  chimeric  proteins  have  been  expressed  in  an 
almost  unlimited  range  of  different  cell  types,  including  neurons, 
in  tissue  slices,  and  even  in  whole  living  organisms  (transgenic 
mice,  rats,  rabbits,  fish,  etc.). 

Often  GFP  chimeras  are  placed  under  tissue-specific  promot¬ 
ers.  The  spectral  variants  of  GFP  that  represent  donor-acceptor 
pairs  allow  us  not  only  to  map  intracellular  protein  localization, 
but  also  to  analyze  their  interactions  in  different  cells  and  subcel- 
lular  compartments.  GFP  chimeras  can  be  targeted  to  different 
cellular  localizations.  As  GFP  itself  is  a  relatively  small  protein 
(~27kDa)  with  a  chromophore  inside  a  cylinder  of  2.4 nm  in 
diameter,  the  centers  of  the  two  chromophores  cannot  get  closer  to 
each  other  than  their  diameter.  Maximal  FRET  between  CFP  and 
YFP  was  measured  at  a  distance  of  4.9  to  5.2 nm,  demonstrating 
FRET  (proximity)  in  a  size  range  of  protein-protein  complexes 
(Tsien,  1998).  Stryer’s  dream  of  FRET  as  a  “Spectroscopic  Ruler” 
started  to  become  a  reality. 

FRET  between  fluoroscein  and  rhodamine  molecules  was  first 
measured  a  long  time  ago  and  R0  values  of  4.9  to  5.4 nm  have  been 
known  and  used  for  the  FITC  (fluorescein  isothiocyanate)  and 
TRITC  (tetramethyl  rhodamine)  pair  as  chromophores  with  high 
energy  transfer  efficiency  (Johnson  et  al.,  1984). 

DsRed,  derived  from  the  Discosoma  coral,  is  genetically  dif¬ 
ferent  from  GFP  (Matz  et  al.,  1999).  An  intrinsic  tendency  of 
DsRed  to  tetramerize  and  its  slow  maturation  compared  to  GFP 
prevented  its  broad  application  for  colocalization  or  FRET  studies. 
However,  DsRed  has  recently  been  “re-evolved”  by  extensive 
mutagenesis,  resulting  in  bright  monomeric  versions,  called 
mRFPl  or  mCherry,  which  show  improved  fluorescence  and  mat¬ 
uration  times  (Campbell  et  al.,  2002;  Shaner  et  al.,  2004).  Using 
GFP  as  a  donor,  these  mRFPs  can  now  be  used  for  FRET 
analyses. 

Small  fluorescent  molecules  (e.g.,  Cy2,  Cy3,  Cy5,  Alexa-488, 
Alexa-543)  can  be  easily  attached  to  bacterial  toxin  molecules 
without  changing  their  activity  and  can  then  be  used  for  monitor- 
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ing  toxin  actions  in  living  cells.  These  dyes  can  either  form  FRET 
pairs  by  themselves  (e.g.,  Cy3  and  Cy5)  or  can  be  combined  with 
GFPs  in  a  FRET  pair  (e.g.,  CFP  and  Cy3,  or  potentially  GFP  and 
Cy5). 

Qualitative  Analysis 

Because  in  many  cases  it  is  difficult  either  to  estimate  the  level  of 
cellular  expression  to  determine  the  donor-acceptor  ratio  or  to 
correct  for  non-transfer  energy  loss  pathways1  in  a  system  as  com¬ 
plicated  as  a  living  cell,  many  researchers  settle  for  qualitative 
data.  Additionally,  lack  of  precise  filters,  suboptimal  laser  excita¬ 
tion  wavelengths,  and  the  limited  speed  of  data  readout  are 
common  sources  of  errors  in  FRET  analysis. 

The  spectroscopic  properties  required  for  successful  FRET 
measurements  in  live  cells  (both  for  GFP-like  chromophores  and 
for  organic  dyes)  include: 

•  A  suitable  spectral  integral  overlap  between  donor  emission 
and  acceptor  excitation. 

•  Fligh  donor  molar  extinction  coefficient. 

•  Fligh  fluorescence  quantum  yield  of  the  donor. 

•  Comparable  rates  for  donor  and  acceptor  photobleaching. 

•  Comparable  maturation  times  and  intensity  of  chromophores 
during  FRET  records. 

•  Known  ratio  of  donor:acceptor  molecules. 

To  avoid  common  mistakes  when  choosing  either  a  single 
probe  or  a  FRET  donor/acceptor  pair,  the  most  important  first  step 
is  a  literature  search.  For  example,  the  strong  pFl  dependence  of 
YFP  makes  it  unsuitable  for  fusions  on  the  luminal  side  of  trans¬ 
membrane  proteins  of  acidic  membranous  organelles,  such  as  lyso- 
somes.  Flowever,  when  YFP  is  attached  to  such  proteins  on  the 
cytosolic  side  it  will  work  well  and  give  bright  fluorescence 
(Elsliger  et  al.,  1999;  Griesbeck  et  al.,  2001).  For  each  chro- 
mophore,  it  is  useful  to  know  whether  the  fluorescent  signal 
obtained  derives  from  the  intrinsic  concentration  of  the  chro- 
mophore,  or  if  it  also  reflects  properties  of  the  microenvironment 
(e.g.,  pFI  dependence). 

Preparation 

Preparations  for  live  cell  experiments  thus  should  include: 

1.  Selection  of  appropriate  fluorescent  probes  and  cell  lines. 

2.  Correct  design  of  experiments  (strategy)  to  answer  the 
question. 

3.  Proper  labeling  or  correct  cloning  of  chimeric  fluorescent 
proteins. 

4.  Finding  a  suitable  method  for  introducing  fluorescent  mole¬ 
cules  into  cells:  transfection/electroporation/microinjection. 

5.  Proper  microscope  setup  for  FRET  analyses  [lasers,  filters, 
charge-coupled  devices  (CCDs),  photomultiplier  tubes 
(PMTs)]. 

6.  Best  conditions  for  data  acquisition  from  live  cells  (ensure 
good  cell  condition  during  the  experiment,  e.g.,  temperature, 
C02,  isotonicity,  illumination  levels). 

7.  Appropriate  image  analysis  (e.g.,  using  programs  such  as 
MetaMorph  4-6,  Imaris,  NIH  ImageJ,  Photoshop,  or  custom- 
made  software  programs)  based  on  controls  in  which  only  the 
donor  or  only  the  acceptor  is  expressed. 


1  For  example,  a  normal  cellular  protein  may  act  as  a  non-fluorescent  acceptor, 
partially  depleting  energy  of  the  donor. 


NANOBIOSCOPY  OF  PROTEIN-PROTEIN 
INTERACTIONS  WITH  FLUORESCENCE 
RESONANCE  ENERGY  TRANSFER 

Methods  of  Fluorescence  Resonance  Energy 
Transfer  Measurement 

FRET  is  never  observed  directly  but  can  only  be  monitored  imme¬ 
diately  after  the  actual  act  of  energy  transfer  from  donor  to  accep¬ 
tor.  Therefore,  all  methods  discussed  here  determine  FRET 
indirectly.  Changes  in  the  degree  of  donor-acceptor  interactions 
result  in  changes  in  FRET  efficiency  which  is  measured  by  com¬ 
paring  two  states:  (1)  donor  signal  in  the  presence  of  acceptor  FDA, 
compared  to  (2)  donor  signal  without  the  acceptor  FD  (i.e.,  after 
bleach).  Several  experimental  FRET  setups  are  described  below. 

Sensitized  Emission  of  Acceptor 

FRET  by  sensitized  (i.e.,  increased)  emission  of  the  acceptor  relies 
on  measurements  of  acceptor  emission  resulting  from  excitation  of 
a  donor  (Bastiaens  and  Jovin,  1998;  Lakowicz,  1999;  Periasamy 
et  al.,  2001).  Sensitized  emission  of  the  acceptor  occurs  when  it  is 
in  close  enough  proximity  to  an  excited  donor  that  it  can  accept 
energy  non-radiatively  from  the  donor  and  then  emit  its  own  flu¬ 
orescence  without  direct  excitation.  This  method  is  simple  but 
always  requires  a  subsequent  correction  for  cross-talk  of  the  donor 
fluorescence  into  the  acceptor  channel,  something  that  is  especially 
difficult  to  measure  for  chromophores  with  closely  related  spectra. 

The  concentration  ratio  of  donor/acceptor  molecules  present  in 
living  cells  will  also  influence  FRET  results.  FRET  data  on  molec¬ 
ular  interactions  are  easier  to  obtain  if  the  donor/acceptor  ratio  is 
in  the  region  of  1 : 1  to  1 : 10  (Gordon  et  al.,  1998;  Herman  et  al., 
2001;  Hoppe  et  al.,  2002).  The  optimal  D/A  pair  must  be  chosen 
for  each  particular  experiment.  Donor-acceptor  pairs  that  are  far 
away  in  wavelength  allow  one  to  avoid  significant  spectral  overlap 
but  usually  have  poor  FRET  efficiencies.  Alternatively,  although 
D/A  pairs  that  share  a  large  spectral  overlap  are  often  difficult  to 
separate  and  to  use  reliably  in  experiments,  they  almost  always 
show  high  FRET  efficiencies. 

The  sensitized  emission  (FRET  signal)  can  also  be  easily 
detected  by  spectrofluorimetry  (Fig.  45.6).  However,  in  this  case, 
the  FRET  signal  will  be  averaged  over  the  entire  cell  population. 
Only  microscopes  equipped  with  the  appropriate  filters  allow  one 
to  resolve  both  FRET  signals  together  with  spatial  information  at 
cellular  or  subcellular  level.  A  FRET  signal  from  a  corresponding 
cellular  image  obtained  with  a  microscope  will  thus  uniquely 
provide  an  additional  magnification  of  the  process,  surpassing  the 
optical  resolution  needed  to  visualize  the  Golgi,  endoplasmic  retic¬ 
ulum  (ER),  lysosomes,  mitochondria,  and  other  cellular  subcom¬ 
partments.  Often,  FRET  efficiency  (Een)  can  be  directly  visualized 
under  the  microscope  throughout  the  entire  cell  and  even  resolved 
at  the  level  of  subcellular  compartments,  for  example,  the  differ¬ 
ential  increase  in  FRET  at  the  Golgi/cytosol  interface  shown  in 
Figure  45.8. 

Currently,  a  typical  living-cell  FRET  experiment  involves 
usually  capturing  the  emitted  fluorescence  from  both  the  donor 
CFP  and  the  acceptor  YFP  after  excitation  of  only  the  donor  (CFP). 
The  increase  in  sensitized  emission  corresponds  to  the  degree  of 
physical  association  between  the  two  fusion  proteins.  In  addition, 
spectral  measurement  or  ratiometric  imaging  of  the  emission 
profile  can  be  performed.  This  should  include  recording  (a)  emis¬ 
sion  of  a  sample  expressing  donor  alone  as  a  control,  and  (b) 
recording  the  interacting  FRET  pair.  Comparison  between  the 
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emission  spectra  of  these  two  samples  allows  one  to  assess  the 
degree  of  FRET  (see  Fig.  45.6). 

Sensitized  emission  of  the  acceptor  is  a  quick  method  for  mon¬ 
itoring  FRET  in  living  cells,  and  perhaps  the  most  promising 
approach  for  analyzing  protein-protein  interactions  that  have  fast 
kinetics.  It  is  also  useful  for  recording  neuronal  activities  and  for 
studying  the  effects  of  drugs  on  specific  cellular  targets.  These 
experiments  require  that  the  detection  of  the  acceptor  emission  is 
not  significantly  contaminated  with  a  non-FRET  contribution  from 
the  donor.  The  only  way  to  observe  FRET  in  a  dynamic  situation 
is  to  have  filters  that  effectively  eliminate  cross-talk  between  the 
emitted  light  from  the  two  fluorophores.  The  underlying  assump¬ 
tion  is  that  the  ratiometric  analysis  of  donor  and  acceptor  signals 
will  allow  one  to  monitor  and  to  calibrate  the  kinetics  of  the  bio¬ 
logical  responses.  Precalibration  of  the  expected  effect  in  the 
correct  time  frame  and  the  correct  range  of  fluorescence  intensi¬ 
ties,  laser  sources,  and  filters  is  required  to  create  an  intensity- 
dependent  lookup  table,  which  can  then  be  used  during  an 
experiment. 

Using  sensitized  emission,  we  were  able  to  monitor  the  time- 
dependent  effect  of  a  bacterial  toxin,  cholera  toxin,  on  the 
oligomerization  of  a  transmembrane  Golgi  receptor  over  a  period 
of  2h  [Fig.  45.7(A)], 

Donor  Fluorescence 

Acquisition  of  a  clean  donor  image,  that  is,  non-contaminated  with 
acceptor  fluorescence,  is  the  most  critical  step  in  this  type  of  FRET 
analyses.  This  is  especially  true  for  CFP  because  the  CFP/YFP 
emission  spectra  are  closely  overlapping  and  CFP  chimeras  are 
usually  less  bright  than  YFP  chimeras.  The  main  requirements  for 
acquiring  a  correct  donor  image  are  proper  donor  excitation  wave¬ 
length,  laser  power  adjusted  to  be  appropriate  for  living-cell  exper¬ 
iments,  and  a  correct  detector  gain  in  the  acquisition  channel.  The 
microscope  must  be  precalibrated  to  ensure  that  the  detector  gains 
are  linear  within  the  working  range  of  the  experiment,  and,  there¬ 
fore,  that  each  channel  is  tuned  separately  for  a  maximal  signal. 
For  each  set  of  experiments,  we  used  a  donor-only  sample  (CFP) 
to  set  the  gain  in  the  donor  channel  and  to  determine  the  minimum 

- ► 

FIGURE  45.7.  (A)  FRET  monitored  by  sensitized  emission  of  acceptor 
revealed  the  time-dependent  state  of  oligomerization  at  the  single  cell  level. 
Vero  cells  were  double  transfected  with  CFP  and  YFP  versions  of  ERD2. 
During  its  intracellular  transport,  cholera  toxin  binds  to  the  KDEL  receptor  and 
induces  receptor  oligomerization,  revealed  here  by  FRET  as  an  increase  in  sen¬ 
sitized  emission  of  acceptor  ( large  arrow).  (For  details,  see  Majoul  et  al ., 
2001.)  (B)  The  distribution  of  a  CFP  signal  in  llnm  lambda  windows  using 
the  LSM-510  META,  single  transfection  with  a  CFP  chimera.  (C)  The  distrib¬ 
ution  of  a  YFP  signal  in  11  nm  lambda  windows  using  the  LSM-510  META, 
single  transfection  with  a  YFP  chimera.  Note  that  the  upper  four  spectral 
windows  contain  the  CFP  signal  non-contaminated  by  YFP,  whereas  a  YFP 
signal  is  best  detected  in  the  middle  windows.  The  spectral  windows  are  very 
important  for  monitoring  the  FRET-induced  increase  in  the  CFP  signal  after 
acceptor  photobleaching.  (D)  Spectral  characterization  of  FRET  induced 
between  CFP  and  YFP  chimeras;  FRET  proximity  leads  to  the  disappearance 
of  the  donor  emission  peak  (X,em  =  474  nm)  and  appearance  of  the  sensitized 
emission  of  acceptor  (A,em  =  526  nm)  (gray  curve).  (E)  Time-dependent  FRET 
induced  by  cholera  toxin  transport  between  cytosolic  ARFGAP-CFP  and  trans¬ 
membrane  ERD2-YFP  revealed  by  spectrofluorimetry.  Disappearance  of  CFP 
(474  nm)  peak  and  the  appearance  of  YFP  (526  nm)  sensitized  emission  peak 
can  be  followed  during  these  records.  Red  curve  shows  typical  CFP  emission 
at  474  nm  at  the  beginning  of  toxin  transport,  indicating  no  or  low  FRET, 
whereas  a  later  time  point  (black  curve)  shows  strong  FRET.  For  details,  see 
Majoul  et  al.  (2001). 
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laser  power  necessary  to  acquire  a  donor  signal  at  maximum  detec¬ 
tor  gain.  The  same  process  was  repeated  for  the  acceptor  channel 
using  an  acceptor-only  (YFP)  sample.  To  acquire  a  FRET  image 
the  same  laser  power  and  wavelength  was  used  as  for  CFP  alone. 

If  FRET  occurs,  it  should  result  in  a  decrease  in  donor  fluo¬ 
rescence,  and  this  can  be  measured  by  different  methods,  in¬ 
cluding  fluorescence  lifetime.  Usually  it  is  done  by  acquiring 
sequential  sets  of  donor  images  and  comparing  donor  intensities, 
/D,  with  /DA  (donor  in  the  presence  of  acceptor)  after  different  (t  + 
T)  and  after  different  amounts  of  donor  photobleaching.  The 
advantage  of  this  approach  is  the  ability  to  follow  FRET-induced 
changes  in  the  donor  fluorescence.  The  disadvantage  is  the  absence 
of  proper  controls  for  acceptor  and  other  non-FRET  ways  of  donor 
deactivation. 

Acceptor  Bleach 

The  best  way  to  prove  that  an  observed  FRET  signal  reflects  a  true 
interaction  between  two  molecules  is  to  remove  (photo-inactivate) 
the  acceptor  (Bastiaens  et  al,  1996;  Bastiaens  and  Jovin,  1998). 
In  the  case  of  true  FRET,  an  increase  in  donor  fluorescence  will 
result;  from  the  portion  of  the  donor  fluorescence  that  was 
recovered  after  accepter  inactivation.  The  ratio  of  the  donor  fluo¬ 
rescence  intensities  before  and  after  acceptor  photobleaching 
(Dbb/Dab)  reports  the  FRET  efficiency.  FRET  efficiency  (EcS)  is 
calculated  as  the  ratio  of  two  intensities  generated  in  the  same 
detection  channel  but  from  the  same  sample  before  and  after 
acceptor  bleach.  Depending  on  how  much  FRET  occurred  in  a  par¬ 
ticular  region  of  interest,  the  ratio  of  the  two  intensities  measured 
on  the  same  cell  in  two  different  regions  of  interest  (e.g.,  a 
bleached  and  an  unbleached  region  of  the  same  cell  treated  with 
bacterial  toxin)  often  varies  (Bastiaens  et  al,  1996). 

The  application  of  the  acceptor  bleach  method  is  often  delicate 
in  living  biological  samples  because  of  the  long  bleaching  times 
involved  and  the  associated  phototoxicity.  We  used  a  special  built- 
in  laser  emitting  at  532  nm  for  acceptor  photo-inactivation,  and  cal¬ 
culated  FRET  from  the  percentage  of  acceptor  bleach  instead  of 
performing  a  total  inactivation  of  the  acceptor.  The  degree  of 
acceptor  bleach  can  be  calculated  from  the  ratio  of  acceptor  after 
and  before  bleach  (Aab/Abb),  images.  In  this  type  of  experiment, 
one  needs  to  ensure  also  that  the  process  of  acceptor  photo¬ 
bleaching  does  not  affect  the  donor  intensity,  otherwise  the  exper¬ 
imentally  measured  FRET  value  will  be  lowered. 

An  example  of  a  strong  increase  in  donor  fluorescence  after 
acceptor  bleach  (Dab),  detectable  even  by  eye,  at  the  Golgi/cytosol 
interface  is  shown  in  Figure  45.8.  This  figure  represents  data  from 
two  different  setups  used  for  measuring  FRET:  a  Leica-based,  two- 
photon  multi-focal,  multi-photon  microscopy  (MMM)  FRET  setup 
(described  in  detail  in  Majoul  et  al.,  2001,  2002b)  and  a  single¬ 
photon  Olympus  FV500  setup.  Here  we  compare  FRET  results 
measured  on  the  same  type  of  biological  samples  consisting  of 
interacting  CFP  and  YFP  fusions  (Fig.  45.8). 

In  both  cases,  the  fluorescent  donor  was  a  CFP  chimera  of  the 
cellular  protein  ARFGAP.  This  normally  predominantly  cytosolic 
protein  becomes  associated  with  membranes  of  the  Golgi  complex 
in  a  biologically  meaningful  way  upon  its  interaction  with  the 
acceptor,  the  YFP-tagged  transmembrane  Golgi  KDEL-receptor 
(Majoul  et  al.,  2001).  We  analyzed  fluorescence  of  the  donor 
ARFGAP-CFP  before  (Dbb)  and  after  acceptor  photo-inactivation 
(Dab)  (Fig.  45.8). 

The  FRET-MMM  setup  included  an  lOOx  oil  objective  and  a 
cooled  back-illuminated  CCD  camera.  For  the  Olympus  setup,  a 
PL  APO  60x  oil  NA  1.4  objective  lens  was  used  at  Aex  =  458  nm 


Olympus  FV500 


FRET-MMM  (2-photon  Leica) 


FIGURE  45.8.  An  example  of  FRET  induced  by  cholera  toxin  transport 
between  cytosolic  ARFGAP-CFP  and  transmembrane  ERD2-YFP  recorded 
under  the  microscope.  It  is  obvious  that  a  cytosolic  donor  can  only  efficiently 
interact  with  the  transmembrane  Golgi  acceptor  at  the  Golgi/cytosol  interface. 
(Upper  panel)  The  Olympus  FV  500  setup  allowed  us  to  resolve  the  increase  in 
CFP  (donor)  fluorescence  at  the  Golgi/cytosol  interface  even  after  partial  pho¬ 
toinactivation  of  acceptor  (compare  Dbb  and  Dab).  (Lower  panel)  The  same 
combination  of  cytosolic  donor  ARFGAP-CFP  and  the  transmembrane  accep¬ 
tor  ERD2-YFP  was  used  for  FRET-MMM  analyses.  Although  we  were  able  to 
see  an  increase  in  donor  fluorescence  after  acceptor  bleach,  the  fine  resolution 
of  FRET  at  Golgi/cytosol  interface  was  not  achieved  in  this  experimental  setup. 
(For  more  details  see  text.) 
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(argon  laser,  10%  power)  and  the  PMT  voltages  set  to  729  V  for 
the  CFP  channel  and  653  V  for  the  YFP  channel.  Acceptor  emis¬ 
sion  was  detected  within  a  narrow  15  nm  wavelength  window 
between  480  and  495  nm.  The  YFP  acceptor  was  photobleached  at 
A.ex  =  532nm  and  emission  was  sensed  between  535  and  565 nm. 
The  FRET-MMM  two-photon  excitation  wavelength  for  the  donor 
(CFP)  was  A,ex  =  820  nm.  Acceptor  bleaching  was  done  using  a 
Nd:YAG  laser  at  532  nm  and  CFP  donor  fluorescence  was  detected 
with  the  a  BP485/17nm  emission  filter  (Zeiss)  in  front  of  a  back- 
illuminated  CCD  camera.  Up  to  >90%  of  acceptor  (YFP)  was 
photobleached  within  72  s.  After  the  irradiation  of  the  acceptor  was 
completed,  the  de-quenched  donor  fluorescence  was  analyzed 
(Dab)  and  compared  to  the  initial  state  of  the  donor  (Dbb)  to  esti¬ 
mate  the  degree  of  donor-acceptor  interactions  in  FRET. 

Using  FRET-MMM,  we  were  able  to  see  the  increase  in  donor 
fluorescence  (Fig.  45.8,  lower  panel,  big  arrows)  but  were  not  able 
to  distinguish  the  Golgi/cytosol  interface.  We  expected  to  see  more 
FRET  at  the  Golgi  membrane/cytosol  interface  because  the 
cytosolic  donor  (ARFGAP-CFP)  can  interact  with  the  transmem¬ 
brane  Golgi  acceptor  (ERD2-YFP)  only  in  this  location,  that  is,  the 
multiple  cytosolic  donor  molecules  accumulating  on  the  Golgi 
membrane  interact  with  the  transmembrane  acceptor.  Remarkably, 
using  the  Olympus  setup  we  could  discriminate  between  outer 
(looking  into  the  cytosol)  and  inner  surfaces  of  the  Golgi  complex 
(Fig.  45.8,  upper  panel).  The  Olympus  setup  revealed  a  much 
stronger  de-quenching  of  the  donor  at  the  Golgi/cytosol  interface 
after  the  acceptor  photoinactivation  (Fig.  45.8,  left  panel,  see  small 
arrows). 

FLUORESCENT  PROTEINS  AS  FLUORESCENCE 
RESONANCE  ENERGY  TRANSFER  PAIRS 

Cyan  Fluorescent  Protein  and  Yellow 
Fluorescent  Protein  —  The  Commonly  Used 
Fluorescence  Resonance  Energy  Transfer  Pair 

As  native  CFP  and  YFP  molecules  normally  do  not  interact,  ideally 
only  a  biologically  meaningful  interaction  between  the  non-GFP 
part  of  chimeras  will  bring  these  chromophores  into  FRET  prox¬ 
imity.  However,  in  over-expressing  cells,  unspecific  stochastic 
interactions  induced  by  a  very  high  local  density  of  CFP- YFP 
chimeras  can  also  generate  an  artificial  FRET  signal.  Such  cells 
should  not  be  used.  Normally  the  interaction  time  of  an  unspecific 
pair  of  CFP  and  YFP  chimeras  should  be  significantly  shorter  than 
that  displayed  by  a  pair  of  CFP  and  YFP  chimeras  undergoing  a 
biologically  meaningful  interaction.2 

To  measure  sensitized  emission  complementary  to  the  FRET 
microscope  data,  we  used  Fluoromax-2  and  Fluoromax-3  spectro- 
fluorimeters  (ISA,  Jobin-Yvon  Instruments,  Edison,  NJ)  as  a 
control  in  the  same  transfected  cell  population  in  bulk  cell  exper¬ 
iments.  CFP  was  excited  at  Xex  =  425  nm,  and  fluorescence  emis¬ 
sion  was  detected  between  450  to  600  nm.  For  comparison,  YFP 
was  excited  separately  at  =  498  nm,  and  its  fluorescence  was 
measured  between  510  and  600  nm.  For  a  time-dependent  analy¬ 
sis  of  the  interactions  between  CFP  and  YFP  fusion  proteins,  the 
culture  plate  was  divided  into  segments  and  the  cells  from  differ¬ 
ent  segments  were  examined  by  spectrofluorimetry  before  and 
after  toxin  application  (Majoul  et  al.,  2001,  2002b). 


2  Except  when  these  fusions  may  co-aggregate  in  over-expressing  cells. 

3  The  458  nm  laser  line  can  be  used  for  CFP  excitation  but  is  not  optimal.  A 
line  at  -420  to  430  nm  would  be  better,  but  these  are  relatively  rare. 


The  ability  to  detect  meaningful  signals  clearly  depends  on  the 
signal/noise  ratio.  For  example,  although  trying  to  detect  GFP  flu¬ 
orescence  in  the  presence  of  the  intrinsically  strong  autofluores- 
cent  background  signal  found  in  transfected  primary  hepatocytes 
often  results  in  a  signal/noise  ratio  of  ~1,  the  same  GFP  expressed 
in  primary  fibroblast-like  cell  cultures  will  produce  a  signal/noise 
ratio  of  — 10 : 1.  Compared  to  fibroblasts,  hepatocyte  cultures  will 
always  present  much  bigger  problems  with  photobleaching  and 
phototoxicity.  An  understanding  of  cell  physiology  and  the  photo¬ 
physics  of  the  dye  used  in  the  microenvironment  of  each  cell  type 
is  essential  for  evaluating  FRET  data  obtained  with  the  fluores¬ 
cence  microscope. 

Cyan  Fluorescent  Protein  or  Green  Fluorescent 
Protein  Forms  a  Fluorescence  Resonance  Energy 
Transfer  Pair  with  mRFPI 

The  development  of  monomeric  forms  of  DsRed  (Campbell  et  al., 
2002;  Shaner  et  al.,  2004)  provided  a  new  partner  for  FRET  analy¬ 
ses.  The  current  leading  FRET  pair  CFP/YFP  usually  requires 
donor  excitation  at  a  wavelength  (A,ex  peak  =  430  nm)  not  found  on 
many  confocals.  Although  confocal  microscopes  equipped  with 
argon-ion  laser  lines  at  458  nm,  488  nm,  and  514nm,  a  green 
heliummeon  (He-Ne)  laser  Aex  =  543  nm  or  a  red  He-Ne  lasers 
=  633  nm  can  be  used,3  those  employing  dual-gas  krypton/argon- 
ion  lasers  with  lines  at  Xex  =  488  nm,  568  nm,  and  657  nm  are  not 
suitable  for  exciting  either  CFP  or  YFP  (excitation  peak  = 
514nm).  Although  CFP  can  be  efficient  excited  by  a  krypton-ion 
laser  at  A,ex  =  413nm  or  violet  laser  diodes  at  A,ex  =  405  nm,  both 
of  these  are  close  to  the  ultraviolet  (UV)  and  potentially  cytotoxic, 
making  them  unsuitable  for  long,  time-dependent  experiments. 

As  the  488  nm  line  is  common  on  most  confocals  and  is  optimal 
for  exciting  GFP,  the  introduction  of  mRFP  provides  us  with  a  new 
FRET  pair  that  will  be  highly  appreciated  by  many  cell  biologists. 

Whereas  the  large  spectral  overlap  between  CFP  and  YFP 
emission  spectra  necessitates  the  use  of  expensive,  narrow  band¬ 
pass  filters  and  sensitive  CCDs,  mRFP,  and  other  members  of  a 
growing  family  of  red-shifted  sea  coral  fluorophores,  emits  in  the 
red.  This  favors  their  use  as  FRET  acceptors  with  either  GFP  or 
CFP  (Galperin  et  al.,  2004).  Integral  spectral  overlaps  (7DA) 
between  CFP/mRFPl  and  GFP/mRFPl  are  critical  for  FRET  mea¬ 
surements  (see  also  Fig.  45.6). 

The  R0  values  typical  of  GFP  and  DsRed  are  ~5  nm  (Patterson 
et  al.,  2000).  Emission  maxima  of  the  donor  (CFP  or  GFP)  and  the 
acceptor  (mRFPI)  measured  on  a  Fluoromax-3  are  shown  in 
Figure  45.9,  together  with  integral  spectral  overlaps.  One  remain¬ 
ing  problem  is  that  mRFP  matures  slowly  in  the  cytoplasm; 
mCherry  seems  to  be  an  improvement  in  this  respect  (Shaner  et 
al.,  2004).  This  is  a  disadvantage  for  both  comparative  and  quan¬ 
titative  FRET  measurements  with  GFP  or  CFP,  but  it  may  be  par¬ 
tially  overcome  by  using  a  FRET  detection  algorithm,  recently 
developed  to  compensate  for  the  slow  acceptor  maturation  in 
CFP/DsRed  or  GFP/DsRed  pairs  (Erickson  et  al.,  2003).  Combi¬ 
nations  of  laser  excitation  and  specific  filter  sets  suitable  for  dif¬ 
ferent  FRET  pairs  of  chimeric  fluorescent  proteins  as  well  as 
fluorescent  dyes  are  depicted  in  Table  45.2. 

Fluorescence  Resonance  Energy 
Transfer-Based  Sensors 

Many  biological  processes  are  successfully  analyzed  with  fluores¬ 
cent  biosensors  where  FRET  between  CFP  and  YFP  reflects  a  wide 
variety  of  events  such  as  phosphorylation  of  specific  sequences, 
activity  of  cellular  kinases  or  small  GTPases,  oscillations  of  intra- 
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cellular  Ca++,  pH,  or  binding  of  specific  molecules  that  increase  or 
abolish  FRET  (Miyawaki  et  al.,  1997;  Miesenbock  et  al.,  1998; 
Kraynov  et  al.,  2000;  Ting  et  al.,  2001;  Del  Pozo  et  al.,  2002; 
Wouters  et  al.,  2001;  Bunt  and  Wouters,  2004).  Quite  often,  a  flu¬ 
orescent  biosensor  is  a  fusion  protein  comprising  three  main  com¬ 
ponents,  a  functional  or  targeting  domain  fused  to  the  C-  or  N- 
terminus  of  enhanced  cyan  and  yellow  fluorescent  proteins.  FRET 
measurable  with  the  biosensor  will  change  in  a  predictable  way  in 
response  to  changes  in  the  intracellular  environment  [pH,  Ca++, 
cyclic  adenosine  monophosphate  (cAMP)  levels,  redox  potential, 
adenosine  triphosphate  (ATP)  levels,  glucose  levels,  etc.]  or  the 
binding  of  a  protein  or  peptide  to  the  biosensor  (reviewed  in  Hahn, 
2003;  Meyer  and  Teruel,  2003;  Bunt  and  Wouters,  2004).  GFP- 
based  biosensors  have  been  successfully  applied  to  analyze  com- 
partmentalization  of  many  cellular  dynamic  processes,  notably 
the  spatio-temporal  analysis  of  cellular  transport,  signaling,  and 
development. 

Phosphoinositide  signaling  can  be  studied  using  GFP-tagged 
pleckstrin  homology  (PH)  domain  constructs.  Different  reporters 
for  serine,  threonine,  and  tyrosine  protein  kinase  activities  have 
been  reported.  Such  GFP-based  phosphorylation  substrates  are 
designed  for  precise  subcellular  targeting  and  to  not  impair  phos- 


CFP/mRFPI  configuration  for  FRET: 


GFP/mRFPI  configuration  for  FRET: 


FIGURE  45.9.  Monomeric  red  fluorescent  protein  (mRFP)  is  potentially  a 
good  FRET  acceptor  for  both  CFP  and  GFP  chromophores.  Excitation  and 
emission  spectra  of  CFP,  GFP,  and  mRFP  together  with  their  integral  spectral 
overlaps  (7Da)  for  CFP/DsRed  and  GFP/DsRed  are  given  for  comparison.  Dif¬ 
ferent  filter  sets  used  to  separate  CFP,  GFP,  and  DsRed  signals  are  indicated. 


TABLE  45.2.  Lasers  and  Filter  Configurations  for  Selected 
Fluorophores  Used 


Laser,  X-Ex 

XEm  Filters  (nm) 

CFP 

Argon,  457 

485/30  or  485/17  (Zeiss) 

YFP 

Argon,  514 

530/50 

Alexa488  or  GFP 

Argon,  488  nm 

515/30  or  535/50 

Cy3  or  Rhod-2  Green 

He-Ne,  543  nm 

590/70 

DsRED/mRFPl 

He-Ne,  543 

590/70 

Cy5 

He-Ne  633  or 

660LP 

He-Ne  594 

660LP 

FRET:  Acceptor  (YFP)  bleach:  external  laser  of  530  ntn 

Filters  for  the  Widefield  FRET:  Donor/Acceptor  /.Fi;/BP(n in);  ivRm(nm); 
Dichroic  (nm) 


Donor/ Acceptor 


BFP/GFP 

365/15  460/50 

535/50 

390 

BFP  YFP 

365/15  460/50 

535/26 

390 

BFP/mRFPl 

365/15  460/50 

610/60 

390 

CFP/mRFPl 

436/20  485/17 

610/60 

455 

CFP/YFP 

436/20  485/17 

535/20 

455 

GFP/Rhod-2  Green 

488/20  535/50 

595/60 

505 

FITC/Rhod-2  Green 

488/20  535/45 

595/60 

505 

FITC/Cy3 

488/20  535/45 

595/60 

505 

Cy3/Cy5 

525/45  595/60 

695/55 

560 

Alexa488/Cy3 

488/20  535/45 

595/60 

505 

phorylation  of  endogenous  substrates.  As  a  result,  they  allow  spatio- 
temporal  resolution  of  phosphorylation  processes  in  live  cells.  For 
example,  imaging  of  FRET-based  reporters  for  protein-kinase  C 
(PKC)  translocation  (with  CFP  and  YFP  fused  to  the  N-  and 
C-terminus  of  PKC),  phosphoinositide  bisphosphate  conversion 
to  IP3,  and  diacylglycerol  has  shown  that  in  HeLa  cells  PKC- 
mediated  oscillatory  phosphorylations  correlate  with  Ca2+- 
controlled  translocation  of  conventional  PKC  to  the  membrane 
without  oscillations  of  phospholypase-C  (PLC)  activity  or  diacyl¬ 
glycerol  (Violin  etal,  2003).  Time-resolved  visualization  of  growth 
factor-induced  activation  of  Ras  and  Rapl  in  living  cells  is  yet 
another  example  of  dissecting  cellular  signaling  events  using  micro¬ 
scopic  techniques  and  FRET  sensors  (Mochizuki  et  al.,  2001). 

A  fluorescent  biosensor  that  is  useful  for  temporal  measure¬ 
ments  of  the  mitotic  clock  has  been  recently  described  (Jones  et 
al.,  2004).  High  throughput  methods  of  FRET  screening  using 
biosensors  will  likely  be  useful  in  pharmaceutical  and  clinical 
screens  for  modulators  of  tyrosine  kinases  and  phosphatases,  and 
many  other  cellular  activities.  The  list  of  useful  biosensors  is 
growing  by  the  day.  Previously  based  mostly  on  biochemical  data, 
real-time  observations  with  FRET  in  living  cells  are  now  becom¬ 
ing  major  sources  for  our  knowledge  of  how  cells  move,  divide, 
and  are  activated  for  coordinated  cellular  actions  in  signaling  and 
intracellular  transport. 

FLUORESCENCE  RESONANCE  ENERGY 
TRANSFER  AND  OTHER 
COMPLEMENTARY  METHODS 

Fluorescence  Resonance  Energy 
Transfer  and  Fluorescence  Lifetime 
Imaging  Microscope 

FLIM  is  a  neat  tool  for  detecting  multiple  fluorophores  in  living 
cells,  particularly  spectrally  overlapping  molecules  such  as  GFP 
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variants,  which  have  different  fluorescence  lifetimes  despite  their 
overlapping  spectra  (Bastiaens  and  Squire,  1999).  The  detection  of 
fluorescence  is  typically  achieved  by  counting  the  number  of 
photons  emitted  by  the  excited  state  of  a  fluorophore.  Alternatively, 
the  lifetime  of  the  excited  state  can  be  measured  by  FLIM 
(Lakowicz,  1999;  Chapter  27,  this  volume).  FLIM  is  an  excellent 
tool  to  measure  FRET  because  the  lifetime  of  the  excited  state 
decreases  strongly  when  FRET  provides  an  additional  means  for 
decay  from  the  excited  state. 

FLIM  can  also  be  used  to  monitor  FRET-based  protein-protein 
interactions.  The  lifetime  of  the  donor  is  the  time  spent  in  the 
excited  state  (x)  before  returning  to  the  ground  state  (Lakowicz, 
1999).  Analyses  of  donor-acceptor  interactions  thus  relies  on  mea¬ 
surements  of  the  length  of  time  (typically  less  than  4  ns  for  GFP 
molecules)  that  the  donor  remains  fluorescent  after  being  excited 
with  a  fast  pulsed  laser.  FRET  can  be  calculated  in  such  an  exper¬ 
iment  according  to  the  following  Eq.  (Lakowicz,  1999): 

Ee s  =  1  ~  Tda/Td  (14) 

Because  the  change  in  the  fluorescence  lifetime  of  the  donor 
can  be  analyzed  independently  of  acceptor  emission,  this  technique 


can  be  used  to  detect  FRET  between  GFPs  with  nearly  identical 
emission  spectra  but  different  x  (Bastiaens  and  Squire,  1999). 
FLIM  measurements  are  also  of  advantage  for  those  donor  mole¬ 
cules  that  are  prone  to  quick  photobleaching. 

Changes  in  x  can  easily  be  detected  by  advanced  modern  tech¬ 
nology  using  the  frequency  or  time  domain,  or  direct  photon  count¬ 
ing  (see  Chapter  27,  this  volume).  A  FLIM  setup,  suitable  for 
measuring  FRET,  as  well  as  some  results  from  living  cells  are 
shown  in  Figure  45.10.  Comparison  between  the  life  time  of  the 
donor  alone  (xD)  and  the  donor  life  time  in  the  presence  of  the 
acceptor  (xDA)  will  provide  information  on  R. 

The  biggest  advantage  of  FLIM  analyses  is  the  independence 
of  donor  lifetime  xD  on  the  concentration  of  the  dye  (except  in 
some  cases  of  homo-FRET).  Nevertheless,  the  complex  calibra¬ 
tion  procedure  for  each  particular  FLIM  setup  will  require  profes¬ 
sional  support.  The  FLIM  approach  has  a  number  of  limitations: 

•  Each  microscope  must  be  calibrated. 

•  Spatial  resolution  may  be  limited  if  multiple  images  must  be 
acquired. 

•  More  photons  must  be  detected  to  determine  x  for  a  pixel  than 
are  required  to  merely  detect  presence  or  absence. 


s 


Frequency  domain 


FIGURE  45.10.  FRET  resolution  by  FLIM.  Upper  panel,  left:  Schematic  presentation  of  FRET  in  the  frequency  domain:  the  pulsed  excitation  frequency  (dark 
blue);  the  delayed  signal  of  the  CFP-donor  (red  curve);  and  the  less-delayed  signal  of  the  donor  when  FRET  is  occurring  (light  blue)  because  FRET  decreases 
the  donor  lifetime.  Right:  The  intensity  of  the  signal  having  the  CFP,  (donor)  time  constant  collected  under  the  same  conditions,  and  showing  that  FRET  causes 
the  donor  (CFP)  intensity  to  decrease  when  both  CFP-  and  YFP-tagged  connexin  molecules  are  oligomerized  in  a  perinuclear  Golgi  region  of  this  cell.  Middle 
right:  Alternative  representation  of  the  data  in  image  above.  Histograms  represent  the  shorter  donor  lifetime  when  FRET  to  an  acceptor  is  occurring  (Tfrft,). 
compared  to  when  only  the  donor  is  present  (TD).  Lower  panel,  left:  Schematic  presentation  of  the  individual  parameters  in  the  time  domain:  Log  (intensities) 
versus  lifetime:  (TD  )  is  longer  than  (Tfret).  Left:  Images  of  FRET  in  the  Golgi  region  of  a  cell:  comparison  between  total  counts  and  counts  sampled  only  from 
the  donor  channel  reveals  the  portion  of  the  donor  molecules  that  are  not  transferring  energy  to  the  acceptor,  and  thus  still  emitting  in  the  donor  channel. 
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Despite  the  existence  of  commercial  hardware  and  software 
programs  for  the  task,  FLIM  data  will  generally  require  more 
complex  (global)  mathematical  analysis  of  acquired  results 
(Bastiaens  and  Jovin,  1998;  Bastiaens  and  Squire,  1999;  Bastiaens 
and  Pepperkok,  2000;  Gerritsen  et  al.,  2001;  Herman  et  al.,  2001; 
and  the  overview  of  FLIM  given  in  Chapter  27,  this  volume). 

Fluorescence  Recovery  After  Photobleaching 
and  Fluorescence  Loss  in  Photobleaching 

Fluorescence  recovery  after  photobleaching  (FRAP)  is  a  technique 
in  which  a  region  of  interest  is  selectively  photobleached  with  a 
high-intensity  laser  and  the  recovery  that  occurs  as  molecules 
move  back  into  the  bleached  region  is  monitored  over  time  with 
lower-intensity  laser  light.  Depending  on  the  protein  studied,  flu¬ 
orescence  recovery  can  result  from  protein  diffusion,  binding/dis¬ 
sociation,  or  transport  processes.  FRAP  experiments  can  thus 
determine  the  kinetic  parameters  of  a  protein,  including  its  diffu¬ 
sion  constant,  mobile  fraction,  transport  rate,  or  binding/dissocia¬ 
tion  rate  from  other  proteins  in  living  cells  (Lippincott-Schwartz 
et  al.,  2001,  2003).  Fluorescence  recovery  is  usually  quantitatively 
monitored  by  a  microscope-mounted  CCD  camera  that  is  cali¬ 
brated  to  the  fluorescence  signal  before  photobleaching. 

Fluorescence  loss  in  photobleaching  (FLIP)  is  a  photobleach¬ 
ing  technique  complementary  to  FRAP.  In  a  FLIP  experiment,  a 
region  of  interest  in  a  cell  is  repeatedly  photobleached  while  fluo¬ 
rescence  in  the  whole  cell  is  continuously  monitored.  Any  cellular 
regions  with  connections  to  the  area  being  bleached  will  lose  flu¬ 
orescence  due  to  lateral  movement  of  mobile  proteins  into  the 
bleached  area.  On  the  other  hand,  the  fluorescence  in  unconnected 
cellular  regions  will  be  unaffected  (Cole  et  al.,  1996).  FRAP  and 
FLIP  techniques  have  been  extensively  used  for  studying  the  traf¬ 
ficking  within  the  secretory  pathway  in  animal  cells,  especially 
ER-to-Golgi  trafficking  (Cole  et  al.,  1996;  Lippincott-Schwartz  et 
al.,  2001;  Presley  et  al.,  2002;  Lippincott-Schwartz  and  Patterson, 
2003).  In  principle,  FRET  techniques  can  be  nicely  combined  with 
both  FRAP  and  FLIP  to  obtain  supplementary  information  on 
protein-protein  interactions  together  with  kinetic  parameters  of 
protein  localization,  for  example,  about  mobility  of  plasma  mem¬ 
brane  receptors  and  lipids. 

Fluorescence  Resonance  Energy  Transfer  and 
Fluorescence  Correlation  Spectroscopy 

Confocal  fluorescence  correlation  spectroscopy  (FCS)  is  a  unique 
tool  to  analyze  processes  that  can  be  recorded  from  a  very  small 
area  of  illumination  in  a  selected  region  of  less  than  1  pm  (Magde 
et  al.,  1972).  FCS  is  thus  suitable  for  quantitative  measurements 
of  the  local  concentrations  and  diffusional  mobility  of  fluorophores 
through  a  small  volume  of  a  living  cell,  for  example,  in  endocytic 
membrane-bound  carriers  or  other  compartments  (Bacia  et  al., 
2002;  Kim  and  Schwille,  2003).  Diffusion  coefficients  measured 
by  FCS  can  be  used  to  calculate  the  approximate  sizes  of  protein 
complexes  in  living  cells,  and  based  on  quantification  of  the 
fluorescence  intensity  of  the  diffusing  complex  the  number  of  mol¬ 
ecules  can  be  estimated,  as  has,  for  example,  been  done  for  the 
Gag  complex  during  retrovirus  assembly  (Larson  et  al.,  2004). 
FRET  and  FCS  can  be  considered  as  complementary  techniques, 
where  FRET  is  used  to  estimate  the  distance  between  the  donor 
and  acceptor  molecules,  and  FCS  to  provide  information  on 
dynamic  properties  and  the  sizes  of  the  protein  complexes 
analyzed. 


Fluorescence  Resonance  Energy  Transfer  and 
Total  Internal  Reflection  Fluorescence 

In  total  internal  reflection  fluorescence  (TIRF)  microscopy,  exci¬ 
tation  light  approaches  an  aqueous  specimen  as  a  coherent  beam 
of  light  that  strikes  the  coverslip/water  interface  at  an  angle  greater 
than  the  critical  angle.  As  a  result,  only  an  evanescent  excitation 
field  penetrates  into  the  water  to  excite  dye  molecules,  such  as 
transfected  GFP  chimeras,  that  are  within  a  few  tens  of  nanome¬ 
ters  of  the  interface.  The  thickness  of  the  excitation  band  depends 
on  the  wavelength  and  the  approach  angle.  Images  are  recorded  in 
widefield  using  intensified-  or  electron  muliplier  (EM)-CCDs. 

As  this  technique  excites  and  visualizes  only  fluorophores  in 
close  proximity  to  the  plasma  membrane,  it  has  the  obvious  advan¬ 
tage  of  strongly  reducing  photobleaching  and  phototoxicity  com¬ 
pared  to  other  microscopy  techniques.  It  also  reduces  background 
light  from  out-of-focus  planes  essentially  to  zero.  Therefore,  the 
parts  of  living  cells  near  the  coverslip  can  be  imaged  over  longer 
time  periods  and  with  high  contrast  and  improved  temporal 
resolution. 

TIRF  is  the  method  of  choice  for  those  who  want  to  resolve 
processes  occurring  close  to  the  plasma  membrane.  Such  processes 
include  endocytosis  and  exocytosis,  cell-cell  signaling,  formation 
of  cell-cell  contacts,  and  release  of  neuromediators. 

If  the  biological  process  under  study  allows  photobleached 
GFP  chimeras  to  be  quickly  exchanged  from  the  non-bleached 
pool  of  chimeras  in  deeper  regions  of  the  cell,  this  results  in  a  very 
stable  fluorescent  signal  for  long-term  observations.  TIRF  can  be 
combined  with  conventional  widefield  fluorescence  or  DIC 
microscopy  for  localized  analyses  of  cellular  processes  and/or 
FRET  measurements  (Bezzi  et  al.,  2004;  Tones  et  al.,  2004). 

Quantum  Dots  and  Fluorescence  Resonance 
Energy  Transfer 

Semiconductor  quantum  dots  were  introduced  recently  into 
microscopy  for  applications  in  cell  biology  (Michalet  et  al.,  2005). 
Although  they  seem  to  be  very  promising  (Lidke  et  al.,  2004),  their 
application  for  FRET  will  be  limited.  Quantum  dots  exhibit  broad 
absorption  profiles:  as  both  quantum  dots  at  565  nm  and  655  nm 
can  be  excited  with  a  single  excitation  filter  435/70,  long-pass 
emission  filters  must  be  used  to  separate  the  signals.  These  param¬ 
eters  will  prevent  simultaneous  use  of  quantum  dots  with  CFP-  or 
GFP-like  live  chromophores  that  could  otherwise  be  complemen¬ 
tary  donors  for  live-cell  FRET  analyses.  Nevertheless,  double¬ 
labeling  with  quantum  dot  conjugates  is  a  powerful  tool  in 
immunohistochemistry  and  immunocytochemistry  applications 
(Wu,  2003). 

CLONING  AND  EXPRESSION  OF  FLUORESCENT 
CONSTRUCTS  FOR  FLUORESCENCE 
RESONANCE  ENERGY  TRANSFER 

Cloning  of  Fluorescent  Chimeras 

After  deciding  to  use  a  FRET  approach  to  study  a  protein  of  inter¬ 
est,  fluorescent  donor-acceptor  pairs  need  to  be  generated.  The 
most  common  approach  is  to  obtain  the  coding  regions  for  the 
chosen  proteins  and  use  commercially  available  vectors  with  mul¬ 
tiple  cloning  sites  (e.g.,  BD  Biosciences  Clontech,  Stratagene, 
Qbiogene,  and  other  companies)  to  generate  in-frame  fusions  with 
the  CFP,  YFP,  GFP,  or  mRFPl  genes.  GFP  fusions  can  be  gener- 
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ated  at  the  N-  or  C-terminus  of  proteins  of  interest,  or  even  inter¬ 
nally.  with  GFP  being  interspaced  between  domains  of  a  protein 
coding  sequence.  It  is  generally  a  good  idea  to  simultaneously 
prepare  both  N-  and  C-terminal  fusions,  hoping  that  at  least  one 
of  these  chimeras  retains  functional  activity. 

One  fairly  common  problem  is  that  the  linker  between  the 
protein  of  interest  and  GFP  may  be  recognized  as  a  nuclear  import 
signal  (NLS)  by  the  cellular  machinery  and  the  chimera  may  thus 
unexpectedly  be  detected  in  the  nucleus  instead  of  its  proper  cel¬ 
lular  location.  This  chance  is  particularly  high  if  the  linker  con¬ 
tains  NLS-like  motifs  such  as  repeats  of  lysines  (K)  or  arginines 
(R).  Such  common  biological  mistakes  in  linker  generation  should 
be  avoided.  Combinations  of  other  amino  acids,  especially  sec¬ 
ondary  structure  breakers  such  as  glycine  (G)  or  proline  (P)  or 
small  amino  acids  such  as  serine  (S),  alanine  (A),  or  threonine  (T) 
are  less  likely  to  create  problems.  However,  there  are  no  dead-sure 
recipes  for  generating  functional  GFP  chimeras,  beyond  following 
standard  protocols  for  the  cloning  itself  (Sambrook  et  al,  1989). 
Therefore,  performing  functional  tests  is  critical. 

After  DNA  sequencing  to  verify  the  constructs,  DNA  for  mam¬ 
malian  cell  transfections  needs  to  be  prepared  with  great  care.  In 
general,  commercial  kits  (e.g.,  Qiagen  Maxiprep)  work  well.  For 
some  cell  lines  (e.g.,  our  favorite  cell  line,  Vero  cells)  removal  of 
endotoxins  (bacterial  lipopolysaccharides)  is  important  and  easily 
accomplished  with  commercial  kits.  Many  transfection  reagents 
routinely  used  to  introduce  plasmid  DNA  into  living  cells  are 
lipophilic  agents  that  penetrate  biological  membranes  but  may  also 
induce  unwanted  side  effects,  such  as  aggregation  or  fusion  of 
intracellular  membranes  (see  below).  This  makes  cells  transfected 
with  these  reagents  often  look  clumpy  and  non-beautiful,  and  the 
appearance  of  intracellular  membranous  organelles  may  also  be 
abnormal.  As  a  word  of  caution,  any  strong  overproduction  of  a 
transfected  protein  (10-  to  100-fold  above  normal  cellular  levels) 
may  impair  cell  function,  an  effect  that  needs  to  be  controlled  by 
appropriate  functional  assays.  (For  useful  www  links  see  Table 
45.4). 

Functional  Activity  of  Expressed  Constructs 

Successfully  cloned  chimeras  will  be  introduced  into  the  cells 
selected  for  initial  test  experiments.  First,  the  expression  levels  and 
protein  stability  of  the  donor  and  acceptor  needs  to  be  tested  in  a 
time  course,  and  the  spectral  properties  of  the  chimera  established. 
Often,  a  newly  generated  chimera  may  be  brightly  fluorescent  but 
has  lost  functional  activity,  displays  the  wrong  localization,  and/or 
has  lost  the  ability  to  interact  with  its  normal  partners  in  a  protein 
complex.  Therefore,  it  is  most  important  to  check  the  functional 
activity  of  chimeras  before  attempting  to  draw  any  conclusions 
from  their  use.  Such  checks  require  extreme  delicacy  because  one 
can  only  attribute  aberrant  behavior  to  the  construct  if  one  is  sure 
that  the  behavior  is  not  being  caused  by  your  method  of  observa¬ 
tion.  If  an  antibody  against  the  protein  under  study  is  available,  the 
GFP  chimera  should  display  the  same  or  a  very  similar  intracellu¬ 
lar  distribution  as  the  endogenous  protein.  If  normal  interactions 
between  the  chimera  with  known  cellular  partner  proteins  can  be 
established  by  co-immunoprecipitation,  this  is  a  good  sign! 

Expression  and  Over-Expression 

Proper  expression  levels  of  chimeras  are  crucial.  In  the  case  of 
many  of  our  CFP/YFP  chimeras  that  have  functions  in  the  secre¬ 
tory  pathway,  we  use  the  intracellular  transport  of  cholera  toxin 
(CTX)  as  a  good  functional  test.  We  learned  that  in  cells  over¬ 
expressing  fluorescent  chimeras,  transport  of  the  toxin  from  the 


TABLE  45.3.  Transfection  Buffer  for  Electroporation, 
Internal  Medium 

IM,  Transfection  Buffer 

120  m M  final  MW:  74.56  g/mol 
Stock  solution  1 .2 M  =>  for  50  mL  4.48  g 

KH2P04  10  mM  final  MW:  136.09  g/mol 
Stock  solution  lOOmM  =>  for  50  mL  680.45  mg 

EGTA  2  m M  final  MW:  380.4  g/mol 
Stock  solution  20  m M  =>  for  50  mL  380  mg 

MgCl2  5  m M  final  MW:  203.31  g/mol 
Stock  solution  50  m M  =>  for  50  mL  508.3  mg 

HEPES  25  mM  final  MW:  238.3  g/mol 
Stock  solution  250  m M  =>  for  50  mL  2.98  g 

CaCl2  0.5  mM  final  MW:  147.2  g/mol 
Stock  solution  15  mM  =>  for  50  mL  110  mg 

To  prepare  50  mL  of  cytomix: 

Mix  5  mL  of  each  stock  solution,  except  for  CaCl2,  only  500  jiL 
Add  H20  until  50  mL,  adjust  pH  =  7.5-7.6 

For  complete  cytomix: 

Add  GSSG  5mM  final  MW:  656.6  g/mol 
Stock  solution  lOOmM  =>  for  lOmL  660 mg 
Add  50pL/mL 

ATP  2  m M  final  MW:  550  g/mol 
Stock  solution  lOOmM  =>  for  lOmL  550 mg 
Add  20  pL/mL 

Modified  from  Majoul  et  al.  (2001). 

1 07— 1 08  cells  washed  in  PBS  are  transferred  into  400  pL  of  total  Internal  Medium 
Mix  for  electroporation  in  Bio-Rad  (green  cuvette)  Cat.  #165-20086. 

Add  20^10  |lL  of  endo-free  plasmid  DNA  (from  stock  of  1  pg/pL). 

Bio-Rad  electroporator:  0.7  kV,  main  position  50/capacitance  extension  200  pF  (see 
on  the  panel  of  electroporator).  Resulting  time  constant  for  this  combination  should 
be  1.44-1.64  to  get  50%-80%  of  finally  transfected  cells. 


plasma  membrane  to  the  ER,  or  even  to  Golgi  compartments,  was 
blocked.  Extending  the  time  after  transfection  (in  our  case  using 
electroporation)  to  16  to  20  h  for  the  transmembrane  protein 
ERD2-YFP,  which  is  the  receptor  for  cholera  toxin  in  the  Golgi 
complex,  led  to  severe  disturbances  not  only  in  the  intracellular 
localization  of  the  chimeras  but  also  resulted  in  impaired  toxin 
transport.  Although  cholera  toxin  was  able  to  recognize  the  CFP 
or  YFP-fused  ERD2  protein  6  to  8h  after  transfection  [Fig. 
45.4(A)],  after  16h  of  expression,  it  was  no  longer  able  to  enter 
the  structures  labeled  by  ERD2  fusions  [compare  Figs.  45.4(A,B)]. 
However,  even  in  these  damaged  cells,  CTX  still  bound  to  the 
areas  of  Golgi  containing  functional  endogenous  receptor  and 
exhibited  only  low  labeling  with  the  chimeric  receptor  [Fig. 
45.4(B),  long  arrows]. 

Conventional  fluorescence  microscopy  can  be  used  to  estab¬ 
lish  the  time  course  of  expression  and  to  select  the  time  when  the 
expression  is  sufficient  to  measure  FRET,  when  the  chimeras  are 
still  localized  in  the  same  areas  as  in  their  endogenous  wild-type 
counterparts,  where  their  functional  activity  can  be  established.  In 
our  hands,  using  CMV  promoter  constructs,  this  was  usually  ~6  to 
12  h  after  electroporation,  depending  on  the  construct.  In  general, 
the  physiological  validity  of  FRET  measurements  in  living  cells 
expressing  a  given  pair  of  CFP/YFP  fusion  proteins  is  strength¬ 
ened  if  the  observed  FRET  signals  are  obtained  in  response  to  a 
known,  meaningful  physiological  trigger,  especially  when  com¬ 
pared  with  other  pairs  of  CFP/YFP  fusion  proteins  that  remain 
unchanged. 
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TABLE  45.4  Links  Useful  for  the  Design  of  FRET  Experiments 


Proteins 

Collection  of  useful  links  related  to  microscopy  around  the  world: 

Molecular  Expressions:  The  Microscopy  Primer.  A  comprehensive  Tutorial  Site  that 
includes  fascinating  “Virtual  Microscopy” 

BIND:  Protein-protein  interaction  database 
Fluorophores  &  other  probes:  Proteins 
MoBiTec:  Tools  for  Cell  and  Molecular  Biology 
Invitrogen:  Cloning  &  expression 
BD  Biosciences  Clontech:  GFP  news 
Protein  sequence  and  classification  database 
Society  for  Molecular  Imaging 


http://www.ou.edu/research/electron/www-vl/long.shtml 

http://micro.magnet.fsu.edu/primer/index.html 

http://bind.ca/ 

http://www.probes.com/ 

http  ://www.  mobitec .  de/ 

http  ://w  w w.  invitrogen  .com/ 

http://www.bdbiosciences.com/clontech/gfp/ 

http  ://pir.  georgetown.edu/ 

http://www.molecularimaging.org 


Optics 

Fluorescent  Spectra  Database 

Chroma  Technology:  Optical  filters  for  biological  applications 
Omega:  Optical  filters  for  biological  applications 
Semrock:  New  generation  of  filters 
AHF  analysentechnik  AG  Optical  filters  including  GFPs 
Sutter:  Filter  changer  and  light  sources  for  microscopes 
Bioimage:  Biological  images  for  scientific  research 
Semrock:  FRET  filters 


http://www.mcb.arizona.edu/IPC/spectra_page.htm 

http://www.chroma.com/ 

http://www.omegafilters.com/ 

http://semrock.com 

http  ://www.  ahf .  de 

http://www.sutter.com/ 

http://www.bioimage.org/pub/collaborators.jsp 

http://www.laser2000.co.uk/semrock/brightline.htm 


METHODS  FOR  INTRODUCING 
CHROMOPHORES  INTO  LIVING  CELLS 

Electroporation 

We  have  described  an  efficient  electroporation  technique  (Majoul 
et  al.,  2001)  that  has  been  routinely  used  during  the  Vancouver  3D 
microscopy  courses  (2002-2005)  for  multiple  transfections  with 
different  plasmids.  With  slight  modifications  (see  Table  45.3),  this 
protocol  is  now  successfully  used  in  many  laboratories  for  trans¬ 
fection  of  a  large  variety  of  cell  lines,  including  primary  cultured 
neurons,  to  study  GFP-highlighted  organelles  in  living  cells  and 
for  FRET  analyses.  Instead  of  using  PBS  (phosphate-buffered 
saline),  which  is  not  very  physiological,  we  use  a  high  potassium 
“internal  media”  buffer  condition  for  the  electroporation  step 
(K-glutamate;  up  to  140 mM,  pH  7.6).  Other  essential  components 
are  fresh  ATP  and  GSSG  (see  Table  45.3  for  details).  We  prefer 
to  use  a  Bio-Rad  electroporator,  green  Gene  Pulser  cuvette 
(BioRad  Cat.  No.  165-2086;  gap  0.2  cm)  and  use  plasmid  DNA  in 
a  total  volume  of  up  to  300  pL  “internal  media”  per  transfection 
reaction.  In  Vero  cells  (a  Green  monkey  kidney  fibroblast  cell 
line),  we  routinely  obtain  90%  or  more  of  successfully  transfected 
cells  with  this  method.  The  advantage  of  electroporation  for  trans¬ 
fer  of  DNA  for  FRET  analyses  is  the  total  absence  of  the  unspe¬ 
cific  fluorescence  background  produced  by  other  transfection 
reagents. 

Transfection  Reagents 

For  transfecting  cells  with  foreign  DNA,  scientists  commonly  use 
different  commercially  available  agents  with  lipid-like  properties, 
able  to  form  micellae  around  the  DNA:  for  example,  Lipofectamine, 
Lipofectin,  Metafecten,  FUGENE  6,  etc.  Transient  transfections  are 
usually  performed  on  routine  cell  lines,  such  as  HEK  297,  COS-5, 
COS-7,  NRK  (normal  rat  kidney),  3T3  fibroblast-like  cultures, 
BHK-21  (hamster  kidney  cell  line),  or  the  Vero  cells  used  for  the 
experiments  described  in  this  chapter.  Cells  prepared  for  trans¬ 
fection  are  typically  grown  in  media  such  as  Dulbecco’s  modified 
Eagle  medium  (DMEM),  supplemented  with  10%  fetal  calf  serum 
(FCS),  glutamine,  and  antibiotics  such  as  penicillin  and  strepto¬ 
mycin.  Exact  transfection  protocols  are  often  supplied  by  the  man¬ 


ufacturer  of  the  transfection  reagents.  Many  of  them  work  well, 
although  certain  cell  types,  especially  neurons,  may  require  differ¬ 
ent  transfection  procedures.  Ballistic  methods  to  introduce  neuronal 
tracers,  plasmid  DNA  or  intracellular  calcium  indicators  into 
neurons  have  also  been  reported,  are  now  in  common  use 
(Grutzendler  et  al.,  2003). 

Although  usually  the  commercial  reagents  do  not  create  prob¬ 
lems  for  biochemical  experiments,  for  microscopic,  single-cell 
analyses,  remnants  of  precipitated  DNA  or  aggregates  of  transfec¬ 
tion  reagents  may  cause  a  strong  fluorescent  background,  espe¬ 
cially  when  two-photon  excitation  is  used.  Tetracyclin-regulated 
promoters  cannot  be  used  in  combination  with  two-photon  excita¬ 
tion  of  CFP,  as  even  traces  of  tetracyclin  create  a  strong  fluores¬ 
cent  background  in  the  cell.  Expression  systems  induced  by  agents 
such  as  ecdysone  or  hygromycine  may  be  more  suitable  in  such 
cases.  We  routinely  express  our  fusion  proteins  of  interest  under 
the  control  of  the  widely  used,  strong  CMV  promotor.  Again, 
selection  of  the  appropriate  time  window  after  transfection,  with 
regard  to  obtaining  good  fluorescence  signals  for  imaging  without 
having  vast  over-expression,  is  crucial  for  best  results. 

Microinjection 

Microinjection  is  an  elegant  way  to  deliver  biological  material  into 
living  cells,  for  example,  DNA,  RNA,  and  proteins  (e.g.,  fluo- 
rophore-labeled  or  unlabeled  recombinant  proteins  or  inhibitory 
antibodies).  Microinjection  was  used  as  early  as  the  1960s  (Kohen 
et  al.,  1966).  One  of  the  first  attempts  to  measure  intracellular  Ca++ 
with  aequorin  also  was  based  on  microinjection  techniques  (Blinks 
et  al.,  1978).  In  the  1970s  and  1980s  microinjection  was  applied 
for  the  transfer  of  dye  between  neighboring  cells  and  used  to  study 
the  permeability  of  gap  junctions  (Conn,  1991)  (see  Fig.  45.11). 
With  the  development  of  antibodies  against  specific  proteins  or 
protein  domains,  microinjection  of  antibodies  to  inhibit  the  func¬ 
tion  of  the  protein  under  study  became  popular.  Microinjection  of 
antibodies  against  COPI  vesicular  transport  machinery  (anti-|3- 
COP  antibodies  raised  against  the  EAGE  sequence  of  this  subunit) 
prevented  the  transport  of  cholera  toxin  through  the  Golgi  appara¬ 
tus  (Majoul  et  al,  1998). 

The  construction  of  plasmids  encoding  specific  proteins  and 
the  development  of  anti-sense  RNA  and  RNAi  provided  another 


804  Chapter  45  •  I.  Majoul  et  al. 


FIGURE  45.11.  (A,  B)  Two  examples 
of  microinjection:  (A)  a  dying  cell  after 
being  impaled  by  the  needle,  with  col¬ 
lapsed  cellular  membranes,  and  (B)  suc¬ 
cessful  microinjection  into  the  cytosol 
of  a  cell.  (C)  Dye  transfer  between 
astrocytes  after  microinjection  of 
calcein  into  a  single  cell  (indicated  by 
arrow );  (D)  The  same  field  viewed  in 
transmission  light.  (E,  F)  A  decrease  in 
dye  transfer  from  a  microinjected  cell 
upon  lowering  of  the  extracellular  free 
Ca^  concentration.  Living-cell  nuclei 
stained  with  DAPI;  (F)  the  same  field 
viewed  in  transmission  light. 


area  for  rapid  development  of  microinjection  techniques.  The  dis¬ 
covery  of  GFP  and  its  color  variants  has  led  to  the  use  of  microin¬ 
jection  for  inducing  expression  of  fluorescent  chimeras  in  many 
different  types  of  cells,  including  cultured  neurons  and  even  live 
organisms  by  injecting  DNA  into  Xenopus  oocytes,  embryos, 
developing  zebrafish,  Drosophila,  and  other  species.  In  microin¬ 
jection  experiments,  Alexa-350  or  Cascade  blue  conjugated  to 
BSA  can  be  co-injected  to  easily  identify  the  injected  cells.  In 
experienced  hands  and  with  availability  of  good  equipment, 
microinjection  into  cells  and  micromanipulation  of  cells  and  nuclei 
with  glass  pipettes,  sometimes  combined  with  electrophysiology, 
can  address  many  complicated  biological  questions  (for  an 
example,  see  Wakayama  et  al.,  2003). 

In  short,  microinjection  can  be  suitable  for: 

1.  Introduction  of  plasmid  DNA:  Mostly  for  cell  imaging,  while 
for  a  macroscale  biochemical  analysis  electroporation  is  more 
advisable. 

2.  Introduction  of  RNA:  si-RNA,  labeled  oligonucleotides,  anti- 
sense  RNA. 


3.  Introduction  of  proteins:  Dominant-negative  functionally 
active  molecules  (e.g.,  dynamin)  or  microinjection  of  Fab  frag¬ 
ments  of  antibodies  because  whole  divalent  antibody  mole¬ 
cules  may  produce  unspecific  effects  by  cross-linking  the 
proteins  of  interest. 

4.  Introduction  of  dyes  and  other  fragments:  Fluorescent 
markers  (e.g.,  labeled  phalloidin  to  visualize  actin),  mem¬ 
brane-permeable  dyes  (e.g.,  calcein)  to  study  cell-cell  con¬ 
tacts,  or  impermeable  agents  (e.g.,  GTP-y-S). 

FUTURE  PERSPECTIVES:  3D  MICROSCOPY, 
BIOLOGICAL  COMPLEXITY,  AND  IN  VIVO 
MOLECULAR  IMAGING 

The  21st  century  should  see  a  huge  wave  of  novel  micro- 
analytical  techniques.  It  is  now  becoming  obvious  that  the  major 
goals  of  the  “post-genomics”  and  proteomics  era  will  be  to  analyze 
protein  function  and  protein-protein  interactions  at  the  single-cell 
level.  We  now  realize  that  structural  anatomy  provides  only  limited 
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information  about  pathophysiological  consequences,  and  that  the 
future  will  involve  a  struggle  to  visualize  complexes  of  single  mol¬ 
ecules  (Ha  et  al.,  1999;  Kusumi  et  al.,  2005),  biochemical  events 
(Hirose  et  al.,  1999;  Weijer,  2003)  and  activities  of  signaling  net¬ 
works  (Hurtley  and  Helmuth,  2003).  Many  of  these  scientific 
questions  are  well-suited  to  molecular  proximity  methods  such  as 
FRET,  FCS,  FRAR  Eventually,  these  types  of  analysis  will  lead  us 
to  a  new  branch  of  cell  biology  —  functional  cell  physiology. 
Imaging  applied  not  only  to  living  cells,  but  also  to  living  organ¬ 
isms,  should  keep  us  occupied  for  the  next  few  decades!  (Fig. 
45.12). 

Today,  in  addition  to  having  a  wide  variety  of  fluorescent 
markers  for  subcellular  compartments  (Lippincott-Schwartz  and 
Patterson,  2003),  we  also  have  a  broad  palette  of  established  fluo¬ 
rescent  biosensors  (Ting  et  al.,  2001)  and  reporter  molecules  that 
can  be  used  to  measure  cellular  activities  (Miyawaki  et  al.,  1997; 
Miyawaki  et  al,  1999;  Zhang  et  al.,  2001;  Sato  et  al.,  2002;  Sato 
et  al.,  2003;  Violin  et  al.,  2003;  Umezawa,  2005,  see  also  Chap¬ 
ters  16  and  17,  this  volume). 


The  importance  of  using  FRET  to  probe  the  functional  assem¬ 
bly  of  multi-subunit  protein  complexes,  such  as  the  neuronal  recep¬ 
tors  AMPA,  NMDR,  the  hexameric  connexins  in  the  plasma 
membrane  or  the  immuno-recognition  complexes,  will  drive  the 
development  of  new  FRET  partners  with  custom-designed  spectral 
properties  (Tsien,  2004;  Galperin  et  al.,  2004).  New  high-speed/ 
high-resolution  microscopes  with  improved  sensitivity  are  under 
constant  development  (Stephens  and  Allan,  2003;  Goldman  and 
Spector,  2005),  and  will  provide  new  tools  transforming  the  single¬ 
molecule  analyses  of  “cellular  imaging”  into  “molecular  imaging.” 

Cellular  processes  in  the  plasma  membrane,  the  nucleus  and 
other  cellular  compartments  such  as  Golgi,  ER,  etc.,  will  be 
resolved  in  real-time  and  at  high  spatial  resolution.  The  resulting 
improvements  in  the  understanding  of  the  cellular  physiology  of 
both  healthy  and  diseased  organisms  (including  tumors),  will 
support  new  frontiers  in  drug  discovery.  These  are  fields  in  which 
FRET  can  be  applied  to  elucidate  the  core  of  many  of  the  biolog¬ 
ical  processes  that  must  be  understood  before  we  move  on  to  clin¬ 
ical  diagnostics  and  drug  screening. 
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FIGURE  45.1 2.  Experiment  performed  during  the  UBC  3D  microscopy  course,  2004.  A  simple  experimental  setup  for  whole-organism  imaging  of  a  transgenic 
GFP  mouse  is  shown  in  (A,  B).  (C)  Fluorescence  of  GFP-expressing  neuronal  cells  in  cerebellum  can  be  seen  under  the  microscope  even  with  5x  magnifica¬ 
tion,  and  this  allows  preparation  of  tissue  slices  for  high-resolution  microscopy;  (D)  the  insert  shows  the  new  “2004  palette  of  non-oligomerizing  fluorescent 
proteins”  (taken  from  Tsien,  2004),  illustrating  the  future  of  live-cell  imaging. 


806  Chapter  45  •  I.  Majoul  et  al. 


IN  VIVO  MOLECULAR  IMAGING 

The  field  of  molecular  imaging  is  developing  rapidly.  Two  rela¬ 
tively  mature  disciplines  —  molecular  biology  and  radiology  are 
coming  together  to  create  an  interdisciplinary  approach  to  the 
biology  of  tissues  and  cells  and  the  interactions  of  molecules. 
Molecular  imaging  will  add  a  new  dimension  to  anatomical 
imaging  and  the  evaluation  of  histological  slides,  providing  us  with 
the  detailed  information  on  the  distribution  and  function  of 
reporters  and  marker  molecules  needed  to  deepen  our  insight 
into  physiological  differences  between  normal  and  diseased 
tissues. 

The  equipment  required  to  provide  this  information  is  now 
becoming  available.  A  new  generation  of  “stick  lens”  objectives 
(Fig.  45.13)  was  recently  introduced  by  Olympus  to  permit  one  to 
image  planes  inside  the  living  animal  without  resorting  to  the  fiber- 
optical  confocal  endoscopes  described  in  Chapter  26  ( this  volume). 
Exciting  developments  in  high-content  screening  are  described  in 
the  next  chapter. 

Many  other  initiatives  are  currently  focused  on  the  develop¬ 
ment  of  high-specificity/high-sensitivity  probes  designed  to 
improve  the  detection  sensitivity  of  molecular  imaging  by  10-  to 
100-fold.  A  feature  of  this  new  generation  of  probes  will  be  the 
development  of  “Molecular  Libraries”  that  will  help  to  overcome 
current  limitations  in  our  ability  to  detect  and  image  specific  mo¬ 


lecular  events  in  animal  models  and  humans.  Methods,  such  as 
microSpect  imaging,  as  recently  described  by  Choi  et  al.  (2005), 
demonstrate  the  potential  for  the  pre-clinical  application  of  in  vivo 
molecular  imaging,  a  topic  also  discussed  in  Chapters  26  and  38. 
The  outlook  looks  both  bright  and  multi-colored! 
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FIGURE  45.1 3.  (A)  Three,  new  “stick  lens”  objectives  have  been  developed  by  Olympus  especially  for  in  vivo  imaging  of  fluorescent  signals  in  whole  animals. 
These  objectives  were  designed  to  be  used  with  a  new,  miniaturized,  tiltable  confocal  scan-head,  the  Olympus  IV100  Intravital  LSM  (B).  The  system  uses 
an  AOTF  to  select  between  488,  561,  633  and  748 nm  lasers.  Three  different  stick  objectives:  27x/NA  0.7  (3.5  mm  diameter,  FOV:  200pm),  20x/NA  0.5  and 
(1.3mm  diameter,  FOV:  200pm),  6x/NA  0.14  (1.3mm  diameter,  FOV:  670pm)  and  a  comprehensive  double-galvanometer  scanner  provided  flexible  raster 
scanning  over  a  wide  magnification  range.  A  fixed,  4-channel  dichroic  beamsplitter  and  3  turrets,  each  with  6  emission  filters,  provides  wavelength  selection  for 
3  PMT  detector  channels.  To  minimize  the  effect  of  invasive  microscopy  on  living  animals,  the  angle  at  which  the  objective  and  the  scanning  head  approached 
the  animal  is  adjustable  (B)  and  the  whole  apparatus  is  enclosed  in  a  light  shield  that  can  also  serve  as  means  of  administering  anesthetic  gases. 
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INTRODUCTION 

The  vast  amount  of  biological  information  emerging  from  large- 
scale  high-volume  genomics  and  proteomics  has  significantly 
changed  biological  research.  The  problem  is,  however,  that  we  cur¬ 
rently  do  not  have  the  methods  to  analyze  the  enormous  complex¬ 
ity  of  cells  or  cellular  systems  within  reasonable  time  intervals 
using  the  traditional  approach  of  hypothesis  formulation  followed 
by  experimental  verification.  One  challenge  is  that  the  data  gained 
from  observing  a  few  cells  is  rarely  suited  for  statistical  evaluation. 
This  limitation  has  stimulated  development  of  new  approaches  to 
the  comprehensive  collection  of  information  with  more  immediate 
biological  importance  at  the  cellular  level  (cytomics).  This  effort 
spans  both  basic  biomedical  research  and  drug  development. 

Recent  advances  in  combinatorial  chemistry,  genomics,  and 
high  throughput  screening  (HTS),  as  they  apply  to  drug  dis¬ 
covery,  have  produced  a  deluge  of  potential  therapeutic  “hits.” 
Conventional  cell-based  HTS  assays  rely  on  measuring  single 
parameters  from  lysates  of  cell  populations  grown  in  multi-well 
high-density  plates  (whole-well  average  measurement),  at  single 
time  points  (end-point  format)  with  very  limited  temporal  resolu¬ 
tion  and  no  spatial  resolution.  Using  standard  or  more  advanced 
plate  readers  it  is  possible  to  reach  the  speed  requirement  of  HT 
primary  campaigns  for  drug  discovery.  Unfortunately,  although 
advances  in  automation  offer  increasing  rates  of  HTS  throughput, 
the  past  few  years  have  witnessed  a  dearth  of  newly  approved 
drugs.  At  the  same  time,  slowness  in  the  identification  and  valida¬ 
tion  of  cellular  targets  potentially  affected  by  leads  originally  iden¬ 
tified  by  HTS  has  created  a  bottleneck  in  the  drug  discovery 
process.  Many  promising  compounds  have  failed  in  costly  animal 
studies  and  in  even  costlier  clinical  trials  farther  down  the  drug 
discovery  pipeline. 

In  response,  new  screening  approaches  have  recently  been  pro¬ 
posed  that  are  referred  to  as  high-content  screening  (HCS).  HCS 
relies  on  more  sophisticated  assays  where  the  higher  quality  (as 
opposed  to  higher  quantity  in  HTS)  of  the  data  being  generated 
is  believed  both  to  increase  the  likelihood  of  discovery  and  to 
accelerate  the  profiling  of  successful  drugs  (Taylor  et  al.,  2001; 
Dove,  2003). 

HCS  readouts  can  include  simultaneous  measurements  of 
several  physiological  parameters  from  individual  cells  living 
within  a  population.  HCS  provides  more  complex  but  also  more 


informative  data  than  single,  simpler  HTS  “presence-or-absence” 
readouts.  Importantly,  high-content  data  allow  for  a  faster,  com¬ 
prehensive,  mechanistic  understanding  of  all  the  cellular  functions 
that  are  affected  by  the  signaling  molecules/drug  candidates  under 
test  and  permit  us  to  understand  their  physiological  relevance,  in 
the  context  of  an  intact  biological  system. 

In  recent  years,  single-cell  imaging  has  been  recognized  as  a 
revolutionary  way  to  look  at  biology  (Comley,  2005;  Cole  et  al., 
2003).  As  a  result,  imaging  holds  a  great  potential  in  HCS  because 
of  the  dynamic  and  multi-dimensional  (multiple  targets  over  time 
and  space)  aspect  of  the  data  generated. 

HTS  technologies  fail  because  they  look  at  cell  populations, 
averaging  results  over  the  entire  well  and  wrongly  assuming  the 
cells  to  be  homogeneous.  Although  such  techniques  have  provided 
valuable  cell  biological  insights,  they  necessarily  miss  or  down¬ 
play  subtle  or  rare  cellular  responses.  As  an  example,  if  a  small 
subset  within  a  cell  population  exhibits  a  cytotoxic  reaction  to  a 
test  compound,  such  unwanted  effect  may  go  undetected  if  only 
the  overall  population  response  is  considered.  This  could  cause  the 
compound  not  to  pass  the  more  stringent  screenings  in  costly 
animal  models  later  in  the  drug  discovery  process.  Single-cell 
imaging  on  the  other  hand,  allows  one  to  appreciate  the  heteroge¬ 
neous  behavior  of  cells  (presence/absence  of  response,  magnitude 
of  response,  temporal/spatial  dynamics)  and  to  characterize  it  in 
the  context  of  the  cells’  innate  biological  variability  (e.g.,  differ¬ 
ent  stages  in  the  cell  cycle,  receptor  expression,  genetic  makeup). 
Consequently,  drug  profiling  can  take  place  in  a  physiological 
background  that  is  more  representative  of  an  in  vivo  scenario  and, 
therefore,  more  likely  to  reflect  the  true  individual  global  response. 

To  fulfill  their  purpose,  HCS  assays  must  be  able  to  perform 
multiplexed  analyses  (preferably  simultaneously  and  in  real  time) 
in  the  same  cell  population  or,  in  a  higher  resolution  imaging  mode, 
within  spatially  distinct  domains  inside  the  same  cells.  A  wealth  of 
non-invasive  fluorescent  probes  have  been  developed  and  are  now 
available  for  monitoring  cellular/subcellular  targets  in  living 
specimens.  In  particular,  fluorescence  imaging  of  signalling  path¬ 
ways  is  a  powerful  new  tool  for  understanding  signal  transduction 
cross-talks  (DeBernardi  and  Brooker,  1998;  Meyer  and  Truel, 
2003),  as  well  as  the  molecular  architecture  and  functionality  of  cell 
systems  conceived  as  integrated  biological  entities  or  cytomes 
(Valet,  2003).  Rather  than  dissecting  cellular  pathways  in  order  to 
study  single  components  in  vitro,  it  is  becoming  possible  to  watch 
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important  biochemical  events  unfold  at  the  cellular  and  even  organ- 
ismal  level.  Confocal  microscopy,  with  its  high-content  information 
yield,  is  the  tool  of  choice  that  provides  the  required  spatial  and  tem¬ 
poral  imaging  resolution.  As  is  well  documented  in  the  other  chap¬ 
ters  of  this  book,  confocal  imaging  techniques  have  the  potential  to 
generate  phenotypic,  functional,  and  molecular  information  via 
quantitative  assessments  of  subcellular,  cellular,  and  tissue  con¬ 
structs  that  have  long  proved  capable  of  elucidating  molecular 
mechanisms.  Contextual  biology  (where/when/how  it  happens/ 
what  happens  then)  is  indeed  the  current  frontier  of  the  life  sciences. 

PLATFORMS  USED  FOR  AUTOMATED 
CONFOCAL  IMAGING 

With  their  larger  pinhole  size,  lamp-based,  spinning-disk  confocal 
systems  provide  greater  scanning  speed  and  light  transmission,  but 
thicker  optical  sections  than  their  laser-based,  line-scanning, 
single-beam  relatives.  However,  for  researchers  who  must  balance 
the  need  for  thin  sectioning  against  the  requirement  for  full-frame, 
high-speed,  real-time  confocal  imaging,  spinning-disk-based  con¬ 
focal  systems  (see  Chapter  10,  this  volume)  are  the  tool  of  choice. 
They  provide  the  ability  to  monitor  rapidly  occurring  events  within 
living  cells  without  compromising  xy-resolution,  and  record  events 
that  would  be  missed  during  the  time  required  to  perform  a  single¬ 
beam  laser  scan. 

Besides  reduced  maintenance  cost,  other  important  advantages 
offered  by  spinning-disk-based  systems  include  using  white  light 
sources  rather  than  lasers  permitting  full  spectrum  excitation  capa¬ 
bility  [from  ultraviolet  (UV)  to  near-infrared  (IR)],  and,  because  of 
the  lower  light  intensity  delivered  to  the  sample  (in  photons/s/|im2), 
reduced  photobleaching  of  the  probes  and  less  phototoxicity  to  the 
specimen  (see  Chapters  38  and  39,  this  volume). 

These  features  are  of  crucial  importance  for  high-content 
imaging  where  multiple  fluorescent  probes  (whose  absorption 
peaks  often  do  not  match  laser  lines)  can  be  efficiently  excited 
throughout  the  spectrum  or  where  prolonged  monitoring  of  fluo- 
rescently  labeled  living  cells  is  needed  to  follow  real-time  translo¬ 
cation  events. 

A  number  of  automated  imaging  platforms  have  been  intro¬ 
duced  to  the  market.  Each  instrument  solves  the  problems  related 
to  imaging  and  specimen  handling,  fluidics/kinetics  capabilities, 
sample  throughput,  and  data  analysis  in  different  ways  to  accom¬ 
plish  HCS.  Some  of  these  imaging  workstations  are  confocal 
(either  laser  line  based  or  spinning  disk  based)  while  some  are  not 
(see  Table  46.1).  It  is  clear  this  is  a  growing  new  segment  of  the 
imaging  community  that  will  evolve  extensively  over  the  next 
decade. 

One  class  of  commercially  available  confocal  imaging  work¬ 
stations  is  represented  by  an  automated  bioimager  that  combines 
full  spectrum,  Nipkow-based  confocal  microscopy  with  kinetic 
and  end-point  single-cell  imaging  capabilities  in  a  compact,  flexi¬ 
ble  benchtop  unit  (PathwayHT,  Bioimaging  Systems,  BD  Bio¬ 
science,  Rockville,  MD).  The  system  is  suitable  for  both  academic 
basic  research  and  pharmaceutical  applications  in  assay  develop¬ 
ment  and  secondary  screening  (Vanek  and  Tunon,  2002). 

When  the  goal  of  the  experiment  is  to  measure  multiple  cellu¬ 
lar  events  at  once,  it  is  advantageous  to  employ  multiple  filter  sets 
and  multiple  light  sources.  For  example,  two  mercury  arcs  allow 
full-spectrum  illumination,  from  350  nm  to  near  IR,  while  16  exci¬ 
tation  filters,  10  dichroic  mirrors,  and  8  emission  filters  (housed  in 
software-controlled  Alter  wheels)  enable  the  use  of  most  available 
dyes,  both  single  wavelength  [e.g.,  green  fluorescent  protein  (GFP), 


Fluo4,  rhodamine,  Alexa  dyes)  and  ratiometric  (e.g.,  fura-2  and 
JC-1).  Multiple  Alter  sets  provide  flexibility  both  in  assay  develop¬ 
ment,  where  fluorescent  probes  with  a  wide  range  of  excitation 
requirements  must  be  tested,  and  in  secondary  screening,  where 
multiple  cellular  events  must  be  measured  in  a  single  assay  to 
provide  improved  speed  and  more  relevant  biological  information. 

The  spinning-disk  confocal  module  (readily  interchangeable 
with  widefleld  mode)  provides  real-time  confocal  imaging  of  fast 
events  in  live  cells  with  high  signal-to-noise  ratio  and  minimal 
photobleaching  and  phototoxicity.  High-resolution  multi-color 
confocal  z-stacks  and  three-dimensional  (3D)  analysis  capabilities 
are  available  for  localization/redistribution  studies  of  multiple  flu- 
orescently  tagged  biomolecules  within  subcellular  compartments. 

Real-time  confocal  imaging  requires  sensitive  cameras,  such 
as  high  quantum  efflciency  charge-coupled  device  (CCD)  cameras 
that  provide  high  resolution  and  optimal  signal-to-noise  ratio. 
Once  the  new  electron  multiplier  (EM)-CCD  cameras  (with  excep¬ 
tional  sensitivity,  greater  speed,  and  minimal  readout  noise) 
become  more  widely  available,  it  is  expected  that  automated 
imaging  platforms  will  greatly  beneflt  from  their  use  (see 
Chapters  2  and  12). 

Other  required  features  include  integrated  temperature  and 
C02  control  to  ensure  the  consistency  of  physiological  data.  On¬ 
stage  liquid  handling  for  automated  drug  delivery  to  living  cells 
provides  an  image-while-you-add  capability  and  allows  one  to 
follow  fast-onset  kinetic  responses  (Fig.  46.1). 

To  permit  imaging  of  adjacent  flelds  while  the  stage  and  the 
sample  remain  stationary,  proprietary  optical  designs  allow  objec¬ 
tives  mounted  below  the  specimen  to  move  in  x-y-z  directions 
while  both  the  stage  and  the  sample  remain  stationary.  This  enables 
suspended  or  poorly  adherent  cells  to  be  imaged  without  mechan¬ 
ical  disruption.  Motorized  x-,  v-  ,  and  z-positioning  of  the  objec¬ 
tive  with  lOOnm  fry)  and  50  nm  (z)  precision  permits  revisiting 
previously  imaged  flelds.  This  feature  allows  (a)  prolonged  time- 
lapse  experiments  to  be  carried  out  simultaneously  on  multiple 
fields  by  imaging  each  field  sequentially  at  different  time  points, 
and  (b)  capture  of  montaged  images  of  samples  larger  than  the 
microscopic  field  of  view.  Montaging  is  useful  when  imaging 
tissue  arrays  or  whole  organisms,  such  as  embryos  of  Caenorhab- 
ditis  elegans,  Arabidopsis,  zebrafish,  and  Drosophila,  and  also 
when  dealing  with  highly  heterogeneous  populations  with  rare 
responders,  where  a  larger  sample  size  is  needed  to  improve  sta¬ 
tistical  significance. 

Depending  upon  the  resolution  required  by  a  specific  applica¬ 
tion,  the  type  of  imaging  substrate  used  (glass/plastic)  and  the 
throughput  desired,  different  magnification  objectives  (2x,  4x, 
lOx,  20x,  40x  dry  and  60x  oil)  can  be  employed.  These  optics  can 
be  optimized  for  the  demands  of  the  assay.  Low  power,  low  NA 
dry  lenses  are  used  for  high  speed  in  non-confocal  applications. 
High  NA  immersion  lenses  matched  to  the  dish  in  which  the  cells 
are  cultured  are  used  for  slower  studies  where  higher  spatial  reso¬ 
lution  is  needed. 

As  with  other  automated  bioimagers,  the  system  is  amenable 
to  robotic  integration  with  industry  standard  plate  handling  and 
batch  processing  devices  and  can  be  operated  in  a  fully  automated 
mode  under  software  control.  A  Windows-based  software  system 
seamlessly  integrates  image  capture,  sample  navigation  (for  multi¬ 
well  plates;  chamber  slides,  standard  microscope  slides,  culture 
dishes,  etc.),  liquid  addition,  and  on-the-fly  image  analysis  (see 
Chapter  14,  this  volume).  A  key  requirement  of  the  system  soft¬ 
ware  is  the  ability  to  drive  a  broad  range  of  endpoints  (biomarker 
identification,  localization,  translocation,  redistribution)  and  kinet¬ 
ics  (ion/second  messenger  real-time  measurements). 
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TABLE  46.1.  Non-Confocal  HCS  Systems 

Amersham  IN  Cell 
Analyzer  1000 

Axon 

ImageXpress 

Beckmann- 

Coulter/Q3DM 

Cellomics 

Array  scan  Vti 

Cellomics 

KineticScan 

Universal  Imaging 
Discovery  1 

URL 

www.axon.com 

www.beckmann.com 

www.cellomics.com 

www.universal- 

imaging.com 

Imaging 

system 

CCD  camera-based 

Non-confocal 

Nikon  objectives 

CCD  camera-based 

Non-confocal 

Nikon  objectives 

CCD  camera-based 
Non-confocal  system 

CCD  camera-based 

Non-confocal 

ZeissApotome 

Grating  optional 

Zeiss  objectives 

CCD  camera-based 

Non-confocal 

Zeiss  objectives 

CCD  camera-based 

Non-confocal  Nikon 
objectives 

Light  source 

Xenon  lamp  and 
filter  wheel 

Xenon  lamp  and 
filter  wheel 

Mercury  lamp 

Mercury  lamp  and 
filter  wheel 

Mercury-xenon 

lamp 

Lamp-based  and 
filter  wheels 

Excitation/ 

emmission 

filter  # 

6/6 

10/10 

10/10 

10/8 

8/8 

10/10 

Binocular 

eyepiece 

No 

No 

No 

No 

No 

No 

Objective 

choices 

4x,  lOx,  20x,  40x 

4x,  lOx.  20x,  40x 

4x  to  40x 

5x,  lOx,  20x, 

40x  &  60x 

5x,  lOx,  20x,  40x 

2x,  4x,  10,  20x,  40x 

Plates 

96,  384  well 

96,  384,  1536  well 

96-1536  well 

&  slides 

96-384  well 

96-384  well 

96,  384,  1536  well 

Environmental 

controls 

Temp  37  or  ambient 

Temp 

co2 

No 

No 

Temp 

C02 

Humidity 

No 

Liquid 

dispensing 

Yes 

Yes 

No 

No 

Yes 

No 

Confocal  HCS  Systems 

Amersham  IN  Cell  Analyzer  3000 

BD  Pathway  HT 

Evotec.  Opera 

URL 

www.  amersham  .com 

www.bdbiosciences.com 

www.evotec-technologies.com 

Imaging  system 

CCD  camera-based  confocal  line  scanning 
Nikon  objectives 

CCD  camera-based  confocal  system 
Nipkow  spinning  disk 

Olympus  objectives 

CCD  camera-based  confocal 
spinning  disk  (Yokogawa) 

Light  source 

2  lasers 

Krypton  and  argon  3  lines 

Mercury  lamp  and 
filter  wheel 

4  lasers  and 

xenon  lamp  for  UV  (non  confocal) 

Excitation/emmission 

filter  # 

3/8 

16/8 

4/6 

Binocular  eyepiece 

No 

Yes 

No 

Objective  choices 

40x 

2x,  4x,  lOx,  20x,  40x,  60x, 

lOx,  20x,  40x,  60x 

Plates 

96-384  well 

96-384  well  &  slides;  slide  holder 
provided 

96,  384,  1536  well 

Environmental  controls 

Temp 

C02 

Humidity 

Temp 

C02 

Temp 

C02 

Liquid  dispensing 

Yes,  limited 

Yes 

Yes,  optional 

TYPES  OF  ASSAYS 

Selected  automated  confocal  imaging  platforms  support  current 
kinetics  (e.g.,  calcium  and  sodium  fluxes,  mitochondria  and 
plasma  membrane  potential  changes,  etc.)  and  endpoint  cell  assays 
(e.g.,  cytotoxicity,  apoptosis,  cell  cycle,  translocations)  using 
common  commercially  available  fluorescent  probes  (Table  46.2) 
Multiplexed  assays  are  only  limited  by  the  optical  compatibility 
of  the  fluorescent  probes  used.  In  particular,  selected  applications 
amenable  to  automated  high-content  imaging  will  benefit  from  the 


advantages  provided  by  confocal  microscopy  including,  but  not 
limited  to,  the  following: 

•  Live/fixed  cell  assay  where  fluorescent  probes  photobleach 
rapidly,  either  because  of  the  chemical  nature  of  the  fluo- 
rophore  or  because  of  high  sampling  frequency  (fast  kinetics 
imaging  with  no  or  minimal  delay  between  two  illumination 
cycles)  [Fig.  46.2(A),  left  panel]. 

•  Live-cell  assays  where  the  fluorescent  probes  used  to  label 
cells  are  not  removed  during  the  imaging  phase.  This  might 
happen  for  several  reasons: 
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FIGURE  46.1.  Automated  single-cell  kinetic  calcium  imaging.  Screenshot  from  PathwayHT  showing  single-cell,  real-time  calcium  fluxes  in  HeLa  cells.  Cells 
were  grown  on  a  96-well  plastic  plate  overnight,  labeled  with  2u M  Fluo-4  and  imaged  (20x,  0.75NA,  at  37°C  and  5%  C02)  in  resting  conditions  and  during/after 
automated  addition  of  ATP  (50  u M),  a  purinergic  receptor  agonist  that  mobilizes  intracellular  Ca2+.  On-the-fly  visualization  of  single-cell-derived  kinetic  traces 
reveals  profound  heterogeneity  in  the  magnitude/dynamic  of  cell  responses  to  ATP.  After  completing  a  user-defined  time-lapse,  average  responses  are  plotted  in 
each  well  enabling  ready  visualization  of  overall  cells’  responsiveness.  Color-coded  single-cell  traces  identify  corresponding  cells  (cell  segmentation  mask  shown) 
allowing  ready  comparison  of  cell  morphological  and  biochemical  patterns. 


TABLE  46.2.  Methods  Suitable  for  HCS  Fluorescence  Analysis  of  Cells 


Application 

Parameter  Measured 

Dyes 

Information 

Areas  of  Study 

Cytotoxicity 

Cell  viability 

Cell  proliferation 

Calcein  AM  —  Live  cells 

Propidium  Iodide  —  Dead  cells 
Hoechst/DAPI  nuclear  dyes 

Number/percentage  of 
live/dead  cells  over  a 
population 

ADME/Tox 

Cancer  biology 

Apoptosis 

Mitochondria  potential  changes 
Phosphatidylserine  extemalization 
Caspase  activation 

Chromatin  condensation/ 
fragmentation 

DNA  strand  breaks 

JC-1,  TMRE 

Annexin  V 

Caspase  sensors  (cytoplasmic 
and  nuclear) 

Hoechst/DAPI 

TUNEL 

Early/intermediate/late 
apoptosis  stage 
Apoptosis/Necrosis 

DNA  damages 

Pharmacology 
Toxicology 
Cell/cancer  biology 
Genotoxicity 

Translocation 

Stimulus-induced  target 
redistribution,  e.g.: 

NF-kB  (cytoplasm  to  nucleus) 

PKCa  (cytosol  to  membrane) 

STATs  (cytosol  to  nucleus) 
MAPKAk2  (nucleus  to  cytoplasm) 
Receptor  internalization  and  fate 

Detection  is  provided  by  antibody- 
based  reagents 

for  endogenously  expressed  targets. 

Alternatively,  targets  are  expressed  in 
cells  as  fusion  protein  with 

GFP,  its  variants  or  other  fluorescent 
protein  (biosensors). 

GFP-parrestin,  CypHer" 

Organelle  changes  in 
fluorescence  intensity/ 
patterns  are  indicators  of 
specific  target  activation 

Cell  signalling 

Pathway  cross-talk 
Immunology 

Receptor  activation 
and  desensitization 

Kinetics 

Calcium 

Sodium 

Membrane  potential  (MP) 
Mitochondria  potential 

Multiplexed 

cAMP 

Fluo-4,  fura-2 

SBFI 

Various  MP  Dyes 

JC-1 

See  below*7 

ACT:Onec 

Kinetic  changes  in  fluorescence 
intensity  (or  ratio)  report  live 
cells  ion/second  messengers 
response  to  stimuli. 

Multiplexing  reveals  real-time 
signaling  cross-talk 

Cell  signaling 
Pharmacology 
Receptor/ion 
channel 

Activation 

Signaling, 

(cross-talk) 

Cell  cycle 

DNA  content  and  replication  status 
Stage-specific  target  expression 
and/or  phosphorylation 

Hoechst/DAPI/Anti-nucleotide 
(e.g.,  BrdU)  antibodies 
Anti-Histone/cyclins/cyclin-dependent 
kinase  antibodies 

Cyclin-fusion  protein 

DNA  content  (2n/4n)  in  Gl, 

G2,  M  phase. 

Cell  cycle  stages  (e.g., 
phosphoH3  histone 
indicates  M  phase) 

Cancer  biology 
Genomics 

Neurite 

outgrowth 

Neurite  extensions 

Neuron  specific  antibody-based 
reagents 

Number/length/branching 
upon  neurite  inducing 
stimuli 

Neuroscience 

Regeneration 

Differentiation 

Examples  of  cell-based  assays  suitable  for  automated  imaging  workstations.  Listed  are  representative  assays  and  fluorescent  probes;  more  specifics  on  dyes  can  be  found 
in  Table  16.1,  this  volume.  Kits,  cell  lines,  and  fluorescent  agents  are  available  from  independent  suppliers  or  imaging  platform  vendors  that  offer  a  variety  of  protocols. 
Examples  include  "GFP-Parrestin  =  Transfluor  technology  (by  Norak  Biosciences  Inc);  CypHer,  pH  sensitive  dye  (Amersham  GE  Healthcare);  ^calcium  and  sodium 
(Fluo-4  and  SBFI);  calcium  and  membrane  potential  (fura-2  and  various  MP  probes);  calcium  and  mitochondria  membrane  potential  (Fura-2  and  JC-1) c  ACT:One  cell  lines 
and  dyes  for  real-time  cAMP  measurement  in  live  cells  (Bioimaging  Systems,  BD  Bioscience). 
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FIGURE  46.2.  Confocal  imaging  of  living  cells.  (A)  Confocal  imaging  of  primary  rat  cerebellar  granule  neurons.  Cells  were  labeled  with  a  membrane- 
potential  dye  and  imaged  confocally  on  PathwayHT  (20x,  0.75NA)  at  37°C  and  5%  C02  without  dye  removal.  Left  panel:  cells  were  imaged  under  continuous 
illumination  either  in  widefield  or  confocal  mode.  By  100  sec,  a  75%  loss  in  the  probe-generated  fluorescence  signal  was  detected  in  cells  imaged  in  widefield 
mode  while  in  confocal  mode,  only  -25%  of  loss  was  detected.  The  reduced  photobleaching  provided  by  lamp-based  confocal  imaging  platforms  allows 
monitoring  responses  whose  fast  onset  might  require  high-frequency  sample  illumination.  Right  panel:  Membrane-potential  depolarization  induced  by  glutamate 
(If) uA7 }  and  KC1  (50mM)  in  primary  neurons.  Trace  represents  mean  ±  SD.  (B)  Real-time  glucocorticoid  receptor-nuclear  translocation.  Living  MMTV  mouse 
cells  stably  expressing  GFP-tagged  glucocorticoid  receptors  (GR)  were  imaged  confocally  on  PathwayHT  (40x,  0.9NA)  at  37°C  and  5%  C02.  GR  nuclear  translo¬ 
cation  was  induced  by  dexamethasone  (In M)  and  followed  over  time;  images  were  captured  every  20 sec.  From  left  to  right:  images  at  0,  5,  10,  15min  after 
dexamethasone  addition.  Cells  were  kindly  provided  by  Dr.  G.  Hager,  NCI,  NIH,  Bethesda,  MD. 


—  the  probe  needs  to  be  in  contact  with  the  cells  for  functional 
measurements  to  be  made  (e.g.,  membrane  potential  dye) 
[Fig.  46.2(A),  right  panel]; 

—  a  homogeneous  assay  format  with  no  washing  steps  is 
needed  to  optimize  cell  responsiveness  and  improve  speed; 

—  active,  postloading  dye  leakage  from  cells.  Drug-resistant 
cancer  cells  are  notorious  for  their  active  extrusion  of  small 
fluorescent  probes  because  they  over-expressed  drug  efflux 
pumps.  The  ability  of  the  confocal  microscope  to  image  a 
single  plane  reduces  the  effect  of  the  background  generated 
by  this  excess  probe,  increasing  signal-to-noise  ratio  and 
assay  sensitivity. 

•  Live-cell  assays  in  which  phototoxicity  might  interfere  with 
biological  functions  (e.g.,  prolonged  time  courses  where  small 
organisms  expressing  GFP  or  other  fluorescent  fusion  proteins 
are  followed  through  multiple  cellular  divisions  or  where  cell 
lineage  is  being  investigated)  [Fig.  46.2(B)]. 

•  Assays  employing  in  vitro  systems  consisting  of  two- 
dimensional  (2D),  mixed-cell  populations  (e.g.,  primary 
neuron  cultures  with  supporting  glial  cells,  or  special-purpose 
feeder  layers  for  differentiating  embryonic  stem  cells). 
Confocal  microscopy  will  allow  imaging  at  specified  focal 
planes  to  emphasize  the  response  of  the  desired  cell  population. 

•  Assays  where  high-resolution  spatial  discrimination  is  required 
for  localizing  fluorescent  markers  to  specific  subcellular  com¬ 
partments.  This  process  could  be  as  simple  as  assigning  a  given 
probe  to  a  given  structure  (target  identification  or  confirmation 
of  organelle  specificity  for  novel  biomarkers)  or  as  sophisti¬ 
cated  as  generating  complete  confocal  image  database  towards 
a  systematics  for  protein  subcellular  localization  (Boland  el  al., 
2001). 


•  Cell  surface  targeting  assays  such  as  those  aimed  at  identify¬ 
ing  biomolecules  that  either  are  constitutively  present  on  the 
outside  of  the  plasma  membrane  (e.g.,  surface  antigens  and 
receptors)  [Fig.  46.3(A)]  or  become  newly  exposed  on  cell 
surfaces  in  response  to  a  stimulus  (e.g.,  phosphatidylserine, 
normally  found  on  the  inner  side  of  the  plasma  membrane,  gets 
externalized  during  drug-induced  apoptosis  and  its  identifica¬ 
tion  by  fluorescent  annexin  V  provides  a  robust  assay  for  early 
apoptosis  classification). 

•  Multi-color  applications  where  different  cell  types  within  a 
tissue  or  different  organelles  or  proteins  within  a  cell  are 
labeled  with  specific  probes  whose  spatial  localization  or  colo¬ 
calization  requires  the  high  spatial  resolution  provided  by  con¬ 
focal  imaging  [Fig.  46.3(B)].  This  scenario  might  include 
fixed  samples  fluorescently  labeled  with  multiple  antibodies 
conjugated  to  either  conventional  organic  dyes  or  recently 
developed  semiconductor  nanocrystals  (quantum  dots;  Watson 
et  al.,  2003;  see  Chapters  16  and  17,  this  volume)  as  wells 
as  live/fixed  cells  expressing  multiple  GFPs,  or  other  fluores¬ 
cent  fusion  proteins,  and  targeting  different  subcellular 
compartments.  Phenotypic  profiling  of  either  tissues  or  cell 
populations  labeled  with  multiple,  organelle-specific  probes 
represents  a  recently  developed  approach  to  screen  mole¬ 
cules  that  alter  specific  cellular  parameters,  allowing  the 
identification  of  compounds  (or  siRNAs)  that  interact  with 
different  systems  in  the  cell  such  as  the  cytoskeleton  or 
various  signal  transduction  pathways  (Yarrow  et  al., 
2003). 

•  Pharmacological  screening  of  the  hepatoxic  potential  of 
compounds  in  primary  liver  cells  or  liver  cells  lines,  using  con¬ 
focal  imaging,  was  reported  to  provide  visual  discrimination 
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FIGURE  46.3.  High  resolution  confocal  imaging  of  surface  and  intracellular  target.  (A)  Peripheral  Blood  Mononuclear  Cells  (PBMC)  labeled  with  an  APC- 
tagged,  anti-CD4  monoclonal  antibody  (BD  Biosciences)  (pseudocolored  in  green)  and  counterstained  with  the  nuclear  dye,  Hoechst  33324  (pseudocolored  in 
red).  Confocal  imaging  (60x  oil,  1.4NA)  clearly  shows  plasma-membrane-specific  localization  of  cell  surface  CD4  receptors  on  a  selected  subset  of  PBMC.  (B) 
HeLa  cells  expressing  three  organelle-targeted  fluorescent  fusion  proteins:  AmCyan-peroxisomes,  ZsYellow-nucleus,  HcRed-Mitochondria  (Clontech,  BD  Bio¬ 
sciences)  were  fixed  and  imaged  on  PathwayHT  (40x,  0.9NA)  in  confocal  mode. 


of  compound-specific  intracellular  fat  deposition  patterns 
(McMillian  et  al.,  2001). 

•  Endpoint  translocation/redistribution  assays  where,  following 
a  relevant  stimulus,  the  labeled  target  moves  from  one  subcel- 
lular  compartment  to  another  (e.g.,  cytoplasm-to-nucleus 
translocation  of  NF-kB  transcription  factor  or  cytoplasm-to 
plasma-membrane  translocation  of  protein  kinase  Cot)  or 
undergoes  changes  in  fluorescence  pattern  [e.g.,  redistribution 
assays  where  the  fluorescence  pattern  of  receptor-bound  GFP- 
(3  arrestin  changes  from  an  evenly  diffuse  cytosolic  pattern  to 
a  more  punctate  one  (pits/vesicles  formation)  upon  agonist- 
induced  internalization  of  the  receptor]  (Fig.  46.4).  Confocal 
microscopy  provides  high-resolution  images  of  the  fluores¬ 
cence  redistribution  patterns  that  allow  more  sophisticated 
image  analysis  algorithms  (such  as  granularity)  to  be  optimally 
applied.  These  in  turn  lead  to  greater  assay  sensitivity. 


•  Assays  based  on  3D  biological  structures  (micro-organs) 
where  confocal  z-stacks  captured  over  time  can  provide  phys¬ 
iologically  relevant  insights  on  cell  proliferation,  differentia¬ 
tion,  drug  responsiveness  within  a  miniaturized  ex  vivo 
environment. 

•  Assays  using  quantum  dots  in  vivo  or  on  fixed  cells  (Chapters 
16  and  17,  this  volume),  which,  because  of  their  narrow  emis¬ 
sion  bands,  allow  one  to  monitor  the  dynamics  and  location  of 
several  different  proteins  at  once. 

An  automatic  confocal  imaging  workstation  that  provides 
control  of  temperature  and  C02,  precise  xy-position  field  re¬ 
visiting,  and  automated  multiple  color  z-sectioning  capabilities 
with  minimal  photobleaching  and  phototoxicity  will  be  the  tool  of 
choice  for  3D  model  imaging  with  applications  in  cancer  biology, 
toxicology,  therapeutics,  and  regeneration. 


NF-kB  Nuclear  Translocation 


|32  Adrenergic  Receptor  Redistribution 


FIGURE  46.4.  Automated  fluorescence  imaging  applied  to  endpoint  translocation  assays.  Left  panel:  Nuclear  translocation  of  the  transcription  factor,  NF-kB. 
HeLa  cells  were  grown  overnight  on  96-well  plastic  plates  and  treated  with  increasing  concentrations  of  IL-la  for  30  min.  Cells  were  fixed,  permeabilized  and 
labeled  with  an  anti-NF-kB  Alexa488-IgG  and  the  nuclear  dye  Hoechst  33342.  Cells  were  imaged  on  PathwayHT  in  an  automated  mode  (20x,  0.75NA)  with 
fluorescence  intensity  being  measured  in  the  nucleus  and  the  cytoplasm.  Images  show  basal  fluorescence  signal  (A)  and  IL-1 -stimulated  increase  in  nuclear 
NF-kB  fluorescence  (B).  Cell-by-cell  data  analysis  was  performed  and  a  dose-response  curve  generated  using  BD  Image  Data  Explorer.  Right  panel:  y2- 
adrenergic  receptor  trafficking  visualized  by  GFP-arrestin  fluorescence  redistribution.  Human  osteosarcoma  cells  (Transfluor  technology,  Xsira  Bioscience, 
Research  Triangle,  NC)  stably  expressing  GFP-arrestin  were  challenged  with  increasing  concentrations  of  isoproterenol,  a  y-adrenergic  receptor  agonist,  fixed 
and  nuclei  counterstained  with  DAPI.  Cells  were  imaged  on  Pathway  HT  (40x,  0.9NA)  and  GFP-p  arrestin  fluorescence  measurements  were  taken  in  cytoplas¬ 
mic  regions.  (A)  basal  fluorescence,  (B)  after  isoproterenol.  Granularity  algorithms  (variation  in  pixel  intensity  and  distribution)  were  applied  for  image  analy¬ 
sis  and  data  were  expressed  as  %  of  cells  exhibiting  a  granularity  value  above  a  user-set  threshold.  Dose  response  curves  were  generated  using  BD  Image  Data 
Explorer. 
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3D  CELL  MICROARRAY  ASSAYS 

The  cell  microarray  (CMA)  is  one  embodiment  of  the  tissue 
microarray  (TMA)  that  is  a  particularly  useful  type  of  prepared 
specimen  for  confocal  high-content  screening. 

A  cell  or  tissue  microarray  is  a  microscope  slide  containing 
between  10  and  1000  individual  samples  arranged  in  a  grid  fashion 
(Braunschweig  et  al.,  2004)  (Fig.  46.5).  This  approach  converts 
imaging  from  a  descriptive  endeavor  to  a  high-throughput  method¬ 
ology  producing  databases  and  requiring  statistical  analysis.  The 
cell-line  array  can  be  an  array  of  different  cell  lines,  or  an  array  of 
cells  grown  under  different  conditions  (or  both)  that  can  be  used 
to  address  a  variety  of  questions.  At  the  simplest  level,  a  cell 
microarray  can  be  considered  a  “Western  blot  on  a  slide,”  but  such 
a  characterization  covers  only  a  small  portion  of  the  information 
that  can  be  obtained  from  such  specimens. 

CMA  can  be  used  with  confocal  microscopy  as  a  tool  to  under¬ 
stand  the  human  cytome.  Utilizing  the  60  cell  lines  the  NCI 
employs  in  drug  testing,  the  Tissue  Array  Research  Program 
(TARP  Lab),  at  the  National  Cancer  Institute,  has  developed  a 
unique  platform,  the  NCI 60  CMA,  which  is  studied  with  a  HCS 
instrument  for  protein  expression,  localization,  and  interaction. 

Most  CMA  cells  are  obtained  from  cell-line  cultures  of  trans¬ 
formed  cells,  however,  primary  cultures  are  also  sometimes 
adequate.  Current  protocols  require  less  than  500,000  cells  as 
starting  material.  The  cells  can  be  from  clinical  samples  such  as 
leukophoresis  or  flow-sorted  specimens.  Cells  grown  in  vitro  must 
be  harvested  by  scraping.  The  cells  are  then  embedded  in  a  low 
melt  agarose  plug,  which  is  then  fixed  and  processed  as  if  it  were 
tissue  to  produce  a  paraffin-embedded  block  (Hewitt,  2004). 
Unlike  most  tissue  specimens,  formalin  is  not  the  only  choice  of 
fixation  (see  Chapter  18,  this  volume).  Ten  percent  formalin, 
although  an  excellent  fixative  for  tissue,  cross-links  the  proteins 
and  nicks  nucleic  acids,  as  well  as  producing  a  significant  amount 
of  background  autofluorescence.  As  an  alternative,  70%  ethanol 
combines  coagulative  fixation  with  dehydration,  and  does  so 
without  cross-linking  proteins  or  nicking  nucleic  acids  (Gillespie 
et  al.,  2002).  An  additional  advantage  of  using  70%  ethanol  as  a 
fixative  is  that  it  is  significantly  easier  to  translate  an  antibody  used 


in  Western  blots  or  for  the  immunostaining  of  frozen  tissue  to 
ethanol  fixed  material.  Tissues  (and  cell-agarose  plugs)  can  be 
fixed  in  70%  ethanol  exactly  as  if  fixed  in  formalin,  using  the  same 
times  and  volumes.  When  it  is  time  to  process  the  tissue  (dehy¬ 
dration  and  replacement  by  paraffin),  only  the  first  stage  on  the 
processor  (10%  formalin)  need  be  skipped,  as  the  second  stage  is 
70%  ethanol.  Although  other  alcohols  and  different  concentrations 
are  associated  with  excessive  shrinkage  and  distortion  of  cellular 
morphology,  70%  ethanol  results  in  shrinkage  comparable  with 
10%  formalin  (Gillespie  et  al.,  2002;  Chapter  18,  this  volume). 

After  processing  and  embedding,  a  3D  cell  pellet  is  obtained. 
This  can  both  be  sectioned  and  viewed  as  such,  or  the  cell  pellet 
blocks  can  be  utilized  for  the  construction  of  a  CMA.  The  methods 
of  arraying  are  straightforward,  and  can  be  fabricated  manually  or 
using  different  levels  of  automatic  instrumentation  (Braunschweig 
et  al.,  2004).  The  recipient  array  block  (CMA  block)  can  then  be 
sectioned  into  slides  for  staining  (Fig.  46.4).  Unlike  tissue  sections 
where  4  or  5  pm  sections  are  preferred,  CMAs  are  optimally  cut 
at  between  7  and  10  pm  to  allow  more  complete  3D  representation 
of  the  cells  on  the  array. 

For  confocal  imaging  of  the  arrays,  appropriate  staining  pro¬ 
tocols  must  be  worked  out.  It  is  essential  that  one  use  an  excellent 
deparaffinization  routine  as  residual  paraffin  will  reduce  the  affin¬ 
ity  of  the  antibodies,  and  will  produce  increased  autofluorescence. 
Most  normal  staining  protocols  are  inadequate,  and  three  changes 
in  xylene  for  5  min  each  should  be  considered  a  starting  point  for 
deparaffinization.  Depending  on  the  targets  and  their  cellular  local¬ 
ization,  the  use  of  4/,6-diamidino-2-phenylindole  (DAPI)  may  or 
may  not  aid  in  imaging. 

In  its  first  embodiment,  the  NCI60  CMA  consisted  of  58  cell 
lines  arrayed  with  1.00  mm  diameter  cores  of  agarose  plugs  in  trip¬ 
licate  cut  as  7  pm  sections  on  tape  slides  from  Instrumedics  [Fig. 
46.5(A)].  The  array  was  designed  to  present  -300  cells/core  for 
analysis  [Fig.  46.5(B)].  Cells  were  prepared  at  low  density  in  an 
agarose  plug,  and  cut  as  thick  sections  (7  pm)  so  that  they  would 
have  a  more  homogeneous  shape  distribution  in  three  dimensions, 
and  had  only  a  few  artificial  cell-cell  contacts.  This  CMA  is  opti¬ 
mized  for  the  investigation  of  cell  signaling  and  protein  localiza¬ 
tion,  and  their  translocation.  It  is  not  designed  for,  and  should  not 
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FIGURE  46.5.  Cell  microassays  (Such  as  NCI  160)  exhibit  different  cell  lines  that  allow  rapid  screening  of  cellular  responses. 
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FIGURE  46.6.  A  single,  1.00mm  core  of  in  vitro  cultured  cells  embedded  in 
agarose  present  on  the  NC160  CMA.  This  is  a  core  of  NCI-H23  cells,  a  cell 
line  derived  from  human  lung  cancer. 


be  used  for,  studies  of  cell  adhesion,  communication,  or  most 
cytoskeletal  studies.  The  dispersion  of  the  cells  within  the  cores 
is  designed  to  make  analysis  simpler.  This  array  is  available  on 
a  limited  basis  from  through  the  Developmental  Therapeutics 
Program  (DTP)  of  the  NCI. 

Confocal  imaging  of  a  complete  CMA  slide  with  58  individ¬ 
ual  cell  lines  (in  triplicate)  requires  an  almost  heroic  effort  and  is 
impractical  without  an  automated  stage.  Identification  of  the  indi¬ 
vidual  cores  requires  a  “scout  image”  of  the  slide.  Depending  on 
the  method  being  applied,  a  low-magnification  image  of  the  slide 
can  be  made  to  identify  and  locate  the  cores.  Alternatively,  a  high- 
magnification,  low-resolution  image  of  the  slide  can  be  made  for 
the  same  purpose.  After  the  cores  have  been  identified  and  located, 
the  instrument  is  then  programmed  to  image  the  cores  individually 
based  on  the  low-magnification  map. 

DATA  MANAGEMENT  AND 
IMAGE  INFORMATICS 

High-content  imaging  generates  a  vast  amount  of  data  that  must 
be  analyzed  using  software  analysis  programs,  either  those  pro¬ 
vided  with  the  commercial  instrumentation  or  custom  written  for 
very  specific  applications.  For  example,  if  two  proteins  are  stained, 
and  DAPI  is  used  (especially  useful  for  the  low-resolution  imaging 
to  locate  individual  cells)  on  an  NCI160  slide,  the  imaging 
demands  on  the  platform  are  significant.  One  could  anticipate 
requiring  58x  10x3  =  1740  images  (58  cell  lines  x  10  z-steps  x 
3  fluorophores)  for  a  first-pass  imaging  run.  Assuming  that  the  field 
imaged  approximately  30  cells  per  core  and  did  not  take  advan¬ 
tage  of  the  triplicate  over-sampling  on  the  array,  this  experimental 
design  would  require  58  x  10  x3x3x5  =  26,100  images  even 
if  it  attempted  to  analyze  only  half  of  cells  presented.  Although 
this  approach  sounds  overwhelming,  it  is  what  must  be  done  to 
deliver  useful  information. 

Automated  high-content  imaging  offers  the  ability  to  interro¬ 
gate  thousands  of  cells,  often  for  multiple  responses,  to  collect  a 
significant  volume  of  data  about  them,  and  finally  to  derive 
meaning  from  this  biological  information  The  data  generated  by 


this  class  of  microscopical  scanners  and  applications,  is  in  the  ter¬ 
abyte  (TB)  range  for  many  industrial  applications  and  is  stored  in 
image  databases,  along  with  the  resulting  analytical  data. 
Oracle/Web-object-based  relational  databases  management 
systems  (ORDMS)  are  the  preferred  solution  for  image  data  man¬ 
agement,  search,  and  analysis.  Secondary  image  representations, 
made  by  subsampling  or  data  compression,  mainly  serve  Web- 
based  viewing  or  control  purposes  and  have  been  developed  as  part 
of  solutions  for  the  adequate  handling  and  mining  of  such  large 
datasets.  The  goal  is  automated  analysis  of  assays  without  user 
intervention.  Given  the  variations  in  staining  and  morphology 
inherent  in  cell  populations,  this  is  a  more  challenging  task  for 
HCS  than  for  more  simple  endpoint  assays  in  HTS. 

The  first  step  in  automated  imaging  is  to  physically  find  the 
specimen,  as  well  as  the  appropriate  image  planes  for  the  sample, 
to  identify  targets  of  interest  (tissue,  cells,  subcellular  structures). 
No  single  slide  (or  plate)  is  absolutely  flat  and  it  should  be 
expected  that  some  adjustment  will  be  required  core  to  core/plate 
to  plate  to  determine  the  imaging  planes.  This  is  followed  by  image 
capture  in  one/multiple  fluorescence  channels.  The  next  step 
involves  using  a  number  of  computational  algorithms  to  segment 
the  image  into  user-defined  regions  of  interest  (ROIs)  such  that 
each  ROI  represents  a  single  cell  or  a  compartment  (e.g.,  nucleus, 
cytoplasm)  within  a  cell  from  which  the  relevant  biological  infor¬ 
mation  (changes  in  fluorescence  intensity/pattern  or  spatial  redis¬ 
tribution)  can  be  extracted  and  quantified,  either  on-the-fly  or 
offline.  Real-time  feedback  provides  information  on  the  quality  of 
the  results,  especially  in  kinetics  assays. 

Because  individual  cells  are  viewed,  it  is  possible  to  separate 
(or  gate)  cells  into  different  response  categories  (Boolean  classifi¬ 
cation).  Cells  are  grouped  (cell  classification)  based  on  a  number 
of  cellular  features  (from  one  or  multiple  probes)  such  as  overall 
fluorescence  intensity,  rate  of  rise/fall  (for  kinetics),  area,  object 
pixel  statistics  (average  intensity,  minimum,  and  maximum),  and 
variation  of  pixel  intensity  within  ROIs  (e.g.,  granulation  algo¬ 
rithm).  Class  of  cells  can  be  color-coded  for  easy  visualization,  and 
average  measurements  over  a  subset  of  cells  can  be  taken.  A  well 
classification  is  then  applied  based  on  the  number  of  cells  that,  in 
each  well,  meet  a  user-defined  threshold.  In  turn,  a  response  heat 
plate  map  that  readily  highlights  cellular  trends  or  hit  compounds 
can  be  generated.  This  process  enables  to  identify  features  that 
best  reflect  specific  biological  responses  and,  therefore,  are  good 
screenable  parameters. 

Typically,  high-content,  automated  image  analysis  involves 
several  user-defined,  software-driven  steps  that  are  compiled  into 
a  template  (or  protocol)  that  is  specific  for  each  assay  type  (e.g., 
kinetics  or  endpoint,  single  dye  or  multiple  dyes).  Easy  access  to 
the  templates  allows  users  to  modify  preset  parameters  that  might 
need  day-to-day  adjustment,  such  as  camera  exposure  time, 
number/volume  of  drugs  being  delivered,  time-course  duration, 
image  capture  frequency,  number/position  of  wells  to  be  tested, 
and  type  of  plates  (96-,  384-well  plate).  Optimized  imaging  pro¬ 
tocols  and  assay  templates  are  stored  (always  in  a  re-settable 
format)  for  use  in  larger  volume  screening  applications.  Such  pro¬ 
tocols  can  be  transferred  to  other  imaging  workstations  ensuring 
coherent,  standardization  of  image  capture  and  analysis  processes 
among  multiple  instruments.  Robust  analysis  procedures  that  rely 
on  the  topology  or  pattern  of  cell  organelles  (see  Chapters  15  and 
47)  and  cells,  and  new  object-oriented  approaches  are  preferred 
solutions  that  have  distinct  advantageous  over  the  prevailing  pixel- 
oriented  methods. 

The  kinetics  and  endpoint  data  generated  by  automated  bioim¬ 
agers  are  typically  analyzed  with  proprietary  software  packages 
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that  are  either  integrated  with  the  system  or  are  third-party 
analysis  packages  (e.g.,  ImagePro  and  Metamorph).  Some 
packages,  such  as  the  BD  Image  Data  Explorer  (BD  Bioscience) 
use  customized  database  engines  to  store  and  access  large,  multi¬ 
dimensional  data  sets  and  apply  sophisticated  statistical  routines 
to  allow  data  visualization  and  analysis.  Analysis  programs 
allow  one  to  quickly  import  raw  data  hies  from  imaging  worksta¬ 
tions  and  associate  them  with  drug  treatment  plate  maps,  con¬ 
taining  information  on  the  drug  treatment  that  each  well  has 
received.  In  order  to  uncover  artifacts  or  highlight  rare  events, 
software  must  be  able  to  provide  true  cell-by-cell  analysis  rather 
than  averaging  cell  response  characteristics  up  front  using  well- 
by-well  data. 

Powerful  data  analysis  programs  can  reduce  large  kinetics  or 
endpoint  image  datasets  into  relevant  pharmacological  data.  Data 
can  then  be  compiled  into  informative  reports  that  are  archived 
with  all  the  analysis  steps  included  to  maintain  data  integrity. 
Bar/line  graphs,  dose-response  curves,  EC50  ,  signal-to-noise  ratio, 
z-scores,  and  other  analysis  tools  useful  to  the  assay  developer  and 
the  cellular  biologist/pharmacologist  can  be  derived. 


CONCLUSION 

Automated  HCS  imaging  instruments  are  hybrid  instruments  that 
bridge  between  low-throughput  manual  microscopes  and  high- 
throughput  plate  readers.  At  present,  automated  confocal  imaging 
technologies  mainly  find  their  niche  in  target  identification/valida¬ 
tion  and  secondary  drug  screenings  where  the  higher  information 
content  provided  by  single-cell  imaging  balances  the  limited 
volume  throughput  (relatively  to  HTS  biochemical  assay).  Further 
refinement  in  hardware  and  software  is  expected  to  narrow  the  gap 
to  produce  an  HTS  imaging  system  capable  to  deal  with  primary 
screening  efficiently. 

The  high-content,  high-speed  cytomics  information  yield  of 
the  disk-scanning  confocal  microscope  makes  it  the  imaging  tool 
of  choice  to  resolve  multiple,  distinctly  labeled  molecular  targets 
in  space  and  time,  as  they  undergo  quantitative  (e.g.,  concentra¬ 
tion)  as  well  as  qualitative  changes  (redistribution  within  cellular 
compartments)  in  single  cells.  With  new  automation,  liquid  han¬ 
dling,  real-time  capabilities,  and  the  ability  to  manage  multiple- 
well  plates  or  cell-line-based  tissue  microarrays,  confocal  HCS 
systems  are  finding  uses  not  only  within  the  domain  of  the  indi¬ 
vidual  microscopist  but  in  higher  throughput  (automated)  imaging 
environments  where  thousands  of  cells  are  being  quantitatively 
interrogated  for  multiple  functional  responses  (Price  et  al.,  2002; 
Giuliano  et  al.,  2004;  Perlman  et  al.,  2004).  As  such,  advanced 
confocal  imaging  technologies,  amenable  to  scaling  and  automa¬ 
tion,  are  opening  new  avenues  in  drug  discovery  and  development, 
toxicology,  and  also  in  functional  genomics  and  proteomics. 
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Automated  Interpretation  of  Subcellular  Location  Patterns 
from  Three-Dimensional  Confocal  Microscopy 

Ting  Zhao  and  Robert  F.  Murphy 


INTRODUCTION 

Confocal  microscopy  has  been  widely  used  for  biological  studies 
because  of  its  unique  ability  to  acquire  images  free  of  most  out- 
of-focus  light.  This  property  is  critical  for  the  acquisition  of  high- 
quality  three-dimensional  (3D)  images  of  cells.  Automated  image 
analysis  and  data  mining  methods  can  be  used  to  extract  rich  infor¬ 
mation  contained  in  such  images.  In  this  chapter,  we  focus  on 
describing  effective  methods  for  automatically  interpreting  3D 
images  of  protein  subcellular  location  patterns. 

Protein  Subcellular  Location 

Proteomics,  the  large-scale  study  of  proteins,  is  the  next  big  step 
in  decoding  genomes  and  achieving  a  thorough  understanding  of 
how  living  systems  work.  An  important  aspect  of  protein  behav¬ 
ior  is  subcellular  localization,  and  the  field  of  location  proteomics 
is  concerned  with  systematically  analyzing  the  complex  processes 
by  which  proteins  are  localized  within  cells  (Chen  et  al.,  2003). 
Context  provided  by  the  location  of  a  protein  can  help  biologists 
infer  possible  functions.  For  example,  a  protein  located  exclusively 
in  the  plasma  membrane  may  serve  as  a  transporter  or  ion  channel, 
while  a  cytoplasmic  protein  may  act  as  a  catalyst.  Recent  projects 
have  described  systematic  approaches  to  the  determination  of 
protein  location  (Kumar  et  al.,  2002;  Dreger,  2003).  However, 
objectively  and  quantitatively  describing  subcellular  location 
patterns  in  cells  using  computational  methods  has  not  been 
emphasized. 

Most  computational  studies  on  protein  subcellular  location  are 
focused  on  prediction  of  locations  from  the  protein  sequence 
(Chou  and  Elrod,  1999;  Chou  and  Cai,  2002;  Park  and  Kanehisa, 
2003;  Bhasin  and  Raghava,  2004).  These  prediction  methods 
became  popular  because  of  the  rapid  increase  in  the  number  of 
available  protein  sequences.  However,  a  significant  problem  with 
using  any  prediction  system  is  that  none  of  the  methods  can  give 
perfect  accuracy.  This  creates  the  risk  that  the  unverified  predicted 
results  may  be  misleading  when  they  are  relied  upon  by  biologists. 
Furthermore,  the  low  resolution  of  current  prediction  schemes  pre¬ 
vents  them  from  distinguishing  proteins  in  the  same  organelle. 
They  can  only  classify  proteins  at  the  predefined  organelle  level, 
that  is,  only  12  categories  were  used  in  a  recent  study  to  describe 
the  organelles  and  compartments  in  an  animal  or  plant  cell  (Chou 
and  Elrod,  1999).  Not  only  can  two  proteins  that  reside  in  differ¬ 
ent  subcompartments  of  the  same  organelle  not  be  distinguished, 
but  minor  (or  previously  unrecognized)  patterns  cannot  be  recog¬ 
nized  and  proteins  that  show  mixed  patterns  (e.g.,  residing  in  more 


than  one  organelle  or  moving  from  one  to  another)  may  not  be  rec¬ 
ognized  correctly. 

Combining  the  rapid  developments  of  microscope  imaging 
techniques  in  biology  with  advances  in  image  analysis  and 
machine  learning,  we  have  over  the  past  few  years  developed  auto¬ 
mated  systems  to  analyze  the  subcellular  distributions  of  proteins 
quantitatively  and  precisely  at  high  resolution  (Boland  et  al.,  1997, 
1998;  Boland  and  Murphy,  2001;  Huang  et  al.,  2003;  Murphy  et 
al.,  2003;  Huang  and  Murphy,  2004b).  Fluorescence  microscopy 
is  the  most  common  method  to  experimentally  determine  subcel¬ 
lular  location  because  it  has  much  higher  resolution  than  subcel¬ 
lular  fractionation,  and  because,  unlike  electron  microscopy,  it 
permits  rapid  imaging  of  living  cells. 

Overview  of  2D  Dataset  Analysis 
From  2DCHO  to  2DHeLa 

Our  work  on  automatic  analysis  of  subcellular  location  patterns 
started  with  two-dimensional  (2D)  images,  initially  of  five  patterns 
in  Chinese  hamster  ovary  cells  (the  2DCHO  dataset)  and  subse¬ 
quently  of  10  patterns  in  HeLa  cells  (the  2DHeLa  dataset).  The 
quantitative  description  and  classification  of  the  2DCHO  dataset 
opened  up  the  area  of  systematic  analysis  of  protein  subcellular 
location  (Boland  et  al.,  1997,  1998).  By  using  four  primary  anti¬ 
bodies,  directed  against  the  Golgi  protein  giantin,  the  lysosome 
protein  LAMP2,  the  yeast  nucleolar  protein  NOP4,  tubulin,  and  a 
DNA  stain,  five  classes  of  subcellular  location  patterns  were 
created.  After  preprocessing  steps  including  deconvolution,  seg¬ 
mentation,  and  background  subtraction,  49  Zernike  moments  and 
13  Haralick  texture  features  were  calculated  as  features  for  each 
image.  The  Zernike  moments  represent  the  decomposition  of  an 
image  into  2D  polynomials  and  the  Haralick  features  capture  sta¬ 
tistical  properties  of  intensity  distribution  in  neighboring  pixels. 
Using  these  two  sets  of  features,  both  global  information  and  local 
information  of  an  image  will  be  captured.  Ten  of  the  62  features 
were  selected  by  stepwise  discriminant  analysis  (SDA)  and  input 
into  a  single  hidden-layer  back-propagation  neural  network 
(BPNN).  An  average  classification  accuracy  of  88%  was  obtained, 
demonstrating  the  initial  feasibility  of  the  application  of  image 
analysis  and  pattern  recognition  methods  to  subcellular  location 
patterns. 

Encouraged  by  the  good  results,  a  more  thorough  study  was 
carried  out  on  the  2DHeLa  dataset  (Boland  and  Murphy,  200 1 ).  This 
covered  most  major  organelles  in  animal  cells  (Fig.  47.1).  In  this 
dataset,  HeLa  cells  were  labeled  for  one  of  nine  proteins  and  in  par¬ 
allel  labeled  with  the  DNA  intercalating  dye  4',6-diamidino-2 
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FIGURE  47.1.  Typical  images  from  the  2DHeLa  dataset  (Boland  and  Murphy,  2001).  The  10  classes  are  an  ER  protein  (A),  the  Golgi  protein  giantin  (B),  the 
Golgi  protein  GPP130  (C),  the  lysosomal  protein  LAMP2  (D),  a  mitochondrial  protein  (E),  the  nucleolar  protein  nucleolin  (F),  the  endosomal  protein  transferrin 
receptor  (H),  the  cytoskeletal  proteins  filamentous  actin  (G)  and  tubulin  (I),  and  DNA  (J). 


phenylindole  dihydrochloride  (DAPI).  Seven  of  the  proteins  are 
located  in  organelles  (endoplasmic  reticulum,  Golgi  complex,  lyso- 
some,  mitochondria,  nucleolus,  and  endosome),  with  two  of  them 
in  the  same  organelle,  the  Golgi  complex.  These  two  patterns  were 
included  to  test  whether  automated  methods  could  distinguish  them. 
The  other  two  proteins  are  found  in  cytoskeletal  structures  (fila¬ 
mentous  actin  and  tubulin).  A  tenth  class  was  created  by  consider¬ 
ing  the  parallel  DNA  images  as  a  class. 

2D  Subcellular  Location  Features 

The  Zernike  moment  features  and  Haralick  texture  features  were 
found  to  be  insufficient  to  describe  all  of  the  patterns  in  the 
2DHeLa  dataset,  and  other  types  of  measures  were  therefore 
implemented.  We  also  implemented  a  standard  nomenclature  for 
referring  to  features  and  sets  of  features  so  that  each  result  could 
be  associated  with  the  features  used  to  create  it  (Boland  and 
Murphy,  2001).  Each  set  was  referred  to  by  the  prefix  SLF  (for 
subcellular  location  features)  followed  by  a  number  (e.g.,  SLF1) 
and  each  feature  within  a  set  was  referred  to  by  the  set  number  and 
an  index  within  it  (e.g.,  SLF1.7). 

The  subcellular  location  features  (SLF)  can  be  divided  into 
different  subsets  according  to  their  properties  (Table  47.1).  SLF1 
consists  of  eight  morphological  features,  five  edge-related  features, 
and  three  convex  hull  features.  These  correspond  roughly  to  attrib¬ 
utes  of  patterns  typically  recognized  by  cell  biologists.  SLF2  is 
formed  by  adding  six  DNA-related  features  to  SLF1.  The  DNA- 
related  features  provide  the  ability  to  distinguish  similar  patterns 
that  differ  in  their  proximity  to  the  nucleus.  For  example,  nucle¬ 
olin  can  be  easily  distinguished  from  other  proteins  using  these 
DNA-related  features.  SLF3  is  the  union  of  SLF1  with  the  49 
Zernike  moment  features  and  13  Haralick  texture  features.  SLF4 
is  the  union  of  SLF3  and  the  six  DNA  related  features. 


TABLE  47.1.  Brief  Description  of  2D  Subcellular 
Location  Features 


Set 

Size 

SLF  Index 

Feature  Description 

SLFl 

16 

SLFl. 1-1. 8 

SLF1.9-1.13 
SLFL  14-1. 16 

Morphological  features 

Edge-related  features 

Convex  hull  features 

SLF2 

22 

SLF2.1-2.16 
SLF2. 17-2.22 

SLFl. 1-1. 16 

DNA-related  features 

SLF3 

78 

SLF3.1-3.16 
SLF3. 17-3.65 
SLF3. 66-3.78 

SLFl. 1-1. 16 

Zernike  moment  features 

Haralick  texture  features 

SLF4 

84 

SLF4. 17-4.22 
SLF4.1-4.16, 
4.23-4.84 

SLF2. 17-2.22 

SLF3.23-3.78 

SLF5 

37 

SLF5. 1-5.37 

Selected  from  SLF4  using  SDA 

SLF6 

65 

SLF6. 1-6.65 

SLF3. 1-3.65 

SLF7 

84 

SLF7. 1-7.9 
SLF7. 10-7.13 
SLF7. 14-7.65 
SLF7. 66-7.78 
SLF7.79 

SLF7.80-7.84 

SLF3. 1-3.9 

Minor  corrections  to  SLF3.10-3.13 

SLF3. 14-3.65 

Modified  Haralick  texture  features 

The  fraction  of  cellular  fluorescence 
not  included  in  objects 

Skeleton  feature 

SLF8 

32 

SLF8. 1-8.32 

Selected  from  SLF7  using  SDA 

SLF12 

8 

SLF12.1-2.8 

SLF8. 1-8.8 

SLFl  3 

31 

SLF13.1— 13.3 1 

Selected  from  SLF7  and  SLF2. 17-2.22 
using  SDA 

SLF15 

44 

SLF15. 1-15.44 

Selected  from  SLF7  and  90  Gabor  and 

wavelet  features 

SLF  16 

47 

SLF16.1-16.47 

Selected  from  SLF7,  90  Gabor  and 
wavelet  features  and 

SLF2. 17-2.22 
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It  is  often  the  case  that  features  anticipated  to  be  valuable  for 
a  given  classification  problem  turn  out  to  have  little  or  no  useful 
information.  In  this  case,  their  presence  can  actually  hinder  the  per¬ 
formance  of  a  classifier.  One  approach  to  this  problem  is  to  select 
an  informative  subset  of  all  available  features,  and  we  have  found 
that  the  stepwise  discriminant  analysis  (SDA)  method  performs 
very  well  for  our  purposes  (Huang  et  al.,  2003).  Thus,  some  SLF 
sets  have  been  created  by  selecting  informative  features  from  one 
of  the  larger  sets  (e.g.,  SLF5  was  selected  from  SLF4). 

The  13  Haralick  texture  features  are  sensitive  to  image  pixel 
resolution  and  number  of  gray  levels.  To  permit  the  features  to  be 
compared  for  images  at  different  resolutions,  a  set  of  improved 
texture  features  were  defined  by  first  downsampling  each  image  to 
1.15pm/pixel  and  256  gray  levels.  This  resulted  in  a  new  feature 
set,  SLF7,  that  consists  of  SLF3  (with  the  13  improved  texture  fea¬ 
tures)  and  six  new  features  that  capture  the  fraction  of  cellular  flu¬ 
orescence  not  included  in  objects  and  object  morphology  as 
reflected  in  object  skeletons. 

To  provide  further  information,  a  set  of  90  features  calculated 
by  transforming  an  image  into  the  frequency  domain  was  added. 
These  features  were  calculated  based  on  the  Gabor  transformation 
and  the  wavelet  transformation.  Subsets  of  these  features  (either 
with  or  without  the  DNA  features)  were  also  selected  (SLF15  and 
SLF16). 

Classification  Results  for  IDHeLa  Dataset 

The  effectiveness  of  the  SLF  for  describing  subcellular  patterns 
can  be  evaluated  by  training  and  testing  classifiers.  With  good  fea¬ 
tures,  different  patterns  should  be  distinguished  reasonably  by  a 
well-trained  classifier.  A  feature  set  that  gives  better  classification 
performance  is  more  suitable  for  describing  location  patterns. 

Since  the  introduction  of  the  first  SLF  sets,  we  have  explored 
additional  feature  sets  and  various  classification  approaches  for  the 
2DHeLa  images.  The  most  common  classifier  we  have  used  was 
a  BPNN  with  one  hidden  layer  and  20  hidden  nodes  (the  same 
architecture  used  for  the  2DCHO  data).  Table  47.2  shows  that  clas¬ 
sification  accuracy  increased  when  more  features  were  added 
(SLF2,  SLF3,  SLF4).  However,  fewer  features  can  give  higher 
accuracy  if  they  are  carefully  selected  (Huang  et  al.,  2003;  Murphy 
et  al.,  2003).  SLF5  selected  from  SLF4  by  SDA  has  fewer  features 
than  SLF4,  but  the  classification  accuracy  is  higher.  Similarly, 
SLF8  and  SLF13  gave  better  performance  than  SLF7. 

Another  way  to  increase  the  classification  accuracies  is  to 
improve  the  classifiers.  We  have  therefore  compared  the  perfor¬ 


TABLE  47.2.  Average  Classification  Accuracy  of  Different 
SLF  Sets  and  Classifiers  on  the  2DHeLa  Dataset 


Set 

DNA 

SDA  from 

Classifier 

Accuracy 

SLF2 

Yes 

_ 

BPNN 

76% 

SLF3 

No 

— 

BPNN 

79% 

SLF4 

Yes 

— 

BPNN 

81% 

SLF5 

Yes 

SLF4 

BPNN 

83% 

SLF7 

No 

— 

BPNN 

74% 

SLF8 

No 

SLF7 

BPNN 

86% 

SLF8 

No 

SLF7 

AdaBoost 

88% 

SLF13 

Yes 

SLF7 

BPNN 

88% 

SLF13 

Yes 

SLF7 

MOE 

90% 

SLF  16 

Yes 

SLF7  +  Gabor  + 

wavelet 

MVE 

92% 

The  classifiers  used  included  back-propagation  neural  network  (BPNN),  mixture-of- 
experts  (MOE),  and  majority  voting  ensemble  (MVE). 


mance  of  eight  different  classification  methods  on  the  2DHeLa 
dataset  (Huang  and  Murphy,  2004b).  The  results  showed  that 
ensemble  methods  increased  the  classification  accuracy  signifi¬ 
cantly.  The  best  classification  accuracy  on  SLF8  was  achieved  by 
Adaboost  (88%),  a  method  that  keeps  modifying  the  training  set 
based  on  how  well  they  can  be  classified  by  a  base  classifier.  For 
SLF13,  Mixture-of-Experts  was  the  most  suitable  one  among  the 
eight  classifiers  (90%).  This  classifier  can  improve  performance  by 
separating  the  training  set  into  many  partitions.  The  eight  classi¬ 
fiers  can  further  be  combined  to  organize  optimal  majority-voting 
ensembles.  The  highest  accuracy  on  2DHeLa  to  date  was  achieved 
by  such  an  ensemble  on  SLF16  (92%). 


HIGH-RESOLUTION  3D  DATASETS 

Since  the  automated  interpretation  of  the  2D  images  was  shown  to 
be  successful,  the  next  step  was  to  extend  the  analysis  to  3D 
images. 

3DHeLa 

The  3DHeLa  dataset  was  used  to  demonstrate  the  advantages  of 
3D  images.  Therefore  all  patterns  used  in  2DHeLa  are  included  in 
3DHeLa  to  make  the  results  from  two  datasets  comparable  (Fig. 
47.2). 

HeLa  cells  were  stained  and  imaged  by  a  three-laser  confocal 
scanning  microscope.  The  dataset  includes  the  same  nine  proteins 
as  in  2DHeLa  and  there  are  50  to  52  3D  images  in  each  class.  In 
addition  to  a  DNA  channel  for  each  image,  a  channel  measuring 
total  protein  was  acquired  at  the  same  time  (by  adding  reactive 
Cy5  dye  to  label  all  proteins  non-specifically).  This  resulted  in 
three-color  3D  images.  The  benefits  of  acquiring  the  two  additional 
channels  are  that  (1)  the  two  channels  can  be  used  to  automatically 
isolate  individual  cells,  (2)  the  DNA  channel  can  be  used  for  cal¬ 
culating  additional  features,  and  (3)  a  tenth  and  eleventh  class 
could  be  created  by  adding  images  from  the  DNA  channel  (class 
DNA)  and  images  from  the  total  protein  channel  (class  Cytoplas¬ 
mic)  to  the  nine  classes  of  proteins. 

For  each  3D  image,  14  to  24  2D  slices  were  acquired  at  a 
Z-  (axial)  interval  of  0.203  p.m.  The  in-plane  intervals  between 
neighboring  voxels  were  0.049  pm. 


3D3T3 

An  additional  source  of  3D  images  is  the  CD-tagging  project, 
which  has  created  a  demonstration  database  for  a  number  of  ran¬ 
domly  tagged  proteins  in  NIH  3T3  cells  (http://cdtag.bio.cmu.edu/ 
www/public/).  By  randomly  inserting  a  green  fluorescent  protein 
(GFP)-encoding  exon  into  the  genome  of  a  cell  using  a  retroviral 
vector,  a  different  protein  can  be  tagged  in  many  different  cells. 
For  the  demonstration  project,  90  different  clones  were  isolated 
and  the  identity  of  the  tagged  gene  was  determined  by  real-time 
polymerase  chain  reaction  (RT-PCR)  sequencing  and  genome 
database  searches.  For  each  clone,  a  number  of  3D  images  were 
acquired  from  living  cells  by  spinning-disk  confocal  microscopy 
(no  parallel  channels  for  DNA  and  total  protein  were  acquired). 
Eight  to  33  images  were  collected  for  each  clone,  with  each  image 
consisting  of  1024  x  1024  X  31  voxels  and  each  voxel  corre¬ 
sponding  to  a  0.11pm  x  0.11pm  X  0.5  pm  region  of  the  sample. 
(See  Fig.  47.3.) 
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FIGURE  47.2.  Typical  three-color  images  from  the  3DHeLa  dataset.  Red,  DNA  channel;  blue,  total  protein  channel;  green,  protein  channel.  (Reprinted  by  per¬ 
mission  of  Carnegie  Mellon  University.) 


Image  Acquisition  Considerations  When  Using 
Automated  Analysis 

Although  the  whole  analysis  procedure  can  be  automated,  the  per¬ 
formance  is  influenced  by  how  images  are  acquired.  Some  impor¬ 
tant  points  must  be  considered  in  image  acquisition  to  get  valid 
results. 

1.  In  each  image  there  should  be  at  least  one  whole  cell.  The 
SLF  are  defined  to  capture  information  on  whole,  individual  cells, 


and  features  from  a  partial  cell  might  be  very  different  from  the 
features  of  the  expected  pattern.  For  3D  images,  the  slices  should 
cover  the  top  and  bottom  of  the  cell.  Some  microscopes  provide 
axial  range  scanning  to  help  decide  the  range  of  scanning.  For 
those  that  don’t,  ensuring  that  the  number  and  position  of  the  slices 
are  appropriate  is  critical. 

2.  If  possible,  cells  being  imaged  should  be  well  separated 
from  each  other  to  facilitate  automated  segmentation.  As  for  a 
partial  cell,  the  features  of  a  region  containing  more  than  one  cell 
are  expected  to  be  different  from  those  of  single  cells. 


Glull  Tubb2-1  Atn5al 


FIGURE  47.3.  Example  images  from  the  3D3T3  dataset.  (Reprinted  by  permission  of  Carnegie  Mellon  University.) 
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3.  Higher  resolution  is  more  desirable.  An  obvious  reason  of 
using  higher  resolution  is  that  more  information  can  be  acquired. 
The  Rayleigh  criterion  indicates  that  the  resolution  is  decided  by 
the  transmitted  wavelength  and  the  numerical  aperture  of  the 
objective  that  is  used.  Because  the  range  of  wavelengths  is  limited, 
the  major  consideration  is  to  use  an  objective  with  a  numerical 
aperture  as  large  as  possible.  This  is  especially  important  for 
increasing  the  axial  resolution  because  it  is  inversely  proportional 
to  the  square  of  numerical  aperture. 

4.  Sampling  should  be  sufficient  to  capture  as  much  informa¬ 
tion  as  possible.  The  two  factors  are  the  voxel  dimensions  and  the 
number  of  gray  levels  in  an  image.  The  voxel  size  in  the  sample 
plane  is  determined  by  the  total  magnification  and  the  spacing 
between  pixels  on  the  camera.  An  image  with  smaller  physical  size 
of  each  pixel  shows  finer  structure  of  a  location  pattern  and  thus 
its  feature  values  are  more  informative.  However,  the  amount  of 
information  that  can  be  obtained  is  limited.  A  common  criterion  is 
to  use  Nyquist  sampling,  or  sampling  at  one  half  of  the  resolution 
from  the  Rayleigh  criterion.  For  520  nm  light  and  a  numerical  aper¬ 
ture  of  1.3,  the  Rayleigh  limit  is  0.244p.m.  Additional  magnifica¬ 
tion  before  the  camera  may  be  required  to  sample  at  half  this  value. 
Given  the  large  dynamic  range  found  in  the  distribution  of  many 
proteins,  using  a  detector  with  greater  than  8-bit  image  depth  is 
also  very  valuable. 

5.  The  number  of  images  in  each  class  should  be  sufficient  to 
describe  the  variation  of  the  pattern.  The  details  of  images  in  the 
same  pattern  are  all  different  because  of  variations  in  size,  shape, 
and  cellular  environment  of  each  individual  cell.  The  ability  of 
numerical  features  to  describe  location  patterns  depends  on  how 
well  the  range  of  variations  is  covered.  Because  essentially  all  clas¬ 
sification  and  clustering  methods  are  statistical  approaches,  suffi¬ 
cient  data  are  required  to  get  significant  results. 

6.  Imaging  conditions  should  be  kept  as  unchanging  as  possi¬ 
ble.  Most  subcellular  location  features  are  sensitive  to  imaging 
conditions  such  as  the  objective  used  and  exposure  time.  Although 
normalization  steps  can  decrease  such  sensitivities,  it  is  always 
more  desirable  to  avoid  such  problems  at  the  data  acquisition  step. 

7.  It  is  desirable  to  acquire  parallel  channels  measuring  DNA 
and  total  protein.  These  channels  can  facilitate  isolating  individual 
cells.  They  can  also  be  used  to  develop  location  features  because 
they  carry  information  of  cell  morphology. 

IMAGE  PROCESSING  AND  ANALYSIS 

The  most  critical  step  of  automatic  interpretation  is  the  extraction 
of  features  from  images.  These  features  should  be  insensitive  to 
cell  rotation  and  translation.  If  images  from  more  than  one  source 
(e.g.,  different  microscopes)  are  to  be  analyzed  together,  the  fea¬ 
tures  should  also  be  defined  in  such  a  way  as  to  compensate  for 
these  differences.  The  procedure  for  moving  from  images  to  fea¬ 
tures  is  described  in  Figure  47.4. 

Segmentation  of  Multi-Cell  Images 
and  Preprocessing 

After  acquiring  images,  preprocessing  steps  are  necessary  before 
feature  calculation.  The  most  important  step  is  image  segmenta¬ 
tion,  which  is  used  to  isolate  individual  cells  from  a  multi-cell 
image.  Without  this  step,  features  defined  on  individual  cells 
cannot  be  extracted  correctly.  It  is  hard  to  work  on  the  targeted 
protein  image  directly  to  isolate  cells  because  we  do  not  know  how 
many  cells  there  are  in  a  particular  image,  and  the  locations  of  the 


FIGURE  47.4.  Feature  calculation  process. 


cell  boundaries  relative  to  the  targeted  protein  depend  on  which 
pattern  the  image  contains.  A  segmentation  method  that  is  suitable 
for  one  pattern  may  fail  for  another.  Fortunately,  acquiring  two 
more  channels,  DNA  and  total  protein,  make  this  problem  much 
easier.  Nuclei  can  easily  be  detected  in  the  DNA  channel  by  thresh¬ 
olding.  Small,  non-nuclear  objects  resulting  from  this  thresholding 
can  be  easily  removed  by  setting  a  second  threshold  on  size  or  inte¬ 
grated  intensity.  Knowing  how  many  cells  are  present  and  where 
their  nuclei  are  greatly  simplifies  the  problem. 

With  this  information,  a  seeded  watershed  method  (Lotufo  and 
Falcao,  2000)  can  be  applied.  The  nuclei  are  used  as  seeds  and  the 
total  protein  image  is  used  to  find  cell  boundaries.  In  our  case,  the 
seeded  watershed  algorithm  was  implemented  by  the  mmcwater- 
shed  function  from  the  SDC  Morphology  Toolbox  for  MATLAB 
(SDC  Information  Systems,  Naperville,  IL).  The  output  from  the 
function  is  a  set  of  masks  (binary  images  showing  each  separated 
region),  each  of  which  corresponds  to  one  cell  (and  one  original 
seed).  It  is  possible  that  partial  cells  will  be  included,  and  there¬ 
fore  a  criterion  based  on  the  distance  to  the  image  edge  is  used  to 
remove  partial  cells. 

By  watershed  segmentation,  50  to  58  individual  cell  images 
per  class  were  obtained  from  the  50  to  52  original  images  per  class 
acquired  for  the  3DHeLa  dataset.  However,  this  segmentation 
method  cannot  be  applied  to  the  3D3T3  dataset  because  there  are 
no  available  images  for  the  DNA  or  the  total  protein  channel. 
Instead,  manual  segmentation  was  used. 

After  finding  masks  for  segmentation,  the  following  steps  of 
preprocessing  are  the  same  for  both  3DHeLa  and  3D3T3: 

1.  Background  fluorescence  is  subtracted  from  each  image. 
This  is  done  by  subtracting  the  most  common  pixel  value  for  each 
image  from  the  values  of  all  pixels  and  then  setting  any  negative 
values  in  the  subtracted  image  to  0.  Some  type  of  background  cor¬ 
rection  is  needed  so  that  integrated  fluorescence  intensities  prop¬ 
erly  reflect  the  fraction  of  cell  fluorescence  occupied  by  each 
object.  The  background  correction  is  done  before  masking  the 
image  to  isolate  individual  cells  so  that  it  can  better  reflect  the 
global  background  (some  masked  regions  may  have  no  back¬ 
ground  pixels  in  them). 

2.  The  segmentation  masks  are  applied  to  each  slice  in  the 
same  3D  image. 

3.  An  automated  thresholding  method  (Ridler  and  Calvard, 
1978)  is  used. 

3D  Subcellular  Location  Features 

Subcellular  location  features  (SLF)  sets  have  been  shown  to  be 
successful  for  describing  2D  images,  therefore,  a  set  of  3D  SLF 
were  defined  on  3D  images  by  extending  the  2D  SLF.  Each  3D 
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image  is  composed  of  several  2D  slices  with  the  same  size.  The 
extension  from  2D  SLF  to  3D  SLF  relies  on  the  differences 
between  a  voxel  and  a  pixel.  First,  a  voxel  is  located  in  a  3D  space 
and  has  three  coordinates,  two  for  the  lateral  plane  and  one  for 
the  vertical  axis.  Second,  each  voxel  has  26  adjacent  voxels  while 
a  pixel  has  only  8  neighbors.  This  is  especially  important  for 
detecting  an  object. 

Three  categories  of  2D  SLF  have  been  extended.  They  are 
morphological  features,  edge  features,  and  texture  features. 

Morphological  Features 

These  features  compose  the  first  set.  SLF9,  that  was  used  for 
classifying  3D  location  patterns  (Velliste  and  Murphy,  2002). 
SLF9.1-14  are  morphological  features  extended  from  2D 
SLF1.1-8  and  SLF2. 17-22.  For  this  purpose,  objects  were  defined 
as  a  contiguous  set  of  above-threshold  pixels  (using  26-neighbor 
connectivity).  The  features  were  defined  as: 

SLF9. 1 :  The  number  of  fluorescent  objects  in  the  image. 

SLF9.2:  The  Euler  number  of  the  image.  This  is  the  difference 
between  number  of  objects  and  number  of  holes  in  the  image. 
SLF9.3:  The  average  object  size.  The  size  of  an  object  is  defined 
as  the  number  of  voxels  in  the  object. 

SLF9.4:  The  standard  deviation  of  the  object  size. 

SLF9.5:  The  ratio  of  size  of  the  largest  object  to  the  smallest. 
Note:  SLF9.3—9.5  describe  the  statistical  distribution  of  object  size 
in  a  cell. 

SLF9.6:  The  average  object  distance  to  the  center  of  fluorescence 
(COF). 

SLF9.7:  The  standard  deviation  of  object  distances  from  the  COF. 
SLF9.8:  The  ratio  of  the  largest  to  the  smallest  object  to  COF 
distance. 

Note:  SLF9.6-9.7  describe  the  spatial  distribution  of  objects  in  a 
cell. 

SLF9.9:  The  average  object  distance  from  the  COF  of  the  DNA 
image. 

SLF9. 10:  The  standard  deviation  of  object  distances  from  the  DNA 
COF. 

SLF9.11:  The  ratio  of  the  largest  to  the  smallest  object  to  DNA 
COF  distance. 

Note:  SLF9.9-9.il  describe  the  spatial  distribution  of  objects  rel¬ 
ative  to  the  nucleus  in  a  cell. 

SLF9.12:  The  distance  between  the  protein  COF  and  the  DNA 
COF. 

SLF9.13:  The  ratio  of  the  volume  occupied  by  protein  to  that  occu¬ 
pied  by  DNA. 

SLF9.14:  The  fraction  of  the  protein  fluorescence  that  colocalizes 
with  DNA. 

SLF9.15  to  SLF9.28  were  designed  to  describe  the  in-plane 
and  axial  distribution  of  the  protein.  This  is  especially  useful  for 
polarized  cells.  The  membranes  of  these  cells  are  divided  into 
sides,  apical,  and  basolateral  domains.  They  contact  different  envi¬ 
ronments  and  some  proteins  must  be  localized  to  one  of  the  two 
domains  to  carry  out  their  function.  For  example,  the  physiologic 
properties  of  transporting  epithelia  are  determined  by  the  polarized 
distributions  of  ion  transporters  (Muth  and  Caplan,  2003).  For  non¬ 
polarized  cells,  the  vertical  distribution  is  also  significant  because 
they  can  become  polarized  when  attaching  to  a  surface  to  grow. 

SLF9.15:  The  average  horizontal  distance  of  objects  to  the  protein 
COF. 

SLF9.16:  The  standard  deviation  of  object  horizontal  distances 
from  the  protein  COF. 


SLF9.17:  The  ratio  of  the  largest  to  the  smallest  object  to  protein 
COF  horizontal  distance. 

SLF9.18:  The  average  vertical  distance  of  objects  to  the  protein 
COF. 

SLF9.19:  The  standard  deviation  of  object  vertical  distances  from 
the  protein  COF. 

SLF9.20:  The  ratio  of  the  largest  to  the  smallest  object  to  protein 
COF  vertical  distance. 

SLF9.21:  The  average  object  horizontal  distance  from  the  DNA 
COF. 

SLF9.22:  The  standard  deviation  of  object  horizontal  distances 
from  the  DNA  COF. 

SLF9.23  The  ratio  of  the  largest  to  the  smallest  object  to  DNA 
COF  horizontal  distance. 

SLF9.24:  The  average  object  vertical  distance  from  the  DNA  COF. 

SLF9.25:  The  standard  deviation  of  object  vertical  distances  from 
the  DNA  COF. 

SLF9.26:  The  ratio  of  the  largest  to  the  smallest  object  to  DNA 
COF  vertical  distance. 

SLF9.27:  The  horizontal  distance  between  the  protein  COF  and  the 
DNA  COF. 

SLF9.28:  The  signed  vertical  distance  between  the  protein  COF 
and  the  DNA  COF. 


Edge  Features 

Edge  features  were  originally  developed  for  the  2DHeLa  dataset 
to  distinguish  proteins  distributed  along  long  and  thin  edges  from 
other  proteins.  For  convenience,  two  edge  features  were  added  to 
3D  SLF  by  a  small  modification.  Edge  detection  was  performed  in 
each  of  the  2D  slices  instead  of  using  a  3D  edge-detection  method 
(so  that  the  different  resolution  of  the  lateral  plane  and  axial  axis 
does  not  have  to  be  considered).  The  Canny  method  (Canny,  1986) 
was  used  for  edge  detection  because  it  is  less  sensitive  to  noise 
than  some  other  methods. 

The  two  edge  features  were  added  to  SLF9. 1-9.8  and 
SLF9. 15-9.20  to  create  SLF11: 

3D-SLF11.15  The  fraction  of  above-threshold  pixels  that  are  along 
an  edge. 

3D-SLF11.16  The  fraction  of  fluorescence  in  above-threshold 
pixels  that  are  along  an  edge. 

Texture  Features 

The  extension  of  texture  features  to  3D  requires  calculating  a  co¬ 
occurrence  matrix  in  13  directions  because  each  pixel  has  26 
neighbors.  The  mean  and  range  over  all  13  directions  were  calcu¬ 
lated  for  each  statistical  property  of  the  co-occurrence  matrix.  This 
resulted  in  26  texture  features  for  each  image: 

SLF1 1.17/30  Average/range  of  angular  second  moment. 

SLF11. 18/31  Average/range  of  contrast. 

SLF1 1.19/32  Average/range  of  correlation. 

SLF1 1.20/33  Average/range  of  sum  of  squares  of  variance. 

SLF1 1.2 1/34  Average/range  of  inverse  difference  moment. 

SLF1 1.22/35  Average/range  of  sum  average. 

SLF1 1.23/36  Average/range  of  sum  variance. 

SLF1 1.24/37  Average/range  of  sum  entropy. 

SLF1 1.25/38  Average/range  of  entropy. 

SLF1 1.26/39  Average/range  of  difference  variance. 

SLF1 1.27/40  Average/range  of  difference  entropy. 

SLF11.28/41  Average/range  of  info  measure  of  correlation  1. 
SLF1 1.29/42  Average/range  of  info  measure  of  correlation  2. 
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AUTOMATED  CLASSIFICATION  OF 
LOCATION  PATTERNS 

The  task  of  classification  is  to  automatically  recognize  the  pattern 
of  an  image.  The  SLF  of  images  whose  pattern  is  known  can  be 
used  to  train  a  classifier  to  recognize  an  image  whose  pattern  is 
not  known  (a  service  called  Subcellular  Localization  Image  Clas¬ 
sifier,  or  SLIC,  is  available  on  our  Web  site  at  http://murphylab. 
web.cmu.edu/services/PSLID/).  The  output  of  the  classifier  can 
only  be  one  of  the  predefined  patterns.  So,  if  we  want  a  classifier 
to  recognize  a  new  pattern,  the  classifier  must  be  fed  with  a  train¬ 
ing  set  corresponding  to  the  new  pattern. 

Classification  of  3DHeLa  Dataset 
Feature  Normalization 

Normalization  was  performed  for  all  features  before  feeding  them 
into  a  classifier.  This  was  done  by  translation  and  scaling  so  that 
each  feature  has  the  mean  0  and  variance  1 .  Otherwise,  some  fea¬ 
tures  will  dominate  others  during  calculation.  For  example,  the 
number  of  objects  in  an  image  (SLF9.1)  can  be  thousands,  but  the 
two  edge  features  are  all  between  0  and  1 .  Although  no  additional 
information  was  generated  after  feature  normalization,  this  step  is 
helpful  for  training  the  classifier.  Some  classifiers  like  a  neural 
network  tend  to  look  for  borders  in  the  subspace  of  dominant  fea¬ 
tures  even  if  these  are  not  good  for  distinguishing  classes.  Because 
the  features  were  divided  into  a  training  set  and  a  test  set,  the  nor¬ 
malization  factors  should  be  calculated  on  the  training  set  and  then 
the  same  factors  are  used  to  normalize  the  test  set. 

Feature  Selection 

As  discussed  above,  a  very  useful  procedure  at  the  feature  level  is 
feature  selection.  As  shown  in  2D  classification,  more  features 
do  not  necessarily  mean  higher  classification  accuracy.  Besides 
requiring  a  longer  time  for  training,  a  classifier  may  overfit  with 
some  noisy  features.  SDA  can  be  used  to  deal  with  this  problem 
by  finding  a  feature  subspace  in  which  different  classes  are  well 
separated  while,  at  the  same  time,  samples  from  the  same  class  are 
close  to  each  other.  The  measurement  of  this  property  is  defined 
as  the  ratio  of  the  within-group  covariance  matrix  to  the  among- 
group  covariance  matrix.  The  SDA  was  implemented  by  the 
stepdisc  function  of  SAS  (SAS  Institute,  Cary,  NC)  with  default 


parameter  values.  The  feature  set  SLF10  is  composed  of  14  fea¬ 
tures  selected  from  SLF9  by  SDA. 

Classification  Results 

As  was  done  for  the  2DHeLa  dataset,  we  have  evaluated  different 
feature  sets  and  classifiers.  At  first  SLF9  and  a  BPNN  with  one 
hidden  layer  and  20  hidden  nodes  was  compared  with  classifica¬ 
tion  results  from  SLF2.  This  resulted  in  91%  accuracy,  which  is 
higher  than  the  counterpart  of  2D  classification  (76%)  and  close 
to  the  best  2D  results  (92%).  When  an  eleventh  cytoplasmic  pattern 
was  added,  the  classification  on  the  11  classes  was  also  91%. 

As  expected,  higher  accuracy  was  obtained  when  SLF  10, 
selected  by  SDA  from  SLF9,  was  used.  This  feature  set  gave  95% 
accuracy  averaged  over  10-fold  cross-validation  (Huang  and 
Murphy,  2004b)  using  the  same  classifier.  The  best  classification 
accuracy  (96%,  Table  47.3)  on  the  feature  set  was  achieved  using 
an  optimal  majority-voting  ensemble  classifier,  in  which  the  eight 
base  classifiers  were  the  same  as  those  used  on  2DHeLa. 

Downsampled  Images  with  Different 
Gray  Scales 

There  are  two  motivations  for  exploring  downsampling  and  gray¬ 
scale  binning  before  calculating  texture  features.  The  first  is  that 
this  is  a  step  to  normalize  images  from  different  sources.  The 
second  is  that  this  may  improve  the  texture  features  by  focusing 
on  the  most  appropriate  spatial  frequencies.  Texture  features  cal¬ 
culated  at  three  different  pixel  resolutions  (0.2,  0.4,  1.0  pm)  and 
three  different  numbers  of  gray  levels  (16,  64,  256)  have  been 
explored  (Chen  and  Murphy,  2004).  To  evaluate  the  contribution 
of  texture  features  under  different  conditions,  results  from  differ¬ 
ent  features  were  compared.  We  found  that  higher  resolution  and 
more  gray  levels  tend  to  give  better  performance  (Table  47.4). 
Figure  47.5  shows  the  changes  of  classification  accuracy  with 
number  of  features.  Each  curve  corresponds  to  a  specific  feature 
set.  Surprisingly,  one  of  the  best  classification  accuracies  was 
achieved  by  only  seven  features  (Table  47.5),  that  were  then 
defined  as  SLF17.  The  average  accuracy  was  98%  (Table  47.6), 
which  is  the  most  accurate  classification  that  has  ever  been 
obtained.  This  means  that  the  10  patterns  can  be  well  described  in 
a  7D  space,  which  even  has  fewer  degrees  of  freedom  than  the 
number  of  the  patterns. 


TABLE  47.3.  Confusion  Matrix  for  Classification  of  11  Classes  of  3DHeLa  Images  Using  the  Feature  Set  SLF10 


True  Class 

Output  of  the  Classifier  (%) 

Cyt 

DNA 

ER 

Gia 

GPP 

LAM 

Mit 

Nuc 

Act 

TfR 

Tub 

Cytoplasmic 

100 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

DNA 

0 

98 

0 

0 

0 

0 

0 

2 

0 

0 

0 

ER 

0 

0 

97 

0 

0 

0 

0 

0 

2 

0 

2 

Giantin 

0 

0 

0 

98 

0 

2 

0 

0 

0 

0 

0 

GPP  130 

0 

0 

0 

4 

96 

0 

0 

0 

0 

0 

0 

LAMP2 

0 

0 

0 

2 

2 

96 

0 

0 

0 

0 

0 

Mitochondria 

0 

0 

0 

4 

0 

0 

95 

0 

2 

0 

0 

Nucleolin 

0 

0 

0 

0 

0 

0 

0 

100 

0 

0 

0 

Actin 

0 

0 

2 

0 

0 

0 

2 

0 

95 

2 

0 

TfR 

0 

0 

0 

0 

0 

6 

4 

0 

2 

85 

4 

Tubulin 

0 

0 

4 

0 

0 

0 

0 

0 

0 

2 

94 

The  results  are  averages  over  10-fold  cross-validation  trials  on  an  optimal  majority-voting  ensemble  classifier.  The  overall  average  accuracy  was  96%.  (Data  from  Huang 
and  Murphy,  2004b.) 
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TABLE  47.4.  The  Contents  of  Each  Cell  in  the  Table  Are  the 
Net  Increase  in  Percentage  Accuracy  Over  a  Classifier 
Using  No  Texture  Features 


Pixel 

resolution 

Number  of  gray  levels 

256 

64 

16 

0.2  pm 

7.7  (16) 

5.5  (22) 

4.7  (15) 

0.4  pm 

7.5  (7) 

5.3  (18) 

3.8  (34) 

1 .0  pm 

3.8  (28) 

2.6  (23) 

0.1  (10) 

The  number  included  in  each  parenthesis  is  the  number  of  features  selected  by  SDA. 
[Data  from  Chen  and  Murphy  (2004).] 


CLUSTERING  OF  LOCATION  PATTERNS: 
LOCATION  PROTEOMICS 

One  goal  of  proteomics  is  to  study  the  relations  between  proteins 
and  group  them  into  meaningful  categories.  This  has  been  done  at 
the  sequence  level  by  sequencing  all  proteins  and  clustering  them 
hierarchically  according  to  their  sequence  similarity.  For  location 
proteomics,  the  clustering  should  be  done  by  their  localization  sim¬ 
ilarity.  Our  work  on  automated  interpretation  of  confocal  micro¬ 
scope  images  of  cellular  proteins  allows  us  to  do  this,  given  sets 
of  images  for  large  numbers  of  proteins.  The  approaches  we  have 
used  are  described  below  for  the  images  in  the  3D3T3  dataset.  We 
refer  to  a  cluster  tree  representing  a  set  of  subcellular  location  pat¬ 
terns  as  a  subcellular  location  tree  (SLT). 

Exclusion  of  Outliers 

To  measure  the  similarity  between  the  locations  of  two  proteins, 
their  images  must  be  acquired  under  similar  conditions.  These  con¬ 
ditions  include  not  only  the  parameters  of  the  imaging  conditions, 
but  also  the  state  of  the  cells.  To  describe  the  typical  location 
pattern  of  a  protein,  images  of  normal  interphase  cells  are  usually 
preferred.  But  biologists  may  want  to  randomly  choose  cells  for 
imaging  to  avoid  observer  bias.  So  some  unusual  examples  of  a 
pattern  were  included  in  the  3D3T3  dataset.  Cells  in  these  images 
include  dead  cells,  dying  cells,  cells  just  before  mitosis,  and  cells 
just  after  cytokinesis.  The  influence  of  the  outliers  increases  with 
their  fraction  in  the  whole  dataset.  Because  of  the  small  size  of  the 
3D3T3  dataset  (8-33  images  per  class),  there  is  high  risk  that  the 
outliers  will  bias  the  clustering  results.  Therefore,  the  outliers  were 
removed  by  a  stringent  procedure  discussed  below. 


TABLE  47.5.  The  Seven  Features  in  SLF17  that  Give  98% 
Classification  Accuracy  on  the  3DHeLa  Dataset 

SLF9.4  The  standard  deviation  of  object  volumes 

SLF11.16  The  fraction  of  fluorescence  in  above  threshold  pixels  that 

are  along  an  edge 

SLF  11.21  The  average  inverse  difference  moment 

SLF11.27  The  average  difference  entropy 

SLF11.28  The  first  average  information  measure  of  correlation 

SLF  11.40  The  range  of  the  first  information  measure  of  correlation 

SLF  11.42  The  range  of  the  second  information  measure  of  correlation 


Determination  of  Optimal  Clustering 

A  number  of  algorithms  and  distance  functions  have  been  used  for 
clustering.  Because  different  feature  sets  and  distance  functions 
result  in  different  clusters  or  trees,  it  is  necessary  to  determine 
which  one  is  the  best.  A  simple  way  is  to  visually  inspect  the  tree 
and  determine  how  consistent  it  is  with  biological  knowledge  and 
the  visual  interpretation  of  the  images.  However,  empirical  evalu¬ 
ation  is  impractical  while  there  are  so  many  classes  in  the  data  (or 
when  not  all  of  the  protein  patterns  are  known!).  Some  measure¬ 
ment  of  goodness  should  be  defined  to  automate  the  process.  The 
basic  criterion  is  having  enough  partitions  to  separate  all  distin¬ 
guishable  patterns  while  at  the  same  time  keeping  indistinguish¬ 
able  patterns  in  the  same  group.  The  trade-off  implies  the  existence 
of  an  optimal  partitioning. 

As  a  starting  point,  all  of  the  3D3T3  images  were  clustered  by 
the  unsupervised  A-means  method  with  varying  cluster  numbers. 
The  optimal  cluster  number  was  decided  using  the  Akaike  Infor¬ 
mation  Criterion  (AIC),  which  balances  the  fitness  and  simplicity 
of  the  model  represented  by  the  clusters.  The  results  showed  that 
the  optimal  number  of  clusters  was  17.  They  also  showed  that  three 
of  the  3T3  clones  had  their  images  spread  over  a  number  of  clus¬ 
ters.  We  interpreted  this  as  evidence  that  these  clones  were  either 
not  pure  cell  lines  or  that  the  tagged  protein  varied  extensively  in 
location  (perhaps  as  a  function  of  cell  cycle).  These  3  clones  were 
therefore  removed  and  all  further  studies  were  based  on  the  87 
clones.  These  initial  clustering  results  were  also  used  to  remove 
outliers:  images  that  were  not  contained  in  the  major  cluster  of 
each  clone  were  removed  from  consideration. 

When  all  data  are  used  to  build  a  cluster  tree,  the  tree  struc¬ 
ture  can  be  very  dependent  on  which  particular  examples  from  a 


TABLE  47.6.  Confusion  Matrix  for  Classification  of  11  Classes  of  3DHeLa  Images  Using  the  Feature  Set  SLF17 


True  Classification 

Output  of  the  Classifier  (%) 

DNA 

ER 

Gia 

GPP 

LAM 

Mit 

Nuc 

Act 

TfR 

Tub 

DNA 

98 

2 

0 

0 

0 

0 

0 

0 

0 

0 

ER 

0 

too 

0 

0 

0 

0 

0 

0 

0 

0 

Giantin 

0 

0 

too 

0 

0 

0 

0 

0 

0 

0 

GPP  130 

0 

0 

0 

96 

4 

0 

0 

0 

0 

0 

LAMP2 

0 

0 

0 

4 

95 

0 

0 

0 

0 

2 

Mitochondria 

0 

0 

2 

0 

0 

96 

0 

2 

0 

0 

Nucleolin 

0 

0 

0 

0 

0 

0 

too 

0 

0 

0 

Actin 

0 

0 

0 

0 

0 

0 

0 

too 

0 

0 

TfR 

0 

0 

0 

0 

2 

0 

0 

0 

96 

2 

Tubulin 

0 

2 

0 

0 

0 

0 

0 

0 

0 

98 

The  results  are  averages  over  10-fold  cross-validation  trials  on  a  BPNN  classifier  with  one  hidden  layer  and  20  hidden  nodes.  The  overall  average  accuracy  was  98%.  [Data 
from  Chen  and  Murphy  (2004).] 
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FIGURE  47.5.  Classification  accuracy  over  10  major  classes  in  SDHeLa 
dataset.  Each  curve  represents  one  of  five  subsets  of  SLF1 1 ,  morphological  fea¬ 
tures  only  (closed  circle),  morphological  and  edge  features  (closed  square),  and 
morphological,  edge,  and  texture  features  computed  at  256  gray  levels  and  dif¬ 
ferent  pixel  resolutions:  0.2  pm  (closed  diamond),  0.4  pm  (*),  and  1.0  pm 
(closed  triangle).  [Data  from  Chen  and  Murphy  (2004).] 


cluster  are  present.  Therefore,  rather  than  building  one  tree,  we 
have  constructed  many  trees  from  random  subsets  of  the  images 
for  each  clone.  The  common  properties  of  these  trees  were 
extracted  to  form  a  consensus  tree  (Chen  and  Murphy,  2005).  In 
order  to  convert  the  tree  into  a  specific  number  of  clusters,  we 
assumed  that  the  number  of  clusters  obtained  by  k-means/AIC 
analysis,  17,  was  correct  and  cut  the  consensus  tree  accordingly. 

Many  clustering  methods  use  some  distance  function  to 
measure  which  observations  are  similar  to  each  other  and  should 
be  grouped.  As  a  criterion  to  choose  an  optimal  distance  function, 
we  reason  that  a  good  distance  function  should  give  consistent 
results  from  different  clustering  methods  (Chen  and  Murphy, 
2005).  Two  other  clustering  methods  independent  of  the  distance 
function  were  also  applied  on  the  data.  One  is  implemented  based 
on  the  confusion  matrix  from  a  classifier  to  group  easily  confused 
clones  together.  The  other  was  done  by  visual  inspection.  Finally, 
we  had  results  from  four  clustering  methods,  consensus  tree,  k- 
means/AIC,  confusion  matrix  based  method  (ConfMat),  and  visual 
inspection.  Then  we  need  a  numerical  value  to  measure  the  agree¬ 
ment  between  clusters  from  any  two  methods.  This  value  was 
described  by  Cohen’s  K  statistic  (Cook,  1998), 

_  Observed  agreement  -  expected  agreement 

K  — - . 

1  -  expected  agreement 


TABLE  47.7.  Evaluation  of  Consistency  of  Clustering 
Methods  Based  on  Different  Distance  Functions 


z-Scored  Euclidean 
Distribution  (k) 

Mahalanobis 
Distribution  (k) 

k- means/ AIC  vs.  consensus 

i 

0.5397 

k- means/ AIC  vs.  ConfMat 

0.4171 

0.3634 

Consensus  Vs.  ConfMat 

0.4171 

0.1977 

&-means/AIC  vs.  visual 

0.2055 

0.1854 

Consensus  vs.  visual 

0.2055 

0.1156 

Data  from  Chen  and  Murphy  (2005). 


Observed  agreement  is  defined  as  the  portion  of  protein  pairs 
where  the  two  clustering  results  agree  and  expected  agreement  is 
defined  as  the  agreement  between  two  random  clusterings  with  the 
same  distribution  frequencies  as  the  two  clusterings  being  com¬ 
pared  (these  are  obtained  by  simulation).  Two  distance  functions, 
z-scored  Euclidean  and  Mahalanobis,  were  tried  for  building  the 
consensus  tree  and  applying  k-means/AIC.  The  results  showed  that 
Euclidean  distance  always  results  in  better  agreement  (Table  47.7). 

The  results  above  allowed  us  to  choose  the  free  parameters  to 
build  a  consensus  tree  that  is  optimal  according  to  the  criteria  dis¬ 
cussed  above.  Figure  47.6  shows  the  consensus  tree  built  using 
these  optimal  parameters.  Visual  inspection  suggests  that  this  tree 
is  consistent  with  available  biological  information.  For  example, 
it  was  found  that  all  nucleolar  proteins,  including  Rpl32, 
Unknown-25,  and  Unknown-32,  are  grouped  together.  The  two 
groups  representing  nucleus  only  and  nucleus-cytoplasm  mixture 
were  also  well  separated. 


STATISTICAL  COMPARISON  OF 
LOCATION  PATTERNS 

Besides  classification  and  clustering,  another  interesting  task  for 
biologists  is  to  determine  whether  two  patterns  are  different.  A  pair 
of  patterns  for  comparison  could  be  either  patterns  from  different 
proteins  or  patterns  for  the  same  protein  under  different  experi¬ 
mental  conditions.  Because  the  difference  between  two  location 
patterns  implies  different  functions  of  the  protein,  comparing  two 
protein  location  patterns  can  help  distinguish  diseased  cells  from 
healthy  ones,  evaluate  drug  effects,  and  find  similarities  between 
two  proteins. 

Although  we  can  tell  something  about  whether  two  patterns 
are  different  or  not  from  a  confusion  matrix  or  a  cluster  tree,  it  is 
hard  to  tell  how  confident  we  are  in  any  conclusion.  A  better  way 
to  compare  two  location  patterns  is  to  use  a  hypothesis  test,  in 
which  the  null  hypothesis  is  that  the  location  patterns  are  the  same. 
Because  we  can  calculate  a  feature  matrix  for  each  pattern,  the  null 
hypothesis  becomes  that  the  two  feature  matrices  are  from  the 
same  population.  The  hypothesis  test  generates  a  value  called  a  test 
statistic  to  measure  the  difference  between  two  groups  of  location 
patterns.  Usually  the  test  statistic  has  a  known  statistical  distribu¬ 
tion  under  the  null  hypothesis.  So  we  can  calculate  a p  value,  which 
is  the  probability  that  the  test  statistic  is  as  much  or  more  extreme 
than  the  observed  one  if  the  null  hypothesis  is  true.  Thus,  the  two 
location  patterns  are  considered  to  be  different  if  the  p  value  cal¬ 
culated  from  their  feature  matrices  is  smaller  than  some  chosen 
criterion,  such  as  0.05. 

There  are  many  ways  to  define  a  test  statistic  and  then  calcu¬ 
late  the  p  value.  A  widely  used  method  is  the  Hotelling  T 2  test, 
which  is  the  multivariate  extension  of  the  t  test.  This  test  results 
in  a  test  statistic  or  F  value  measuring  the  difference  between  the 
means  of  the  two  feature  matrices  by  considering  the  correlation 
of  the  features.  For  this  test,  the  total  number  of  images  must  be 
greater  than  the  features.  This  is  reasonable  because  sufficient  data 
is  needed  to  describe  the  correlation  and  variance  of  the  features. 

This  test  has  been  applied  on  the  2DHeLa  data  (Roques  and 
Murphy,  2002).  It  can  be  easily  applied  to  the  3D  images  because 
after  feature  calculation  both  2D  images  and  3D  images  are  rep¬ 
resented  by  a  feature  vector.  The  Hotelling  T2  test  on  3DHeLa 
showed  that  the  p  values  for  any  pair  is  almost  0  (unpublished 
data),  which  means  that  all  patterns  are  statistically  different.  The 
smallest  F  value  (96.8;  p  <  0.01)  was  from  the  gppl30-giantin  pair, 
which  is  consistent  with  the  results  of  2DHeLa. 
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FIGURE  47.6.  An  optimal  consensus  tree  for  the  87  3D3T3  clones.  The  text  columns  show  protein  names  (if  any),  location  description  assigned  by  visual  exam¬ 
ination,  and  location  description  from  protein  data  bases.  The  tree  is  also  available  in  a  Web  page  that  can  display  representative  images  for  any  branch  (http:// 
muiphylab.web.cmu.edu/services/PSLID/).  From  Chen  and  Murphy  (2005). 


A  Web  service  for  comparison  of  two  location  patterns. 
Statistical  Image  Experiment  Comparator  (SimEC)  is  also  avail¬ 
able  through  http://murphylab.web.cmu.edu/services/PSLID/. 

IMAGE  DATABASE  SYSTEMS 

The  methods  described  in  this  chapter  are  all  designed  to  be  carried 
out  in  an  automated  fashion  on  large  numbers  of  images.  Indeed, 
all  of  them  perform  significantly  better  when  the  number  of 
available  images  increases.  They  also  rely  on  knowledge  of  the 
conditions  under  which  an  experiment  was  done  and  each  image 
was  acquired.  It  is  therefore  natural  to  consider  an  image  database 
system  as  the  most  appropriate  framework  to  utilize  these  tools. 
We  have  described  such  a  system  (PSLID,  for  Protein  Subcellular 
Location  Image  Database)  for  storing  images  and  their  annotations 
with  a  focus  on  providing  tools  for  analyzing  subcellular  patterns 


as  an  integral  part  of  the  database  (Huang  et  al.,  2002).  Other 
image  database  systems  have  been  described  with  a  focus  on  cat¬ 
aloguing  published  images  from  both  fluorescence  and  electron 
microscopy  (Bioimage)  (Carazo  and  Stelzer,  1999)  or  on  support¬ 
ing  images  from  many  fluorescence  microscope  systems  and  bio¬ 
logical  applications  (OME)  (Swedlow  et  al.,  2003).  A  number  of 
efforts  to  improve  and  expand  the  capabilities  of  databases  for 
microscope  images  are  ongoing,  and  it  is  anticipated  and  hoped 
that  a  consensus  on  standards  for  such  systems  will  emerge  to  min¬ 
imize  duplication  of  effort  (see  Chapter  50,  this  volume). 

FUTURE  DIRECTIONS 

In  this  chapter  we  have  introduced  how  automated  image  analysis 
methods  can  be  used  to  interpret  3D  confocal  microscope  images. 
The  core  of  the  whole  process  is  to  convert  an  image  to  a  numer- 
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ical  feature  vector.  The  conversion  allows  us  to  apply  any  avail¬ 
able  multivariate  analysis  method  to  the  data.  As  we  have  shown, 
we  can  perform  classification,  clustering  and  comparison  on  these 
images  automatically.  The  whole  procedure  provides  a  high 
throughput  tool  for  analyzing  data  from  3D  confocal  microscopy. 
Given  these  past  successes,  it  is  worth  considering  directions  for 
future  work  in  this  area. 

The  images  in  the  3D3T3  dataset  were  segmented  manually 
into  single  cell  images  because  of  the  difficulty  of  automated 
segmentation  without  parallel  DNA  or  total-protein  channels. 
The  problem  becomes  more  difficult  for  tissue  images  in  which 
cells  are  not  well  separated.  Segmentation  of  tissue  images  has 
been  accomplished  by  collecting  parallel  DNA  and  membrane 
protein  images  and  using  active  contour  methods  (De  Solorzano 
et  al.,  2001).  An  alternative  that  does  not  require  the  parallel 
images  is  to  develop  SLF  that  are  insensitive  to  the  number  of  cells 
in  the  image.  We  can  calculate  such  features  directly  from  multi¬ 
cell  images  without  segmentation.  This  approach  has  shown  to  be 
successful  for  2D  images  containing  multiple  cells  (Huang  and 
Murphy,  2004a). 

The  SLT  shown  in  Figure  47.6  was  obtained  only  for  location 
patterns  in  interphase  cells.  To  build  such  a  tree,  the  images  of  cells 
under  unusual  states  such  as  mitosis  were  removed  as  outliers.  But 
this  does  not  mean  that  these  outliers  do  not  contain  useful  infor¬ 
mation.  When  adequate  images  are  acquired  for  each  state,  a  bigger 
database  containing  location  patterns  for  different  cell  states  can 
be  created.  This  is  another  advantage  of  determination  methods 
over  prediction. 

Protein  localization  is  a  dynamic  process.  For  example,  some 
proteins  are  localized  to  ER  or  ER  membrane  after  synthesis,  and 
are  then  sorted  to  different  compartments.  The  characteristics  of 
such  a  sorting  procedure  cannot  be  described  unless  we  do  the 
analysis  on  the  images  of  a  3D  time  series.  We  are  currently 
extending  our  methods  from  3D  to  4D  (space  and  time).  Similarly, 
we  can  extract  features  describing  movement  of  proteins  and 
define  a  new  set  of  SLF  that  will  highlight  these  features. 

Because  we  have  shown  that  SLF  is  useful  for  both  3DHeLa 
and  3D3T3,  one  interesting  question  is  whether  the  analysis  would 
be  valid  if  the  datasets  were  mixed,  for  example,  building  a  SLT 
for  all  the  patterns  in  the  two  datasets.  This  suggests  an  even  more 
challenging  task:  to  develop  a  feature  set  insensitive  to  cell 
type,  tagging  method,  or  imaging  method.  This  is  an  important 
goal  given  the  number  of  different  confocal  microscopes  being 
used  and  the  number  of  different  cell  types  being  studied.  Nonethe¬ 
less,  when  applied  under  controlled  conditions,  the  methods 
described  here  have  been  proven  to  be  more  objective,  repro¬ 
ducible,  and  sensitive  for  characterizing  the  proteome  of  a  partic¬ 
ular  cell  type  or  comparing  protein  distribution  patterns  in  the 
presence  and  absence  of  a  drug  than  were  visual  examination 
methods. 
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Display  and  Presentation  Software 

Felix  Margadant 


INTRODUCTION 

The  final  destination  of  much  of  the  data  you  coax  out  of  your 
microscope  is  the  screen  in  front  of  a  digital  projector.  Demon¬ 
stration  software  such  as  PowerPoint,  Keynote,  QuickTime,  and 
MediaPlayer  will  convert  your  digital  micrographs  and  movies 
into  a  format  suitable  for  projection.  What  effect  does  this  con¬ 
version  have  on  the  data  you  plan  to  present?  We  have  endeavored 
to  assess  what  can  be  properly  presented  and  identify  the  most 
notorious  pitfalls  of  two-dimensional  (2D)  display  software. 

This  chapter  supplements  information  in  Chapter  32  about  the 
performance  of  the  display  itself.  The  final  result  any  viewer  sees 
depends  both  on  the  display  software  output  and  the  quality  of  the 
projector.  Unfortunately,  it  is  impossible  to  separate  the  quality  of 
the  result  from  the  nature  of  the  images  or  animations  to  be  dis¬ 
played.  Some  image  data  will  lose  more  quality  and  look  inferior 
on  some  displays  using  some  software  than  others.  Therefore,  we 
will  first  categorize  the  images  into  classes.  In  this  chapter,  we  con¬ 
centrate  on  the  output  the  software  generates  rather  than  the  visible 
result  on  the  screen.  This  way  we  have  a  logical  separation  between 
the  rendering  of  images  and  the  viewer’s  visual  impression. 

We  will  start  by  considering  the  display  of  still  images  in  terms 
of  the  resolution  reproduced,  artifacts,  intensity  dynamics,  and 
spectral  reliability  (i.e.,  color  changes  introduced).  Later,  for  the 
dynamic  case  (i.e.,  movies),  we  will  assess  the  frame  rate  artifacts, 
frame  drops,  and  format  issues.  As  we  shall  see,  the  choice  of  data 
format  is  a  much  more  critical  issue  for  motion  pictures  than  for 
static  images. 

In  short,  this  chapter  is  a  list  of  shortcomings  of  display  tools 
and  display  file  formats.  In  contrast  to  a  few  years  ago,  we  found 
no  serious  flaws  in  the  way  that  static  images  were  handled  by  any 
of  the  viewing  software  inspected.  The  rendering  artifacts  that  at 
one  time  were  created  solely  by  naive  programming  are  now  gone. 
The  pool  of  cryptic  and  incoherent  graphics  formats  has  shrunk 
to  a  manageable  number  of  common  formats.  And  even  for  the 
animation  formats,  Darwinian  selection  has  preferred  those  that 
actually  work. 

As  a  result,  it  makes  sense  to  focus  on  a  few  encoder  tech¬ 
niques  and  file  formats  and  thereby  provide  a  decent  coverage  of 
the  fundamentals.  Because  all  the  common  tools  grew  from  the 
few  surviving  concepts,  the  file  formats  used  actually  do  work  on 
many  computer  systems  and  will  probably  remain  accessible  for 
several  years.  We  cannot  cover  the  myriad  software  add-ons  used 
to  tune  the  basic  display  software. 

The  display  errors  of  particular  interest  are  those  made  when 
images  are  resized  to  fit  into  the  slide  windows  of  the  presentation 
software.  Most  of  our  measurements  relate  to  Microsoft  Power¬ 
Point  and  MediaPlayer,  Apple  Keynote,  and  QuickTime. 


Although  we  usually  create  a  new  presentation  using  a  com¬ 
puter  system  with  a  screen  resolution  that  is  likely  to  be  better  than 
1024  X  768  (XGA)  pixels,  projection  systems  rarely  exceed  XGA 
resolution.  All  the  software  we  investigated  kept  the  presentation 
freely  scalable.  When  you  import  a  1024  x  1024  image  of  any 
format,  PowerPoint,  Windows  Viewer,  or  QuickTime  decodes  it  by 
means  of  a  software  tool  specific  to  the  image  format  and  then 
resizes  it  by  means  of  the  native  viewer  to  fit  the  window  that  the 
presentation  uses.  If  this  window  or  the  screen  resolution  is  resized 
again,  then  the  software  re-applies  the  same  scaling  mechanism  to 
the  first  result  and  this  degrades  the  image  in  the  ways  described 
later  in  this  chapter.  None  of  these  viewers  modify  the  original 
data,  which  remains  unaffected  by  these  tools.  However,  the  image 
displayed  is  still  the  result  of  the  universal  decoding  mechanism 
and  the  universal  resizing  mechanism  we  investigate  next. 

As  we  shall  describe  in  more  detail,  the  resizing  method  per¬ 
forms  an  interpolation  between  the  original  pixel  intensities.  The 
values  are  treated  as  if  they  linearly  evolve  between  the  actual 
image  pixels,  and  if  several  pixels  have  to  be  downsized  into  one, 
their  intensities  get  averaged  in  such  a  way  that  the  “old”  pixels 
located  closer  to  the  location  of  the  “new”  pixel  is  contribute  more. 
The  term  used  for  this  method  is  smart  bilinear  resampling.  The 
claim  to  be  “smart”  comes  from  its  ability  to  stretch  an  image  as 
well  as  average  it  down  to  a  lower  resolution.  The  averaging  is 
crucial  to  prevent  loss  of  brightness  information.  It  is  called  bilin¬ 
ear  as  it  works  in  two  dimensions  simultaneously.  Bilinear  resam¬ 
pling  is  popular  as  it  is  very  fast.  It  is  also  available  in  hardware, 
and  rescaling  introduces  almost  no  distortions  to  the  images. 
Hence,  it  is  applicable  to  a  vast  class  of  different  images  such  as 
all  photographs.  For  micrographs,  however,  better  methods  are 
known.  Therefore,  if  you  know  that  resizing  will  be  necessary,  it 
is  wise  to  resample  your  images  before  inserting  them  into  the 
presentation. 

The  most  common  error  in  using  presentation  software  is  not 
using  enough  pixels.  One  cannot  squeeze  all  of  the  information  of 
a  1024  x  1024  confocal  image  into  a  small  patch  of  a  1024  x  768 
PowerPoint  model  page.  Even  if  the  image  array  does  match  the 
screen  resolution,  it  will  nevertheless  very  likely  be  scaled  because 
the  presentation  software  will  try  to  achieve  a  perfect  match  of  the 
geometry  (i.e.,  the  shape  and  aspect  ratio  of  the  presentation 
window)  at  the  expense  of  matching  the  image  data  pixel-for-pixel 
to  the  resolution  of  the  screen.  Movie  players  do  this  too  when 
operated  in  full-screen  mode,  and  hence  it  is  important  to  investi¬ 
gate  the  effects  of  minor  scale  changes  as  well. 

These  matters  lead  to  a  list  of  precautions: 

•  Never  use  under-sampled  image  material,  because  it  will  not 
properly  scale  —  not  even  by  5%  —  and  cannot  be  processed 
properly. 


Felix  Margadant  •  Zurich,  Switzerland 
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•  If  you  have  to  use  such  material,  force  it  into  a  consistent  form. 
Use  a  smoothing  filter  such  as  a  Gaussian  with  a  radius  of 
larger  than  0.5  (i.e.,  0.7)  to  make  the  image  consistent  for 
further  operations.  This  will  not  give  you  back  a  proper  image 
but  transformations  won’t  further  harm  that  image  to  the  same 
extent.  Deconvolution  of  confocal  and  two-photon  data,  as 
is  recommended  elsewhere  in  this  book,  will  also  ensure  a 
properly  sampled  image. 

•  Even  when  correctly  sampled,  do  not  scale  down  the  image 
any  more  than  absolutely  necessary.  Reducing  image  size  by 
50%  or  more  will  likely  show  aliasing  artifacts.  As  the  origi¬ 
nal  image  noise  worsens  those  effects,  try  never  to  scale  down 
by  more  than  25%. 

•  If  you  are  really  short  of  space,  it  is  better  to  crop  the  image, 
and  cut  away  the  bits  that  are  not  absolutely  essential,  than  to 
simply  rescale. 

•  Scale  the  contrast  so  that  it  looks  acceptable  in  a  bright  room. 
If  you  cannot  see  the  feature  of  interest  on  your  screen  under 
these  conditions,  the  projection  system  is  unlikely  to  do  better 
(see  Chapter  32,  this  volume). 

•  When  using  contrasting  colors,  keep  in  mind  that  your  monitor 
and  the  projector  do  not  have  the  same  calibration  and  the  latter 
may  not  preserve  hues.  Macintosh  users  are  often  surprised 
when  the  color  consistency  of  their  system  does  not  transpose 
to  the  projector. 

•  If  planning  to  use  compression,  test  the  results  by  zooming  in 
to  enlarge  details  first.  If  compression  artifacts  do  emerge, 
lower  the  compression  ratio. 

When  we  show  movies,  the  presentation  program  does  even 
more  work  in  the  background.  In  particular,  because  movies  are 
such  big  files,  compression  becomes  necessary  but  serious  com¬ 
pression  requires  many  computer  cycles,  impairs  the  visual  quality, 
and,  as  we  will  demonstrate,  can  slow  the  presentation. 

When  you  play  your  presentation,  the  software  renders  all  the 
files  that  go  into  each  slide  to  make  a  1024  x  768  video  stream 
that  is  passed  from  the  CPU  to  the  graphics  board  and  from  there 
to  the  monitor  or  projector. 


TESTING 

To  quantify  the  impact  of  display  software  or  imaging  algorithms 
in  general,  we  use  a  feedback  mechanism  based  on  simple  artifi¬ 
cial  images,  called  the  reference  that  are  fed  to  the  displaying  soft¬ 
ware  (also  referred  to  as  the  viewer)  and  a  screen  capturing 
program  (named  the  grabber  or  screen  capture)  that  reads  the 
content  of  the  display  memory  at  a  specified  time  and  hence 
obtains  an  unfalsified  copy  of  what  will  actually  be  displayed.  The 
difference  between  the  reference  and  the  grabbed  result  allows  us 
to  assess  the  quality.  Metrics  for  the  quality  are  a  matter  of  taste 
and  we  use  the  maximum  norm  and  the  Gaussian  norm  for  reasons 
of  economy  and  broad  acceptance;  that  is,  we  measure  the 
maximum  deviation  and  the  root  mean  square  of  the  reference,  if 
and  only  if  a  norm  can  be  used.  If  the  images  cannot  be  matched 
geometrically,  we  only  measure  the  distortions.  The  reason  for  not 
assessing  the  quality  of  any  resized  image  is  that  to  do  so  one 
would  have  to  decide  how  to  resample  the  reference,  and  this 
would  introduce  errors  specific  to  the  resampling  algorithm 
chosen. 

Resampling  to  change  size  can  either  be  thought  of  as  chang¬ 
ing  the  pixel  dimensions  of  the  display  raster  or  as  changing  the 
amount  of  the  original  image  data  that  will  displayed  as  a  single 


pixel.  The  choice  of  the  algorithm  used  to  generate  a  pleasant- 
looking  new  image  is  driven  by  speed  requirements,  and  by  a  fair 
bit  of  superstition  about  the  efficacy  of  different  mechanisms  of 
resampling.  The  dominant  paradigm  in  computer  graphics  is  that  a 
pixel  is  a  point  measurement  that  is  sampled  at  its  center  coordinate 
(this  is  the  official  notion  for  the  MPEG  movie  standards),  whereas 
for  a  microscopic  image,  this  is  the  light  distribution  in  the  speci¬ 
men,  convolved  with  the  point  spread  function  (PSF)  and  either 
sampled  from  the  time-data  stream  from  the  photomultiplier  tube 
(PMT),  or  integrated  over  the  area  of  a  charge-coupled  device 
(CCD)  pixel.  Nyquist  sampling  implies  that  the  signal  from  a  point 
object  will  be  spread  over  12  to  16  2D  pixels  at  the  detector  plane 
(see  Fig.  24.  A6,  this  volume),  and  therefore  that  the  signal  in  each 
of  these  pixels  represents  only  part  of  the  image  of  a  point.  Because 
optical  systems  can  be  thought  of  as  working  in  reverse,  we  turn 
this  situation  around  to  see  that,  if  a  single  pixel  on  the  CCD  is 
imaged  back  onto  the  object,  the  image  of  the  pixel  will  cover  an 
area  about  12  to  16  pixels  in  size.  This  means  that  the  data  recorded 
in  each  pixel  basically  represents  signal  generated  from  a  volume 
in  the  object  that  is  the  size  and  shape  of  the  PSF,  an  area  signifi¬ 
cantly  broader  than  the  physical  dimension  of  that  pixel.  This 
feature  (“fuzzy  edges”)  allows  a  variety  of  linear  image-process¬ 
ing  procedures  to  be  carried  out  without  seriously  damaging  the 
image  data.  However,  if  the  image  is  downsampled  (i.e.,  com¬ 
pressed  into  a  smaller  number  of  pixels  so  that  the  smallest  features 
becomes  <3  pixels  wide),  this  relative  immunity  from  harm  is  lost. 

To  assess  timing  errors  in  motion  displays,  we  used  a  simple 
optical  probe:  a  photodiode  in  a  tube  mounted  to  the  display  (Fig. 
48.1).  By  intentionally  encoding  specific  brightness  fluctuations 
into  the  test  movie  (a  blinking  beacon  and  a  sinusoidal  flash)  we 
can  observe  timing  errors.  If  the  computer  time  source  is  accurate, 
however,  almost  everything  observed  with  the  diode  could  also 
have  been  measured  with  the  grabber. 

One  notable  exception  relates  to  the  hardware-accelerated 
movie  players  that  employ  modern,  built-in  digital  rights  manage¬ 
ment  (DRM).  DRM  basically  is  a  copy  protection  mechanism  for 
digital  movie  material  and  it  is  designed  to  prevent  movie  displays 
from  being  captured.  As  we  wish  to  assess  the  performance  by 
doing  exactly  this  and  as  the  DRM  mechanism  is  active  not  only 
when  playing  copyrighted  material  but  is  also  incorporated  into 
some  presentation  software,  it  can  prevent  us  from  grabbing  the 
output  from  the  test  material. 

DRM  works  by  taking  over  some  features  of  the  graphics  hard¬ 
ware  so  that  the  rendered  movie  never  gets  copied  into  the  video 


display  surface 


FIGURE  48.1.  Test  setup  for  display  speed  and  sensitivity. 
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memory  (the  method  is  referred  to  as  overlaying).  To  gain  quanti¬ 
tative  insight  into  the  movie  playback,  we  must  first  disable  this 
hardware  feature.  However,  to  assess  playback  speed  and  conti¬ 
nuity,  we  must  use  overlaying,  as  it  improves  the  rendering  per¬ 
formance  of  the  player  software,  something  that  we  assess  using 
the  external  timing  signal. 

During  testing,  it  became  obvious  that  performance  limitations 
usually  had  less  to  do  with  the  viewer  software  than  with  the  under¬ 
lying  universal  graphics  library  that  is  part  of  the  operating  system 
and  hence  is  shared  among  all  or  almost  all  the  graphics  programs 
running  on  a  given  computer.  If  the  display  module  is  provided  by 
the  operating  system,  we  refer  to  it  as  a  native  viewer  in  Windows 
terminology.  Furthermore,  an  image  or  movie  file  format  is  inter¬ 
preted  by  dedicated,  shared  software.  This  software  program  is 
referred  to  as  “codec”  (abridged  for  coder/decoder). 

Although  these  universal  software  modules  keep  quality  high 
and  independent  of  the  viewer  used,  they  also  make  it  very  diffi¬ 
cult  to  assess  errors  because  both  the  test  image  generator  and  the 
viewer  that  is  being  tested  likely  rely  on  the  same  basic  software 
codec. 

Codecs  are  always  paired  with  native  viewers:  the  native 
image  viewer  uses  a  universal  codec  to  handle  JPEG  images,  the 
native  movie  player  will  use  codecs  for  the  movie  formats  used. 
This  layered  concept  is  not  new  (introduced  in  1970),  but  recently 
its  usage  has  become  far  more  pervasive.  In  the  end,  many 
presentation  packages  make  use  of  the  underlying  graphics  sub¬ 
system.  On  a  Windows  machine,  this  consists  of  the  Direct  X  and 
Direct  3D  system;  Unix  platforms  mostly  use  X  Windows;  and 
Macs  offer  the  QuickDraw3D  system  (see  Apple  developer  site). 
The  graphics  subsystem  in  turn  tries  to  engage  the  particular  hard¬ 
ware  features  of  the  particular  graphics  card  that  is  part  of  each 
individual  computer,  frequently  referred  to  as  graphics  engine 
or  just  the  engine.  For  recent  technology  of  graphics  subsystems, 
see  http://developer.nvidia.com/page/home  or  http://www.ati.com/ 
developer/index.html. 

Use  of  a  fast  graphics  card  can  have  a  very  positive  impact  on 
the  rendering  speed  of  the  system  but  it  also  decouples  the  per¬ 
formance  of  the  graphics  almost  entirely  from  the  overall  system 
performance.  As  a  result,  we  were  unable  to  relate  processor  power 
directly  to  the  frame  rate  of  embedded  movies.  Even  worse,  there 


seemed  to  be  no  immediate  link  between  the  performance  of  the 
graphics  engine  and  the  movie  frame  rate,  as  the  entire  chain 
of  devices  —  disk,  memory,  processor,  graphics  bus,  graphics 
engine  —  multi-linearly  controls  the  actual  performance. 

One  result  is  that  the  quality  of  the  image  displayed  on  the 
screen  may  depend  more  on  the  hardware  installed  on  the  com¬ 
puter  driving  the  projector  and  on  how  this  hardware  is  configured 
than  it  does  on  the  program  you  use.  Because  Apple  specifies 
minimum  quality  standards  for  the  graphics  engine  (http:// 
developer.apple.com/),  there  seemed  to  be  no  (noticeable)  differ¬ 
ence  between  software  and  hardware  rendering. 

"Static"  Image  Performance 

The  setup  for  testing  the  ability  of  the  program  to  display  static 
images  is  trivial  because  the  resulting  image  can  be  grabbed  any¬ 
where.  The  resolution  reference  test  image  is  a  sinusoidal  grid  at 
various  spatial  frequencies  and  all  quantities  of  interest  can  be 
directly  extracted  from  the  grabbed  image.  Another  reference  used 
to  illustrate  under-sampling  is  an  image  of  sparse  dots  [Fig. 
48.2(B)].  Because  it  is  more  difficult  to  interpret  results  related  to 
the  latter  pattern  most  of  the  figures  below  concentrate  on  the  grid 
[Fig  48.2(A)], 

To  obtain  meaningful  results,  it  is  of  utmost  importance  to  use 
properly  sampled  images  with  finite  line  width  (i.e.,  lines  at  least  4 
pixels  wide)  (Bracewell,  1995),  as  all  modern  compression  and 
coding  algorithms  will  fail  when  applied  to  images  with  single-pixel 
shot  noise  or  steep  pixel-to-pixel  transitions  as  we  shall  illustrate. 

The  native  viewer  for  Windows  XP  is  the  Windows  and  Fax 
Viewer;  for  OS  X  it  is  QuickTime.  The  first  finding  is  that,  not  sur¬ 
prisingly,  when  image  data  is  displayed  without  data  reduction,  the 
native  viewer  does  not  change  the  image  intensity  content  at  all  as 
long  as  the  image  has  exactly  the  same  raster  size  as  the  graphic 
window  in  the  slide.  Any  image  with  three  or  fewer  channels  and 
8  or  less  bits  per  channel,  is  displayed  as  is.  This  also  means  that 
the  output  of  different  viewers  is  indistinguishable,  and  as  all  Pow¬ 
erPoint  versions  later  than  the  1997  edition  use  the  same  underly¬ 
ing  graphics  library  for  viewing  static  images,  images  that  have  not 
been  scaled  in  any  way  are  reproduced  accurately;  that  is,  the  image 
grabbed  at  the  output  of  the  computer  is  identical  to  the  source. 
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FIGURE  48.3.  Strong  photometric  gamma  switched 
off  (left)  and  on  (right)  for  a  linear  ramp.  The  visual 
brightness  is  red;  the  absolute  value  plotted  in  green. 


Brightness 

This  equality  does  not  extend  to  the  brightness  correction  infor¬ 
mation  that  is  stored  with  each  image.  Such  information  is  a  feature 
supported  by  both  the  JPEG  and  TIFF  formats.1  This  photometric 
information  (also  termed  photometric  interpolation)  is  intended 
to  standardize  the  actual  brightness  of  a  digital  value  stored  with 
the  image.  This  is  not  done  as  an  absolute  physical  value  —  which 
would  be  difficult  to  display  and  print  —  but  as  a  ratio  to  the 
peak  brightness  displayed.  Hence,  all  packages  make  adventurous 
assumptions  about  the  dynamic  range  and  linearity  between  the 
darkest  and  the  brightest  values  displayed.  These  assumptions 
range  from  assuming  that  everything  is  fine  (i.e.,  completely  ignor¬ 
ing  the  photometric  information  as  the  Windows  native  viewer 
does)  to  properly  scaling  the  image  to  the  photometric  ramp  (as  in 
Photoshop).  When  the  display  contains  a  stored  profile  describing 
its  color  response,  these  editing  tools  will  scale  the  output  to  take 
this  profile  into  account. 

For  example,  when  Windows  2000  is  used  with  modern  graph¬ 
ics  cards,  the  Direct  X  library  includes  built-in  correction  for 
the  display.  This  has  always  been  an  intrinsic  part  of  the  system  for 
Apple  machines  and  hence  is  never  an  issue. 

The  Nvidia  and  ATI  graphics  drivers  refer  to  this  feature  as 
monitor  gamma,  gamma  being  the  inverse  of  the  brightness 
response  of  the  display.  Fortunately  for  the  user,  these  values  are 
transparent  and  can  be  read  by  the  software.  If  a  monitor  contains 
a  gamma  specification  file,  Photoshop  and  Paintshop  image  editors 
will  use  these  preset  display  values  and  not  add  their  own  correc¬ 
tion  system.  In  the  end,  the  results  can  vary  considerably  if  this 
feature  is  used  (see  Fig.  48.3). 

For  the  user,  all  of  this  hidden  activity  has  a  very  limited 
impact.  Micrographs  represent  a  quantity  related  to  photon  counts 
and  hence  do  not  carry  any  photometric  correction  information. 
There  is  no  photometric  meaning  to  microscopic  images  other  than 
that  the  brightness  scale  should  be  as  linear  as  possible  and  this  is 
close  to  the  case  where  no  photometric  scaling  is  used.2 

Conclusion:  do  not  actively  adjust  the  photometric  information 
until  one  has  finalized  the  image  editing  for  printing.  To  do  other¬ 
wise  may  contaminate  your  results  without  adding  anything  to 


1  http://www.jpeg.org/  and  http://www.digitalpreservation.gov/formats/fdd/ 
fdd000022.shtml. 

2  The  linearity  (or  not!)  of  both  the  display  and  the  visual  system  is  discussed 
at  greater  length  in  Chapters  4  and  32. 


your  image  quality.  Faulty  calibration  ramps  (or  ramps  that  are 
considered  faulty  by  an  editing  program)  can  sometimes  reset  your 
rich,  beautiful  image  to  black. 

Once  the  layout  is  ready  for  publishing,  any  photometric  infor¬ 
mation  available  assures  that  the  results  stay  constant  as  long  as 
the  images  are  used  with  the  same  set  of  programs  (i.e.,  an  image 
saved  in  Photoshop  will  look  much  the  same  on  different  displays 
with  known  profiles).  To  test  this  claim,  we  started  with  an  image 
that  was  the  inverse  of  the  gamma  ramp  shown  in  Figure  48.3(B) 
and  processed  it  through  the  same  gamma-correction  photometric 
ramp  so  that  the  output  should  mimic  the  linear  ramp.  The  origi¬ 
nal  data  is  shown  in  Figure  48.4  (top)  and  the  reconstructed  ramp 
is  seen  at  the  bottom.  It  looks  as  expected,  except  that,  because  the 
computation  is  done  in  integer  numbers,  the  ramp  looks  coarser 
because  of  round-off  errors  The  colored  bars  mark  lines  of  identi¬ 
cal  brightness.  These  marks  are  shifted  but  the  effect  is  quite  mild 
and  hard  to  spot  by  eye. 


Resolution:  Changing  the  Display  Size  of 
Your  Images 

Desktop  and  notebook  displays  are  now  available  at  decent  reso¬ 
lutions.  XGA  (1024  x  768  pixels)  represents  the  very  low  end, 
SXGA  (1280  x  1024),  SWXGA  (1400  x  1050),  UXGA  (1600  x 
1200),  UWXGA  (1900  x  1200),  and  Apple’s  1200  x  800,  1280  x 
854,  and  1440  x  900  can  be  found  in  low-  to  medium-priced 
devices.  Desktop  screens  come  in  these  and  higher  resolutions. 
Projection  systems,  however,  feature  SVGA  (800  x  600)  or,  more 
commonly,  XGA  (see  Chapter  32,  this  volume).  Higher  resolutions 
are  still  rare  even  though  SXGA  projectors  do  exist.  Briefly:  a 
beautiful  micrograph  that  is  1024  square  will  simply  not  fit  on 
today’s  projector.  PowerPoint  and  the  native  viewers  “solve”  this 
issue  by  down-sampling  the  image  if  needed,  using  the  graphics 
library  and  the  graphics  hardware.  Using  the  test  setup  to  compare 
different  resampling  algorithms  (Nikolaidis  and  Pitas,  2000),  it  is 
obvious  that  presentation  software  uses  bi-linear  interpolation,  as 
illustrated  in  Figure  48.5  It  is  a  fast  resampling  method  with  the 
only  disadvantage  that  it  does  not  correspond  to  any  physical 
system,  that  is,  it  cannot  be  simulated  by  building  an  optical  system 
that  changes  the  resolution  in  the  same  way. 

Although  the  inherent  and  unavoidable  danger  of  all  down- 
sampling  is  the  loss  in  resolution,  a  fine  feature,  correctly  sampled 
at  the  resolution  limit,  will  no  longer  be  correctly  sampled  when 
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scaled  down.  This  can  be  visualized  by  observing  the  effect  of  the 
process  on  the  line  grid  image  that  originally  was  properly  sampled. 

To  illustrate  the  damage  done  by  down-sampling  in  Power¬ 
Point  —  or  linear  interpolation  in  general  —  we  force  a  25%, 
50%,  and  75%  scaling  of  the  line  grid  reference  image  in  Figure 
48.6. 

Reducing  the  resolution  of  the  line  grid  by  25%  to  50%  hurts 
the  resolution  but  it  does  not  totally  destroy  the  image  content  as 
the  undulation  between  high  and  low  persists  through  the  scaling 
even  though  the  peak  values  of  the  resulting  images  are  less  than 
they  were  in  the  reference.  This  is  because,  while  the  50%  scaled 
image  is  no  longer  properly  sampled,  it  is  not  yet  aliased :  it  is 
the  smallest  scale  at  which  all  details  of  the  original  image  will  still 
be  present.  When  new  structures,  such  as  patterns,  appear  during 
the  manipulation  of  a  raw  image,  the  phenomenon  is  referred  to  as 
aliasing.  When  a  Nyquist-sampled  image  is  down-sampled  to  25% 
of  the  original  size,  it  will  suffer  from  severe  aliasing  even  if  the 
down-sampling  is  performed  correctly  through  the  interpolation. 
As  the  intensity  of  four  neighboring  pixels  is  averaged,  the  image 
becomes  homogeneous.  This  (correct)  algorithm  for  reducing 
image  content  to  fit  it  into  a  smaller  window  is  built  into  the  modern 
libraries;  however,  on  systems  using  older  software,  the  interpola¬ 
tion  can  still  go  wrong,  as  Figure  48.7(D)  shows. 

The  ability  to  sample  over  more  than  just  the  nearest-neigh- 
boring  pixels  when  scaling  down  is  called  anisotropy 
(http://www.opengl.org).  Because  the  fast  preview  modes  of  Pho¬ 
toshop  or  Paintshop  are  often  used  without  anisotropy  to  increase 
speed,  this  phenomenon  can  still  be  observed,  but  it  is  not  an  issue 
for  presentation  software. 


All  these  cases  were  “well  behaved”  in  the  sense  that  down- 
sampling  them  did  not  introduce  any  structure  but  merely  lowered 
the  image  quality.  Flowever,  even  with  properly  sampled  images, 
it  is  easy  to  commit  true  mistakes.  Figure  48.6(B)  illustrates  the 
common  failures  of  digital  image  operations.  Scaled  to  40%  of  its 
original  resolution,  the  image  content  breaks  down  regardless  of 
the  interpolation  method  chosen.  Even  though  the  down-sampling 
was  done  correctly,  the  original  structure  simply  cannot  be  dis¬ 
played  anymore. 

Although  the  50%  image  still  conveys  the  original  information 
[Fig.  48.6(C)],  it  cannot  be  manipulated  any  further.  Any  rotation 
or  shifting  more  complex  than  a  90°  rotation  or  shifting  an  integer 
number  of  pixels  will  then  destroy  content,  as  Figure  48.7(B) 
proves. 

Far  stronger  under-sampling  pattern  artifacts  (for  regular  struc¬ 
tures,  these  are  called  moire  patterns)  can  occur,  as  Figure  48.7(C) 
shows  for  the  30%  scale  image.  Instead  of  displaying  fine  lines, 
the  image  exhibits  slow  undulations.  Although  similar  errors  occur 
from  non-regular  patterns  —  such  as  micrographs  —  they  are 
harder  to  recognize.  Except  when  the  scanning  process  itself 
creates  some  regular  pattern,  excessive  down-sampling  is  more 
likely  to  break  up  structures  or  attach  them  to  neighbors  than  to 
create  a  large-scale  pattern  (see  Fig.  48.8). 

How  can  we  prevent  such  things  from  going  wrong?  As  XGA 
can  safely  be  regarded  as  the  low-end  display  for  computers  and 
is  the  dominant  display  for  projection  systems,  if  scaling  your 
Nyquist-sampled  original  image  down  to  an  XGA  display  does  not 
force  you  below  the  50%  threshold,  then  details  should  remain 
visible.  So  a  1024  x  1024  Nyquist-sampled  micrograph  can  be 


FIGURE  48.5.  When  a  step  image  (top)  is  zoomed  by  PowerPoint,  it  results  in  a  ramp  (center).  The  intensity  plot  (bottom)  shows  a  ramp  consisting  of  three 
linear  segments. 
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FIGURE  48.6.  Down-sampling  in  PowerPoint.  From  the  left:  original  image,  25%  smaller,  50%  smaller,  and  75%  smaller. 


A  B  C  D 


FIGURE  48.7.  Classical  aliasing  gallery:  (A)  The  reference  image  sampled  at  40%  of  its  original  resolution;  (B)  the  50%  size  image,  rotated  by  30°;  (C)  the 
reference  image  reduced  to  28%  of  its  size;  (D)  the  image  at  70%  if  its  resolution  with  nearest-neighbor  interpolation  instead  of  linear  interpolation. 
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FIGURE  48.8.  Operations  that  do  not  produce  aliasing  in  properly  sampled  image  data.  If  you  subject  your  beautiful  micrograph  to  any  of  these  operations, 
keep  the  worst  case  in  mind  and  do  not  allow  it  to  be  scaled  down  by  more  than  a  factor  of  V2. 


A  B  C  D 

FIGURE  48.9.  Up-sampling:  (A)  a  slightly  under-sampled  image  gets  up-sampled  by  30%  with  either  full  anisotropy  (B)  or  basic  linear  interpolation  (C),  as 
used  by  PowerPoint.  For  comparison,  the  pixel  resolution  in  (D)  is  reduced  by  30%  to  see  the  similarity  of  the  processes. 
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shown  in  a  presentation  as  long  as  it  occupies  at  least  two  thirds 
of  the  display’s  height.  However,  you  can  only  put  two  such  (now 
512  X  512)  images  side  to  side  for  comparison  if  you  utilize  the 
entire  width  of  the  screen.  If  you  want  borders,  you  must  crop  the 
images  to  cut  away  all  but  a  region  of  interest  (ROI).  This  is  often 
a  good  idea  anyway  as  those  at  the  back  of  the  hall  can  usually  not 
see  the  finest  details  on  the  screen. 

The  process  of  scaling,  rotating,  removing  distortion,  and 
aligning  images  all  lower  the  effective  resolution  of  the  digital 
image,  but  the  order  can  be  important.  If  you  shrink  your  image 
into  fewer  pixels,  first,  and  then  rotate  or  distort  it,  you  will  now 
be  working  on  an  under-sampled  image  and  the  damage  will  be 
more  severe:  Shrink  last! 

Digital  images  have  a  V2x  coarser  resolution  in  the  diagonal 
direction  than  along  the  axes  because  of  the  rectangular  grid,  a 
property  not  shared  with  analog  media.  If  an  image  is  turned  by 
an  angle  -45°  <  a  <  45°  away  from  the  axis,  the  values  along  the 
grid  will  be  mapped  onto  a  raster  of  cos(a)  times  the  resolution 
with  the  Yj. 2  worst  case  for  45°. 

Translating  a  digital  image  a  fractional  number  of  pixels  can 
also  blur  the  image.  Fortunately,  the  effect  is  less  pronounced  on 
properly  sampled  data  [Fig.  48.8(D)]  than  is  the  effect  of  scaling 
and  rotation  [Fig.  48.7(A,B)],  but  the  resulting  images  lose 
resolution,  as  the  half-sized  images  of  the  rotated  reference 
[Fig.  48.8(C)]  and  the  shifted  image  [Fig.  48.8(D)]  show.  The  latter 
shows  no  loss  at  all  but  the  shift  is  no  longer  noticeable,  except 
for  a  mild  change  in  intensities. 

Even  if  the  damage  done  in  Figure  48.8(C)  appears  minor,  you 
need  to  remember  that  here  the  operation  was  done  in  a  single  step 
with  maximum  care  and  had  the  same  result  been  reached  as  the 
result  of  a  number  of  separate  steps,  errors  would  have  accumu¬ 
lated  at  each  step  (Fig.  48.9). 

Other  types  of  operations,  such  as  changes  in  histogram,  linear 
filters,  convolution,  and  deconvolution,  do  not  impair  the  resolu¬ 
tion  and  little  care  has  to  be  taken  when  using  them.  Median  filters, 
anisotropic  filters,  and  image  editing  do  harm  image  resolution 
and,  as  the  images  may  no  longer  be  properly  sampled  after  these 
steps,  caution  has  to  be  applied.  Even  minor  changes  of  a  page 
setup  —  without  any  intention  to  rescale  a  picture3  —  can  result 
in  changing  the  resolution  of  the  image. 

Compared  to  down-sampling,  up-sampling  is  relatively  harm¬ 
less.  Therefore,  the  safe  approach  is  to  keep  the  image  near  the 
target  system’s  resolution  and  assume  that  small  changes  in  scale 
will  not  harm  the  content. 

Figure  48.10  shows  that  this  happy  outcome  cannot  be  fully 
achieved  if  the  image  itself  is  not  properly  sampled.  This  means 
that  even  if  we  can  make  sure  that  the  images  never  get  reduced 
in  size  during  a  presentation,  proper  sampling  of  the  original  image 
is  still  essential  to  obtain  a  consistent  outcome. 

Compression 

Although  compression  is  often  unnecessary  for  still  image  presen¬ 
tations,  it  also  has  some  significance  for  storing  large  datasets  and 
hence  we  briefly  touch  it  here.  Classically,  compression  algorithms 
are  split  into  two  groups.  The  first  type  are  pure  compression 
methods  (so-called  no  loss  methods)  that  try  to  exploit  redundan¬ 
cies  such  as  homogeneous  regions  in  the  image  to  use  less  infor¬ 
mation  to  encode  the  image.  The  second  type  are  data-reduction 
algorithms  that  try  to  identify  and  remove  image  information  that 


3  For  example,  changing  the  printer  reproduction  ratio. 


FIGURE  48.10.  Manipulation  of  the  slightly  under-sampled  image.  From  the 
original  (A),  a  30°  rotation  (B)  and  a  half-pixel  shift  were  performed  with  sine 
interpolation  (C)  and  with  linear  interpolation  (D).  The  artifacts  arising  are  least 
pronounced  in  (C).  but  all  result  in  some  loss  of  resolution. 

is  not  needed  for  conveying  the  essential  information.  As  such  a 
compressor  cannot  really  know  what  it  is  that  you  want  in  an  image, 
using  such  a  procedure  is  a  common  source  of  artifacts. 

As  light  microscope  images  are  inherently  noisy  and  noise  has 
no  redundancy,  it  is  important  to  ALWAYS  deconvolve4  your  3D 
confocal  data  before  trying  to  display  it.  Doing  so  not  only  sup¬ 
presses  single-pixel  noise,  but  also  effectively  averages  the  signal 
from  many  voxels  to  reduce  Poisson  noise.  Noisy  images  cannot 
be  compressed  noticeably  by  means  of  pure  compressors 
(Castleman,  1995).  Exceptions  to  this  rule  are  images  with  many 
“black”  pixels,  as  dark  regions  exhibit  little  noise.  It  is  trivial  to 
test  if  your  images  actually  can  be  compressed.  Start  with  a  format 
such  as  TIFF  that  offers  pure  compressors  to  squeeze  the  image 
and  then  compare  the  size  of  compressed  image  to  the  original. 

What  is  lost  when  compressing  microscope  images  using  lossy 
methods?  All  modern  compressors  rely  on  psycho-visual  percep¬ 
tion  compression  (Watkinson,  2001),  that  is,  they  try  to  obliterate 
any  image  information  that  the  human  eye  is  bad  at  reliably  picking 
up.  The  most  famous  such  reduction  is  the  “spectral  coding"  which 
relies  on  the  observation  that  the  spatial  resolution  of  the  human 
eye  for  color  changes  is  much  lower  than  for  intensity  changes. 
How  well  the  compressors  take  advantage  of  this  fact  is  shown  in 
Figure  48.11. 

Most  compressors  exploit  the  connectivity  of  neighboring 
pixels.  If  the  image  is  noisy  or  not  properly  sampled  and  hence  does 
not  exhibit  this  connectivity,  the  effort  to  compress  will  either  fail  to 
achieve  high  compression  or  —  much  more  likely  —  introduce 
errors  to  the  image.  It  will  also  often  NOT  produce  a  smaller  image 
file. 

How  does  compression  affect  the  display  of  micrographs  that 
not  only  should  look  pretty  but  must  also  convey  correct  bright- 


4  Or  at  least  Gaussian  filter  for  2D  data. 
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FIGURE  48.1 1 .  A  random  color  dot  image  (A)  gets  mildly  compressed  (B).  Although  the  overall  hue  is  preserved  (for  a  display  of  unity  gamma),  the  individ¬ 
ual  colors  get  completely  washed  out.  If,  however,  the  original  image  is  properly  lowpass  filtered  to  remove  the  single-pixel  noise  (C),  the  compressed  image 
(D)  looks  very  comparable. 


ness  information?  We  focus  on  a  small  200  pixel  square  subsec¬ 
tion  of  a  properly  sampled  micrograph  (with  some  noise)  and 
assess  what  compression  is  doing  (Fig.  48.12).  First,  because  the 
image  contains  a  large  black  background  with  relatively  low  noise, 
the  image  can  be  compressed  with  little  loss  by  a  factor  of  3.97.5 

Even  though  we  get  very  small  error  values  even  for  the 
strongest  compression,  the  visual  result  is  poor  as  the  fine  details 
are  lost  and  this  is  what  we  perceive  as  quality.  For  image 
processing,  this  has  substantial  consequences:  even  mildly 
compressed  images  are  no  longer  suitable  for  deconvolution, 
segmentation,  or  quantification.  However,  for  presentations, 
compressed  versions  of  properly  sampled  images  will  usually 
leave  a  good  visual  impression. 

MOTION  PICTURES 

Athough  motion  pictures  inherit  all  the  artifacts  that  plague  still 
pictures,  the  image  changes  so  fast  that  the  flaws  are  usually  less 
noticeable.  On  the  other  hand,  they  add  timing  flaws.  The  domi¬ 
nant  problems  are  non-steadiness  in  the  picture  flow  (jittering)  and 
dropped  frames  because  of  lack  of  computer  power.  In  addition, 
there  are  implementation  mistakes  that  can  create  incorrect  frame 
rates  or  random  intervals  between  frames.  The  latter  are  easy  to 
detect  and  usually  require  the  movie  file  to  be  re-created. 

As  for  image-viewer  programs,  the  movie  player  software  is 
only  responsible  for  a  fraction  of  the  possible  problems.  Once  a 
movie  is  created,  certain  flaws  can  no  longer  be  compensated  for 
by  the  player  software.  Because  movies  that  exhibit  aliasing  in 
resolution  or  timing  cannot  be  resurrected  by  better  player  tools, 
correctly  coding  the  movie  will  be  our  first  focus.  The  second  case 
of  interest  is  when  player  and  data  are  set  up  correctly  but  do  not 
match  the  hardware  systems  they  run  on. 

Movies  also  add  another  entire  dimension  to  the  problem:  that 
of  computing  resources.  Movies  use  large  amounts  of  both  storage 
and  computing  power,  and  playing  a  movie  at  the  required  frame 
rate  can  push  a  computer  over  its  limits.  Also,  unless  the  player 
software  is  dedicated  to  playback-from-disk  mode,  discontinuities 
in  playback  are  likely  if  the  entire  movie  cannot  be  held  in  com¬ 
puter  memory.  These  considerations  restrict  the  number  of  formats 
that  can  be  used  for  playing  very  long  time  series. 


5  Such  compression  is  possible,  at  least  in  part,  because  all  the  “black  pixels” 
can  be  specified  as  black  using  fewer  bytes  than  recording  a  row  of  zeros. 
However,  the  process  still  entails  a  “little  loss”  because  setting  the  “almost 
black”  pixels  to  zero  is  a  small  loss.  If  the  data  has  been  3D  deconvolved 
before  being  compressed,  few  pixels  will  actually  be  zero,  and  one  must  set 
a  threshold,  a  process  that  may  also  entail  some  small  loss  of  data. 


Of  the  plethora  of  animation  formats  available,  we  investigate 
only  the  most  popular  ones.  There  are  also  the  metaformats 
QuickTime  and  Microsoft’s  AVI  that  are  really  just  descriptions  of 
the  parameters  of  the  movie  such  as  frame-rate,  resolution,  color 
matching,  etc.,  but  that  do  not  decode  the  movie  data  itself.  Within 
these  formats,  there  is  also  a  parameter  that  refers  to  the  codec  to 
be  used  to  interpret  the  movie  data.  Hence,  these  metaformats  can 
be  reduced  to  the  understanding  of  the  codecs  involved.  To  keep 
things  nice  and  complex,  the  word  QuickTime  is  used  both  as  the 
name  of  a  codec  and  also  as  the  name  of  a  metaformat.  Below,  we 
talk  about  the  basic  problems  of  the  codecs  rather  than  the  conve¬ 
nience  of  the  player  software  (i.e.,  MediaPlayer  and  QuickTime 
player). 

The  codecs  investigated  are  QuickTime,  Cinepac,  MPEG  1,  2, 
and  4  (Ebrahimi  and  Pereira,  2002)  [including  the  new  MPEG-4 
offspring,  H.264  (Richardson,  2003)],  and  the  image-series  format. 
The  last  mentioned  features  no  movie  compression  at  all  but 
simply  consists  of  a  series  of  frames.  The  frames  themselves  can 
still  be  compressed  but  there  is  no  link  between  the  frames  other 
than  enumeration.  The  format  comes  in  a  few  popular  forms, 
including  TIFF  series  (which  simply  uses  the  TIFF  image  standard 
for  frame  handling),  Motion-JPEG  that  uses  the  JPEG  standard 
(which  is  incorporated  in  the  mini-DV,  digital  camcorder  format) 
and  the  Digital  Betacam  (broadcasting)  formats.  These  compres¬ 
sion  standards  will  now  be  discussed  in  more  detail. 

As  we  shall  see,  the  following  considerations  have  to  be 
respected: 

•  Shrinking  movies  results  in  the  same  artifacts  as  shrinking 
stills.  If  scaling  down  is  required,  it  is  wiser  to  shrink  the 
frames  by  proper  down-sampling  before  they  are  encoded  into 
the  movie  format.  However,  cropping  movie  formats  should 
also  be  considered  (Fig.  48.13). 

•  Movies  can  also  be  under-sampled  temporally.  Depending  on 
the  recording  device,  this  can  be  much  harder  to  prevent  than 
spatial  under- sampling.  It  becomes  obvious  when  structures 
move  enough  between  frames  that  they  cannot  be  properly 
associated  in  the  individual  frames.  Our  vision  system  com¬ 
pensates  a  substantial  amount  of  temporal  aliasing  by  utilizing 
other  criteria  such  as  shape,  hue,  and  texture  to  label  moving 
objects,  but  this  skill  is  limited  to  a  few  objects  per  scene.  Tem¬ 
poral  aliasing  can  be  reduced  either  by  increasing  the  record¬ 
ing  frame  rate  or  by  reducing  the  complexity  of  the  scene  with 
proper  color  staining  or  labeling. 

•  If  none  of  these  are  possible,  the  best  remedy  is  to  play  the 
animation  at  a  very  low  frame  rate  so  that  the  visual  cortex 
gains  time  to  understand  the  scene  in  each  frame.  The  result 
resembles  a  slide  show  more  than  an  animation. 
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FIGURE  48.12.  Compression  artifacts.  (A)  is  part  of  a  single-channel,  173  Kbytes  original  image.  (B-E)  show  files 
compressed  by  1.6,  19.5,  32.5  and  54.4  time  (respectively).  The  Gaussian  error  norms  for  these  files  over  the  peak 
area  are  25%,  38%,  45%,  and  70%;  over  the  object  they  are  0.4%,  1%,  1.4%  and  6.5%  and  over  the  entire  image  they 
are  <0.1%,  <0.1%,  0.4%,  and  1.6%.  (F-I)  Below  each  compressed  image  is  a  difference  image  that  shows  the  direct 
difference  in  brightness  and  color  from  the  original.  The  brightness  of  the  difference  images  has  been  increased  to 
make  it  visible  in  print.  Inserts  show  enlarged  views. 


•  The  temporal/spatial  aliasing  dualism  of  movies  also  works  in 
the  opposite  direction.  If  a  movie  is  sampled  properly  in  time, 
it  conveys  an  impression  of  having  higher  spatial  resolution 
than  a  still  image  of  the  same  pixel  dimension.  This  is  obvious 
if  you  digitize  the  TV  signal.  A  single  frame  will  appear  much 
coarser  than  the  equivalent  animated  picture.  This  means  that 
for  movies  at  high  frame  rates,  the  resolution  can  actually  be 
lowered  without  seriously  reducing  the  apparent  image  quality. 

•  There  are  only  a  few  universal  movie  formats.  If  you  stick  with 
the  widespread  MPEG  family  of  animations,  frame  rates  and 
resolutions  are  specified  and  your  movie  will  have  to  be 
trimmed  to  fit  both.  The  reward  for  doing  so  is  that  you  obtain 
a  document  that  will  play  reliably  on  almost  all  platforms. 

•  If  it  is  compressed,  movie  performance  depends  on  the  com¬ 
plexity  of  the  movie,  as  both  decoding  effort  and  memory 
requirements  increase  with  complexity.  The  exceptions  are 
the  MPEG-1  and  MPEG-2  formats  that  offer  complexity- 
independent  performance. 

•  Movies  may  easily  exceed  the  memory  resources  of  the  com¬ 
puter  and  even  when  they  don’t,  they  may  take  long  to  time 
load.  As  a  result,  it  can  be  hard  to  tell  whether  the  movie  is 
just  slow  in  loading  or  if  it  will  not  play  at  all  because  the  pre¬ 
sentation  machine  is  not  sufficiently  equipped.  The  MPEG  and 
QuickTime  formats  start  playing  after  loading  only  a  small  part 
of  the  presentation,  but  only  MPEG  guarantees  proper  play¬ 
back  performance  if  played  from  the  disk. 


A  B  C 


D  E  F 


FIGURE  48.13.  A  3  X  3  color  sample  is  zoomed  to  4  X  4  and  5x5  pixels. 
Linear  interpolation  (A-C)  as  it  is  offered  by  movie  playing  hardware  versus 
high  order  interpolation  (D-F).  The  main  advantage  of  high  order  interpolation 
is  that  the  borders  stay  sharper,  even  though  the  image  looks  no  more  con¬ 
vincing  on  such  a  strongly  zoomed  scale.  (Try  looking  at  it  from  across  the 
room.) 
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As  with  static  images,  the  output  of  motion  players  can  be 
grabbed  by  software  grabbers  unless  the  player  has  the  rights- 
management  software  installed  (as  mentioned  above).  The  grab¬ 
bing  setup  is  synchronized  to  the  refresh  rate  of  the  display  and 
during  each  displayed  frame,  a  small,  constant  region  of  the  screen 
is  captured  as  a  static  image  that  then  can  then  be  compared  to  the 
content  of  the  original  movie  file.  To  allow  for  exact  matching, 
frame  numbers  were  written  into  the  test  movie  file.  These  can  be 
read  by  the  grabbing  software,  permitting  a  one-to-one  compari¬ 
son  to  reveal  when  a  frame  is  displayed  or  not  displayed  because 
it  was  dropped.  This  simple  setup  allows  analysis  of  rendering  arti¬ 
facts,  jittering,  and  frame  drops  all  at  once.  It  is  important  to  rec¬ 
ognize  that  the  frame  rate  must  not  be  higher  than  the  refresh  rate 
of  the  display.  This  is  not  entirely  an  academic  problem,  as  NTSC 
videocams  can  record  at  60  Hz  and  some  LC  displays,  especially 
the  very  high  resolution  ones,  are  limited  to  a  lower  refresh.  In 
addition,  many  projectors  use  30  Hz  displays,  assuming  that  this 
is  sufficient  for  rendering  full-frame  PAL  or  NTSC  at  25  Hz  and 
30  Hz,  respectively. 

It  is  expected  that  the  frame-by-frame  (2D)  quality  of  movie 
players  will  be  worse  than  in  viewers,  because  movie  players  rely 
on  fast  codecs,  supported  by  the  graphics  hardware.  There  is  only 
a  problem  if  the  movie  is  scaled  down  during  the  presentation.  As 
a  result,  it  is  even  more  important  to  avoid  downsizing  movies  than 
it  is  for  stills.  Besides  the  speed  penalties,  shrinking  movies  can 
result  in  coding  and  playback  artifacts,  some  of  which  we  show 
below. 

Coding  Limitations 

It  is  trivial  but  crucial  to  keep  in  mind  that  the  same  fundamental 
restrictions  hold  for  movies  as  for  stills.  You  cannot  safely  display 
a  movie  with  a  higher  format  resolution  than  your  display  window. 
All  the  players  are  armed  with  down-sampling  algorithms  that  — 
due  to  the  strict  timing  requirements  —  are  texture  based  (resort¬ 
ing  to  the  Direct  X  or  QuickDraw  library)  and  use  linear  inter¬ 
polation.  The  price  is  high:  uncontrollable  loss  of  image  detail  even 
at  constant  (or  increased,  for  the  sake  of  down-sampling)  comput¬ 
ing  power.  Both  QuickDraw  and  DirectX  use  the  texture  hardware 
to  zoom  images  up  and  down  and  this  will  result  in  sampling 
errors,  if  the  hardware  anti-aliasing  has  been  switched  off.  Figure 
48.6(D)  shows  a  4-pixel  sinusoidal  pattern  that  has  been  down- 
sampled  by  a  factor  of  4.  Although  the  proper  outcome  for  this 
pattern  is  a  homogeneous  average  intensity,  the  hardware  sampling 
on  the  Windows  machine  used  is  set  to  only  handle  interpolation 
between  neighboring  pixels  and.  hence,  the  result  ends  up  as 
shown.  To  show  the  same  outcome  on  Apple  hardware,  the  down¬ 
sizing  has  to  be  more  severe,  exceeding  a  shrink  factor  of  4. 
Although  this  might  happen  if  many  proper  512  square  movies  are 
laid  out  on  a  single  PowerPoint  or  Keynote  slide,  it  seems  rather 
unlikely. 

Most  movie  formats  do  not  cover  really  high  resolutions  prop¬ 
erly  so  users  are  saved  from  that  mistake  by  system  restrictions.6 
The  only  way  to  keep  control  over  your  movie  details  is  to  limit 
its  resolution  to  the  worst  display  you  are  likely  to  use. 

Apart  from  resolution,  movies  face  a  frame  rate  limit.  Most 
projectors  are  capable  of  handling  full  frames  at  full  resolution  at 


6  However,  this  situation  is  in  great  flux,  as  Apple  recently  moved  up  its  H.264 
codes  to  cover  the  two  MPEG  high  levels,  these  formats  can  now  actually  be 
created  in  a  high-definition  standard.  Of  course,  they  cannot  yet  be  played  on 
common  consumer  hardware  or  XGA  projectors. 


30  to  60  Hz,  except  for  the  very  high  resolution  models.  Cathode 
ray  tubes  (CRTs)  and  liquid-crystal  displays  (LCDs)  offer  even 
higher  refresh  rates,  likewise  high-speed  cameras.  However,  as 
human  vision  cannot  resolve  these  higher  frame  rates,  it  is  safe  to 
down-sample  them  to  60  Hz  if  real  time  is  required  or  use  slow 
motion  if  every  frame  counts.  It  should  be  obvious  that,  when 
building  a  movie,  one  must  respect  the  computational  power  of  the 
display  system.  Not  doing  so  results  in  jerky  image  motion  and  the 
unpredictable  behavior  of  a  player  running  at  an  arbitrary  frame 
rate  and  trying  to  reduce  image  contents  using  the  fastest  (i.e., 
lowest  quality)  method  available. 

Up-Sampling  or  Frame  Rate  Matching 

Assuming  now  that  we  have  a  healthy  movie  hie  designed  to  run 
at  frame  rate  /  with  a  horizontal  resolution  X  that  the  display  can 
accommodate.  We  then  face  a  temporal  scenario  that  is  similar  to 
the  spatial  scenario  we  encountered  for  stills:  the  display  not  only 
exhibits  a  constant  resolution,  its  frame  rate  is  also  fixed.  Although 
both  can  be  changed,  doing  so  requires  setup  effort  and  cannot  be 
done  on  the  fly.  Given  a  fixed  refresh  rate,  the  player  software 
behaves  in  the  most  canonical  way  possible:  that  is,  at  a  60 Hz 
refresh,  frames  start  -16.67  ms  apart.  On  the  other  hand,  an  8  Hz 
movie  that  issues  a  new  frame  every  125  ms  produces  a  display 
table  that  looks  like  Table  48.1,  and  this  is  what  one  obtains  from 
a  sequence  of  grabbed  images  —  each  7  to  8  cycles,  the  frame 
counter  increases. 

Although  the  player  and  movie  files  are  both  perfectly  correct, 
the  images  of  the  movie  cannot  be  displayed  for  125  ms  each  as 
designed.  The  first  picture  (no.  0)  is  displayed  for  133  ms;  the  next 
one  gets  swapped  at  250  ms  and  hence  stays  for  117  ms.  the  third 
one  for  133  ms,  etc. 

In  many  cases,  the  16ms  difference  in  display  time  will  be 
hardly  noticeable ,  a  feature  exploited  by  low-end  conversion 
between  TV  standards,  where  25  Hz  get  mapped  onto  30  Hz  or  vice 
versa. 

If  however,  you  encode  two  labels  in  alternating  frames  and 
display  them  in  red  and  blue,  then  one  of  the  channels  will  appear 

133ms  __  |  |^7  a|most  14%  brighter  than  the  other  one.  Not  a 

117  ms 

disaster,  but  definitely  not  what  one  intended  to  convey.  Worse 
still,  each  time  the  sequence  is  played,  randomly,  either  the  red  or 
the  blue  feature  will  appear  brighter. 

There  are  two  lessons  to  be  learned  from  this:  Do  not  use 
the  compressed  movie  formats  for  anything  but  their  designed 
purpose.  If  you  need  to  add  stereo  or  highlighting  effects,  either 
the  movie  format  has  to  support  them,  or  you  have  to  code  them 
into  the  frames,  and  cannot  rely  on  the  interframe  time  lapse  as  it 
is  unstable. 

A  more  important  message  is  to  only  use  frame  rates  that  are 
literally  in-sync  with  your  display.  So  for  a  60  Hz  display,  use  any 


TABLE  48.1.  Synchronization  Between  Frame  Count  and 
Display  Cycles 


Time 

0  ms 

16.7  ms 

33.3  ms 

50  ms 

66.7  ms 

83.3  ms 

Refresh 

0 

1 

2 

3 

4 

5 

Frame 

0 

0 

0 

0 

0 

0 

Time 

100  ms 

116.7  ms 

133ms 

150ms 

166.7  ms 

183.3ms 

Frame 

0 

0 

1 

1 

1 

1 
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integer  fraction  of  60  Hz,  such  as  6  Hz  or  10  Hz.  Because,  even  in 
countries  with  50  Hz  TV  standards,  projection  systems  support  and 
prefer  60  Hz  modes  when  connected  to  a  computer,  it  is  much 
easier  to  trim  your  movie  for  a  digital  presentation  than  for  your 
home  television.  As  some  CRTs  use  high  and  exotic  refresh  rates 
such  as  85  Hz,  92  Hz,  or  120  Hz.  it  is  futile  to  trim  a  movie  to  their 
needs. 

If  you  cannot  match  the  refresh  rate  of  the  display,  you  can 
compute  the  variance  of  the  frame  display  time:  with  tR  the  refresh 
time  (i.e.,  the  inverse  of  the  refresh  rate)  and  fF  >  tR  the  frame 
display  time  and  the  refresh  cycles  per  frame  q:  =  t¥/tR,  the  time 
mismatch  will  be  tM  =  (q  -  u/l)tR  (Eq.  1).  If  tM  >  0,  then  the 
maximum  timing  error  is  I  fR/fM  I  ■  tM  (Eq.  2).  From  Table  48.1,  one 
would  expect  that  the  theoretical  maximum  error  would  be  near 
2  •  tR  and  that  this  would  be  worse  at  the  start  and  the  end  of  a  movie 
clip.  However,  when  we  captured  a  series  of  out-of-register  frames 
it  became  clear  that  the  movie  player  had  assigned  each  frame  to 
its  closest  refresh  period  rather  than  to  the  next  one  in  sequence, 
that  is,  they  use  a  centric  update  that  reduces  the  worst  frame 
timing  error  by  half. 

Despite  this  smart  feature,  Equation  1  has  two  noteworthy  con¬ 
sequences.  If  there  is  any  timing  error  whatsoever,  the  maximum 
error  will  always  be  larger  than  half  a  refresh  cycle.  Even  more 
unexpectedly,  any  apparently  rational  ratio  between  frame  rate  and 
refresh  rate  will  result  in  a  largest  possible  timing  error  of  one 
entire  refresh  cycle!  So  it  is  not  wise  to  somewhat  align  the  frame 
rate;  they  need  to  be  identical. 

Practically,  this  means  that  if  you  cannot  align  your  movie 
frame  rate  to  the  refresh  rate  of  the  display  (something  that  you 
might  not  know)  then  a  timing  error  of  up  to  one  refresh  cycle  is 
nearly  unavoidable.  This  timing  error  is  less  dramatic  if  the  refresh 
rate  is  significantly  higher  than  the  movie  frame  rate.  A  7  Hz  movie 
will  look  OK  on  all  types  of  displays,  whereas  the  50  Hz  anima¬ 
tion  noticeably  varies  in  playback  speed  when  viewed  on  a  60  Hz 
device  (often  referred  to  as  breathing ). 

We  keep  hammering  this  issue  because,  very  often,  one  has  a 
choice  as  to  how  fast  your  animation  plays  back,  and  slowing  it 
down  might  actually  improve  the  visual  impression.  Also,  some 
advanced  movie  formats  (such  as  MPEG-2  and  MPEG-4)  do  not 
encode  individual  movie  frames  but  evolve  one  frame  from  the 
previous  one.  These  formats  can  produce  frames  at  any  desired 
point  in  time,  not  just  at  the  original  frame  rate.  This  feature  is 
called  pull  down  and  is  used  for  standards  conversion  between  tele¬ 
vision  formats.  In  the  case  of  MPEG-2,  it  actually  can  be  per¬ 
formed  in  real  time  on  the  very  fastest  single-CPU  machines 
available.  It  does  not  yet  offer  the  quality  of  offline  pull  down  but 
will  definitely  become  a  method  of  choice  in  the  future.  The 
MPEG-2  player  from  Philips  already  offers  this  feature.  So  far  no 
MPEG-4  player  can  do  online  pull  down.  As  an  intermediate  solu¬ 
tion,  one  could  create  the  same  movie  for  each  of  the  refresh  rates 
one  is  likely  to  face,  one  for  your  native  display  speed  and  one  for 
60 Hz  projectors. 

Motion  Picture  Artifacts 

Besides  the  patch  and  pixelation  artifacts  inherited  from  still 
images,  movies  face  timing  subsampling  artifacts  (temporal  arti¬ 
facts).  If  a  scene  evolves  too  much  from  one  frame  to  another,  the 
movie  will  be  plagued  with  temporal  aliasing.  As  in  the  case  of  an 
image  with  insufficient  resolution,  this  has  absolutely  nothing  to 
do  with  the  encoding  of  the  movie  or  technical  restrictions  and 
depends  solely  on  the  mismatch  between  movie  acquisition  and 
human  perception.  Fortunately,  because  human  vision  is  outstand¬ 


ingly  tolerant  with  temporal  artifacts  and  can  compensate  for  it, 
temporal  aliasing  catches  much  less  attention  than  subsampling 
images  spatially.  In  addition,  automated  software  that  can  be 
applied  to  time  sequences  is  still  much  less  widespread  than  image- 
processing  algorithms  for  still  images.  Finally,  because  movie 
formats  are  seldom  used  for  image-processing  applications,  we 
have  limited  the  issue  of  temporal  artifacts  to  coding  errors. 

A  movie  with  a  fixed  data  rate  cannot  always  perfectly  encode 
the  amount  of  change  between  two  frames.  The  different  formats 
cope  with  this  problem  in  very  different  ways.  The  MPEG  formats 
have  switches  to  handle  this  scenario.  MPEG-4  can  be  set  to  “scene 
change  detection"  that  will  cue  restarting  encoding  at  steep  scene 
changes.  MPEG-2  can  be  allowed  to  arbitrarily  increase  its  data 
rate  (called  variable  bitrate  encoding )  when  a  scene  change  re¬ 
quires  this.  Both  solutions  come  at  the  expense  of  higher  storage 
requirements.  The  other  movie  compressors  also  invest  more  in 
coding  data  to  handle  changes  but  they  do  so  much  less  con¬ 
sciously,  to  the  extent  that  image  noise  can  drive  their  coding  effort 
to  the  same  level  as  real  scene  changes.  Conversely,  limiting  their 
noise  encoding  restricts  their  scene  update  accuracy. 

Naturally,  none  of  this  holds  if  one  uses  a  coding  format  that 
does  not  include  any  compression.  TIFF  series  are  lossless  as  they 
offer  no  data  reduction  either  within  the  frame  or  over  a  sequence 
of  frames.  For  today’s  large  micrographs  of  one  or  more  channels 
and  megapixel  resolution  per  frame,  these  formats  have  little  rel¬ 
evance  except  for  programs  that  allow  us  to  pan  and  zoom  small 
windows  in  these  datasets. 

The  second  type  of  artifact  comes  not  from  the  information 
encoded  in  the  file  but  from  insufficient  machine  performance  for 
decoding  the  animation  or  from  the  limited  capabilities  of  the 
decoder.  This  can  be  a  very  serious  restriction  when  playing  back 
large  frames  at  high  rates,  especially  if  one  must  use  an  unknown 
computer.  All  MPEG-4  formats,  independent  of  resolution,  are 
computationally  expensive.  Do  not  expect  an  aged  computer  to 
perform  seamless  MPEG-4  playback. 

At  rigid  timing,  the  player  will  drop  frames,  causing  motion 
under-sampling  to  become  even  worse.  In  addition,  when  forced 
to  play  all  frames,  the  player  may  not  decode  all  the  image  infor¬ 
mation  and  may  play  coarse  images  with  ripples  in  them.  The 
results  look  very  similar  to  badly  encoded  movie  formats  in 
general. 

As  outlined  above,  decoders  usually  do  an  excellent  job  of 
squeezing  the  best  possible  result  from  the  movie  format  and  com¬ 
puter  resources  available. 

The  same  cannot  be  said  about  encoders  (also  referred  to  as 
compressors)  because  the  task  of  the  compressor  is  infinitely 
complex  and  as  long  as  the  compressors  do  not  understand  what 
they  are  encoding,  they  have  to  rely  on  heuristics  to  decide  what 
image  information  they  can  throw  away  without  visible  harm  to 
the  movie  quality.  After  all,  this  is  the  only  way  to  reduce  the  data. 

The  criteria  for  useless  information  are  unique  to  each  com¬ 
pressor  but  all  are  optimized  to  handle  cinema  movies  or  TV 
shows,  a  condition  that  sets  clear  limits  on  the  color  space  they 
must  handle  and  how  well  they  replay  motion.  Unfortunately, 
cinema-optimized  encoders  are  notoriously  bad  at  displaying  small 
moving  objects  against  a  large  constant  background,  that  is,  images 
that  look  like  a  fluorescent  object  on  a  black  background.  This  is 
why  some  encoders  have  a  special  setting  to  encode  sports  shows, 
a  setting  that  can  be  useful  to  process  time  sequences  of  micro¬ 
graphs.  Without  this  optimization,  small  moving  objects  drag 
strong  ripple  artifacts  around  with  them.  On  the  other  hand,  fast 
motion  compensation  comes  at  the  price  of  a  generally  higher 
noise  level.  MPEG-2  encoders  can  achieve  both  low  noise  and 
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small  object  motion  at  once  but  only  by  requiring  higher  data 
throughput.  It  is  usually  worthwhile  pay  the  price  of  a  large  data 
file  as  it  usually  costs  less  than  more  computing  power. 

The  MPEG  Formats 

A  notable  feature  of  movie  formats  is  that  the  more  advanced 
formats  can  arbitrate  between  performance  and  quality,  and  are 
able  to  sacrifice  resolution  and  sharpness,  in  order  to  maintain  the 
frame  rate.  Depending  on  the  data  to  be  displayed,  this  can  be 
either  useful  or  noxious.  Two  major  approaches  exist  to  implement 
this  so-called  rigid  timing.  MPEG-1  (as  it  is  implemented  in  the 
Video  For  Windows  format,  also  known  as  H.261)  and  MPEG-2 
(also  named  H.262)  limit  the  complexity  of  the  movie  format  (also 
referred  to  as  bitrate  capping)  upon  coding  the  movie.  If  content 
changes  too  much  between  frames,  capping  can  only  be  achieved 
by  jettisoning  some  of  the  resolution.  MPEG-2  can  encode  two 
different  bitrates  at  once,  called  the  basic  and  the  helper  stream. 
If  performance  allows  for  it,  the  helper  stream  is  decoded  as  well 
and  this  helps  to  improve  the  quality.  If  the  resources  are  insuffi¬ 
cient,  the  basic  stream  supplies  the  output.  The  concept  is  not  very 
flexible  and  it  can  only  either  boost  resolution  or  lower  noise.  The 
unique  feature  of  MPEG-2  is  that  because  the  helper  stream  neither 
has  to  be  decoded  nor  read,  playback  performance  is  assured. 

Because  of  this,  frame-rate  tests  for  MPEG-1  and  MPEG-2 
yield  trivial  results  —  independent  of  the  scene  complexity,  the 
frame  rate  stays  constant  and  frame  dropping  will  only  occur  when 
CPU  power  is  very  limited  and  any  sort  of  movie  playback  is  not 
really  possible  anyway.  On  current  low-end  2  GHz  systems,  the 
CPU  load  is  below  25%  of  the  CPU  resources  even  for  very 
complex  movies  (see  description  of  entropy)  and  hence  not  worth 
investigating. 

MPEG-4  incorporates  MPEG-2  modes  but  adds  the  capability 
of  limiting  the  format  complexity  during  playback.  Therefore,  an 
MPEG-4  file  can  be  generated  at  a  high  bitrate  and  then  be  played 
back  at  a  much  lower  one  (Ebrahimi  and  Pereira,  2002).  In  con¬ 
trast  to  the  MPEG-2  helper  stream,  however,  all  data  must  be  read 
to  decode  an  MPEG-4  stream  and  this  can  impede  playback  per¬ 
formance.  Consequently,  the  player  software  has  a  substantial 
influence  on  MPEG-4  performance  (Walsh  and  Bourges-Sevenier, 
2002). 

Many  custom  players  such  as  the  QuickTime  or  DivX  players 
use  the  settings  of  the  MPEG-4  codec  and  the  movie  format  and 
do  not  curtail  the  bitrate.  If  playback  content  is  too  complex  to  be 
mastered  at  full  frame  rate,  the  players  drop  frames.  The  DivX 
player  incurs  a  significant  jitter  because  it  only  notices  the  timing 
loss  after  the  frame  has  been  rendered,  and  CPU  cycles  that  could 
have  been  used  to  render  the  next  frame  have  just  been  wasted. 
However,  both  players  can  be  performance-tuned  by  changing 
the  codec  settings  while  leaving  the  movie  file  untouched.  The 
Windows  Media  Player  reduces  the  content  complexity  within  a 
few  frames,  so  that  jitter  and  frame  loss  occur  only  briefly  (adap¬ 
tive  bitrate  adjustment  sometimes  called  elastic  bitrate).  As  useful 
as  these  procedures  are,  they  make  it  hard  for  the  user  to  tune  the 
trade-off  between  the  available  performance  and  the  image. 

The  long  startup  time  and  the  huge  computing  requirements 
require  one  to  justify  when  to  use  MPEG-4.  A  small  6  MB 
MPEG-4  file  included  into  PowerPoint  did  consume  an  initial 
startup  delay  of  2.5  s,  allocate  28  MB  of  system  memory,  and  cause 
78%  CPU  load  while  playing.  Due  to  the  serial  coding  of  the 
format,  however,  memory  allocation  will  not  exceed  some  40MB, 
even  for  long  movies.  Also  startup  times  will  not  vary  with  file 
size.  However,  the  decoding  effort  prior  to  anything  being 


displayed  can  be  cumbersome  during  a  presentation,  as  can  the  high 
CPU  load. 

The  use  of  MPEG-4  in  microscopy  should  therefore  be 
restricted  to  what  it  does  best:  playing  long,  highly  compressed 
movies  at  guaranteed  video  rates.  Like  MPEG-2,  MPEG-4  can 
play  back  an  unlimited  number  of  frames  from  the  disk  without 
interruption.  Unlike  MPEG-2  however,  it  eats  many  more  CPU 
cycles  and  the  number  depends  strongly  on  the  movie  content.  The 
complexity  of  the  MPEG-4  resource  requirement  is  touched  in  the 
benchmarks  listed  below.  Of  more  concern,  one  cannot  be  sure  that 
an  MPEG-4  file  that  runs  smoothly  on  one  system  will  perform 
well  or  even  acceptably  on  a  slightly  less-powerful  system.  The 
only  recommendation  we  can  give  here  is  to  play  the  movie  on 
your  test  system  at  maximum  quality  and  run  the  CPU  meter  (in 
the  TaskManager  in  Windows  systems).  Check  that  the  CPU  load 
does  not  exceed  that  of  the  presentation  system.  Be  sure  to  leave 
plenty  of  margins  when  planning  to  play  an  MPEG-4  file  this  way. 
In  contrast  to  MPEG-2  hardware,  MPEG-4  accelerators  are  still 
scarce  and  —  to  make  things  worse  —  depend  on  the  player 
software. 

MPEG-4  exists  in  two  completely  different  levels  of  operation: 
the  more  widespread  H.263  level,  referred  to  as  MPEG-4  part  2, 
and  the  widely  hailed  H.264  (MPEG-4  part  10)  (Richardson, 
2003).  H.263  achieves  its  very  high  compression  rate  by  using  a 
plethora  of  different  compression  techniques  that  all  rely  on  certain 
assumptions  about  the  movie  content.  For  example,  H.263  can 
encode  2D  scene  rotation,  panning,  dimming,  and  even  object  rota¬ 
tion  without  storing  any  image  information  whatsoever.  Sadly,  few 
of  these  abilities  matter  for  micrographs. 

On  the  other  hand,  H.264  deploys  a  new  psycho-visual  coding 
technique  that  is  more  effective  than  MPEG-2  but  abandons  the 
scene  understanding  of  H.263.  Therefore,  it  compresses  less  than 
H.263  but  is  also  less  prone  for  artifact  creation.  As  a  consequence, 
H.263  only  works  well  in  microscopy  in  sparse  scenes  and  when 
the  moving  objects  are  not  too  small.  When  maximum  compres¬ 
sion  is  not  essential  there  will  be  no  serious  use  for  H.263  in 
microscopy. 

Because  modern  Apple  systems  support  the  H.264  decoders  as 
standard,7  one  cannot  use  such  a  system  to  assure  your  movie  will 
play  well  on  an  inferior  machine. 

For  PCs,  the  situation  is  less  encouraging  but  easier  to  control. 
Although  later  Radeon  or  X-series  cards  from  ATI  support  the 
decoding  of  MPEG-4,  they  do  so  to  a  lesser  degree  than  the  Apple 
H.264  decoders  but  still  boost  performance.  To  get  a  better  esti¬ 
mate,  the  hardware  support  for  the  MPEG-4  codec  can  be  switched 
off  and  hence  one  can  get  a  more  reproducible  performance 
estimate. 

MPEG  Display  Formats 

The  most  critical  drawback  of  the  MPEG  formats  is  that  they  come 
in  very  few  display  raster  sizes.  MPEG-1  is  available  only  in  352 
X  288  resolution  and  hence  limited  to  coarse  VHS  quality  movies 
only.  The  upside  is  that  it  can  be  universally  played  and  is  not  very 
computing  power  hungry  (decoding  effort  about  one  third  of  main 
format  MPEG-2,  that  is  also  well  within  modest  computer  limits). 
The  so-called  main  profile  of  MPEG-2  is  defined  for  the  resolu¬ 
tions  (called  levels)  352  x  288  (low),  720  x  576  (main),  1440  x 


7  H264  offers  the  same  seamless  playback  and  guaranteed  frame  rates  as 
MPEG-2  does  but  at  roughly  half  the  bandwidth. 
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1152  (high  1440),  and  1920  X1152  (high)  and  for  two  frame  rates 
30Hz  (NTSC)  or  25  Hz  (PAL).  There  are  three  more  MPEG-2  pro¬ 
files  but  they  offer  no  additional  resolutions  or  frame  rates.  The 
hardware  installed  on  most  new  graphics  boards  decodes  the  main 
profile  but  does  not  go  beyond  the  720  X  576  resolution.  If  your 
movie  can  be  played  back  at  one  of  these  two  frequencies  or  an 
integer  fraction  of  it,  then  MPEG-2  offers  low  load  for  the  com¬ 
puter,  universal  playback,  and  color  calibration  for  many  systems, 
and  can  be  written  to  a  DVD.  In  a  presentation,  it  still  may  not  be 
wise  to  play  a  2  frame/s  (fps)  movie  as  an  NTSC  MPEG-2  because 
doing  so  may  inflate  a  single  frame  to  15  frames.  Also,  as  the  jitter 
measurement  in  Table  48. 1  shows,  it  can  be  cumbersome  to  encode 
a  27  fps  movie  to  either  PAL  or  NTSC.  For  the  same  reasons  that 
most  projectors  offer  a  30  Hz  refresh  rate,  the  NTSC  format  in  a 
presentation  will  often  play  more  fluently  than  the  PAL  format.  If 
the  movie  is  played  full  screen,  then  this  limitation  does  not  apply 
as  the  refresh  will  be  switched  to  the  25  Hz  needed  for  PAL. 

The  MPEG-4  format  allows  for  arbitrary  formats  but  playback 
software  will  not  support  deviations  from  the  MPEG-2  levels.  The 
codecs  we  tested  would  work  on  the  main  and  the  low  MPEG-2 
level  for  both  MPEG-2  and  MPEG-4.  None  would  support  the  two 
high  levels,  and  one  MPEG-4  codec  even  crashed  when  using 
alternate  resolutions. 

The  MPEG  standard  (Watkinson,  2000,  pp.  9-16,  47)  assumes 
that  the  analog  image  data  is  filtered  in  such  a  way  that  it  cannot 
change  more  rapidly  than  the  resolution  that  the  digital  movie 
contains. 

In  micrographs,  this  can  be  assured  by  not  under-sampling  the 
images.  Only  those  digital  micrographs  that  are  properly  sampled 
and  not  too  noisy  constitute  suitable  targets  for  either  JPEG  or 
movie  compression  algorithms.  Unlike  JPEG  series,  high-quality 
MPEG  compression  also  requires  proper  temporal  sampling  to 
avoid  compression  artifacts.  Consequently,  MPEG  compressors 
incorporate  digital  smoothing  filters  to  handle  noisy  source  images 
and,  when  an  MPEG-compressed  animation  appears  unnaturally 
sharp  or  hard  edged  with  weak  ripples  and  creases,  this  is  a  limi¬ 
tation  of  the  standard  and  not  of  the  player  software. 

Very  High  Resolutions 

Except  for  the  new  H.264  standard,  there  is  no  widespread  stan¬ 
dard  for  playing  movies  at  1000  pixel  square  or  higher  resolution 
as  is  needed  by  microscopists.  The  other  MPEG  formats  are  prac¬ 
tically  limited  to  the  MPEG-2  main  level,  the  DVD’s  resolution. 
Although  QuickTime  can  assemble  single  frames  of  any  resolution 
in  use  today  into  movies  (there  is  a  filesize  limit  and  a  214  total 
pixels  resolution  limit)  apart  from  the  dated  QuickTime  compres¬ 
sor,  it  does  not  interface  to  any  advanced  movie  standard  to  com¬ 
press  and  store  the  result. 

Microsoft’s  AVI  format  offers  some  of  the  same  capabilities 
but  with  the  even  worse  restriction  that  they  provide  no  compres¬ 
sor  at  all  and  merely  accommodate  playback.  Movie  authoring 
software  such  as  Adobe  Premiere,  Final  Cut  Pro,  or  Jasc  Anima- 
tionShop  can  write  megapixel  movies,  and  so  do  the  tools  of  most 
microscopy  software.  When  it  comes  to  compression  ratios,  Quick¬ 
Time  performs  better  than  the  antique  Sorensen  or  Cinepak  coders 
used  to  support  the  high  resolutions  for  AVI.  Apple  Final-cut  Pro 
can  now  create  HDTV  MPEG-4  (H.264)  and  H.262  movies. 

Movie  Compression  and  Entropy 

To  obtain  some  generally  valid  results  and  establish  a  concept  of 
movie  performance  and  quality,  it  is  necessary  to  focus  on  the 


complexity  of  the  movie.  The  more  information  a  movie  conveys 
(not  necessarily  the  same  as  useful  information!)  the  harder 
it  is  to  compress,  to  decompress,  and  to  play  back  in  real  time. 
Complex  movies  will  both  suffer  from  lower  frame  rates  and 
more  compression  artifacts.  In  addition,  they  will  not  compress  as 
well.  Therefore,  the  compression  factor  can  be  used  as  a  scale  for 
the  movie  complexity.  The  compression  depends  on  the  entropy  of 
the  movie  information  —  the  more  random  the  images  and 
animation  are,  the  less  it  can  be  compressed  without  apparent 
degradation.  To  get  “clean”  measurements,  we  have  used  an  image 
generator  that  yields  image  sequences  in  which  the  randomness  is 
controlled.  As  a  result,  the  image  or  movie  content  will  be  per¬ 
fectly  random  to  test  the  movie  format  but  the  local  changes  in 
the  image  and  the  fluctuations  in  time  will  be  more  rapid  with 
higher  entropy.  In  information  theory,  one  can  either  use  this 
“channel  entropy”  when  talking  about  compression  or  when 
assessing  it  by  taking  the  autocorrelation  of  the  image  sequence. 
This  measure  can  be  scaled  to  1  for  a  completely  constant  image 
sequence  and  0  for  one  where  neighboring  pixels  have  no  rela¬ 
tionship  to  each  other  in  space  and  time.  To  benchmark  the  players 
and  the  formats,  we  devise  three  movies,  one  with  autocorrelation 
0,  that  is,  completely  random  signal  (Format  A).  One  with  the 
highest  useful  complexity,  that  is,  a  movie  that  is  Nyquist  sampled 
in  space  and  time  but  is  as  random  as  possible  within  these 
restraints  (Format  B).  And  finally,  a  sparse  movie  as  they  are 
popular  in  fluorescence  labeling,  with  a  real  two-channel  sequence 
forced  to  proper  sampling  in  space  and  time  (Format  D).  Very 
often,  temporal  and  spatial  sampling  is  not  performed  properly. 
Because,  despite  all  efforts  to  preach  the  ultimate  importance  of 
proper  sampling,  under-sampled  material  still  enjoys  popularity, 
we  also  include  an  under-sampled  version  of  Format  D  as 
Format  C. 

For  the  compressor,  A  is  harder  to  handle  than  B,  B  harder  than 
C,  and  C  harder  than  D.  Instead  of  some  meaningless  autocorre¬ 
lation  factor,  we  give  the  compression  ratio  ( q )  as  a  percentage, 
that  is,  the  size  of  the  compressed  movie  file  divided  by  the  size 
of  the  uncompressed  image  series.  Unfortunately,  decompression 
mostly  behaves  inversely  and  a  well-compressed  movie  (low  q) 
will  take  more  computer  cycles  to  play  than  a  more  complex 
one  that  is  compressed  less.  The  exceptions  are  the  MPEG-2  and 
H.264  formats  with  a  variable  bitrate,  as  here  compression  and 
playback  effort  is  symmetric.  Because  they  all  guarantee  25  to 
30  frames  per  second,  we  do  not  include  playback  speed  in  the 
benchmarks. 


Performance  Benchmark 

Speed  benchmarks  give  you  a  very  coarse  idea  of  what  is  doable 
in  movie  animation.  We  intentionally  do  not  use  the  latest  Nvidia 
GeForce6  series  or  the  ATI  X600  and  X800  engines  that  are 
common  in  G5  Apple  machines  because  we  need  to  get  the  pre¬ 
sentation  running  on  an  average  system  available  today.  Hence,  we 
test  a  notebook  with  a  2-year-old  GeForce4MX  mobile  engine  and 
a  desktop  with  a  GeForce  5600.  Both  cards  do  not  support  hard¬ 
ware  movie  features  for  non-MPEG  playback  and  hence  we  should 
obtain  movie  performance  that  is  somewhat  related  to  system 
power. 

Compression  Ratios  for  TV-Sized  MPEG  Movies 

VBR  is  variable  bitrate,  that  is,  a  compression  that  adapts  to  the 
movie  content.  CBR  stands  for  constant  bitrate,  meaning  that  the 
compression  ratio  is  held  constant,  independent  of  the  content. 
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TABLE  48.2.  Compression  of  TV-Sized  Movies  (720  x  576) 


Format/Type 

A 

B 

C 

D 

MPEG-2  CBR 

2.3% 

2.3% 

2.3% 

2.3% 

MPEG-4  CBR 

0.8% 

0.8% 

0.8% 

0.8% 

MPEG-2  VBR 

2.5% 

2.5% 

2.1% 

2.1% 

MPEG-4  VBR 

2.0% 

2.0% 

1.7% 

1.7% 

TABLE  48.4.  Playback  Frames  per  Second  for  PAL  TV-Sized 
Movies  (30-s  Window) 


Format/Type 

720  x  576  A 

720  x  576  B 

720  x  576  C 

720  x  576  D 

QuickTime  NC 

27  fps 

27  fps 

27  fps 

27  fps 

QuickTime  JPG 

22  fps 

22  fps 

24  fps 

24  fps 

QuickTime  QT 

32  fps 

34  fps 

34  fps 

34  fps 

AVI  Cinepac 

60  fps 

60  fps 

60  fps 

60  fps 

Please  note  that  the  two  modes  cannot  be  compared  to  each  other 
as  they  serve  completely  different  quality  needs.  CBR  is  used  when 
there  is  only  a  limited  bandwidth  that  can  be  transferred.  VBR  is 
used  when  one  needs  at  least  a  certain  quality  level.  The  settings 
used  here  are  default  settings  for  2-h  DVDs  (M2  settings),  result¬ 
ing  in  a  5000  to  6000  kbit/s  bitrate.  The  CBR  ratios  are  trivial 
as  the  movie  size  is  given  via  the  bitrate.  A  high-quality  DVD 
movie  would  use  around  6000  kbit/s.  The  limit  of  the  MPEG-2 
standard  (even  for  VBR  movies)  is  15,000  kbit/s,  including  all 
soundtracks.  The  MPEG-2  compressors  one  can  buy  generally 
cap  at  9200  kbit/s.  Commercial  DVD  movies  will  not  go  below 
4000  kbit/s. 

The  raw  movie  material  assumed  in  developing  Table  48.2 
would  consume  a  datarate  of  30  Hz  X  720  width  X  576  height  X  8 
bits  per  sample,  that  is,  298,500 kbits/s. 

MPEG-4  goes  far  below  this  mark.  Movies  are  often  encoded 
at  600  to  700 kbit/s  and  1500  kbit/s  is  considered  high  quality. 

These  numbers  (Tables  48.3 — 48.6)  list  file  size  and  frames  per 
second  of  different  formats.  Determining  quality  involves  examin¬ 
ing  the  artifacts  introduced  and  for  most  rich  micrographs,  the 
uncompressed  and  motion  JPG  formats  may  very  well  be  the 
methods  of  choice,  despite  their  lack  of  speed  and  high  memory 
requirements.  Cinepac  may  be  a  good  and  universal  choice  for  a 
Web  page  but  hardly  ever  lives  up  to  the  needs  of  displaying 
micrographs. 

Important  note:  the  numbers  for  QuickTime  are  QT  6.5  bench¬ 
marks,  QT  5  actually  has  higher  frame  rates  for  the  uncompressed 
case  (17.5  fps)  and  identical  rates  for  motion  JPEG.  Because  QT  6 
(and  higher)  is  more  efficient  while  compressing  and  supports 
MPEG-4,  we  use  the  latest  version. 

As  in  the  case  of  displaying  static  images,  one  must  realize  that 
the  presentation  system  may  have  a  different  screen  resolution 
from  the  system  you  tuned  the  movie  on.  In  this  case,  the  content 
must  either  be  zoomed  or  —  much  more  likely  —  downsized  to 
match  the  projection  system.  Respect  the  restrictions  on  down- 
sampling  given  early  in  this  chapter  and  also  keep  in  mind  that 
down-sampling  will  cost  CPU  power,  though  less  than  5%  on  a 
2  GHz  system.  If  you  must  run  your  animation  on  an  unknown 
computer,  make  sure  that  you  have  plenty  of  leeway. 


TABLE  48.3.  Compression  Ratios  for  PAL  TV-Sized  Movies 
(720  x  576) 


Format/Type 

q  of  A 

q  of  B 

q  of  C 

q  of  D 

QuickTime  NC 

100% 

100% 

100% 

100% 

QuickTime  JPG 

27% 

18% 

3.5% 

3.5% 

QuickTime  QT 

17% 

18% 

3.5% 

3.5% 

AVI  Cinepac 

7% 

7% 

5% 

4.5% 

Storing  Your  Presentation  for  Remote  Use 

Besides  texts  and  drawings.  Keynote  and  PowerPoint  presenta¬ 
tions  can  include  image  and  animation  data  as  well  as  the  complex 
scripts  needed  to  present  them.  Images  and  animations  can  either 
be  stored  within  the  presentation  or  they  can  be  linked  to  the  latter 
by  reference.  In  the  latter  case,  PowerPoint  or  Keynote  will 
retrieve  the  original  image  or  movie  by  a  path  and  file  name  (the 
link),  and  then  display  it  by  rules  stored  within  the  presentation. 
The  display  rules  at  a  minimum  consist  of  dimensions  and  posi¬ 
tions  but  can  also  contain  timing  and  movement,  triggers  that  react 
to  certain  other  events  or  scripts. 

Due  to  the  native  viewer  concept,  PowerPoint  hosts  images  in 
their  original  format  and  hence  does  not  compress  or  reformat  the 
image  data  for  storage.  Any  resizing  and  resampling  is  done  only 
when  the  image  or  movie  is  displayed.  This  means  that  any  image 
included  in  a  PowerPoint  presentation  will  increase  the  PPT  file 
by  the  size  of  the  original  image  (plus  a  base  overhead  of  about 
4kB  for  the  displaying  rules)  and  will  have  to  be  decompressed 
and  resized  each  time  it  is  displayed  in  a  slide.  The  data  of  each 
image  is  then  cached  by  PowerPoint  so  that,  the  display  is  faster, 
the  second  time  it  is  shown,  even  though  the  image  must  still  be 
decompressed  each  time.  As  a  result,  it  is  better  to  only  use  images 
that  have  no  more  resolution  than  is  actually  needed.8 

There  are  several  reasons  for  only  storing  references  within  a 
presentation.  The  premier  one  is  that  only  a  single  copy  of  an 
image  or  a  movie  has  to  be  stored  no  matter  how  many  different 
presentations  may  use  it.  This  also  assures  update  consistency,  as 
changes  to  a  file  will  be  available  in  all  the  presentations  that 
include  it. 

The  main  drawback  is  that  keeping  the  references  consistent 
when  transporting  a  presentation  to  a  different  computer  is  not 
trivial  because  the  path  referenced  in  the  document  must  also  func¬ 
tion  properly  on  the  presentation  computer. 

There  is  no  perfect  solution  to  this  situation  and  all  approaches 
are  plagued  with  obvious  shortcomings: 

1 .  PowerPoint  offers  a pack-and-go  mode.  The  file  made  using 
this  command  includes  the  images  and  animations  (obtained  from 
their  references).  This  is  the  fastest  (and  fairly  safe)  way  to  com¬ 
plete  the  task  but  it  leaves  one  with  a  single-use  presentation.  Should 
you  perform  changes  and  add  material  to  the  presentation,  it  will  be 
cumbersome  to  revert  it  back  into  a  presentation  with  links  again. 
The  inclusions  have  to  be  deleted  manually  and  replaced  by  new, 
valid  links.  So  if  there  is  the  chance  that  you  will  want  to  update 


8  An  exception  might  be  if  one  is  using  Keynote  or  PowerPoint  not  primarily 
for  a  presentation  but  instead  as  a  way  of  storing  a  number  of  figures  for  a 
printed  publication.  This  can  be  a  convenient  means  of  accumulating  and 
annotating  the  figures  and  in  this  case,  you  can  store  the  images  at  the  reso¬ 
lution  appropriate  for  their  final  use  in  the  article. 
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TABLE  48.5.  Compression  Ratios  for  Large  Movies  (1024 
Square) 


Player/Format 

q  of  A 

q  of  B 

q  of  C 

q  of  D 

QuickTime  NC 

100% 

100% 

100% 

100% 

QuickTime  JPG 

24.2% 

15.1% 

2.75% 

2.75% 

QuickTime  QT 

16.6  % 

16.7% 

2.9% 

2.9% 

AVI  Cincepac 

5.2% 

5.2% 

4.1% 

4.0% 

your  presentation  after  packing,  this  approach  will  not  work  well. 

Also,  failures  of  this  procedure  have  been  reported.  It  works 
flawlessly  for  images  and  PowerPoint  animation  software  but  large 
movies  that  may  be  processed  by  codecs  not  originally  shipped  with 
Windows,  sometimes  do  not  play.  This  is  particularly  common  with 
movies  formats  that  contain  references  in  themselves  such  as  do 
MPEG-4  movie  streams.  Even  under  OSX.  non-standard  codecs 
can  lead  to  incomplete  wrapping  of  the  presentation. 

The  danger  here  lies  in  the  fact  that  this  situation  cannot  be 
avoided  by  packing  then  testing  the  presentation,  as  the  references 
on  the  machine  the  packing  is  performed  on  are  still  valid  and 
failure  will  only  occur  once  the  presentation  is  loaded  onto  the 
target  computer.  A  safe  way  to  force  a  valid  test  is  to  unlink  the 
references  —  if  all  your  included  data  is  distributed  in  a  path  (that 
would  be  a  directory  name  in  Windows  or  a  folder  in  OSX)  say 
“all,”  rename  “all”  as  “former_all.”  Then  running  the  packed  pre¬ 
sentation  will  actually  invalidate  improper  links  and  the  error  will 
show  when  the  presentation  is  run. 

2.  Moving  the  entire  filesystem  (or  file  tree)  is  a  very  flexible 
option  for  people  who  try  to  avoid  version  conflicts  between  pre¬ 
sentations.  If  you  keep  all  the  data  you  work  with  and  intend  to 
present  in  a  certain  path  (a  drive  and  directory  tree  under  Windows 
or  a  mounted  folder  under  OSX),  then  you  will  be  able  to  move 
this  file  system  between  different  computers  and  keep  all  refer¬ 
ences  in  your  presentations  valid.  For  Mac  users,  this  goal  can  be 
achieved  almost  trivially  by  having  a  portable  disk  with  a  name 
that  does  not  occur  on  any  of  the  target  computers.  So  when  the 
disk  with  the  cryptic  name  is  plugged  to  an  alien  Mac,  it  will 
appear  as  a  folder  accessible  from  the  desktop.  All  presentations 
can  then  be  moved  without  any  changes. 

PCs  have  an  unpleasant  restriction  that  the  drive  name 
assigned  to  such  a  disk  must  be  free  on  the  target  computer.  So  it 
is  not  wise  to  use  any  disk  drive  letter  either  between  A  and  E,  as 
they  tend  to  be  occupied  by  disks,  optical  drives,  and  memory 
sticks,  or  using  the  last  characters  V  through  Z  as  they  may  also 
be  used  for  images  of  network  drives.  Other  than  that,  the  more 
remote  letters  work  reliably  and  using  a  drive  letter  far  from  these 
zones  of  confusion  usually  works  well. 

Newer  operating  systems,  such  as  Windows  2000  and  XP, 
allow  the  use  of  “soft  links”  similar  to  the  folder  concept  of  the 
Apple  operating  system.  You  can  link  your  portable  drive  with  a 
soft  link  from  the  desktop  and  when  transporting  presentations 
simply  re-create  that  soft  link  on  the  target  machine.  This  approach 


will  not  work  reliably  with  dated  PowerPoint  versions  as  soft  links 
are  often  resolved  to  their  absolute  paths.  Given  that  you  use  a 
drive  X:  with  all  your  presentations,  hence  using  the  basic  path 
X:\all_my_presentations,  you  could  soft  link  it  from  the  desktop, 
say  as  “all_my_presentations.”  PowerPoint  97  may  still  store  the 
references  with  the  drive  name  though  and  hence  limit  portability. 
However,  as  long  as  this  link  name  is  not  in  use  on  the  target 
system,  this  provides  a  very  reliable  transportation  mechanism  for 
newer  PowerPoint  versions. 

3.  Approach  2  won't  work  if  either  you  cannot  use  your  own 
disk  or  want  to  hand  out  a  presentation.  Sharing  a  presentation  is 
made  much  easier  if  the  sharing  is  planned  in  from  the  start.  If  all 
images  and  animations  for  a  presentations  are  made  available  in  a 
single  directory  —  or  folder  —  from  the  beginning,  then  simply 
copying  that  folder  will  make  the  presentation  mobile.  This 
approach  is  referred  to  as  a flat file  structure,  in  contrast  to  the  direc¬ 
tory  tree  which  is  hierarchical.  For  older  PowerPoint  versions  (espe¬ 
cially  the  still-popular  95  and  97  releases),  this  can  lead  to  a  soft 
error  the  first  time  the  presentation  is  run  and  PowerPoint  will  feel 
obliged  to  ask  each  reference  for  an  updated  location.  The  newer 
versions,  however,  will  deal  with  the  flat  file  structure  smoothly. 

Note,  however,  that  with  this  approach,  the  person  who 
receives  the  presentation  has  copies  of  the  images  and  movies 
available  in  native  format.  Whether  or  not  you  want  to  give  them 
away,  PowerPoint  itself  does  not  prevent  your  data  from  being 
extracted.  PowerPoint  files  are  documented  and  even  a  pack-and- 
go  presentation  can  be  disassembled  into  its  components  again. 

To  obtain  more  sophisticated  protection  of  your  images  and 
animations,  make  sure  to  incorporate  a  copyright  watermark  into 
the  scans  and  the  movies.  Adobe  PhotoShop  and  Premiere,  as  well 
as  some  plug-ins  for  QuickTime,  will  perform  this  task.  Water¬ 
marks  are  very  difficult  to  remove  with  current  tools.  Another 
strong  protection  measure  is  to  lower  the  resolution  of  the  image 
material.  Scans  reduced  to  half  their  size,  say  512  square  will  look 
fine  on  a  projector  but  will  look  inferior  when  printed.  Animations 
in  MPEG-1  format  can  even  look  great  despite  the  fact  that  they 
are  of  far  lower  resolution  (352  x  288  pixels  at  native  resolution) 
than  the  time  series  from  which  they  were  obtained  (usually  of 
megapixel  resolution).  Both  measures,  watermarks  and  lowered 
resolution,  provide  the  best  available  protection  of  your  data  at  this 
stage.  More  advanced  protection  algorithms,  such  as  steganogra- 
phy,  which  encodes  additional  information  into  file  formats,  is 
sometimes  removed  when  “saved  again”  on  some  tools.  Field 
marks  or  any  other  watermarks  that  cover  the  entire  image  are  very 
safe  but  do  impair  the  image  quality.  They  imprint  a  noise  pattern 
on  the  image  that  is  unique  to  a  password  you  choose  and  the 
pattern  is  chosen  in  a  way  that  standard  filters,  such  as  smoothing 
or  noise  reduction,  will  not  remove.  In  contrast  to  normal  water¬ 
marks,  however,  it  is  not  visible  to  the  eye  and  hence  has  no 
meaning  for  daily  copyright  issues.  It  also  does  not  survive  reli¬ 
ably  in  print  and  so  will  not  stop  others  from  using  it. 

In  Apple  Keynote  2,  the  Save  As  command  yields  a  window 
that  asks  if  you  want  to  copy  theme  images,  audio,  or  movies  into 


TABLE  48.6.  Playback  Frames  per  Second  for  Large  Movies  (30-s  Window) 


Player/Format 

1024  x  1024  A 

1024  x  1024  B 

1024  X  1024  D 

1024  x  1024  D 

QuickTime  NC 

1 1 .5  fps 

Same 

Same 

Same 

QuickTime  JPG 

7.0fps 

8.0fps 

8.0  fps 

8.0  fps 

QuickTime  QT 

16.0  fps 

15.5  fps 

17  fps 

17  fps 

AVI  Cinepac 

26.5  fps 

27.0fps 

25  fps 

25  fps 

844  Chapter  48  •  F.  Margadant 


the  document  and  this  seems  likely  to  produce  a  file  similar  to  the 
PowerPoint  pack-and-go  file. 

Taking  Your  Presentation  on  the  Road:  Digital 
Rights  Management  and  Overlaying 

When  playing  animation  formats,  some  advanced  features  of  the 
native  player  software  can  cause  substantial  inconsistencies.  All 
players  are  optimized  to  provide  high  performance  and  will  exploit 
hardware  features  if  the  drawing  libraries  (DirectX  and  Quick¬ 
Draw)  allow  this.  One  of  the  very  popular  features  shared  by  the 
two  main  players  in  the  field  —  literally,  for  once  —  is  dubbed 
overlaying.  For  this,  the  content  of  the  movie  frames  is  not  drawn 
directly  into  the  video  memory  but  into  a  secondary  buffer  named 
the  overlay.  When  the  graphics  engine  refreshes  the  display,  it 
reads  the  contents  of  the  overlay,  if  one  is  present,  rather  than  the 
contents  of  the  main  image  buffer.  Overlays  must  be  of  rectangu¬ 
lar  shape  and  the  hardware  places  strict  limits  on  how  many  over¬ 
lays  there  can  be  in  a  display  frame.  The  limit  can  actually  be 
as  low  as  1  or  0.  The  advantage  of  doing  this  is  that  the  display 
does  not  have  to  be  redrawn  when  the  movie  plays.  This  permits 
a  much  more  naturally  integrated  movie  window  that  will  respond 
to  scaling  and  moving  much  like  any  other  application,  and  that 
hides  its  thirst  for  computing  power. 

However,  on  some  hardware,  overlaying  becomes  a  problem 
when  a  projector  is  added  in  parallel  to  the  primary  display  (i.e., 
on  a  laptop,  the  number  of  overlays  for  desktop  machines  is  often 
more  than  1  and  hence  the  problem  may  not  arise  —  repeat 
“may”).  Under  these  conditions,  sometimes  the  overlay  only  works 
for  the  primary  display  and  the  movie  output  on  the  projector  is 
blank  (actually,  black). 

Overlaying  can  be  turned  off  in  MediaPlayer:  choose  Tools  — » 
Options,  switch  to  the  Performance  tab  and  choose  Advanced,  and 
there,  under  Video  Acceleration,  you  can  disable  Use  Overlays. 
There  is  a  second  overlay  switch  for  DVD  playback  below  the  first 
one  that  you  might  have  to  deactivate  if  you  play  your  movie  from 
a  DVD. 

And  in  QuickTime:  go  to  Edit  —>  Preferences  — »  QuickTime 
Preferences,  toggle  to  Video  Settings,  and  deactivate  Enable 
DirectDraw. 

As  this  will  disable  all  acceleration  via  the  graphics  library  and 
hence  impair  performance,  it  is  not  the  generic  setting  to  be  used 
all  the  time.  It  is  just  a  possible  solution  to  the  blank-movie 
window  problem  described  above. 

A  much  more  general  problem  that  seems  to  be  dramatically 
increasing  over  time  is  the  Digital  Rights  Management  mechanism 
built  into  the  standard  players.  This  is  a  protection  mechanism 
intended  to  uphold  copyrights  and  inhibit  duplication  of  copy¬ 
righted  material  and  is  capable  of  preventing  you  from  grabbing 
screen  shots  and  sequences  (clips)  from  DVDs  or  other  restricted 
movies. 

The  problem  is  that,  as  there  is  no  open  publication  of  what 
DRM  does  and  how  it  affects  your  system,  the  normal  user  can 
have  problems  when  the  player  interprets  your  movie  as  being 
copyrighted  even  when  it  isn’t.  This  type  of  error  can  happen  when 
using  advanced  formats9  in  QuickTime  and  MPEG2  and  the  results 


9  If  you  use  an  animation  editing  tool  such  as  Premiere,  you  can  copyright  the 
output  material  which  gives  you  a  different  file  format  from  the  original  data. 
Editing  lingo  is  production  format  for  the  original  file  and  consumer  format 
for  the  distributed,  that  is,  the  copyrighted  file.  These  latter  files  are  suscep¬ 
tible  of  being  handled  more  strictly  by  DRM  mechanisms. 


can  vary  from  not  being  able  to  play  the  movie  in  PowerPoint  to 
just  not  being  able  to  display  it  on  a  projector. 

A  lesser  nuisance  is  that  movie  startup  may  be  delayed  while 
the  player  tries  to  check  to  see  if  it  is  authorized  to  play  the  format 
involved. 

The  safest  way  to  prevent  problems  of  this  kind  is  to  keep  your 
movies  in  the  original  format  when  playing  them.  Even  playing  a 
DVD  as  part  of  a  presentation  may  lock  the  player  to  a  fixed  frame 
rate  and  format.  If  the  movie  files  are  present  in  the  original,  simple 
format  (MPEG  files  or  AVIs  that  you  wrote  yourself),  it  is  very 
unlikely  that  you  will  encounter  these  problems. 

Finally,  resist  the  temptation  to  update  the  version  of  your 
native  players  just  before  a  presentation.  Continuously  escalating 
DRM  requirements  may  make  unplayable  a  movie  that  had  been 
playable  on  an  earlier  version.  Although  the  chances  that  this  will 
happen  are  small,  they  are  fatal  for  a  presentation.  In  general,  it  is 
inherently  risky  to  update  your  player  as  some  of  your  movie 
formats  may  no  longer  be  supported  or  the  new  player  may  insist 
that  your  formats  must  comply  with  certain  rules  and  will  not  play 
them  unless  they  do.  This  applies  particularly  to  QuickTime  files 
and  MPEG-2  or  MPEG-4  formats. 

At  present,  most  player  software  still  behaves  benignly.  They 
will  play  all  formats  to  the  projector  if  the  overlaying  is  switched 
off,  and  when  they  cannot  retrieve  a  license,  they  search  for  a 
movie  file,  and  then  play  this  format.  One  notable  exception 
involves  movies  on  DVDs,  and  these  come  with  severe  restrictions 
and  may  not  even  be  able  to  be  played  at  the  frame  rate  of  your 
projector.  However,  the  topic  of  DRM  continues  to  evolve  quickly 
and,  at  present,  the  only  way  to  retain  control  over  your  anima¬ 
tions  is  to  keep  them  in  the  original  formats  that  you  can  edit  (i.e., 
the  production  format).  In  other  words,  you  can  use  the  Quick¬ 
Time,  MPEG-4,  and  MPEG-2  formats,  but  you  must  keep  them 
original  (i.e.,  after  writing  them,  do  not  package  them  or  write 
them  onto  a  DVD  or  VCD),  unless  you  are  sure  that  you  have 
not  included  or  activated  an  any  copyrights  when  producing  the 
movie. 

If  you  do  have  to  resort  to  DVDs,  make  sure  that  you  do  not 
create  a  Region  Code  that  will  prevent  the  DVD  from  playing  on 
another  continent  than  the  one  on  which  the  DVD  was  created. 
This  problem  can  be  a  particular  nuisance  when  assembling  pre¬ 
sentations  for  use  at  international  conferences.  The  DVD  feature 
to  avoid  at  all  costs  is  User  Prohibitions,  that  is,  locking  the  way 
the  DVD  has  to  be  played  and  accessed.  These  constraints  will 
prevent  the  player  from  playing  even  an  excerpt  from  the  movie 
and  they  will  also  lock  down  the  frame  rate  of  the  playback.  Unless 
you  can  make  sure  that  these  features  are  switched  off  when 
creating  your  DVD,  you  run  the  risk  that  others  may  not  be  able 
to  play  your  presentation  and  you,  yourself,  may  not  be  able  to 
play  it  on  any  other  system  or,  at  least  not  in  the  sequence  you 
intend  to.  To  be  prepared,  it  is  always  wise  to  carry  a  copy  of  your 
production  files  with  you. 

Good  Luck! 
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When  Light  Microscope  Resolution  Is  Not  Enough: 
Correlational  Light  Microscopy  and  Electron  Microscopy 

Paul  Sims,  Ralph  Albrecht,  James  B.  Pawley,  Victoria  Centonze, 

Thomas  Deerinck,  and  Jeff  Hardin 


INTRODUCTION 

Early  Correlative  Microscopy 

Notwithstanding  the  many  ways,  amply  documented  elsewhere  in 
this  book,  in  which  fluorescent  light  microscopy  can  elucidate  bio¬ 
logical  structure  and  function,  there  are  times  when  the  available 
spatial  resolution  is  just  not  sufficient  to  answer  the  biological 
question.  Consequently,  there  is  a  need  to  follow  up  the  initial  light 
microscope  (LM)  findings  by  subsequently  viewing  the  LM  spec¬ 
imens  in  the  transmission  or  scanning  electron  microscope  (TEM 
or  SEM).  Correlative  microscopy  of  this  type  has  a  long  history. 
In  1973,  shortly  after  LM  stains  had  been  developed  that  identi¬ 
fied  T-  and  B-lymphocytes,  Wetzel  viewed  the  same  exact  cells, 
first  in  the  LM  and  then  in  the  SEM  to  show  that  being  a  T  or  a  B 
cell  bears  no  relation  to  whether  the  cells  appeared  to  be  “rough” 
or  “smooth”  when  viewed  in  the  SEM  (Wetzel  et  al.,  1973). 

Somewhat  later,  Sepsenwol  used  time-lapse  studies  in  the  LM 
followed  by  high-voltage  electron  microscopy  (HVEM)  and  SEM 
to  study  the  unusual  protein  on  which  Ascaris  sperm  motility  is 
based  (Pawley  et  al.,  1986;  Sepsenwol  et  al.,  1989;  Sepsenwol  and 
Taft,  1990). 

Early  4D  Microscopy 

Albrecht  and  his  group  tracked  the  motion  of  colloidal 
gold-labeled  proteins  on  the  surface  of  activated  platelets,  using 
time-lapse,  rectified,  differential  interference  contrast  (DIC),  LM 
(Fig.  49.1),  before  determining  the  final  position  of  the  gold  par¬ 
ticles  using  both  low-voltage  SEM  (LVSEM;  Figs.  49.2,  49.3,  and 
49.4;  Pawley,  1990,  1992)  and  HVEM  (Figs.  49.5  and  49.6; 
Albrecht  et  al.,  1989,  1992;  Loftus  et  al.,  1984). 

This  pioneering  work  did  much  to  elucidate  the  mechanism  of 
clot  formation.  It  was  possible  in  part  because  the  platelet  is  small 
enough  to  be  viewed  in  the  HVEM  as  a  critical-point-dried  (CPD) 
whole  mount  and  in  part  because  the  colloidal  gold  used  to  label 
the  surface  receptors  of  interest  could  be  seen  in  the  LM,  the  SEM 
and  the  HVEM.  The  LM  allowed  one  to  watch  the  movement  of 
the  Au-labeled,  fibrinogen  receptors;  the  LVSEM  at  1  and  5kV 
allowed  one  to  see  how  these  markers  were  bound  to  the  surface 
(Pawley  and  Albrecht,  1988);  and  stereo  views  from  the  HVEM 
allowed  one  to  correlate  these  surface  changes  with  changes  in  the 
location  and  orientation  of  the  cytoskeleton.  The  details  of  these 
early  studies  are  found  in  the  captions  of  Figures  49.1  through  49.9. 


More  recently,  this  group  extended  the  technique  by  adding  an 
additional,  sensitive  charge-coupled  device  (CCD)  camera  and  an 
ingenious  system  of  dichroic  filters  to  permit  them  to  image  living 
cells  using  ultraviolet  (UV)  fluorescence  at  the  same  time  that  they 
were  being  viewed  using  rectified  DIC.  This  has  allowed  them  to 
follow  the  motion  of  the  20  nm  gold  particles  in  the  DIC  image 
while  monitoring  the  intracellular  Caw  concentration  using  fura-2. 
As  shown  in  the  previous  images,  the  binding  of  fibrinogen  to  the 
receptor  (and  associated  receptor  cross-linking)  triggers  a  cen¬ 
tripetal  movement  (edge  to  center)  of  the  receptor-ligand  com¬ 
plexes  over  the  platelet  surface.  The  new  instrumentation  has 
allowed  them  to  determine  that  the  movement  of  the  receptors 
across  the  platelet  surface  coincides  with  a  Caw  transient  (Fig. 
49.10)  and  to  do  this  on  a  platelet  that  they  can  view  subsequently 
by  LVSEM. 

CORRELATIVE  LIGHT  MICROSCOPE/ELECTRON 
MICROSCOPE  TODAY 

Light  Microscope  and  Electron  Microscope  Have 
Different  Requirements 

Because  the  structural  details  of  electron  microscope  (EM) 
samples  must  be  preserved  in  much  greater  detail  and  because 
almost  all  EM  specimens  must  be  viewed  in  vacuo,  procedures  for 
preparing  them  differ  markedly  from  those  used  in  the  LM.  Cor¬ 
relative  studies  can  be  segregated  in  several  ways.  The  first  way  is 
based  on  the  order  in  which  observations  are  made: 

1.  The  LM  observations  are  made  first,  on  fixed  or  living  cells, 
and  these  are  then  prepared  for  EM.1 

2.  The  preparation  is  fixed  and  stained  before  LM  observation. 

3.  Cells  incorporating  fluorescent  markers  are  prepared  for  thin- 
section  TEM,  but  these  are  viewed  in  the  LM  just  before  being 
viewed  in  the  EM. 


1  Because  the  act  of  observing  a  biological  specimen  in  an  EM  subjects  it  to  a 
flux  of  radiation  so  high  that  virtually  all  the  organic  molecules  present  (such 
as  dyes  or  stains)  are  irretrievably  damaged,  there  is  seldom  much  point  in 
viewing  a  specimen  in  an  LM  after  it  has  been  viewed  in  an  EM.  The  excep¬ 
tion  to  this  rule  is  the  quantum  dot,  a  fluorescent  label  that,  as  discussed  below, 
is  not  destroyed  by  being  observed  in  the  TEM. 
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FIGURE  49.1.  Time-series  of  DIC  images  showing  initial  binding  of  Au-conjugated  fibrinogen  (black)  to  the  platelet  surface  membrane  over  the  subjacent 
peripheral  web  and  outer-filamentous  zone  of  the  cytoskeleton  of  a  fully  spread,  substrate-adherent  platelet.  This  label  binds  to  the  integrin  receptor  for  fibrino¬ 
gen  on  platelets.  Over  several  minutes,  bound  labels  are  transported  over  the  surface,  towards  the  platelet  center  coming  to  rest,  still  on  the  membrane  surface, 
but  now  overlying  the  inner  filamentous  zone.  The  platelet  is  then  fixed,  stained  with  osmium  and  uranyl-actetate,  and  dried  by  the  critical-point  procedure  for 
subsequent  LVSEM  and  HVEM.  Bar  =  1  pm. 


FIGURE  49.2.  Surface  image  of  the  same  uncoated  platelet  as  Figure  49.1  FIGURE  49.3.  SEM  at  5kV,  still  in  the  SE  mode.  The  increased  beam  pene- 
viewed  in  LM  at  1  kV  accelerating  voltage  in  a  modified  Hitachi  S-900  SEM.  tration  clearly  demonstrates  the  location  of  the  gold  particles,  bright  spots, 
It  shows  the  platelet  surface  and  labels  in  detail.  What  is  actually  seen  is  relative  to  stained  internal  cytoskeletal  structures.  However,  surface  structure 
the  platelet  surface  and  the  fibrinogen-covered  individual  gold  particles.  is  less  apparent.  Bar  =  1  pm. 

Bar  =  1  pm. 
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FIGURE  49.4.  SEM  at  20 kV  and  somewhat  higher  magnification.  This 
demonstrates  the  relationship  of  the  labels  on  the  membrane  surface  to  the 
underlying  cytoskeletal  organization.  Bar  =  0.25  pm. 


FIGURE  49.6.  A  higher  magnification  HVEM  of  the  same  area  of  the  platelet 
as  seen  in  Figure  49.4.  P,  peripheral  web;  OF,  outer  filamentous  zone;  IF,  inner 
filamentous  zone;  G,  granulomere;  mt,  microtubules;  dark  arrowheads,  margin 
of  inner  filamentous  zone;  white  arrowheads  in  Figure  49.3  point  to  labels 
trapped  under  the  platelet.  Bar  =  0.25  pm. 


FIGURE  49.5.  HVEM  stereo-pair  whole  mount  of  the  same  platelet,  which  offers  a  clear  view  of  the  platelet  cytoskeleton.  The  gold  labels,  black  spots,  are 
clearly  seen  in  relationship  to  the  subjacent  platelet  cytoskeleton.  Bar  =  1  pm. 


FIGURE  49.7.  DIC  imaging  tracking  of  movement  of  individual  label  particles  across  the  surface  of  a  fully  spread  platelet. 


FIGURE  49.8.  The  same  platelet,  following  fixation,  staining,  and  dehydra¬ 
tion,  is  seen  via  LVSEM  at  1.5  kV  accelerating  voltage.  The  position  of  the 
individual  tracked  particles  at  the  time  of  fixation  relative  to  the  platelet  surface 
can  be  seen. 


FIGURE  49.9.  The  same  platelet,  HVEM 
stereo-pair,  demonstrating  the  position  of  the 
particle  labels  relative  to  internal  structure.  P, 
peripheral  web;  OF,  outer  filamentous  zone; 
IF,  inner  filamentous  zone;  G,  granulomere 
zone;  m,  microfilament  bundles.  Arrows  point 
to  individual  labels  also  seen  in  Figure  49.8, 
asterisk  indicates  particle  tracked  in  Figure 
49.7  and  seen  in  Figures  49.8  and  49.9.  Labels 
fixed  in  transit  generally  still  appear  over  the 
outer  filamentous  zone  while  labels  that  have 
completed  their  movement  are  generally  seen 
over  the  inner  filamentous  zone.  Bar  =  1.0  pm. 
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FIGURE  49.10.  (A)  and  (B)  show  a  platelet  imaged  simultaneously  via 
DIC  and  UV  fluorescence.  In  (A),  the  diffraction  images  of  colloidal  gold- 
fibrinogen  particles  can  be  followed  over  time  as  they  move  across  the  platelet 
surface.  (B)  Fura-2  340nm/380nm  ratio  images  that  provide  a  measure  of  the 
free  internal  [Ca-^]  as  the  process  proceeds  (blue  represents  resting  [Ca**]  levels 
while  red  and  white  show  increasing  [Ca++]).  Although  the  initial  fibrinogen¬ 
binding  produces  no  increase  in  free  [Ca-^]  (left  panels),  once  the  movement 
of  the  fibrinogen-receptor  complexes  is  initiated,  [Ca++]  is  seen  to  increase 
(middle  panels)  until  the  movement  is  complete  (right  panels).  (C)  An  LVSEM 
image  of  the  same  platelet  shows  the  final  position  of  the  gold-fibrinogen- 
receptor  complexes  (bright  spots)  relative  to  platelet  surface  structure. 


Another  way  of  dividing  correlative  methods  is  between  those 
using  the  TEM  and  those  using  the  SEM.  A  final  criterion  that 
might  be  used  to  characterize  correlative  LM/EM  studies  is 
whether  it  is  essential  to  view  exactly  the  same  cell  using  all 
methods  or  if  it  is  sufficient  to  use  the  EM  image  merely  to  find 
out  the  general  type  of  structures  being  labeled  in  the  LM  prepa¬ 
ration.  The  following  sections  will  include  a  brief  review  of  recent 
studies  illustrating  all  of  these  approaches. 


Finding  the  Same  Cell  Structure  in 
Two  Different  Types  of  Microscope:  Light 
Microscope/Scanning  Electron  Microscope 

Because  both  the  LM  and  the  SEM  permit  one  to  view  quite  large 
specimens,  it  is  often  easier  to  find  the  exact  cell  previously  viewed 
in  an  LM  using  an  SEM  rather  than  a  TEM.  Centonze  and  co¬ 
workers  used  this  combination  to  investigate  the  effect  of  fluores¬ 
cence  recovery  after  photobleaching  (FRAP)  on  the  molecular 
structure  of  the  labeled  structures.  In  FRAP,  a  structure  or  volume 
that  has  been  labeled  with  fluorescent  dye  in  a  living  cell  is  inten¬ 
tionally  bleached  and  one  then  measures  the  rate  at  which  fluo¬ 
rescence  returns  to  the  bleached  area  (see  Chapters  5,  8,  9,  17,  and 
45,  this  volume).  Alternatively,  one  may  wish  to  derive  inferences 
from  the  motion  of  the  bleached  area.  One  well-studied  example 
is  the  motion  of  a  band  bleached  across  a  mitotic  apparatus  made 
of  fluorescent  tubulin.  In  this  case,  the  band  did  not  move  even 
though  mitosis  proceeded  (Gorbsky  et  al.,  1987).  Centonze  used 
the  SEM  to  investigate  this  further. 

To  do  this,  gold  was  evaporated  through  an  EM  finder  grid 
onto  the  surface  of  a  glass  coverslip  to  create  a  fiduciary  pattern 
that  could  be  seen  in  both  the  LM  and  the  SEM.  These  coverslips 
were  then  mounted  into  a  hole  in  the  bottom  of  a  plastic  Petri  dish 
using  silicone  grease  and  isolated,  fluorescent  microtubules  were 
allowed  to  adhere  to  the  glass.  After  obtaining  reference  transmis¬ 
sion  and  fluorescence  LM  images,  some  of  the  microtubules  not 
located  over  the  gold  were  bleached,  using  546  nm  light  from  an 
argon-ion  laser,  for  a  measured  period  of  time  at  a  known  power 
level.  The  preparation  was  then  fixed,  critical-point  dried,  coated 
with  ion-beam-sputtered  Pt,  and  viewed  in  a  high-resolution,  low- 
voltage  SEM  at  1.5  kV.  Figure  49.11  shows  the  results.  While  a  10 
ms  pulse  caused  little  visible  damage,  30  ms  caused  total  destruc¬ 
tion.  Fortunately,  as  is  shown  in  Figure  49.12(A),  it  is  possible  to 
produce  significant  bleaching  using  a  pulse  only  1  ms  in  duration, 
a  period  thought  unlikely  to  cause  severe  structural  damage.  It 
should  also  be  noted  that  dissolution  or  disruption  of  microtubules 
could  also  be  induced  by  repeated  illumination  under  conditions 
used  for  normal  fluorescence  observations. 

Figure  49.13  shows  this  system  applied  to  bleaching  micro¬ 
tubules  in  a  living  cell  grown  on  a  marked  coverslip.  In  this  case. 


Control  10ms  30ms  50ms 


FIGURE  49.1 1 .  Low-voltage  scanning  electron  micrographs  of  single  fluorescent  microtubules  that  had  earlier  been  subjected  to  irradiation  by  a  546 nm  bleach¬ 
ing  beam  for  the  times  listed.  Any  exposure  above  10ms  destroyed  the  microtubule. 
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FIGURE  49.12.  Fluorescence  micrographs  showing  single  microtubules,  composed  of  in  vitro  polymerized  microtubules  made  from  rhodamine-conjugated 
tubulin,  across  which  a  bar  has  been  bleached  by  exposure  to  a  546  nm  laser  beam  for  the  noted  times.  (A)  shows  that  substantial  bleaching  is  produced  by  a 
1  ms  exposure,  a  level  lOx  lower  (B)  than  that  shown  to  produce  structural  damage  in  Figure  49.11.  The  plots  below  each  image  show  the  intensity  along  a  line 
down  the  center  of  the  microtubule  (upper  trace)  and  along  the  straight  white  line,  showing  the  background  signal  (lower  trace). 


FIGURE  49.13.  (A)  Phase-contrast  image  of  a  spread  cell  containing  microtubles  made  out  of  rhodamine-conjugate  tubulin.  (B)  Fluorescence  image  of  the 
same  field.  (C)  Bleached  bar  and  (D)  fluorescence  image  of  the  same  areas  of  the  specimen  after  it  has  been  prepared  for  SEM.  (E)  Tiled  montage  of  low- voltage 
SEM  images  in  which  one  can  see  that  all  the  microtubules  visible  in  the  LM  images,  even  those  in  the  bleached  zone  (brackets),  remain  physically  intact  after 
the  bleaching  event. 
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fibroblast  cells  were  grown  on  gold-sputtered,  locator  coverslips 
and  injected  with  fluorescent  derivatized  tubulin.  After  the  injected 
tubulin  was  allowed  to  incorporate  into  the  microtubule  cytoskele- 
ton  for  at  least  1  h,  the  injected  cells  were  relocated  using  phase- 
contrast  microscopy.  Fluorescent  microtubules  were  imaged  both 
before  and  after  photobleaching  a  discrete  region  [Fig.  49.13(D)]. 
Because  the  fixation  protocol  involved  lysing  the  cells  in  Triton 
X-100  detergent  before  fixation  in  2%  glutaraldehyde  in  a  cyto 
skeletal-stabilizing  buffer  containing,  Pipes,  Hepes,  EGTA  and 
Mg”  (PFIEM)  buffer,  the  montage  of  SEM  images  shows  the 
cytoskeleton,  rather  than  the  cell  membrane.  In  this  case,  bleach 
levels  were  low  enough  to  create  a  photobleach  mark  without  dis¬ 
rupting  the  microtubules  so  that  all  of  the  microtubules  visible  in 
the  fluorescence  image  are  visible  in  the  SEM  montage.  LVSEM 
images  of  specimens  bleached  with  higher  power,  or  for  a  longer 
time,  showed  that  the  affected  microtubules  could  be  completely 
ablated.  The  ablation  was  documented  in  three  ways:  (1)  the  fluo¬ 
rescence  recovery  rates  were  observed  to  be  different  in  the 
bleached  region;  (2)  treatment  with  fluorescent  anti-tubulin  pro¬ 
duced  no  staining  in  the  photobleached  region  because  the  struc¬ 
tures  were  destroyed;  and  (3)  SEM  images  clearly  show  no 
microtubule  structures  in  the  photobleached  region  (data  not 
shown).  This  correlational  study  should  give  pause  to  anyone  con¬ 
vinced  that  the  ONLY  possible  effect  of  photobleaching  is  that  the 
dye  in  the  irradiated  volume  becomes  non-fluorescent.  More  insight 
into  photodamage  mechanisms  is  provided  in  Chapters  38  and  39. 

Finding  the  Same  Cell  Structure  in  Two 
Different  Types  of  Microscope:  Light 
Microscope/Transmission  Electron  Microscope 

Although  correlative  microscopy  uses  both  light  and  electron 
microscopy  to  examine  the  same  sample,  each  microscopic  method 
produces  contrast  in  a  very  different  way.  When  viewing  living  bio¬ 
logical  specimens  in  the  LM,  one  commonly  uses  phase,  DIC,  or 
more  recently,  fluorescence  imaging.  However,  in  the  TEM,  none 
of  these  factors  produces  significant  contrast,  and,  in  addition,  the 
TEM  specimen  must  be  considerably  thinner  than  most  cells.  To 
produce  enough  mass-thickness  contrast  to  make  biological  struc¬ 
tures  visible  in  the  TEM,  one  must  somehow  decorate  them  with 
heavy  metal  stains,  such  as  uranyl-acetate  and  lead  compounds.  As 
a  result,  LM/TEM  correlative  methods  generally  depend  on  the  use 
of  some  technique  that  will  deposit  heavy  metals  at  or  near  the  site 
of  the  fluorescent  dye.  Possibilities  include  double  labeling  with 
both  fluorescent  and  Au-conjugated  antibodies  and  using  light  cap¬ 
tured  by  the  fluorescent  dye  to  initiate  a  chemical  reaction  that  later 
results  in  the  deposition  of  a  heavy  metal.  Finally,  quantum  dots 
are  particularly  useful  because  they  are  both  fluorescent  and 
directly  visible  in  the  TEM  (Niesman  et  al.,  2004). 

Correlative  LM/TEM  techniques  are  also  complicated  by  the 
fact  that  the  visible  area  of  a  TEM  grid  is  usually  less  than  2  mm 
in  diameter  and  quite  a  lot  of  this  area  is  obscured  by  grid  bars. 
Moreover,  it  is  in  general  hard  to  keep  track  of  changes  in  the  ori¬ 
entation  of  specific  structures  in  the  LM  specimen  as  the  sample 
passes  the  many  steps  needed  to  prepare  it  for  thin-section  TEM. 
This  can  make  it  very  difficult  to  find  exactly  the  same  feature 
using  both  methods,  especially  if  the  handedness  of  the  image  has 
been  changed  by  the  section  having  been  mounted  upside-down. 

Making  LM  Labels  Visible  in  the  Transmission 
Electron  Microscope 

There  are  a  few  techniques  that  allow  the  same  stained  sample  to 
be  used  for  both  light  and  electron  microscopy.  One  of  the  most 


successful  methods  for  doing  this,  called  fluorescence  photo-oxi¬ 
dation,  was  developed  by  Maranto  (1982)  and  later  extended  to 
immunolabeling  and  in  situ  hydridization  by  Ellisman  (Deerinck 
et  al.,  1994).  It  relies  on  the  fact  that,  when  certain  fluorescent 
dyes,  such  as  eosin,  are  excited  in  the  presence  of  diaminobenzi- 
dine  (DAB),  the  reactive  oxygen  produced  by  the  triplet-excited 
fluorescent  compound  causes  the  DAB  to  form  a  deposit  very  close 
to  the  reaction  site.  This  deposit  can  then  be  stained  with  con¬ 
siderable  specificity  with  osmium  tetroxide  (Fig.  49.14),  rendering 
it  visible  by  both  transmitted  light  and  electron  microscopy. 
Because  the  reaction  is  limited  to  only  the  region  near  the  excited 
fluorescent  dye,  the  precise  area  can  be  located  in  the  epoxy- 
embedded  specimen  by  light  microscopy  and  then  prepared  for 
thin  section  electron  microscopy. 

More  recently,  this  group  has  adapted  the  tetracysteine  genetic 
marking  technique  to  produce  the  reactive  oxygen  needed  to 
deposit  the  DAB  in  labeled  cells  (Griffin  et  al.,  1998;  see  Chapter 
16,  this  volume).  In  this  technique,  a  tetracysteine  tag  sequence  is 


FIGURE  49.14.  (A)  Using  immunofluorescently  stained  bovine  aortic  epithe¬ 
lial  cells  for  both  light  and  electron  microscopy,  cultured  cells  were  labeled 
with  an  antibody  to  beta-tubulin  followed  by  a  secondary  antibody-eosin  con¬ 
jugate.  Following  confocal  imaging  of  the  eosin  fluorescence,  the  specimen  is 
intensely  illuminated  in  the  presence  of  diaminobenzidine.  The  resulting  reac¬ 
tive  oxygen  creates  a  reaction  product  that  can  be  subsequently  visualized  by 
electron  microscopy  (B).  (Image  kindly  provided  by  the  laboratory  of  Mark 
Ellisman  at  the  National  Center  for  Microscopy  and  Imaging  Research,  Uni¬ 
versity  of  California,  San  Diego.) 
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FIGURE  49.15.  Using  a  genetically  encoded  tetracys- 
teine  tag  for  labeling  proteins  for  light  and  electron 
microscopy,  cultured  cells  expressing  a  recombinant 
version  of  the  major  gap  junction  protein  Cx43  that  con¬ 
tains  a  small  tetracysteine  tag.  Living  cells  are  stained 
with  the  biarsenical  red  fluorescent  compound  ReAsH 
(A).  Following  imaging,  the  cells  are  fixed  and  the 
specimen  is  intensely  illuminated  in  the  presence  of 
diaminobenzidine.  The  resulting  reactive  oxygen  creates 
a  reaction  product  that  can  be  subsequently  visualized 
by  electron  microscopy  (B).  (C)  is  a  higher  magnifica¬ 
tion  view  of  (B).  (Images  kindly  provided  by  the  labo¬ 
ratory  of  Mark  Ellisman  at  the  National  Center  for 
Microscopy  and  Imaging  Research,  University  of 
California,  San  Diego.) 


introduced  into  the  gene  of  the  protein  of  interest  in  such  a  way 
that  the  four  cysteines  can  bind  to  a  bi-arsenical  derivative  of  flu¬ 
orescein  (green  fluorescence)  or  resorufin  (red  fluorescence). 
Living  cells  can  be  bathed  in  a  solution  of  these  membrane- 
permeant  dyes  without  damage  as  long  as  the  arsenicals  are  neu¬ 
tralized  with  small  vicinal  dithiols  such  as  2,3-dimercaptopropanol 
or  1,2-ethanedithiol.  The  dye  becomes  fluorescent  only  when  it 
bonds  specifically  to  the  tetracysteine  moiety.  Like  eosin,  the  red- 
fluorescent  version  of  this  dye  (called  ReAsH),  is  capable  of  pro¬ 
ducing  reactive  oxygen  and  depositing  DAB.  The  result  is  a 
genetically  encoded  marker  that  can  be  seen  in  both  LM  (by  fluo¬ 
rescence)  and  TEM  (by  specific  staining)  (Gaietta  et  al.,  2002;  Fig. 
49. 15).  LM/EM  correlation  can  be  obtained  by  embedding  the  cells 
still  attached  to  the  coverslip  and  making  high-  and  low-magnifi¬ 
cation  images  of  them  with  a  confocal  microscope  before  the  cov¬ 
erslip  is  removed.  These  fluorescent  images  can  them  be  correlated 
with  low-magnification  TEM  images  of  serial-section  ribbons 
made  starting  from  what  had  been  the  plastic/coverglass  interface. 

This  same  group  has  also  had  success  labeling  structures  in 
such  a  way  that  the  label  can  be  seen  in  both  LM  and  EM  using 
quantum  dots.  These  nanocrystals  are  both  fluorescent  (Nisman 
et  al.,  2004)  and  directly  visible  in  thin-section  TEM  (Giepmans, 
2005;  Fig.  49.16).  One  of  the  major  advantages  of  using  quantum- 
dot  fluorophores  is  that  different  colors  of  quantum  dots  can  be 

- ► 

FIGURE  49.16.  Using  quantum  dots  as  labels  for  light  and  electron 
microscopy,  cultured  cells  were  labeled  with  an  antibody  to  beta-tubulin  fol¬ 
lowed  by  a  secondary  antibody-quantum  dot  655  conjugate  (cell  nuclei  were 
counterstained  with  Hoechst  33342).  Following  confocal  imaging  (A),  the 
specimen  is  prepared  for  electron  microscopy  (B).  The  crystalline  core  of  the 
quantum  dots  are  readily  visible  by  electron  microscopy.  (Image  kindly  pro¬ 
vided  by  the  laboratory  of  Mark  Ellisman  at  the  National  Center  for  Microscopy 
and  Imaging  Research,  University  of  California,  San  Diego.) 
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discriminated  by  both  light  and  electron  microscopy  (via  size  and 
shape  differences),  making  double  or  even  triple  labeling  possible. 
Furthermore,  they  are  not  so  readily  destroyed  by  electron  irradi¬ 
ation  as  are  organic  dyes.  As  a  result,  it  is  possible  to  obtain  fluo¬ 
rescent  images  from  the  sections  after  they  have  been  viewed  in 
the  TEM.  On  the  other  hand,  the  fluorescence  of  quantum  dots  is 
destroyed  if  heavy  metal  stains  are  used  to  produce  contrast. 

Both  of  these  techniques  allow  one  to  test  and  optimize  anti- 
body-labeling  parameters  first  by  LM  before  continuing  the  more 
laborious  processing  for  electron  microscopy. 

Marrying  Fluorescence  with  TEM  Replicas  to 
Analyze  the  Cytoskeleton 

Phase-contrast  and  fluorescence  microscopy  have  been  used  to 
visualize  actin  dynamics  at  the  leading  edge  of  living  cells  in  which 
various  cytoskeletal  proteins  have  been  labeled  with  GFP.  Svitkina 
and  colleagues  followed  up  such  studies  by  removing  the  mem¬ 
branes  with  an  extracting  buffer,  and  then  shadowing  the  prepara¬ 
tion  with  Pt  after  it  had  been  critical-point  dried  to  visualize  actin 
filament  structure  in  the  TEM.  This  led  them  to  propose  a  filopo- 
dial  initiation  model  (Svitkina  et  al.,  2003).  Such  studies  provide 
additional  insights  into  the  detailed  topology  of  cytoskeletal 
assemblies  in  cells.  Because  processing  for  EM  is  performed  on 
cells  with  a  known  history  based  on  LM  and  potential  shrinkage 
artifacts  are  carefully  monitored  during  fixation,  this  approach  pro¬ 
vides  greater  confidence  that  the  more  highly  resolved  EM  images 
directly  reflect  events  imaged  using  LM. 

FluoroNanoGold  for  Cryosections  to  be 
Viewed  by  LM,  then  TEM 

In  addition  to  its  utility  in  analyzing  living,  migrating  cells, 
viewing  specimens  first  using  LM  followed  by  TEM  also  has  other 
uses.  In  some  cases  (e.g.,  immunostaining),  it  is  far  easier  to  label 
the  structures  of  interest  in  fixed  and  sectioned  specimens  using 
LM  techniques.  Knowing  that  a  given  specimen  contains  the 
features  of  interest  provides  more  confidence  that  subsequent 
processing  for  TEM  will  yield  a  specimen  worth  analyzing  in 
detail. 

Although  it  has  long  been  clear  that,  compared  to  chemical  fix¬ 
ation,  cryopreparative  techniques  both  arrest  cellular  processes 
faster  (milliseconds  vs.  seconds)  and  preserve  biological  structure 
down  to  the  molecular  level  better,  the  complexity  of  the 
equipment  and  procedures  needed  to  freeze  even  modest-sized 
specimens  without  creating  ice-crystal  artifacts  has  delayed  its 
widespread  use  in  light  microcopy  (Biel  et  al.,  2003).  However, 
because  cryotechniqes  are  unsurpassed  when  it  comes  to  preserv¬ 
ing  antigenicity,  in  1998  Takizawa  and  colleagues  stained  thin, 
cryosections  with  FluoroNanoGold  (FNG)  antibodies  for  correla¬ 
tive  LM  and  TEM  on  the  same  thin  section  (Takizawa  et  al.,  1998). 
Ultrathin  cryosections  were  cut  from  a  frozen  suspension  of  fixed 
cells  embedded  in  gelatin/sucrose.  The  sections  were  picked  up  on 
finder  grids,  stained  with  FNG,  and  viewed  first  by  fluorescence 
LM  and  then  by  TEM.  FluoroNanoGold  labels  consist  of  ondeca 
gold  conjugated  with  a  fluorescent  dye.  As  the  ondeca  gold  con¬ 
tains  only  11  gold  atoms,  it  often  penetrates  better  than  the  large 
colloidal- Au  labels.  On  the  other  hand,  it  is  difficult  to  see  even  in 
TEM  unless  it  has  been  decorated  by  enhancement  using  silver  or 
gold  salts. 

Several  examples  of  this  technique  have  been  reported.  To 
determine  the  diameter  of  transcription  sites  in  the  nuclei  of  HeLa 
cell,  Pombo  and  colleagues  made  cryosections  100  to  200  nm 
thick,  stained  them  with  fluorescent  antibodies  for  imaging  in  the 


LSCM,  and  then  re-embedded  them  in  Epon  to  image  the  same 
structures  by  TEM  (Pombo  et  al.,  1999).  In  2001,  Robinson 
reviewed  the  techniques  then  used  for  correlative  LM  and  TEM  in 
cryosections  and  the  advantages  of  viewing  thin  sections  by  LM 
(Robinson  et  al.,  2001). 

Any  selective  staining  protocol  must  face  the  problem  of  how 
to  see  structure  not  stained  with  the  selective  agent.  One  solution 
is  to  use  an  energy-filtering  TEM  (EFTEM)  to  maximize  the  con¬ 
trast  between  unstained  protein  and  embedding  resin.  Ren  and  co¬ 
workers  used  this  technique  to  image  Quetol-embedded  cells  in 
which  promyelocytic  leukemia  (PML)-bodies  had  been  antibody- 
labeled  with  Cy-3.  They  found  cyanine  and  Alexa  dyes  to  be  stable 
in  this  resin  (Ren  et  al.,  2003)  and  also  used  quantum  dots  for  cor¬ 
relative  fluorescence  and  EFTEM  (Nisman  et  al.,  2004). 

Green  Fluorescent  Protein  Methods 

The  introduction  of  the  genetically  based  marker,  green  fluores¬ 
cent  protein  (GFP),  has  revolutionized  the  study  of  cell  biology 
(see  Chapter  16,  this  volume).  In  2003,  Luby-Phelps  showed 
that  GFP  fluorescence,  which  was  destroyed  by  most  TEM  embed¬ 
ding  techniques,  survived  embedding  in  LR  White  (Luby-Phelps 
et  al.,  2003).  Adjacent  thin  and  1  pm  sections  were  cut  from  prepa¬ 
rations  of  zebrafish  eyes  and  viewed  using  the  TEM  and  LM, 
respectively. 

Recently,  we  have  extended  this  approach  to  thin  sections. 
Specifically,  we  were  interested  in  the  relationship  between  two 
junction  proteins,  HMP-1  [ Caenorhabditis  elegans  (Ce)  alpha- 
catenin]  and  DLG-1  ( Cediscs  large).  Confocal  fluorescence  LM 
had  shown  that  DLG  localizes  near  HMP-1.  To  find  out  if  these 
molecules  colocalized  at  the  EM  level,  we  imaged  HMP-1  ::GFP 
in  a  thin  section  by  LSCM  and  immuno-gold-labeled  DLG- 1  in  the 
same  section.  EM  localization  of  DLG-1  gold  particles  with  HMP- 
1::GFP  expression  is  consistent  with  LM  data  and  confirms  the 
gold  labeling  parameters  needed  to  label  DLG.  Thin  sections, 
adhered  to  EM  finder  grids  and  floating  on  coverslips,  were  first 
imaged  in  an  LSCM  using  both  fluorescence  and  backscattered 
light  (BSL),  to  reveal  the  section  surface  [Fig.  49.17(A)],  and  flu¬ 
orescence,  to  show  HMP-1  ::GFP  [Fig.  49.17(B)],  The  combina¬ 
tion  of  BSL  (displayed  in  red)  and  GFP,  in  green  [Fig.  49.17(C)], 
shows  the  location  of  GFP  within  each  embryo  and  is  useful  for 
locating  a  specific  embryo  in  the  TEM  [Fig.  49.17(D)].  At  higher 
magnification,  the  boxed  area  shows  the  20  nm  gold  labeling  due 
to  DLG-1  [Fig.  49.17(F)], 

We  have  also  imaged  AJM-1::GFP  (a  novel  Ce  junction  mol¬ 
ecule)  in  the  pharynx  with  20nm-gold  anti-GFP  in  LR-Gold  thin 
sections  after  chemical  fixation  [Fig.  49. 18(A)].  Although  colloidal 
gold  particles  have  been  shown  to  quench  the  fluorescence  of  mol¬ 
ecules  in  close  proximity  to  them  (Kandela  et  al,  2004),  we  have 
found  that  rhodamine-linked  secondary  antibodies,  not  actually 
conjugated  to  the  gold-label  [Fig.  49.18(B)]  still  fluoresce  [Fig. 
49.18(C)]  (Sims  and  Hardin,  2004).  Like  the  ReAsH  system  men¬ 
tioned  above,  this  procedure  also  allows  one  to  visualize  a  genetic 
marker  in  a  thin  section,  using  both  fluorescence  LM  and  a  spe¬ 
cific  stain  visible  in  TEM. 

Using  Phalloidin  as  a  Correlative  Marker 

As  GFP  fluorescence  had  been  shown  to  survive  dehydration 
and  embedding  in  methacrylate  resin,  we  were  curious  if  rho- 
damine-phalloidin  might  also  survive  this  process.  C.  elegans 
embryos  that  had  been  bleached  to  remove  the  eggshell,  were  fixed 
and  incubated  in  buffer  containing  saponin  and  rhodamine- 
phalloidin.  After  low- temperature  embedding  in  LR-Gold,  lOOnm 
sections  were  cut  and  imaged  by  confocal  laser  scanning 
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FIGURE  49.1 7.  (A)  BSL  image  of  a  thin  section  on  an  EM  finder  grid.  (B)  HMP-1::GFP  from  the  same  thin  section.  (C)  A  red-green  merge  of  BSL  and  GFP 
images  which  is  useful  to  orient  fluorescence  to  specific  embryos.  (D)  TEM  of  the  same  area  imaged  in  (A-C)  with  the  same  embryo  highlighted  by  the  boxed 
area.  (E)  Higher  magnification  TEM  montage  of  the  comma-staged  embryo  also  observed  in  all  previous  figures.  (F)  Boxed  area  in  (E)  with  20  nm  gold  label¬ 
ing  DLG-1,  a  junctional  protein  that  colocalizes  with  AJM-1. 


FIGURE  49.18.  (A)  Postembedding  AJM-1::GFP  observed  at  epithelial 
junctions  in  thin  sections  (lOOnm)  of  fixed  C.  elegans  larvae.  A  brief  fixa¬ 
tion  followed  by  freezing  permits  infiltration  and  embedding  of  larva  in  LR- 
Gold  (methacrylate)  embedding  resin.  Both  GFP  fluorescence  and 
antigenicity  endure  the  rigors  of  partial  dehydration  and  low  temp  embed¬ 
ding.  GFP  fluorescence  can  still  be  detected  in  thin  sections  cut  from  this 
original  block  over  1  year  after  embedment.  (B)  Rhodamine  goat  anti-rabbit 
(secondary  antibody)  bound  to  a  rabbit  anti-GFP  antibody.  If  one  couples 
both  Au  and  a  fluorophore  to  the  same  antibody,  the  Au  quenches  the  fluo¬ 
rescence.  (C)  20  nm  gold  particles  conjugated  to  rhodamine  goat  anti-rabbit 
antibody  label  AJM-1  ::GFP  at  junctions.  Mixed  conjugates  can  be  useful  to 
confirm  specific  labeling  by  LM,  but  results  in  reduced  gold  particle  label¬ 
ing  seen  by  electron  microscopy.  Bar  =  1  pm. 
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FIGURE  49.19.  TEM  cross-section  of  a  C.  elegans  larva,  en  Moc-stained  with  rhodamine-phalloidin  and  viewed  simultaneously  in  backscattered  (A)  and  flu¬ 
orescent  (B)  light  on  a  confocal  microscope.  (C)  Overlay  of  (A)  and  (B).  (D)  A  low-magnification  TEM  (bar  =  2pm).  The  two  boxed  areas  are  seen  at  higher 
magnification  in  (E),  a  close-up  of  the  intestine  (bar  =  1  pm)  and  (F)  a  hypodermal  adherens  junction  (bar  =  0.5  pm).  Phalloidin  staining  is  seen  decorating  the 
intestinal  microvilli  and  the  four  muscle  quadrants  on  the  sides  of  the  worm. 


microscopy  (CLSM).  ABSL  image  [Fig.  49.19(A)]  and  a  fluores¬ 
cent  image  [Fig.  49.19(B)]  were  merged  together  [Fig.  49.19(C)] 
showing  actin-rich,  phalloidin-stained  muscle  quadrants  and 
intestinal  microvilli  in  red.  The  contrast  from  the  rhodamine  signal 
was  increased  and  overlayed  on  a  low-magnification  TEM  image 
[Fig.  49.19(D)].  The  boxed  regions  in  Figure  49.19(D)  are  shown 
at  higher  magnification  in  Figure  49.19(E),  covering  the  intestinal 
microvilli,  and  Figure  49.19(F),  showing  an  adherens  junction. 
Overall,  this  study  demonstrates  that,  at  least  for  proteins  as  con¬ 
centrated  as  actin  in  muscle  and  microvilli,  one  can  assess  which 
structures  are  labeled  by  analyzing  only  the  fluorescent  LM 
images. 

Cryo-Immobilization  Followed  by 
Post-Embedding  Confocal  Laser  Scanning 
Microscopy  on  Thin  Sections 

Chemical  fixation  is  known  to  be  slow  and  to  introduce  a  wide 
variety  of  artifacts.  Unfortunately,  the  only  alternative  is  cryo- 
preservation,  a  technique  that  depends  for  its  success  on  freezing 
the  important  part  of  the  specimen  without  forming  detectable  ice- 
crystal  artifacts.  At  atmospheric  pressure,  ice  crystals  can  only  be 
avoided  by  using  cryoprotectants,  such  as  sucrose  and  polyethyl¬ 


ene  glycol,  or  by  freezing  the  tissue  very  fast  indeed  (>105K/s; 
Studer  et  al.,  1989;  McDonald  et  al.,  1993).  As  using  a  cryopro- 
tectant  requires  at  least  some  prefixation,  it  is  not  a  great  improve¬ 
ment  on  other  types  of  chemical  fixation.  However,  fast  freezing 
is  also  complex;  largely  because  of  the  high  heat-of-fusion  of  water 
and  the  low  thermal  conductivity  of  ice,  even  cryogens  as  efficient 
as  liquid-He-cooled  copper  are  unable  to  extract  the  heat  from  the 
specimen  fast  enough  to  prevent  ice-crystal  artifacts  from  more 
than  the  outer  10  to  15  pm  of  the  specimen. 

Objects  the  size  of  C.  elegans  (60  pm  diameter)  can  only  be 
successfully  frozen  using  a  high-pressure  freezer  (Studer  et  al. 
1989;  McDonald  et  al.,  1993).  In  this  device,  pressurized  LN2  is 
used  to  cool  a  specimen  about  2  mm  in  diameter  and  200  pm  thick 
from  ambient  temperature  to  77  K  in  about  20  ms,  under  a  transient 
pressure  spike  of  about  2000  bar.  Under  these  conditions,  water 
freezes  at  a  lower  temperature  (~253  K  rather  than  273  K),  and  ice 
crystals  propagate  more  slowly  for  various  reasons,  including  the 
fact  that  the  viscosity  of  the  water  increases  with  ambient  pressure. 
The  result  is  that  a  relatively  high  fraction  of  such  specimens  are 
frozen  without  noticeable  artifact. 

For  TEM  observation,  freezing  is  followed  by  either  freeze 
fracture/metal  shadowing/carbon  replication,  or  freeze  substitution 
with  ethanol  or  acetone  followed  by  “normal”  epoxy  embed¬ 
ment  in  Lowicryl  HM-20.  With  the  latter  approach,  the  plastic- 
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FIGURE  49.20.  (A)  TEM  longitudinal  section  of  an  adult  C.  elegans.  (B)  The  same  section  viewed  by  fluorescent  confocal  microscopy  showing  that  the 
fluorescence  of  AJM-1::GFP  has  not  been  eliminated  by  embedding  in  LR-Gold.  Boxed  areas  in  (A)  and  (B)  are  shown  at  higher  magnification  in  (C)  and  (D), 
where  the  fluorescence  image  has  been  overlaid  in  green  over  the  TEM  image.  As  the  section  was  also  labeled  with  anti-GFP-gold,  these  insets  show  good  agree¬ 
ment  between  the  distribution  of  these  two  markers  in  pharynx  (C)  and  around  the  gut  (D).  Bar  =  (A,  B)  10pm;  (C)  0.5pm;  (D)  1  pm. 


embedded  specimen  can  be  viewed  directly  using  CLSM,  and 
this  image  can  be  used  to  choose  the  best  area  of  the  block  to  be 
sectioned  for  TEM  (Biel  et  al.,  2003). 

Cryopreparation  of  C.  elegans 

Because  the  eggshell  and  the  vitelline  membrane  reduce  pene¬ 
tration  by  chemical  fixatives,  C.  elegans  embryos  are  difficult  to  fix 
chemically.  As  a  result,  we  have  adapted  the  freeze-substitution 
protocol  to  prevent  it  from  damaging  frozen  GFP  specimens. 
Although  Ward  observed  that  GFP  fluorescence  was  completely 
extinguished  if  specimens  were  placed  in  absolute  ethanol  (Ward, 
1998),  Walther  and  Ziegler  (2002)  found  that  adding  1%  to  5% 
water  to  the  freeze-substitution  medium  increased  the  visibility  of 
membranes,  and  we  reasoned  that  a  similar  approach  might  pre¬ 
serve  GFP  structure.  We  found  that  freeze  substitution  in  95% 
ethanol/5  %  water,  followed  by  low-temperature  embedding  in  LR 
Gold,  preserves  AJM-1::GFP  in  thin  sections  (Fig.  49.20).  Figure 
49.20(B)  shows  a  thin  section  of  a  specimen  containing 
AJM-1::GFP  and  also  stained  with  Au-conjugated  anti-GFP, 


imaged  with  a  disk-scanning  confocal  microscope.  In  Figure 
49.20(A),  an  image  of  GFP  fluorescence  is  overlaid  on  a  low- 
magnification  TEM  image.  Higher  magnification  TEMs  of  the  two 
areas  highlighted  in  Figure  49.20(B)  are  shown  in  Figure 
49.20(C,D).  The  green  overlays  roughly  colocalize  with  the  20  nm 
gold  label. 

This  technique  allows  one  to  combine  the  ability  of  fluores¬ 
cence  LM  to  search  large  areas  of  the  section  rapidly  to  identify 
rare  stained  structures  and  then  find  and  view  these  rare  structures 
in  the  TEM.  The  result  confirms  that  this  HPF  protocol  preserves 
GFP  fluorescence  and  specimen  immunoreactivity  and  also  allows 
one  to  correlate  AJM-1::GFP  with  specific  cellular  structures 
visible  only  at  the  ultrastructural  level. 

Another  application  of  GFP  and  cryopreservation  involves 
identification  of  wild-type  and  mutant  embryos  prior  to  cryo¬ 
preservation  and  processing  with  immunogold.  Although  C. 
elegans  embryos  containing  mutant  copies  of  the  ajm- 1  gene  arrest 
soon  after  the  2-fold  stage,  an  average  60%  of  them  can  be  rescued 
to  viability  if  the  hermaphrodite  (parent)  is  microinjected  with  an 
extrachromosomal  array  that  includes  a  functional  copy  of  the 
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FIGURE  49.21 .  (A)  A  transmitted  light  image  of  living  embryos  embedded  in  agar.  (B)  A  brightest-pixel  projection  (ImageJ)  of  AJM-1::GFP  expression  in  the 
same  embryos  in  (A).  Transmitted  light  and  fluorescence  images  were  obtained  simultaneously  on  a  Bio-Rad  1024  confocal  using  488  nm  excitation.  Embryos 
older  than  comma  stage  which  do  not  express  GFP,  lack  the  ajm- 1  gene.  (C)  Overlay  of  the  fluorescent  projection  over  the  transmitted  light  image.  Red-shaded 
area  is  a  2-fold  embryo  which  lacks  ajm- 1.  (D)  A  TEM  image  of  the  same  embryos  after  F1PF  and  embedding  in  Epon.  Same  area  red-highlighted  in  (C)  and 
(D)  is  shown  in  (E),  where  arrows  point  to  epithelial  cell  membrane  separations  associated  with  the  loss  of  ajm- 1. 


ajm-l  gene,  in  this  case  ajm- 1  fused  to  the  GFP-coding  region.  The 
GFP  label  allows  the  observer  to  determine  which  of  the  living 
embryos  carry  the  ajm- 1  gene  and  which  do  not.  Although  one 
could  make  a  similar  discrimination  by  applying  Au-conjugated, 
anti-GFP  antibodies  to  the  thin  sections,  optimal  fixation  and 
staining  could  be  compromised  by  the  need  to  preserve  the  anti¬ 
genicity  of  the  GFP.  Correlating  the  LM  fluorescence  image  of  a 
living  embryo  with  the  TEM  image  of  a  section  from  this  embryo 
avoids  this  problem,  particularly  if  rescued  and  non-rescued 
embyos  are  next  to  one  another  on  the  same  TEM  grid.  Any 
observable  differences  are  more  likely  to  be  real  when  one 
knows  that  the  high-pressure  freezing,  fixation,  and  staining  are 
identical. 

Figure  49.21(A)  shows  a  transmitted  light  image  of  a  group  of 
10  embryos,  6  of  which  have  been  rescued  (Koppen  et  al.,  2001; 
Simske  et  al.,  2003).  Although  the  non-rescued  embryos  look 
normal  in  transmitted  light,  they  are  clearly  marked  by  the  absence 
of  GFP  fluorescence  [Fig.  49.21(B,C)].  Following  high-pressure 
freezing,  freeze  substitution,  embedding  in  Epon,  and  sectioning, 


the  same  group  of  embryos  can  be  visualized  at  low  magnification 
by  TEM  [Fig.  49.21(D)].  The  small  red-shaded  boxes  in  Figure 
49.21(C,D)  indicate  the  area  shown  at  higher  magnification  in 
Figure  49.21(E).  Arrows  point  to  separations  at  epithelial  cell  junc¬ 
tions  believed  to  be  associated  with  the  loss  of  ajm-l. 

Tiled  Montage  Transmission  Electron 
Microscope  Images  Aid  Correlation 

Many  modern  TEMs  incorporate  both  electronic  image  recording 
and  motor-driven  stage  motion.  This  combination  of  features 
greatly  facilitates  LM/TEM  correlational  studies  by  making  it 
much  easier  to  obtain  high-resolution  images  over  a  wide  area  of 
the  specimen.  Figure  49.22  shows  such  a  tiled  montage  of  a  spec¬ 
imen  prepared  in  the  same  way  as  that  shown  in  Figure  49.21.  This 
image  covers  an  area  30  X  50  pm  in  size  and  consists  of  36  images 
recorded  at  an  original  magnification  of  3.5x.  The  green  overlay 
represents  AJM-1::GFP  fluorescence  recorded  on  a  Bio-Rad  1024 
confocal  microscope. 
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FIGURE  49.22.  A  tiled  montage  TEM  image  of  a  specimen  prepared  in  the  same  way  as  that  shown  in  Figure  49.11.  The  original  montage  covered  an  area  of 
the  specimen  50  x  32  pm  in  size  and  consisted  of  36  images  recorded  at  80  kV  using  an  original  magnification  of  3400x  on  a  Phillips  120  using  a  Soft  Imaging 
Systems,  Keenview,  CCD  camera  coupled  to  the  phosphor  by  a  3.4x  fiber-optic  taper.  The  green  overlay  represents  AJM-1::GFP  fluorescence  recorded  on  a  Bio- 
Rad  1024  confocal  microscope. 
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CONCLUSION 

Although  during  the  1950s  and  1960s  light  microscopy  languished 
beneath  the  high-resolution  shadow  of  the  electron  microscope, 
starting  in  the  early  1970s,  it  flowered  as  a  variety  of  new  techni¬ 
cal  improvements  were  introduced:  video-enhanced  DIC,  video- 
intensified  fluorescence,  improved  CCD  cameras,  and  myriad  new 
fluorescent  probes.  Suddenly,  the  fact  that  one  could  observe  func¬ 
tion  in  wholly  new  ways  seemed  to  more  than  offset  the  ability  to 
image  the  internal  features  of  intracellular  organelles.  These  LM 
developments  were  soon  followed  in  the  1980s  and  1990s  by  the 
widespread  use  of  a  variety  of  confocal  microscopes,  and  more 
recently  by  the  advent  of  GFP  and  its  many  relations.  This  entire 
book  is  a  testament  to  the  fact  that  live-cell  fluorescent  light 
microscopy  is  blossoming  as  never  before. 

On  the  other  hand,  we  lose  nothing  by  admitting  that  much  of 
the  business  of  cells  occurs  at  a  size  scale  that  is  much  smaller  than 
that  which  can  be  imaged  with  the  light  microscope.  When  this 
occurs,  it  is  useful  to  remember  that  help  is  at  hand;  given  the  effort 
needed  to  boost  LM  spatial  resolution  by  a  factor  of  2,  it  is  salu¬ 
tary  to  acknowledge  how  much  more  clearly  one  can  see  when  it 
is  increased  by  a  factor  of  40. 
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INTRODUCTION 

Modern  computerized  and  automated  microscope  technology 
enables  researchers,  technicians,  and  quality-assurance  personnel 
to  collect  large  numbers  of  high-quality  images.  However,  despite 
the  addition  of  digital  storage  (see  Chapter  32,  this  volume )  and 
networking,  microscopes  are  still  generally  used  in  a  stand-alone 
manner  insofar  as  the  output  is  simply  a  huge  number  of  image 
files.  There  is  little  attempt  at  proper  cataloguing,  and  little  meta¬ 
data  beyond  the  filename  to  identify  and  describe  the  contents  of 
each  image  (Gonzalez-Couto  et  al.,  2001). 

This  is  not  only  a  problem  for  the  individual  scientist  who 
wants  to  document  his  or  her  work  and  master  the  flood  of 
data  that  accumulates  over  the  years,  or  for  the  manager  of  a 
microscopy  facility,  who  has  to  maintain  the  huge  amounts  of  data 
generated  by  different  users  on  different  instruments.  It  also  con¬ 
cerns  the  scientific  community  on  a  global  scale:  in  contrast  to 
textual  documents  such  as  journal  articles,  microscopy  data  is  dif¬ 
ficult  to  discover  (usually  based  only  on  keyword  searches  in  lit¬ 
erature  databases,  rather  than  on  image-specific  information).  In 
addition,  getting  copies  of  the  data  usually  relies  on  a  personal  cor¬ 
respondence  with  the  author,  in  which  he/she  is  asked  to  send 
the  data  and  any  relevant  information  regarding  the  experiment 
(Anonymous,  2004). 

The  first  steps  towards  a  better  archive  infrastructure  are  the 
networking  of  instruments  so  that  data  can  be  easily  transferred 
from  one  point  to  the  other,  and  the  introduction  of  databases  both 
at  the  individual  microscope  level  and  at  the  large-scale  level  to 
hold  the  metadata  that  allows  others  to  manage  and  use  the  data. 
The  latter  two  developments  go  hand  in  hand:  Having  local  solu¬ 
tions  does  not  solve  the  global  problem  of  data  access,  and  having 
a  universal  database  for  microscopy  data  requires  local  solutions 
to  develop  an  efficient  data  submission  procedure. 

This  chapter  outlines  the  developments  made  in  both  areas.  It 
discusses  the  benefits  of  integrating  a  database  into  a  microscope 
to  store  information  about  both  the  instrument  and  the  image  data 
generated  by  it,  and  it  presents  preliminary  efforts  made  to  develop 
a  database  infrastructure  at  the  global  level. 

DATA  AND  METADATA  MANAGEMENT 
IN  MICROSCOPES 

In  early  confocal  microscope  systems,  information  about  the 
machine  configuration  was  contained  within  a  single  text  file  that 
could  be  read  and  written  at  any  time  during  data  acquisition  or 


processing.  Such  a  flat  file  format  avoided  the  need  for  recompi¬ 
lation  of  the  entire  microscope  program,  but  as  its  length  increased, 
it  rapidly  became  unsuitable  for  human  viewing. 

Online  configuration  of  a  number  of  system  parameters  was 
possible,  often  by  running  a  setup  file  containing  a  subset  of  the 
information  required  to  obtain  the  image.  However,  the  informa¬ 
tion  was  intended  as  an  aid  to  simply  getting  an  image,  rather  than 
adjusting  the  large  number  of  machine  variables.  Indeed,  storage 
of  some  component  settings  was  impossible  or  useless  because 
these  were  machine  settings  that  did  not  provide  information  about 
what  was  actually  happening  to  the  specimen.  Moreover,  if  a  hard¬ 
ware  component  were  replaced,  then  apparently  identical  machine 
settings  could  result  in  different  image  data,  for  example,  because 
filter  selection  was  implemented  by  reference  to  filter  wheel  posi¬ 
tions,  rather  than  to  actual  filters. 

Image  intensity  data  was  saved  as  a  stream  of  bytes,  and  infor¬ 
mation  about  a  particular  image  was  mostly  limited  to  saving  pixel 
spacing  information  in  an  adjacent  text  file.  Thus,  reading  such 
data  into  image  processing  programs  often  required  some  pro¬ 
gramming  effort  and,  in  most  cases,  manual  intervention.  Cur¬ 
rently,  most  light  microscopes  generate  a  small  metadata  set  that 
helps  retrieve  the  image  data  and  refers  mainly  to  storage  related 
issues  (i.e.,  format,  number  of  lines/pixels,  date,  etc.).  Tradition¬ 
ally,  developers  have  not  offered  the  possibility  of  storing  much 
extra  information  because  they  could  see  only  limited  opportuni¬ 
ties  for  its  later  use. 

Standard  file  systems  permitted  restricted  searches  of  the  text 
files,  often  using  laboriously  constructed  string  searches  where  the 
results  were  returned  in  a  similarly  user-unfriendly  manner. 

Recent  Developments 

More  recently,  it  has  become  standard  practice  to  store  image  data 
in  more  sophisticated,  generic  file  formats  that  may  also  hold  a 
number  of  parameters,  for  example,  in  an  expanded  TIFF  format. 
This  offers  the  advantage  of  being  automatically  readable  by  a 
large  number  of  image-processing  programs,  where  simple  mea¬ 
surements  can  be  performed. 

TIFF  is  actually  an  excellent  example  because  one  can  add  as 
many  tags  as  one  wants  and  could  add  a  lot  of  metadata.  Whatever 
is  not  recognized  by  the  program  reading  the  file  is  ignored.  The 
increasing  number  of  proprietary  raw  image  formats  created 
by  camera  manufacturers  has  recently  prompted  Adobe 
(http://www.adobe.com)  to  release  a  unified  TIFF-based  file 
format  (DNG,  Digital  Negative)  that  supports  a  wide  range  of 
metadata  declared  by  manufacturers. 
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Recent  initiatives  to  remedy  these  problems  led  to  the  devel¬ 
opment  of  MPEG-7  (Pereira  and  Koenen,  2001)  as  the  new  stan¬ 
dard  for  content  description  of  multi-media  files.  MPEG-7 
provides  descriptive  elements  ranging  from  basic  features,  such  as 
colors,  textures,  and  shapes,  to  complex  information  about  motion. 
However,  its  complexity,  and  its  lack  of  clarity  for  those 
illiterate  in  metadata  management,  are  an  impediment  to  its 
implementation. 

Although  standard  database  technology  has  been  available  for 
many  years,  it  is  only  recently  that  commercial  relational  database 
management  systems  (RDBMS)  have  ventured  from  mainframe 
and  workstation  systems  to  systems  suitable  for  use  on  the  desktop. 
Improvements  in  the  quality  of  such  systems  and  in  the  range  of 
interfaces  now  available  for  development  environments  now  offer 
microscopists  the  chance  to  store  data  in  personalized  databases. 
Moreover,  there  are  now  more  data  processing  programs,  such 
AVS  Express  (http://www.avs.com),  available  to  access  such  data¬ 
base  stores. 

Even  the  ubiquitous  MS-Office  package  can  access  databases 
from  within  Word,  Access,  and  Excel.  Nevertheless,  currently 
available  systems  are  far  from  providing  seamless  integration  of 
image  data  acquisition  and  analysis  (Andrews  et  al.,  2002). 

Most  recently,  microscopes  with  integrated  RDBMS  have  been 
developed  (Salmon  et  al.,  1999)  and  some  of  these  are  incorpo¬ 
rated  into  systems  such  as  the  LSM510  (Carl  Zeiss  Jena),  and  the 
compact  confocal  camera  (CCC),  the  microscope  used  for  research 
at  the  European  Molecular  Biology  Laboratory  (EMBL).  These 
machines  use  an  RDBMS  to  describe  and  store  their  hardware  con¬ 
figurations  to  an  unprecedented  level  of  accuracy  and  complete¬ 
ness.  Parts  can  be  changed  without  affecting  performance,  and  the 
machine  is  automatically  configured,  which  requires  tagging  the 
components  such  that  they  can  be  recognized  by  the  software. 
Image  data  and  associated  metadata  are  all  managed  via  databases, 
and  common  software  such  as  Word,  Access,  and  Excel  can  be 
used  to  access  the  databases. 


IMAGE  INFORMATION  MANAGEMENT 

Image  information  has  to  be  managed  both  at  the  local  level,  at  the 
scientist’s  desk,  and  at  the  global  level,  for  example,  at  scientific 
journals.  With  the  advent  of  digital  microscopy,  many  scientists 
have  replaced  their  folders  of  glossy  prints  or  slides  by  shelves 
with  hundreds  of  CDs/DVDs  and  drawers  full  of  tapes.  Some  may 
have  adopted  desktop-based  image  management  systems,  either 
using  software  coming  with  their  instrument  (e.g.,  Zeiss’  LSM 
Image  Browser  or  Leica’s  IM1000)  or  commercial  packages  which 
are  becoming  more  widespread  due  to  the  increased  usage  of 
digital  photography  (see  Appendix).  Alternatively,  they  may  use 
an  image  archiving  system  tailored  to  microscopy  such  as  DDL’s 
Aequitas  or  RVC’s  Research  Assistant  (see  Appendix). 

At  the  other  end,  journals  have  started  to  make  the  image  data 
submitted  by  authors  available  electronically  (Anonymous,  2004). 
Initially  this  was  done  as  an  annual  CD  containing  supplementary 
data,  now  as  files  for  download  from  the  publishers’  Web  sites 
(e.g.,  in  Science’s  Supporting  Online  Material  or  Nature’s  Supple¬ 
mentary  Information).  This  also  enabled  publishers  to  provide  data 
that  could  not  be  published  in  the  printed  journals  due  to  its  dimen¬ 
sionality  or  its  huge  volume.  However,  almost  none  of  this  image 
data  is  managed  in  a  database,  and  —  at  least  for  the  external  user 
of  the  Web  site  —  there  is  no  way  to  bypass  the  individual  articles 
and  any  relevant  keywords  appearing  in  the  text  body  and  directly 
access  all  images  on  one  publisher’s  site  relating  to  a  particular 


topic,  such  as  fluorescence  microscopy  of  GFP-labeled  proteins  in 
the  Golgi  apparatus.  It  is  also  not  possible  to  search  for  a  particu¬ 
lar  topic  across  the  Web  sites  of  different  publishers  (in  case  of  the 
two  leading  journals  mentioned  above,  a  link  to  the  image  section 
is  shown  only  together  with  the  individual  article). 

Increased  usage  of  laboratory  information  management 
systems  (LIMS)  (Nakagawa,  1994)  that  typically  use  a  database 
format  to  store  information  demonstrates  how  the  continual 
improvement  of  data  collection  and  processing  operations  is  also 
vital  for  running  many  types  of  laboratory  procedures,  such  as 
screening  assays.  As  legal  demands  increase,  LIMS  may  become 
mandatory  for  demonstrating  conformance  to  stringent  quality 
requirements.  Potential  benefits  for  LIMS  users  include  smoother 
work  flow,  better  access  to  information,  improved  quality  of 
results,  better  control  of  costs,  and  increased  processing  speed. 
And,  last  but  not  least,  these  systems  replace  the  traditional  labo¬ 
ratory  books  in  which  scientists  keep  their  handwritten  notes. 

THE  AIMS  OF  MODERN  MICROSCOPE 
SYSTEM  DESIGN 

We  have  pointed  out  two  important  aspects  to  microscope  design: 
On  one  side  of  the  system  there  should  be  a  networked  microscope 
that  functions  within  database  structures.  On  the  other  side  there 
should  be  an  image  database  that  provides  easy  querying  and 
access  to  both  data  and  metadata. 

That  the  database  system  plays  a  key  role  as  the  mediating 
component  between  data  acquisition,  image  processing,  and  data 
analysis  software  has  been  recognized  by  the  Open  Microscopy 
Environment  (OME,  http://www.openmicroscopy.org),  a  project 
that  seeks  to  improve  networking  of  biological  microscopy  data  and 
to  transfer  the  data  from  the  instrument  to  image  processing  and 
analysis  software  for  quantitative  analysis  (Andrews  et  al.,  2002; 
Swedlow  et  al.,  2003).  There  is  actually  an  additional  element  in 
this  data  flow:  the  software  must  maintain  contact  between  the 
acquisition  parameter  and  the  processed  result  so  that  the  chain 
from  data  acquisition  to  the  global  database  remains  unbroken. 

The  complete  infrastructure  envisaged  for  the  management  of 
information  is  shown  in  Figure  50.1.  With  computer-controlled 
microscopes,  information  about  the  operational  state  of  the  instru¬ 
ment  can  be  stored  automatically.  A  single  program  should  control 
a  family  of  instruments  and  organize  the  interaction  between  them 
and  the  user.  The  details  of  this  interaction  are  retrieved  from  a 
database  that  describes  the  hardware  as  completely  as  possible, 
storing  at  least  all  variable  parameters  and  periodically  accessing 
results  to  see  if  benchmarks  of  resolution  and  stability  are  met. 
Changes  in  the  microscope  hardware  are  reflected  in  changes  to 
database  entries  and  affect  the  user  interface.  To  this  end,  the  com¬ 
ponents  of  the  microscope  should  be  labeled  such  that  the  software 
can  recognize  them  automatically,  as  it  is  now  done  in  Zeiss 
Axiolmager. 

Further  information  concerning  the  specimen  and  the  experi¬ 
ment  being  performed  can  be  gathered  from  the  user  in  a  non-intru- 
sive  manner  by  fully  integrating  the  data  collection  process  with 
the  microscope  control  program.  This  information  is  saved  in  a 
database,  together  with  a  pointer  to  the  image  data  such  that  the 
user  can  always  be  offered  seamless  navigation  from  the  acquisi¬ 
tion  parameters  to  the  image  data. 

Figure  50.1  doesn’t  make  any  statement  regarding  the  nature 
of  the  archive.  It  may  be  a  central  file  server  or  a  network  of 
distributed  servers.  As  the  major  concern  with  modern  digital 
microscopy  is  the  size  of  the  data  and  the  required  bandwidth,  a 
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FIGURE  50.1 .  The  life  cycle  of  microscopy  data  and  metadata.  Data  from  one  or  more  microscopes  is  stored  on  a  file  server  (1).  Technical  information  is  auto¬ 
matically  written  into  a  database  (2).  The  user  adds  details  on  the  biology  of  the  experiment  (3).  Metadata  and  data  are  reused  by  image  processing  software, 
resulting  data  is  saved  on  the  file  server,  and  the  metadata  database  is  updated  (4).  Metadata  is  transferred  to  a  global  database,  such  as  Bioimage  (5),  while  the 
image  data  stays  on  the  author’s  server  or  is  archived  somewhere  else  (6).  Third  parties  access  the  metadata  in  the  Bioimage  database,  which  directs  them  to 
the  data.  The  Bioimage  user  can  use  tools  that  access  the  metadata  to  get  and  process  the  data  (7).  Note  that  the  microscope  can  be  considered  as  a  tool  in  the 
sense  of  the  diagram:  using  the  metadata  saved  with  the  original  data,  it  can  reconfigure  itself  and  repeat  the  experiment.  The  black  arrows  symbolize  pointers 
to  the  data.  The  left-hand  side  is  a  local  computer  system,  while  the  right-hand  side  is  a  global  system.  The  three-components  concept  for  databases  of  biolog¬ 
ically  relevant  data  (BioAgency,  BioArchive,  and  BioTools)  was  introduced  in  Carazo  and  Stelzer  (1999). 


distributed  storage  system  will  be  preferable  for  a  large  project 
with  global  focus. 

Use  of  an  integrated  microscope  and  database  system  can 
provide  the  following  benefits  (Salmon  et  «/.,  1999): 

•  Fast,  simple,  and  automatic  machine  configuration  can  be 

achieved  by  associating  a  complete  set  of  configuration  param¬ 
eters  with  each  dataset.  The  microscope  can  then  be  adjusted 
and  optimized  by  repeatedly  changing  the  parameters,  rescan¬ 
ning  the  same  sample  area  into  a  new  scan  window,  and  ana¬ 
lyzing  the  effect  of  these  changes.  The  best  image  can  then  be 
quickly  selected  by  the  user  and  the  associated  parameters 
reused  as  a  start  point  for  all  subsequent  scans.  Moreover,  func¬ 
tions  to  reuse  subsets  of  parameters  can  also  be  created.  Thus, 
preselected  areas  can  be  quickly  scanned  with  the  same  illu¬ 
mination  and  detector  sensitivity  parameters,  or  the  same  area 
can  be  scanned  with  different  detector  sensitivity  parameters. 
Loops  can  also  be  inserted  to  check  that  the  data  flow  has  not 
changed  so  much  as  to  imply  a  hardware  malfunction  requir¬ 
ing  operator  intervention. 

•  Configuration  of  different  machines  to  the  same  state  is 

possible  if  the  microscope  configuration  interface  uses  real- 
world  physical  units  rather  than  arbitrary  units,  for  example, 
the  actual  laser  power  at  some  location  close  to  the  microscope 
objective  rather  than  the  voltage  used  to  set  the  laser  power 
output.  Also,  the  microscope  magnification  should  be  cali¬ 
brated  rather  than  relying  on  the  magnification  value  engraved 
into  the  side  of  the  objective  lens.  Finally,  storing  the  detected 
signal  data  in  units  of  photons  or  photoelectrons  would  permit 
one  to  make  a  close  estimate  of  its  statistical  accuracy.  With 
parameters  properly  handled  as  physical  values,  users  are  no 
longer  tied  to  finishing  an  experiment  within  a  single  session 
or  using  a  particular  machine,  and  collaborating  scientists  can 
compare  images  acquired  under  conditions  that  are  strictly 
comparable. 


•  Improved  analysis  of  results  can  be  achieved  by  storing  more 
information  about  the  system  and  using  more  sophisticated 
models.  Although  this  is  often  assumed,  many  properties  of  a 
light  microscope  are  not  described  by  a  constant  relationship 
(e.g.,  resolution  is  not  constant  over  the  entire  field  of  view  and 
often  changes  with  focus  plane).  Saving  such  information 
allows  for  a  better  calibration,  and  makes  room  for  the  sub¬ 
sequent  development  of  algorithms  to  compensate  for  this  vari¬ 
ance.  For  example,  because  of  aberrations  induced  by  refractive 
index  mismatch  (Hell  et  ai,  1993),  image  signal  intensity  is 
reduced  the  deeper  the  focus  plane  moves  into  the  sample. 
Image  processing  programs  such  as  AVS  can  improve  image 
data  using  information  about  the  immersion  medium,  the  optical 
system,  and  the  position  of  data  samples  relative  to  optical 
boundaries,  such  as  the  coverslip.  Although  the  corrected  image 
data  has  a  more  uniform  contrast,  making  images  easier  to  view 
and  giving  a  considerable  improvement  in  the  results  of  three- 
dimensional  (3D)  rendering  techniques  such  as  ray  tracing,  it 
is  not  reasonable  to  imagine  that  such  programs  can  correct  for 
all  the  vagaries  that  beset  fluorescence  microscopy  (Pawley, 
2000). 

•  Improved  access  to  data  is  possible  when  more  information 
is  stored  in  a  modern  RDBMS.  Such  programs  are  not  only 
optimized  to  perform  fast  queries  on  the  data  they  contain,  but 
typically  also  offer  an  array  of  tools  and  interfaces  that  serve 
both  novice  and  expert  programmers  in  the  task  of  data  pro¬ 
cessing.  RDBMS  also  offer  presentation  interfaces  to  many 
commonly  used  office  programs,  as  well  as  dynamically  gen¬ 
erated  HTML  files.  Users  can  perform  sort,  select,  retrieve,  and 
compare  operations  on  image  archives,  querying  not  only  the 
filenames,  but  also  the  entire  spectrum  of  information  associ¬ 
ated  with  the  image. 

•  Semi-automatic  submissions  to  other  databases  becomes 
possible  when  more  information  is  stored  in  a  database.  In  the 
past,  users  were  satisfied  if  they  could  import  the  image  files 
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into  image-processing  programs.  However,  now  that  more 
details  can  be  stored  with  the  source  microscope  images,  users 
can  make  more  useful,  semi-automatic  submissions  to  data 
warehouses.  For  example,  for  biological  applications,  it  is 
highly  valuable  to  export  data  to  databases  that  also  store 
biology-related  information,  such  as  in  the  Biolnrage  Database 
described  below. 

•  Remote  monitoring  and  maintenance  is  possible  if  the  actual 
state  of  the  instrument  is  stored  and  made  available  via  a  com¬ 
puter  network.  A  system  administrator  can  examine  the  infor¬ 
mation  online,  and  users  can  benefit  from  fast  fault  diagnosis, 
rectification,  or  advice  on  the  microscope  configuration. 

•  Repetition  of  experiments  is  a  key  requirement  in  science  that 
is  much  easier  to  satisfy  if  the  experimental  process  can  be 
automated.  By  saving  data  and  constructing  systems  that  use 
this  information  to  repeat  the  experiment  automatically,  the 
potential  for  error  is  reduced. 

•  Performance  and  development:  A  well-defined  database 
structure  enables  not  only  faster  runtime  performance,  but 
also  simplifies  programming  and  program  structure,  speeding 
development  time  of  the  overall  machine  control  program. 

INSTRUMENT  DATABASE  MODEL 

The  database  must  contain  sufficient  information  to  enable  the 
microscope  control  program  to  facilitate  the  communication 
between  the  user  and  the  instrument.  This  technical  information 
can  be  divided  into  four  types  (Fig.  50.2).  Additional  information 
describing  the  biological  specimen  is  required  to  meet  the  funda¬ 
mentals  of  solid  scientific  work. 

•  The  first  group  of  tables  describing  the  microscope  comprises 
essentially  datasheet-like  descriptions  of  items  that  could 
potentially  be  used  within  the  instrument.  This  information  is 
general,  and  therefore  valid  for  any  machine  within  a  family 
of  instruments  that  is  served  by  the  database.  The  collection  of 
tables  describing  all  possible  components  is  like  a  library  for 


FIGURE  50.2.  The  four  instrument  metadata  levels.  The  most  generic  level 
( 1 )  holds  datasheet-like  information  on  potential  components  that  may  be  used 
in  a  certain  microscope  (e.g.,  three  different  lasers:  a  12  mW  argon-ion  laser 
with  several  wavelengths,  a  534 nm  1-rnW  He-Ne  laser,  and  a  633nm  lOmW 
He-Ne  laser).  The  next  level  (2)  is  for  the  documentation  of  the  parts  actually 
available  in  the  microscope  (only  two  of  the  lasers).  These  parts  are  associated 
with  an  entry  at  the  third  level  (3),  that  holds  calibration  details  (these  will  be 
different  for  each  instance  of  a  part).  In  our  example,  the  actual  maximum  laser 
power  of  the  488  nm  line  is  only  2.5  mW  (25%  of  the  calibrated  value  of 
lOmW,  which  is  the  sum  of  the  power  of  all  laser  lines).  Thus,  a  setting  of  20% 
in  an  experiment  means  that  a  power  of  0.5  mW  was  used,  which  is  documented 
at  the  fourth  level  (4).  While  the  first  three  levels  are  instrument  specific,  the 
fourth  level  is  experiment  specific  and  does  not  need  to  be  stored  in  the  instru¬ 
ment  database. 


machine  construction.  Typically,  entries  provide  information 
on  how  to  control  a  part,  how  to  incorporate  it  into  the  system, 
and  indicate  its  expected  performance. 

•  Each  particular  machine  contains  either  0,  1,  or  many  instances 
of  a  specific  component.  Each  of  these  available  parts  requires 
a  unique  database  entry  through  which  it  can  be  referenced  and 
controlled.  Because  many  parts  require  calibration,  separate 
calibration  details  must  also  be  included.  Available  parts  and 
their  calibration  form  the  second  and  third  groups. 

•  Every  time  a  sample  is  scanned  and  data  is  collected,  parame¬ 
ters  of  some  parts  such  as  the  photomultiplier  tube  (PMT) 
voltage  or  the  laser  power  are  set  to  values  that  may  be  dif¬ 
ferent  from  those  used  for  other  scans.  To  record  these  tem¬ 
porally  changing  values,  a  fourth  group  of  tables  is  required. 
These  can  exist  in  a  separate  database  because  they  do  not 
relate  to  a  particular  machine. 

•  Any  specimen  examined  under  a  microscope  has  been  sub¬ 
jected  to  processing  steps  prior  to  observation.  It  may  also  be 
subject  to  processing  steps,  such  as  the  addition  of  chemicals, 
between  observations.  In  addition,  different  features  or  areas 
of  interest  may  be  imaged.  Finally,  information  obtained  by  the 
microscope  may  be  subject  to  one  of  a  number  of  postpro¬ 
cessing  steps  such  as  deconvolution.  Therefore,  backward 
references  to  other  instruments  that  looked  at  or  manipulated 
the  sample  before,  and  forward  references  to  what  will  or 
might  happen  next,  complete  the  information  network. 

The  design  of  the  model  to  hold  this  metadata  is  a  time- 
consuming  task,  but  it  has  to  be  done  carefully  if  the  model  is  to 
remain  optimal  over  the  long  term.  Recently,  ontologies  have  been 
developed  that  describe  the  knowledge  in  many  scientific  areas  in 
a  systematic  computer-processable  form  (for  a  review,  see  Bard  and 
Rhee,  2004).  Such  a  framework  enables  consistent  indexing  of 
experimental  data  and  allows  systems  in  different  domains  to  com¬ 
municate  with  each  other:  Without  it,  one  cannot  exchange  infor¬ 
mation  without  loss.  The  development  of  ontologies  for  the  different 
expertise  areas  of  microscopy  is  the  major  milestone  that  must  be 
passed  before  the  integration  of  databases  for  different  individual 
instruments  can  work  seamlessly  (Pouchard  and  Walker,  2001). 

SYSTEM  REQUIREMENTS 

The  complete  description  of  a  confocal  microscope  such  as 
EMBL's  CCC  contains  about  ninety  tables.  Fortunately,  compo¬ 
nent  specifications  and  part  descriptions  do  not  change  during  the 
lifetime  of  the  microscope,  and  take  up  only  a  few  megabytes  of 
disk  space.  On  the  other  hand,  the  user’s  metadata  database 
increases  in  size  by  a  few  kilobytes  every  time  an  image  is  saved, 
so  even  with  thousands  of  experiments,  the  amount  of  data  stored 
in  the  tables  will  remain  relatively  small. 

Users  should  be  able  to  back  up  their  archives  on  a  networked 
computer,  and  then  view  them  on  any  computer  they  choose.  Thus, 
the  database  program  should  be  separate  from  the  archive  files, 
easy  to  install,  and  preferably  either  available  on  a  number  of  dif¬ 
ferent  computing  platforms  or  accessible  via  an  HTML  interface 
that  can  be  accessed  with  any  Web  browser. 

IMAGE  DATABASE  MODEL 

The  model  of  any  database  for  biological  microscopy  data  has 
to  accommodate  metadata  from  the  following  five  areas  (Lindek 
et  al.,  1999): 
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•  Organizational/administrative  metadata,  such  as  people,  insti¬ 
tutes,  manufacturers  and  suppliers,  etc. 

•  Data  and  file  metadata,  such  as  number  of  pixels,  image  loca¬ 
tion  pointers,  etc. 

•  Technical/instrumentation  metadata,  such  as  information  about 
the  microscope,  its  components,  and  their  settings  during  data 
acquisition. 

•  Biological  metadata,  such  as  sample  taxonomy  (general  infor¬ 
mation)  and  age  (sample  specific  information). 

•  Experiment  metadata,  such  as  preparation  steps  and  their  phys¬ 
ical,  chemical,  and  biological  parameters. 

Because  the  emphasis  of  these  databases  is  on  the  (biological) 
content  of  the  images,  not  as  many  technical  details  would  be 
available  as  on  an  individual  microscope.  In  addition,  images  from 
a  wide  spectrum  of  microscope  techniques  have  to  be  stored  and 
collated.  Component  parameters  therefore  have  to  be  restricted  to 
those  that  are  characteristic  of  a  large  set  of  instrument  types  and 
that  are  necessary  to  accurately  document  the  image  acquisition 
process  and  to  subsequently  evaluate  the  data.  On  the  other  hand, 
general  databases  may  store  considerably  more  information  about 
the  specimen,  sample  preparation  techniques,  and  the  features 
visible  in  individual  images. 

The  improved  analysis  of  results  and  the  better  access  to  data 
that  are  regarded  as  the  benefits  for  microscope/database  integra¬ 
tion  are  also  advantages  of  global  image  databases.  Furthermore, 
if  they  are  multi-specimen  and  multi-technique,  they  allow  the 
comparison  of  complementary  information  obtained  from  similar 
samples  observed  with  different  techniques,  providing  different 
levels  of  resolution  and  visualizing  different  types  of  interaction. 

To  be  efficient,  image  repositories  at  the  desktop,  in  the  labo¬ 
ratory,  or  at  the  global  level  cannot  stand  in  isolation.  First,  as  dis¬ 
cussed  above,  they  need  to  be  connected  to  the  instruments 
that  produce  the  data;  second,  they  need  to  be  associated  with 
other  databases  (Martone  et  al.,  2002);  and  third,  they  need  to 
be  incorporated  into  a  network  of  scientific  resources.  Such  a 
network  is  being  set  up  for  the  life  sciences  by  E-BioSci 
(http://www.e-biosci.org).  It  aims  to  bring  together  the  various 
databases  used  in  the  laboratories  on  a  daily  basis;  literature, 
sequences,  structures,  images,  etc.  (Grivell,  2001). 


SELECTED  PROJECTS 

While  there  is  no  global  microscopy  database  available  at  the  time 
of  writing  this  chapter,  there  are  some  projects  that  are  more  or 
less  mature  and  have  the  potential  of  playing  that  role  (Table  50. 1 ). 
One  major  example  is  the  Bioimage  Database,  now  managed  by  a 
team  in  Oxford,  United  Kingdom.  Initially  designed  as  a  database 
for  all  kinds  of  biological  microscopy  data,  it  has  recently  been 
expanded  to  include  other  types  of  biological  image  data. 
However,  while  less  ambitious  because  more  restricted  in  scope, 
other  projects  also  deserve  mention  in  this  section. 

Bioimage 

Bioimage  is  a  database  of  multi-dimensional  digital  images  for  the 
life  sciences  that  at  the  time  of  writing  is  being  restructured  but 
will  soon  accept  image  data  from  a  variety  of  instruments  (from 
microscopy  to  satellite  remote  sensing)  relating  to  all  aspects  of 
biology  (from  ultrastructural  biology  to  wildlife  conservation).  In 
its  first  phase  (1996-1999),  six  European  research  groups  and  two 
industrial  partners  collaborated  on  a  publicly  funded  project  inves¬ 
tigating  the  possibility  of  storing,  in  a  single  database,  data  gener¬ 
ated  from  very  different  specimens  (ranging  in  size  from  whole 
biological  organisms  down  to  macromolecules)  using  very  differ¬ 
ent  microscopes  (light,  electron,  and  atomic  force  microscopes, 
etc.).  The  aim  was  to  design  a  database  system  providing  hitherto 
unprecedented  levels  of  comparison  and  data  access  to  emphasize 
different,  and  complementary  structural  aspects  of  similar  objects. 

During  the  transitional  period  that  followed  (2000-2001),  the 
consortium  partners  looked  for  a  new  orientation  of  the  database 
that  might  lead  to  a  sustainable  business  model.  This  led  to  the 
integration  of  Bioimage  into  the  ORIEL  project  (2002-2004): 
Online  Research  Information  Environment  for  the  Life  Sciences 
(http://www.oriel.org),  an  EC-funded  E-BioSci  research  project  to 
integrate  Internet-based  biological  information  resources  for  the 
scientific  community.  Because  E-BioSci  embraces  all  life  sciences 
topics,  this  meant  expanding  the  scope  of  the  Bioimage  Database 
to  cover  non-microscopical  image  data  such  as  wild-life  photog¬ 
raphy,  behavioral  biology,  ecology,  etc.  At  the  same  time,  the 
ability  to  store  detailed  technical  information  was  reduced. 


TABLE  50.1.  Image  Management  Software 


Project 

URL 

Reference 

Bioimage 

www.bioimage.org 

Carazo  and  Stelzer,  1999 

Biomedical  Image  Library 

ted. hul. harvard. edu:8080/ted/deliver/home?_collection=bil 

Scientific  Image  DataBase 

sidb .  sourceforge .  net 

Quartz  Imaging 

www.quartzimaging.com 

ARKive 

www.ARKive.org 

Bristol  BioMed  Image  Archive 

www.brisbio.ac.uk 

Wellcome  Trust  Medical 

medphoto .  Wellcome  .ac.uk 

Photographic  Library 

IWF  Contentport  /  Digiclip 

www.iwf.de 

Getty  Images 

www.gettyimages.com 

Corbis 

www.corbis.com 

Fly  Base 

www.flybase.org 

The  FlyBase  Consortium,  2003 

FlyView/FlyMove 

flyview.uni-muenster.de ;  fly  move .  uni-muenster.  de 

Janning,  1997;  Weigmann  et  al.,  2003 

Mouse  Databases  at  Jackson  Lab 

www.jax.org 

Bult  et  al.,  2004 

Mouse  Atlas 

genex.hgu.mrc.ac.uk 

Burger  et  al.,  2004 

Open  Microscopy  Environment 

www.openmicroscopy.org 

Goldberg  et  al.,  2000 
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One  of  the  most  critical  issues  for  the  Bioimage  Database  (and 
for  all  global  image  databases)  is  data  harvesting.  Because  it  is 
designed  as  a  global  database,  data  entry  requires  significant  cura- 
tion,  even  though  large  parts  can  either  be  automated  or  delegated 
to  submitters.  The  process  is  simplified  if  submitters  are  not  solely 
individual  scientists,  as  initially  foreseen,  but  also  organizations 
entering  lots  of  data  in  well-defined  formats.  Bioimage  therefore 
focuses  on  three  types  of  data  source:  The  first  type  consists  of  cor¬ 
porate  submitters,  such  as  scientific  journals  or  learned  societies, 
that  send  their  image  data  to  the  Bioimage  Database  on  a  regular 
basis.  The  second  type  contains  individuals,  research  groups,  and 
museums  with  large  image  collections  who  wish  to  make  these  col¬ 
lections  widely  available.  The  third  type  contains  other  databases 
which  would  provide  their  metadata  and  links  to  their  own  image 
data. 

Biomedical  Image  Library 

Harvard  University’s  Biomedical  Image  Library  (BIL)  took  a  com¬ 
pletely  different  approach  from  the  top-down  approach  chosen  by 
Bioimage.  BIL  focused  on  local  data  in  a  standard  format  (TIFF) 
and  started  with  a  few  microscopy  techniques  (light,  fluorescent, 
or  confocal  microscopy).  This  is  an  approach  that  a  lot  of  smaller 
projects  are  likely  to  take.  The  big  risk  in  doing  so  is  that  the 
data  model  designed  for  a  restricted  set  of  data  will  not  be  able  to 
cope  effectively  with  the  wide  variety  of  metadata  in  the  outside 
world.  At  the  time  of  writing  this  chapter,  BIL  is  accepting 
data  from  any  Harvard  researcher  and  has  a  publicly  accessible 
Web  interface. 

Scientific  Image  DataBase 

One  of  the  most  recent  initiatives  at  the  interface  between  acade¬ 
mia  and  industry  is  the  Scientific  Image  DataBase  (SIDB),  a 
Web-based  archive  for  microscopy  data  initiated  by  Scientific 
Volume  Imaging  B.V.  (www.svi.nl)  and  developed  at  the  Univer¬ 
sities  of  Utrecht  and  Leiden.  The  current  version,  0.4,  released  in 
lune  2005,  is  available  from  SourceForge:  http://sourceforge.net/ 
projects/sidb.  SIDB  has  the  merit  of  being  an  open  source  and  based 
on  free  tools  for  processing  and  archiving  images. 

Other  Projects 

In  electron  microscopy  (EM),  the  basic  problems  are  the  same  as 
for  other  microscopy  techniques,  though  some  companies  now 
provide  software  packages  for  EM  instrumentation  and  image 
management  at  the  laboratory  level  (e.g..  Quartz  Imaging  has  com¬ 
piled  a  well-designed  suite  of  software  tools  for  digital  image  man¬ 
agement).  Recently,  a  couple  of  papers  were  published  which 
described  the  efforts  to  set  up  databases  for  EM  data  (Metoz  et  al., 
2001;  Fellmann  et  al.,  2002;  Martone  et  al.,  2002). 

Image  databases  are  also  set  up  in  many  fields  related  to  bio¬ 
logical  microscopy.  Topical  image  archives,  such  as  ARKive, 
provide  well-documented  still  images  and  videos  of  endangered 
species,  and  intends  to  ultimately  cover  the  more  than  12,000 
species  on  IUCN’s  Red  List.  Among  purpose-oriented  image 
databases,  there  is  an  increasing  number  of  projects  aiming  at  pro¬ 
viding  teaching  materials,  such  as  the  Bristol  BioMed  Image 
Archive,  a  password-protected  collection  of  about  8000  medical, 
dental,  and  veterinary  images  or  the  Wellcome  Trust  Medical  Pho¬ 
tographic  Library,  a  resource  of  images  on  the  history  of  medicine 
and  modern  biomedical  sciences.  At  the  other  end  of  the  spectrum, 


there  are  commercial  databases  marketing  videos  and  photographs, 
such  as  IWF’s  Contentport  (also  known  under  its  former  name, 
Digiclip)  or  the  general-purpose  photography  collections  at  Getty 
Images  or  Corbis.  In  addition,  there  are  many  subject-  and  species- 
specific  biological  databases  such  as  FlyBase,  Fly  View/Fly  Move, 
the  Mouse  Databases  at  Jackson  Laboratory,  or  the  Mouse  Atlas, 
that  integrate  image  data  with  other  types  of  information  related 
to  the  subject. 

Finally,  there  are  projects  that  aim  at  the  integration  of 
microscopy  data  such  as  the  Open  Microscopy  Environment 
(OME)  that  is  developing  an  open  format  for  preserving  much  of 
the  metadata  that  is  usually  lost  when  the  images  are  transferred 
from  one  system  to  the  other,  for  example,  from  the  microscope  to 
the  data-analysis  software  package. 

CRITERIA  AND  REQUIREMENTS  FOR 
MICROSCOPY  DATABASES 

User  Interface 

The  users  of  local  microscopy  databases  are  different  from  those 
that  use  global  databases,  primarily  because  the  former  are 
involved  in  the  research  and  know  what  is  in  the  database,  while 
the  latter  are  exploring  an  unknown  universe.  Another  factor  is  that 
the  former  are  mainly  data  producers  and  therefore  likely  to  be 
more  interested  in  technical  parameters,  while  the  others  are 
mainly  consumers  who  are  more  interested  in  the  scientific  content 
of  the  data.  Despite  these  differences,  there  are  some  commonali¬ 
ties  in  their  approach  to  using  an  image  database. 

Most  users  browse  a  database  by  microscopy  technique,  spec¬ 
imen  type,  biological  process,  or  taxonomical  name.  This  can  be 
facilitated  by  providing  appropriate  metadata  fields  for  these  cat¬ 
egories.  A  single  description  field  that  provides  a  place  for  a  full- 
text  narrative  description  of  the  object  is  not  appropriate. 

Any  database  software  has  to  provide  a  search  interface  that 
enables  quick  iterative  searching  (i.e.,  one  that  returns  rapid  results 
and  allows  easy  revision),  and  optimally  provides  two  different 
types:  a  simple  search  interface  and  an  advanced  one  where  fields 
can  be  specified  and  controlled  vocabularies  can  be  used. 

The  results  have  to  be  as  informative  as  possible.  All  impor¬ 
tant  parameters  have  to  be  displayed  (e.g.,  the  names  and  initials 
of  all  the  authors,  not  just  of  the  first  one;  the  number  of  other 
images  associated  with  the  one  found,  etc.). 

Groupings  of  images  should  not  be  based  only  on  key  meta¬ 
data  (same  data  acquisition  date,  same  instrument,  same  author, 
same  sample,  etc.)  —  there  has  to  be  a  mechanism  to  associate  one 
image  with  others  to  form  a  project  (i.e.,  images  that  share  a 
common  grant  number). 

Query  by  Content 

While  the  storage  of  the  images  and  their  associated  metadata  in 
a  database  opens  up  a  wide  range  of  search  options,  the  biggest 
advantage  of  such  systems  lies  in  the  possibility  of  querying  by 
content.  Query  by  content  is  the  search  for  image  feature  based  on 
what  is  visible  in  the  image  (e.g.,  for  all  images  showing  round 
objects  of  0.1mm  diameter;  de  Alarcon  et  al.,  1999).  In  some 
cases,  the  system  could  rely  on  metadata  that  has  been  extracted  a 
priori.  In  other  cases  (where  the  processing  of  the  images  is  less 
time  consuming  or  where  too  much  data  would  be  generated  for 
storage)  it  would  compute  such  information  on  the  fly.  Chapter  47 
discusses  how  such  data  might  be  extracted. 
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Metadata  Structure 

Support  for  controlled  vocabularies  has  to  be  provided,  especially 
for  multi-user  databases.  One  of  the  most  frequent  reasons  for 
missing  hits  when  entering  a  specific  keyword  is  that  of  using  a 
synonym  or  a  different  spelling  of  the  word  that  was  stored.  To 
remedy  this,  controlled  vocabularies  must  be  created  for  fields 
such  as  the  microscopy  technique,  the  taxonomic  name,  etc.  Note 
that  the  development  of  a  hierarchical  vocabulary  allows  for  the 
easy  extension  of  queries  (searching  for  “confocal  fluorescence 
microscopy”  if  “confocal  two-photon  fluorescence  microscopy” 
does  not  give  enough  hits,  and  that  would  also  include  “confocal 
three-photon  microscopy”). 

It  is  also  very  important  that  the  database  knows  that  “confo¬ 
cal  two-photon  fluorescence  microscopy”  is  both  a  type  of 
“confocal  microscopy”  and  a  type  of  “fluorescence  microscopy.” 
Otherwise  it  would  not  be  able  to  return  a  complete  set  of  hits  to 
a  query  for  “confocal  microscopy”  data. 

Another  reason  for  beginning  to  use  controlled  vocabularies  as 
early  as  possible  is  to  reduce  the  effort  required  to  maintain  the 
database.  Many  large,  scientific  databases  started  years  ago  as 
small  one-man  or  one-laboratory  projects  and,  as  they  grew,  they 
had  to  be  migrated  from  one  DBMS  to  another  or  their  model  had 
to  be  re-engineered.  This  is  often  when  curators  realize  how  messy 
and  uncontrolled  the  fields  have  become  due  to  ignorance  or  lack 
of  diligence. 

Digital  Rights  Management 

The  usage  of  microscopy  image  data  from  a  database  (e.g.,  for 
teaching  purposes)  is  only  possible  for  a  third  party  if  the  rights 
for  such  usage  have  been  granted.  Because  greater  attention  is  now 
being  paid  to  correct  observation  of  image  copyright,  this  means 
that  any  database  needs  to  be  aware  of  digital  rights,  facilitate  their 
management,  and  provide  a  clear  permitted-use  policy. 

FUTURE  PROSPECTS 

In  this  chapter,  we  have  pointed  out  the  benefits  of  integrating 
image  databases  with  digital  microscopes  and  provided  the  basic 
information  required  for  achieving  this  integration.  Similar  bene¬ 
fits  could  clearly  be  gained  by  applying  the  same  principles  to  any 
data  acquisition  system,  and  indeed  the  coverage  of  the  prototype 
CCC  confocal  system  at  EMBL  has  already  been  extended  to 
include  a  wider  range  of  scientific  instruments. 

This  task  is  ultimately  one  for  microscope  manufacturers.  They 
should  aim  at  providing  a  unified  user  environment  in  which  the 
user  does  not  need  to  know  anything  about  the  microscope,  not 
even  its  brand,  to  carry  out  his  experiments.  This  would  not  only 
be  an  advantage  to  the  scientist  who  can  now  concentrate  on  the 
experiment  instead  of  worrying  about  the  technology;  it  is  also  a 
plus  for  the  manufacturers,  who  could  increase  the  compatibility  of 
their  instruments.  Systems  with  a  large  number  of  control  param¬ 
eters  would  clearly  benefit  and  allow  better  online  control  of  the 
machine.  Other  parties  involved  include  software  manufacturers, 
who  currently  provide  inadequate  database  connectivity  (Goldberg 
et  al.,  2000).  Instead,  they  should  reuse  the  parameters  provided  by 
the  instruments  and  encourage  the  addition  of  further  information 
that  can  be  used  by  their  analysis  packages.  There  is  still  a  need 
for  BioTools  (Fig.  50.1),  tools  for  data  access  and  manipula¬ 
tion  (Boudier  and  Shotton,  2000),  data  analysis  (Machtynger  and 
Shotton,  2002)  and  visualization  (Pittet  et  al.,  1999),  data  inter¬ 
change  (Boudier  and  Shotton,  1999),  and  database  connectivity. 


The  path  that  Bioimage  has  paved  shows  quite  clearly  that 
microscopy  data  is  only  one  piece  of  the  puzzle.  In  line  with  devel¬ 
opments  in  completely  different  areas  such  as  proteomics,  where 
the  structure  and  sequence  databases  have  recently  been  brought 
closer  together,  there  is  a  broader  perspective  which  comprises 
both  microscopy  data  and  macroscopical  photographs  via  images 
generated  by  medical  instrumentation,  and  which  spans  the  gamut 
from  structural  to  behavioral  biology. 

Bioimage  shows  another  way  to  go:  as  there  is  no  way  to  put 
all  data  into  a  single  central  database,  it  is  essential  to  share  meta¬ 
data  among  all  specialized  databases,  with  topical  databases,  with 
educational  projects,  with  commercial  scientific  image  data  pro¬ 
viders,  and,  of  course,  with  companies  generating  image  data  of 
general  scientific  interest  in  the  course  of  their  work. 

In  view  of  all  the  concrete  advice  that  has  been  given,  it  may 
seem  surprising  that  image  databases  are  still  in  their  infancy 
(Martone  et  al.,  2002).  But  compared  with  the  historical  past  of 
textual  databases  and  taking  into  account  that  visual  information 
represents  70%  of  all  life  science  data  generated  (Gonzalez-Couto 
et  al.,  2001),  this  is  probably  as  it  should  be. 
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APPENDIX 

The  increased  popularity  of  digital  photography  has  boosted  the 
development  of  software  packages  for  managing  and  archiving 
digital  photographs.  These  systems,  which  range  between  $40  and 
$800  in  price,  are  affordable  for  every  scientist  —  the  question  is 
whether  they  are  able  to  fulfill  the  needs  of  the  microscopist  who 
wants  to  manage  his  image  files.  In  this  area  the  photo  manage¬ 
ment  software  has  to  compete  with  tools  offered  by  microscope 
manufacturers  (e.g.,  Zeiss’  LSM  Image  Browser  or  Leica’s 
IM1000)  and  more  expensive  scientific  image  and  data  manage¬ 
ment  systems  such  as  Dynamic  Data  Links’  Aequitas1  or  RVC’s 
Research  Assistant.2 

This  Appendix  intends  to  give  a  short  overview  of  the  features 
of  the  latter  software  tools  (Table  50. Al)  because  a  scientist  who 
doesn’t  own  a  microscope  and  who  is  faced  with  the  decision  to 
choose  an  image  management  system  will  more  likely  opt  for  one 
of  these  systems  rather  than  a  microscope  manufacturer’s  solution. 
Many  of  the  impressions  are  based  on  recent  comprehensive 
reviews  in  the  magazine  c’t  (Trinkwalder,  2002;  Trinkw alder, 
2004).  A  detailed  account  of  the  software  is,  however,  beyond  the 
scope  of  this  Appendix,  as  new  versions  and  updates  are  released 
almost  every  year  and  many  additional  features  will  have  been 
incorporated  during  the  first  year  this  book  is  released.  For 
example,  communication  with  other  programs  to  import  or  export 
data  and  metadata  currently  seems  to  be  a  general  shortcoming, 
but  it  is  expected  that  this  will  soon  be  overcome.  The  same  prob¬ 


1  The  software  hasn't  been  updated  for  5  years  and  it  is  not  clear  if  further 
releases  can  be  expected. 

2  This  software  is  often  updated  and  the  most  recent  release  is  from  October 
2004. 
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ably  holds  for  platform  dependence:  currently,  iView  is  only  avail¬ 
able  for  Macintosh;  IMatch,  ThumbsPlus,  Aequitas,  and  Research 
Assistant  are  Windows-only  programs.  Cumulus  has  even  a  server 
version  that  runs  on  Unix  systems. 

The  programs  all  work  according  to  the  same  principle.  They 
do  not  store  the  image  data  itself,  but  just  hold  a  thumbnail  picture 
of  the  image  and  a  link  to  the  original  file  in  the  file  system  (in 
this  respect  they  work  similarly  to  the  BioAgency-BioArchive 
tandem,  the  first  of  which  stores  the  metadata  and  a  pointer  to  the 
data  stored  in  the  second  system).  They  provide  a  means  to  map 
the  file  system  onto  one  or  more  independent  structures  that  the 
user  creates  himself,  based  on  categories. 

Most  important  for  microscopy  data  management  are  tools  to 
create  additional  categories  and  keywords,  and  to  organize  them 
in  a  hierarchy  rather  than  merely  in  an  alphabetical  list.  Here,  the 
solutions  offered  by  the  software  range  from  poor  to  excellent. 
While  ACDSee  does  not  even  allow  one  to  create  user-defined 
fields,  IMatch,  for  example,  provides  means  to  build  property  sets, 
which  are  actually  database  entities  that  are  independent  of  the 
image  itself  (e.g.,  a  person  or  an  experiment)  and  which  can  be 
related  to  the  image,  thereby  offering  one  the  possibility  of 
setting  up  a  relational  database  to  get  away  from  an  image-centric 
model. 

Image  formats  are  a  big  issue,  and  development  is  partly 
focused  on  importing  data  directly  from  imaging  hardware.  In  this 
respect,  Aequitas  and  Research  Assistant  promise  to  offer  connec¬ 
tivity  to  equipment  used  in  microscopy.  Some  software,  on  the 
other  hand,  does  not  even  provide  video-handling  capabilities 
(IMatch  and  ThumbsPlus  do  not  even  provide  the  full  range  of  .avi, 
.mpeg  and  .mov  file  handling). 

Query  features  are  important  to  find  and  retrieve  images  effi¬ 
ciently.  Here  again,  the  solutions  range  from  too  simple  to  more 
complex,  the  best  marks  being  for  programs  like  Cumulus  and 
Portfolio  that  are  very  database-near.  Aequitas  even  offers  support 
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TABLE  50.A1.  Image  Management  Software 


Cumulus 

Portfolio 

Imatch 

ThumbsPlus 

iView 

ACDSee 

Aequitas 

Research 

Assistant 

URL 

www.canto.com 

www.extensis. 

www.photools. 

www.cerious. 

www.  i  view- 

www.  acdsy  stems. 

www. ddlinks.co.uk 

www.rvc.nl 

com 

com 

com 

multimedia. 

com 

com 

Pros 

Good  video 

Good  metadata 

Good  metadata 

Good  video 

Good  video 

Microscopy- 

handling 

management 

management 

handling 

handling 

tailored 

Good  metadata 

Good  user 

Good  user 

Good  metadata 

Good  metadata 

management 

interface 

interface 

management 

management 

Good  user 

Support  of 

Support  of 

interface 

hardware  typical 

hardware 

for  microscopy 

typical  for 

microscopy 

Cons 

Upper  price  range 

Only  available 

Only  available 

Only  available 

Poor  metadata 

Only  available  for 

Only  available 

for  photo 

for  Windows 

for  Windows 

for  Macintosh 

management 

Windows 

for  Windows 

software 

Unusual  user 

Poor  search 

Quite  expensive 

interface 

capabilities 

for  SQL  searches  and  IMatch  allows  sophisticated  regular  expres¬ 
sions.  A  plus  in  some  programs  (IMatch,  ThumbsPlus,  iView, 
Aequitas)  is  the  ability  to  query  by  example,  which  provides  a 
quick  way  to  find  visually  similar  images.  However,  with  fluores¬ 
cence  images  being  false  colored,  preferably  in  green  and  red,  and 
with  scientists  having  collections  of  dozens  of  almost  identical 
images,  this  feature  is  not  very  helpful  in  fluorescence  microscopy. 


Another  extra  seems  to  be  more  useful:  Some  programs  have  a 
macro  editor,  which  allows  scripting  more  complex  applications. 

Finally,  the  most  recent  article  by  Trinkwalder  (2004)  also  has 
some  benchmarks  for  the  cataloguing  of  Photoshop,  TIFF,  and 
IPEG  files.  The  differences  are  sometimes  considerable,  both 
depending  on  the  hie  type  and  the  software,  but  apparently  they 
only  rule  out  the  usage  of  IMatch  for  Photoshop  hies. 
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INTRODUCTION 

Biofilms  consist  of  microbial  cells  (bacteria,  fungi,  and  algae)  that 
are  fixed  in  a  spatially  defined  relationship  to  one  another,  typically 
involving  attachment  to  a  substratum  by  means  of  extracellular 
polymeric  substances  (EPS).  Afew  real-world  examples  of  biofilms 
include  dental  plaque,  slimes  on  rocks,  slimes  occurring  in  liquid 
distribution  systems  (e.g.,  drinking  water  supplies  and  sewers),  and 
natural  bacterial  populations  on  plant  and  animal  surfaces  (e.g.,  skin 
microflora,  epiphytic  and  rhizosphere  microorganisms).  In  nature, 
the  vast  majority  of  microbial  biomass  is  attached  to  a  substratum. 
The  most  complex  biofilms  are  mixtures  of  phylogenetically  and 
physiologically  different  organisms.  These  mixed  populations  can 
be  investigated  from  the  community  perspective,  that  is,  in  their 
totality  as  the  sum  of  their  parts,  or  at  the  level  of  their  component 
members,  that  is,  as  collections  of  single  cells.  Even  a  pure-culture 
biofilm  formed  from  a  population  of  clonal  bacterial  cells  can  have 
qualities  that  are  worthy  of  examination  at  the  single-cell  level 
(Rainey  and  Travisano,  1998;  Singh  etal.,  2002;  Boles  eta].,  2005). 
Because  bacterial  cells  typically  measure  at  most  5  |im  in  the  longest 
dimension,  observation  of  biofilms  using  a  microscope  can  range 
over  several  orders-of-magnitude:  from  a  few  micrometers  (single 
cell)  to  several  millimeters  (the  extent  of  many  biofilms  in  in  vitro 
and  in  natural  systems). 

While  biofilm  microscopy  owes  much  to  biological  micro¬ 
scopy,  it  is  a  unique  field  for  which  general  biological  and  micro¬ 
biological  techniques  have  been  adapted.  We  present  here  a 
collection  of  approaches  and  methodologies,  many  of  which  orig¬ 
inated  in  eukaryotic  cell  biological  investigations  but  which  have 
been  adapted  to  studies  of  microorganisms  on  substrata. 

SAMPLE  PRESENTATION 

Flowcells  and  Other  Perfusion  Chambers 

To  obtain  single-cell  resolution  in  bacterial  biofilms,  one  must  use 
lenses  with  high  numerical  aperture  (NA):  oil-  or  water-immersion 
lenses,  or  the  highest  NA  (0.9)  water-immersible  lenses  (dipping 
lenses,  no  coverslip  required).  Also,  sample  preparation  techniques 
which  are  routine  for  other  biological  material  and  which  include 
fixation  and  embedding/sectioning  are  less  frequently  employed  for 
in  vitro  biofilm  research  because  the  biofilms  are  generally  thin 


enough  (<50pm)  that  the  optical  sectioning  provided  by  confocal 
microscopy  is  sufficient  to  examine  much  of  the  sample’s  z- 
dimension.  Together,  these  factors  have  resulted  in  a  preference 
for  the  examination  of  hydrated,  living  biofilms  grown  in  perfu¬ 
sion  chambers  known  as  flowcells  or  flow  chambers.  The  basic 
flowcell  design  is  very  simple:  grooves  are  cut  into  a  plastic  block 
and,  using  silicone  sealant,  a  microscopy  coverglass  is  attached 
covering  the  grooves.  The  coverglass  thus  constitutes  one  wall  of 
the  perfusion  chamber  while  also  providing  an  optically  correct 
substratum  through  which  the  developing  biofilm  is  viewed.  It  is 
useful  to  design  the  flowcell  so  that  a  standard  commercially 
available  coverslip  (e.g.,  24  x  50mm)  can  be  used,  although  many 
coverglass  suppliers  are  willing  to  prepare  custom  sizes. 
Autoclaveable  homemade  designs  exist  (Palmer  and  Caldwell, 
1995;  Kolenbrander  et  al.,  1999)  and  some  commercially  available 
systems  are  autoclaveable  (see,  e.g.,  http://cu.imt.net/~mitbst/ 
Flow_Cells.html).  However,  many  plastics  warp  after  repeated 
autoclaving  and  therefore  the  simplest  designs  are  sterilized  using 
ethylene  oxide  gas  or  are  simply  disinfected  with  bleach  or  ethanol. 
After  the  assembled  flowcell  is  disinfected,  the  experimental  bac¬ 
teria  are  introduced  into  the  chamber,  after  which  sterile  medium 
is  pumped  through  the  chamber  to  remove  unattached  bacteria 
while  providing  nutrients  for  growth  of  the  developing  biofilm. 
Growth  directly  on  the  coverslip  means  that  high-NA  immersion 
lenses  can  be  used  to  maximum  advantage.  After  microscopic 
examination  and  termination  of  the  experiment,  the  coverglass  is 
removed,  the  flowcell  is  cleaned,  and  a  new  coverglass  is  attached. 
Historically,  these  chambers  have  been  used  on  upright  micro¬ 
scopes,  although  many  designs  work  just  as  well  on  inverted 
platforms.  Various  flowcell  designs  are  shown  in  Figure  51.1.  Many 
commercially  available  perfusion  chambers  can  be  used  for  biofilm 
work,  the  most  important  criterion  being  the  ability  to  pump  liquids 
through  the  chamber  under  conditions  of  laminar  flow  (no  mixing 
through  turbulence).  Equations  important  for  these  calculations  are 
available  in  the  literature  (Zinn  et  al.,  1999).  A  disadvantage  of  the 
basic  design  is  that  transmitted  light  microscopy  is  difficult  or 
impossible.  However,  other  approaches  exist  (different  flowcell 
designs,  use  of  glass  capillaries)  that  can  make  use  of  transmitted 
light  to  varying  degrees  (Bos  et  al.,  1994;  Palmer  and  Caldwell, 
1995;  Stoodley  et  al.,  1999). 

Once  a  flow  chamber  design  is  chosen,  the  chamber  must  be 
integrated  into  a  setup  that  will  move  liquid  through  the  chamber. 
The  typical  approach  is  to  feed  the  medium  from  a  reservoir  to  the 
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FIGURE  51.1.  Flow  cells  as  they  have 
developed  over  the  last  1 5  years  at  the  Center 
for  Biomedical  Microbiology,  BioCentrum- 
DTU,  Denmark.  The  design  at  the  lower  left 
is  currently  in  use. 


chamber  with  a  peristaltic  pump  placed  between  the  reservoir  and 
chamber.  Everything  on  the  upstream  side  of  the  chamber  (includ¬ 
ing  the  chamber  itself)  must  be  disinfected  or  sterilized.  The 
element  most  likely  to  fail  in  such  a  setup  is  the  tubing  on  which 
the  peristaltic  pump  acts.  Repeated  autoclaving  of  this  tubing 
invariably  leads  to  degradation  in  the  tubing’s  mechanical  proper¬ 
ties  and  thus  to  altered  flow  rates  or  even  tubing  failure  (leaks). 
Furthermore,  the  peristaltic  action  encourages  bubble  formation 
within  the  tubing;  bubble-trap  designs  exist  to  prevent  passage  of 
bubbles  into  the  flow  chamber  —  a  detailed  description  of  this  type 
of  system  is  presented  below.  Finally,  should  an  obstruction  occur, 
the  high-pressure  capability  of  the  pump  can  cause  the  chamber  to 
leak,  a  happenstance  not  always  well  tolerated  by  the  optical 
components. 

As  an  alternative,  if  the  pump  is  placed  downstream  of  the  flow 
chamber  and  the  medium  reservoir  is  slightly  elevated  above  the 
height  of  the  flowcells  and  downstream  tubing,  a  siphon  can  be 
started  prior  to  inoculation  by  using  a  syringe  to  pull  the  medium 
through  the  line  space  ahead  of  the  pump;  the  primed  tubing  is 
then  attached  the  to  the  pump,  which  acts  more  as  a  metering 
device  than  as  the  driving  force  for  liquid  transport.  In  the  most 


extreme  example,  the  pump  can  be  eliminated  entirely  by  using  a 
set  of  hanging  flasks  on  either  end  of  the  system  (Bos  et  al.,  1994). 

Pumps  are  selected  on  four  criteria:  stability,  flow  rate,  number 
of  channels,  and  number  of  rollers.  Stability  is  quite  important;  the 
pump  should  be  able  to  run  unattended  for  extended  periods  of 
time.  Required  flow  rates  will  depend  on  flowcell  dimensions 
(laminar  flow,  see  previous),  and  the  best  pumps  accommodate 
tubing  of  different  bore  diameters  within  the  pump  head  thus  pro¬ 
viding,  through  a  combination  of  pump  speed  and  bore  diameter, 
as  much  as  a  five  order-of-magnitude  range  in  flow  rate.  Versatile 
pumps  have  multiple  channels  (multiple  tubes  through  which 
liquid  is  pumped);  the  cheaper  pumps  have  two  to  three  channels 
while  the  more  expensive  pumps  have  as  many  as  32  channels. 
Multiple  channels  are  required  to  simultaneously  run  flow  cham¬ 
bers  with  different  medium  compositions,  and  even  to  run  multi¬ 
ple  chambers  with  the  same  medium.  Chambers  connected  in 
series  are  subject  to  changes  in  the  medium  induced  by  bacterial 
metabolism  in  upstream  chambers,  and  chambers  connected  in  par¬ 
allel  (but  to  the  same  main  feed  via  splitters  inserted  downstream 
of  the  pump)  are  subject  to  variation  in  flow  rate  from  chamber  to 
chamber.  Another  feature  of  precision  pumps  is  a  high  number  of 
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rollers  (e.g.,  8-10)  in  the  pump  head.  The  greater  the  number  of 
rollers,  the  smoother  the  flow  (peristalsis  is  less  vigorous). 
Although  syringe  pumps  are  alternatives  to  peristaltic  pumps, 
single- syringe  pumps  must  be  refilled.  Dual-syringe  pumps  such 
as  are  used  in  high-pressure  liquid  chromatography  require  no 
refilling,  but  have  the  major  disadvantage  of  providing  only  a 
single  output  channel. 

Water-Immersible  Lenses 

Bacteria  grow  under  almost  any  condition  presented  by  nature,  and 
retrievable  substrata  can  greatly  simplify  the  examination  of 
biofilms  grown  under  complex  experimental  regimes.  To  examine 
biofilms  produced  under  a  particular  set  of  conditions  within  a  fer¬ 
menter  or  within  a  natural  environment  (e.g.,  a  stream,  a  lake,  or 
the  oral  cavity),  one  need  only  insert  a  “coupon”  that  can  be 
retrieved  at  a  later  time  for  microscopic  examination.  Two  prob¬ 
lems  exist  in  such  an  approach.  First,  the  coupon  invariably  passes 
through  the  air-liquid  interface  at  least  once  in  its  route  to  the 
microscope,  and  is  thereby  subject  to  shear  stress  that  may  remove 
biomass.  Second,  the  sample  must  be  either  investigated  with  a 
water-immersible  lens  (dipping  lens,  no  coverslip  required),  or  the 
sample  must  have  a  coverslip  placed  on  it  with  resultant  pressure 
distortion.  If  the  biofilms  are  very  thin,  that  is,  essentially  mono- 
layers  of  bacterial  cells,  then  distortion  arising  during  careful  addi¬ 
tion  of  a  coverslip  can  be  minimal;  such  is  the  case  in  recent 
experimental  work  on  epiphytic  biofilms  (Monier  and  Lindow, 
2003,  2004).  A  gasket  can  be  used  to  hold  the  coverslip  at  a  fixed 
distance  from  the  slide  to  eliminate  pressure  directly  on  the 
biofilm;  however,  a  gap  between  the  biofilm  surface  and  the  under¬ 
side  of  the  coverslip  is  thereby  created  which  typically  precludes 
use  of  low-working-distance  (high-NA,  immersion)  lenses.  Bio¬ 
films  therefore  represent  exactly  the  sort  of  sample  for  which  all 
major  microscope  suppliers  offer  high-resolution  (ca.  60  X  0.9  NA) 
water-immersible  lenses  that  also  have  working  distances  of 
several  millimeters.  These  lenses  can  resolve  fluorescently  labeled 
single  bacterial  cells  and  can  be  used  to  examine  very  irregular  or 
tilted  substrata,  such  as  human  tooth  enamel  (Palmer  et  al.,  2003). 
They  can  also  be  used  to  directly  examine  biofilms  on  particles 
such  as  sand  grains  or  chromatographic  supports;  as  with  other 
substrata,  the  difficulty  is  not  so  much  in  the  microscopy,  but  rather 
in  bringing  the  substratum  to  the  microscope  without  altering  the 
biofilm  (or  scratching  the  lens!). 

Upright  Versus  Inverted  Microscopes 

Throughout  the  preceding  text,  it  should  be  clear  that  an  upright 
microscope  is  the  most  versatile  platform  for  biofilm  microscopy, 
primarily  because  it  can  easily  accommodate  water-immersible 
lenses.  Biofilms  can  be  grown  in  vitro  under  stagnant  conditions 
if  nutrients  are  provided  by  replenishing  the  medium.  Thus,  multi¬ 
well  plates  (e.g.,  Whatman  catalog  no.  7716-2375)  and  chamber 
slides  (e.g.,  Nunc  LabTek  coverslip  chambers)  designed  for 
inverted  microscopy  can  be  used  for  biofilm  research  provided  that 
the  plates/slides  incorporate  a  substratum  that  is  amenable  to  high- 
NA  immersion  objectives.  On  the  other  hand,  as  small-well-format 
multi-well  plates  (e.g.,  >96  wells/plate)  cannot  be  viewed  with 
immersible  lenses  on  upright  microscopes  because  the  lens  tip 
diameter  is  typically  larger  than  the  well  diameter,  these  must  be 
examined  using  an  inverted  microscope.  However,  a  dipping  lens 
can  be  used  on  an  inverted  platform  to  examine  non-coverslipped 


samples  by  building  up  a  ring  of  wax  or  some  other  easily  remov¬ 
able  material  (certain  types  of  gaskets  can  be  useful)  around  the 
lens  tip.  A  convex  puddle  of  water  can  be  formed  on  top  of  the 
lens  face  and  contained  within  the  ring.  The  lens  is  then  moved  up 
towards  the  substratum  to  be  imaged  (which  is  placed  with  the 
biofilm  on  the  underside  or  facing  downwards  towards  the  objec¬ 
tive)  until  the  water  contacts  the  substratum. 

Setup  of  a  Flow  Chamber  System  Setup  —  A 
Practical  Example 

The  flow  chambers  are  produced  in  translucent  technical  polycar¬ 
bonate.  Each  channel  has  a  depth  of  1  mm  and  the  base  has  a  size 
similar  to  a  standard  microscope  slide.  Such  designs  are  compati¬ 
ble  with  typical  microscope  slide-mounting  hardware,  and  the 
biofilm  can  be  examined  using  either  reflected  light  or  epi- 
fluorescence  confocal  modes.  The  flowcell  is  assembled  by  using 
a  single-use  syringe  to  apply  threads  of  silicone  glue  (e.g..  Super 
Silicone  Sealant,  3M,  St.  Paul,  MN)  to  the  top  of  the  polycarbon¬ 
ate  base.  Long,  continuous  threads  of  glue  should  be  applied  to 
cover  the  area  between  and  around  the  channels,  and  the  syringe 
should  not  touch  the  base,  otherwise  the  silicone  glue  will  smear. 
The  coverslip  is  placed  on  top  of  the  silicone  and  pushed  carefully 
until  the  silicone  spreads  over  the  whole  area  of  the  base  between 
the  channels  [Fig.  51.2(A)].  Allow  the  silicone  to  solidify  before 
use,  preferably  overnight. 

Silicone  tubing  with  dimensions  of  1  mm  inner  and  3  mm  outer 
diameter  is  used  throughout  the  system  except  at  the  pump,  where 
the  stronger  Marprene  tubing  (Watson-Marlow,  Wilmington,  MA) 
is  required  to  withstand  the  extensive  wear  of  the  pump  rollers. 
The  complete  system  with  all  the  tubing  is  diagrammed  in  Figure 
51.2(C). 

Bubble  traps  [Fig.  51.2(B)]  are  constructed  using  5mL 
syringes  mounted  on  top  of  a  piece  of  polycarbonate  with  inlet  and 
outlet  (the  inlet  holes  are  situated  higher  than  the  outlet  holes) 
inside  the  syringe.  The  syringes  are  closed  at  the  top  with  a  Luer- 
lock  stopper  (not  shown  in  Fig.  51.2),  during  normal  operation. 
The  system  effluent  is  collected  in  bottles  that  must  be  placed  at 
or  above  the  level  of  the  flow  channels  so  that  any  leak  in  a  flow 
channel  will  not  result  in  the  entrance  of  air.  Also,  the  ends  of  the 
effluent  tubes  must  be  well  above  the  surface  of  the  waste  liquid 
to  prevent  reverse  siphons  out  of  the  effluent  collection  bottle. 

The  fully  assembled  system  is  disinfected  with  0.5%  hypo¬ 
chlorite  because  ethanol  dissolves  3M  Super  Silicone  Sealant.  The 
system  is  first  filled  up  to  the  bubble  traps.  The  stoppers  should  be 
off  and  stored  in  70%  ethanol.  When  the  traps  are  filled,  continue 
flow  for  approximately  5  to  10  min  at  a  high  flow  rate,  during 
which  the  feed  liquid  pours  out  of  the  top  of  the  bubble  traps, 
leaving  the  downstream  part  of  the  system  dry.  Once  the  stoppers 
are  inserted  into  the  bubble  traps,  the  rest  of  the  system  is  filled. 
Bubbles  are  removed  from  the  system  by  carefully  tapping  the  flow 
cells  while  the  pump  is  running  (5-10min).  Continue  sterilization 
for  2  to  4  h  with  low  or  no  flow. 

The  flow  system  is  washed  to  remove  the  hypochlorite  by 
filling  and  emptying  the  system  two  to  three  times,  as  described 
above,  with  a  total  of  1  to  2  L  of  autoclaved  double-distilled  water. 
To  ensure  removal  of  all  disinfectant,  the  system  is  purged  with  air 
between  the  disinfectant  and  sterile  water  runs.  Water  is  then  run 
through  the  system  overnight  at  a  low  flow  rate,  after  which  the 
system  is  filled  with  medium  and  the  laminar  flow  rate  is  set  to 
approximately  0.2mm/s  (approximately  3  mL/h/channel).  If  the 
carbon  source  is  very  hydrophobic,  then  medium  is  run  in  the 
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FIGURE  51.2.  Flow  chamber  (A)  and  bubble  trap  (B)  design.  Complete 
system  (C).  This  system  was  designed  at  BioCentrum-DTU,  but  many  varia¬ 
tions  exist.  (C)  was  drawn  by  J.A.J.  Haagensen  and  K.B.  Barken. 


system  overnight  to  saturate  the  silicone  tubing.  If  the  experimen¬ 
tal  temperature  is  above  ambient,  the  medium  supply  must  be  equi¬ 
librated  to  that  temperature  or  bubbles  may  appear  in  the  system. 
Because  autoclaving  removes  dissolved  gases  from  the  medium, 
the  medium  supply  can  be  placed  at  the  experimental  temperature 
without  prior  cooling.  Sterilization  of  the  flow  system  can  also  be 
done  by  autoclaving  at  121°C.  Polycarbonate  flow  chambers  will 
not  warp  in  the  autoclave,  but  other  materials  may. 


After  sterilization  and  filling,  the  flowcells  can  be  inoculated 
with  microorganisms.  Before  inoculation,  flow  is  stopped  and  the 
tubing  between  the  flow  chamber  and  the  bubble  trap  is  clamped 
off.  The  outer  surface  of  the  tubing  at  the  injection  site  is  disin¬ 
fected  with  70%  ethanol.  A  0.5  mL  syringe  is  filled  with  250  |iL  of 
an  appropriately  diluted  overnight  culture  of  the  cells.  The  needle 
is  inserted  into  the  tubing  as  near  as  possible  to  the  flow  channel 
inlet  and  the  bacteria  are  carefully  injected.  After  inoculation,  the 
tubing  surface  is  again  disinfected,  and  silicone  is  used  to  seal  the 
injection  site.  The  flowcell  is  placed  with  the  glass  surface  facing 
downward  to  encourage  contact  between  the  bacteria  and  the  cov- 
erglass.  After  1  h,  the  chamber  is  turned  upright,  the  clamps  are 
removed,  and  medium  flow  is  started. 


MAKING  BACTERIA  FLUORESCENT 
Fluorescent  Proteins 

Classic  methods  for  labeling  bacteria  involve  the  addition  of 
staining  reagents  to  the  cells.  Fluorescent  proteins  (FP)  have  elim¬ 
inated  the  need  for  the  addition  of  any  external  component.  The 
properties  of  FP  are  described  in  depth  elsewhere  in  this  volume, 
and  we  will  list  here  only  the  key  features  important  for  bacterial 
biofilm  work. 

Green  fluorescent  protein  (GFP)  from  the  jellyfish  Aequorea 
victoria  has  been  studied  from  the  early  1960s  and  has  been  used 
as  a  tool  in  biology  since  its  cloning  (Chalfie  et  al.,  1994).  GFP  is 
a  small  protein  (238  amino  acids;  27kDa)  which  is  excited  by 
ultraviolet  (UV)  to  blue  light  and  which  can  be  expressed  in  many 
bacteria.  The  excitation  maximum  of  the  protein  is  395  nm  with  a 
minor  peak  at  470  nm.  Emission  maximum  is  507  nm  with  a  shoul¬ 
der  at  540  nm  (Shimomura  et  al.,  1962;  Morin  and  Hastings,  1971). 
Small  amounts  of  oxygen  are  needed  for  the  newly  synthesized 
protein  to  become  fluorescent  in  a  maturation  process  that  requires 
several  hours,  after  which  it  remains  largely  stable  for  days 
(Tombolini  et  al.,  1997).  However,  it  is  this  very  stability  that  pre¬ 
cludes  use  of  wild-type  GFP  for  dynamic  gene  expression  studies. 
In  response  to  these  various  problems,  mutations  in  the  GFP  gene 
sequence  were  developed  that  resulted  in  altered  spectral  proper¬ 
ties  (Table  51.1).  Shorter  maturation  times  were  noted  with  these 
variants. 

Although  green  or  blue  fluorescence  may  suffice  for  many 
applications,  FP  that  emit  in  the  red  or  orange  were  desired.  Red 
fluorescence  can  be  seen  from  GFP  when  it  is  synthesized  anaer¬ 
obically  (Heim  et  al,  1994),  but  any  exposure  to  oxygen  results 
in  rearrangement  and  change  to  wild-type  green  fluorescence. 
Although  Aequorea  GFP  could  not  be  mutagenized  to  emit  outside 
of  a  relatively  limited  range  of  wavelengths,  other  organisms 
(corals  in  particular),  provided  FP  with  interesting  spectral 
properties  (Table  51.2). 


TABLE  51.1.  Spectral  Variants  of  Aequorea  victoria  Green 
Fluorescent  Protein 


Name 

Ecitation  (nm) 

Emission  (nm) 

Color 

BFP  (Blue  FP) 

380  (UV) 

440 

Pale  blue 

CFP  (Cyan  FP) 

433/453 

475/501 

Cyan 

YFP  (Yellow  FP) 

513 

527 

Greenish-yellow 

*Data  from  Heim  and  Tsien  (1996),  Heim  et  al.  (1995),  Ormo  et  al.  (1996),  and 
Siemering  et  al.  (1996). 
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TABLE  51.2.  Fluorescent  Proteins  from  Organisms  Other  than  Aequorea  Victoria 


Name 

Organism  of  origin 

Excitation  (nm) 

Emission  (nm) 

Color 

Reference 

Am  Cyan 

Anemonea  majano 

458 

489 

Cyan 

Matz  et  al.,  1999 

ZsGreen 

Zoanthus  sp. 

493 

505 

Green 

Matz  et  al.,  1999 

Zs  Yellow 

Zoanthus  sp. 

529 

539 

Yellow 

Matz  et  al.,  1999 

DsRed 

Discomonas  sp. 

558 

583 

Red 

Matz  et  al.,  1999 

AsRed2 

Anemonia  sulcata 

576 

592 

Red 

Lukayanov  et  al.,  2000 

HcRed 

Heterectis  crispa 

588 

618 

Far  Red 

Gurskaya  et  al.,  2001 

In  addition  to  excitation/emission  wavelength,  maturation 
times  and  stability  are  properties  that  determine  if  a  particular 
protein  can  be  used  for  dynamic  studies.  Maturation  times  are  gen¬ 
erally  shortened  by  random  mutagenesis,  while  other  methods  are 
used  to  make  the  proteins  less  stable.  Unstable  variants  have  been 
constructed  by  the  addition  of  short  oligopeptides  to  the  C- 
terminal  end  of  the  protein,  thereby  making  the  mutant  protein  a 
target  of  indigenous  proteases  and  shortening  its  lifetime  to  several 
minutes  (Andersen  et  al.,  1998;  Li  et  al.,  1998).  In  biofilm  exper¬ 
iments,  such  degradable  FP  have  been  used  to  demonstrate  differ¬ 
ential  gene  expression  in  Pseudomonas  bacterial  microcolonies 
(Sternberg  et  al.,  1999;  Christensen  et  al.,  2002). 

Most  fluorescent  molecules  will  bleach  under  intense  or 
lengthy  illumination.  In  FRAP  (fluorescence  recovery  after  photo- 
bleaching),  an  area  of  the  biofilm  is  bleached  and  then  observed 
over  time  for  fluorescence  recovery  as  unbleached  fluorophore  dif¬ 
fuses  or  is  transported  into  the  dark  area.  Cell  or  solute  movements 
in  defined  regions  can  thus  be  studied.  An  example  using  two  iso¬ 
genic  strains  of  P.  aeruginosa  demonstrates  the  role  of  pili.  The 
wild-type  strain  has  pilus-dependent  motility  (the  pilA  gene)  and 
is  expressing  YFP  The  pilA  mutant  is  non-motile  and  is  express¬ 
ing  CFP.  Figure  51.3  shows  the  cell  migration  following  a  bleach¬ 
ing  of  a  small  area  in  the  biofilm:  the  pilA  mutant  stays  and  the  wt 
moves  into  the  bleached  area,  indicating  a  pi/A-dependent  active 
relocation  of  cells  in  a  biofilm  (Haagensen  et  al.,  submitted). 
Because  the  relocating  cells  originate  outside  of  the  bleached  area, 
photodamage  to  cells  during  bleaching  is  considered  irrelevant  in 
this  experiment. 


STAINS 

Nucleic  Acid  Stains 

The  traditional  nucleic  acid  stains  4'.6-diani id ino-2-phenyl indole 
(DAPI)  and  Acridine  Orange  stain  bacteria  well  when  used  alone 
or  in  sequential  scanning  modes;  however,  they  have  some  disad¬ 
vantages  for  confocal  work.  Excitation  of  DAPI  requires  expen¬ 
sive  ultraviolet  or  two-photon  lasers;  the  UV  excitation  wavelength 
can  result  in  high  autofluorescence  and  therefore  can  make  simul¬ 
taneous  multi-channel  imaging  difficult.  Simultaneous  imaging  is 
also  difficult  with  Acridine  Orange  because  the  stain  is  extremely 
bright  and  has  a  long,  red  emission  tail.  In  fact,  Acridine  Orange 
displays  red  emission  when  bound  to  RNA  and  green  emission 
when  bound  to  DNA,  a  trait  that  can  be  advantageous  when  used 
as  the  single  dye  in  a  sample  (see  Fig.  39.1,  this  volume). 

SYTO  stains  (cyanine  dyes  produced  by  Molecular  Probes, 
Inc.,  Eugene,  OR)  bind  to  nucleic  acids  in  the  cells,  but  the  dif¬ 
ferent  SYTO  stains  differ  in  penetration  efficiency,  binding  affin¬ 
ity  and  spectral  properties.  In  particular,  bacterial  strain-dependent 
differences  exist.  For  example,  the  red  fluorescent  nucleic  acid 
stain  SYTO  62  works  well  in  combination  with  GFP  or  other  flu¬ 
orescent  proteins.  SYTO  62  stains  Pseudomonas  spp.  weakly,  but 
strong  staining  occurs  with  Acinetobacter  C6  and  with  polymor¬ 
phonuclear  leucocytes  (PMN).  This  difference  was  used  in  inves¬ 
tigation  of  interactions  between  Acinetobacter  C6  and  KT2440,  as 
well  as  in  experiments  on  the  influence  of  PMN  on  P.  aeruginosa 
(Fig.  51.4). 


FIGURE  51.3.  FRAP  in  biofilms.  One-day-old  biofilm  established  using  a  1:1  mixture  of  YFP-tagged  P.  aeruginosa  PAOl  wild  type  and  CFP-tagged  non- 
motile  pilA  mutant  PAOl.  (A)  xy- image  of  the  biofilm  structure  before  bleaching.  (B)  The  bleached  microcolony.  (C)  The  microcolony  40min  after  bleaching. 


Confocal  Microscopy  of  Biofilms  —  Spatiotemporal  Approaches  •  Chapter  51  875 


FIGURE  51.4.  Differential  staining  of  bacteria  and  eukaryotic  cells  using 
SYTO  62  (excitation,  633  nm;  emission,  676  nm)  and  GFP.  SYTO  62  (1  |xL  of 
a  5mM  stock  in  DMSO)  is  dissolved  in  1  mL  of  medium  or  0.9%  NaCl.  Five 
to  15  min  prior  to  microscopy,  0.5  mL  of  the  diluted  SYTO  62  is  injected  into 
the  flow  chamber.  (A)  Spatial  distribution  of  cells  in  a  biofilm  containing 
Acinetobacter  C6,  stained  with  SYTO  62,  and  P  putida  KT2440,  tagged  with 
GFP.  Image  acquired  after  1  day  of  flow-chamber  growth  (S.K.  Hansen,  unpub¬ 
lished).  (B)  P.  aeruginosa  expressing  GFP  in  interaction  with  SYTO 
62-stained  polymorph  nuclear  neutrophils  (red)  (Bjarnsholt  et  al.,  2005). 


Live/Dead  Stain 

The  BacLight  Live/Dead  Bacterial  Viability  Kits  (Molecular 
Probes)  provide  a  rapid  assay  of  bacterial  viability  based  on  mem¬ 
brane  integrity.  The  kits  contain  two  nucleic  acid  stains:  SYTO  9, 
a  proprietary  formula;  and  propidium  iodide  (PI),  a  stain  with  an 


extensive  history  in  eukaryotic  viability  determination.  PI,  a 
charged  molecule,  does  not  go  across  polarized  cell  membranes, 
whereas  SYTO  9  crosses  all  cell  membranes.  The  basis  of  the 
viability  determination  rests  in  the  combined  application  of  the 
dyes.  Cells  that  have  intact  polarized  membranes  are  permeable 
only  to  SYTO  9,  whereas  cells  lacking  sufficient  polarization  are 
permeable  to  both  stains.  PI  has  a  higher  binding  affinity  than 
does  SYTO  9  and  therefore  preferentially  stains  the  nucleic  acid 
of  those  cells  in  which  both  stains  penetrate  (dead  or  inactive 
cells).  It  is  possible  to  use  a  single  laser  line  (488  nm)  or,  to  use 
multiple  laser  lines  from  an  argon  laser  (488,  514nm)  to  excite 
both  dyes  (SYTO  9  excitation  maximum,  480  nm;  PI  excitation 
maximum,  530 nm).  The  emission  wavelengths  are  well  separated 
(SYTO  9  maximum,  500 nm;  PI  maximum,  635).  In  this  way,  even 
single-laser  confocal  microscopes  or  epi-fluorescence  microscopes 
(480 nm  LP  filter)  can  be  used  for  detecting  both  stains.  However, 
if  a  green  helium-neon  laser  (543  nm)  is  used  together  with  the 
488  nm  line  and  sequential  excitation,  then  spectral  overlap  can  be 
greatly  reduced. 

Under  conditions  unfavorable  to  bacterial  growth,  cells  can 
display  an  intermediate  staining  in  which  PI  and  SYTO  9  fluores¬ 
cence  occur  simultaneously  and  single  cells  appear  yellow-green 
to  yellow  when  examined  through  the  oculars  with  a  fluorescein 
long-pass  filter.  Yellow  cells  are  rare  in  flowcell  experiments 
because  growth  conditions  are  generally  good  (see  Hansen  et  al., 
2000,  for  an  exception),  but  they  are  frequently  seen  with  bacteria 
stained  directly  from  soil  or  other  natural  habitats.  Figure  51.5 
shows  examples  of  typical  Live/Dead  staining  and  of  yellow  cells. 
Molecular  Probes  provides  information  on  the  precise  protocols  by 
which  the  Live/Dead  kit  is  calibrated,  as  well  as  on  the  bacteria 
for  which  these  protocols  have  been  tested.  Investigators  are  rec¬ 
ommended  to  calibrate  staining  for  their  strain-of-interest  because 
the  ratio  of  SYTO  9  to  propidium  iodide  which  provides  optimal 
results  varies  with  strain.  Because  of  this,  it  is  difficult  to  interpret 
staining  in  natural  mixed-species  populations. 

Combination  of  FP-tagged  bacteria  with  PI  staining  makes 
possible  discrimination  of  live  (FP-containing)  from  dead  (PI- 
positive)  cells.  FP  fluorescence  is  not  detectable  when  PI  is  present 
(Fig.  51.6).  It  is  also  possible  to  add  PI  directly  to  the  medium  feed 
(a  smaller  reservoir  of  medium  is  used  to  limit  the  cost  of  the  stain¬ 
ing).  The  biofilm  will  be  continuously  stained  and  viewing  can  take 
place  directly  over  time  without  additional  sample  handling. 

Fluorescence  In  Situ  Hybridization 

In  situ  hybridization  techniques  allow  for  identity  and/or  activity 
measurements  of  individual  cells.  Fluorescence  in  situ  Hybridiza¬ 
tion  (FISH)  is  typically  used  in  biofilm  research  to  provide 
spatially  resolved  taxonomic  information  and  is  normally  based  on 
small  subunit  (16S)  ribosomal  RNA  (rRNA)  gene  sequences. 
These  sequences  consist  of  regions  that  are  either  highly  conserved 
between  species  or  highly  variable.  In  most  cases,  one  can  design 
probes  that  hybridize  with  rRNA  from  only  a  single  species  or 
even  subspecies,  as  well  as  probes  that  target  a  broader  taxonomic 
unit  (genus,  kingdom).  In  this  way,  it  is  possible  not  only  to 
identify  the  organisms  but  also  to  reveal  their  precise  location 
within  the  three-dimensional  (3D)  community.  FISH  has  been 
used  extensively  in  environmental  microbiology,  enabling  investi¬ 
gators  to  map  complex  biofilm  communities  (Daims  et  al.,  2001; 
Thurnheer  et  al.,  2004).  Assuming  quantitative  labeling  (i.e.,  all 
ribosomal  rRNA  molecules  are  labeled),  it  is  possible  to  assess 
the  growth  rate  of  a  cell  because  the  amount  of  rRNA  is  correlated 
with  overall  protein  synthesis  rate.  However,  because  not  all 
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FIGURE  51.5.  Live/Dead  (Molecular  Probes,  Eugene,  OR;  catalog  no.  L7012)  staining  of  a  Streptococcus  gordonii  biolilm  after  typical  overnight  growth  in  a 
flowcell  and  of  a  S.  mutans  biofilm  after  overnight  growth  under  starvation  conditions.  Molecular  Probes  protocol  “Staining  of  bacteria  in  suspension”  was  fol¬ 
lowed.  Maximum  projection  images  of  confocally  acquired  image  stacks.  (A)  SYTO  9  channel;  .S’,  gordonii.  (B)  PI  channel;  S.  gordonii.  (C)  Overlay  of  (A)  and 
(B).  Inset  shows  3x  zoom  of  center  of  sample  for  single-cell  resolution.  (D)  Overlay  of  SYTO  9  and  PI  channels  for  S.  mutans  biofilm  showing  intermediate 
staining  of  poorly  growing  cells.  Scale  bar  in  (C)  applies  to  all  images  except  the  inset. 


ribosomes  are  actively  synthesizing  protein,  actual  growth  rate 
may  be  lower  than  the  theoretical  maximum  rate  as  determined  by 
this  assay. 

The  use  of  FISH  in  confocal  microscopy  requires  not  only  that 
the  sample  be  labeled,  but  also  that  its  3D  properties  are  retained. 
Microbial  biohlms  grown  in  flowcells  presented  a  problem  because 
the  multiple  washing  steps  involved  in  FISH  might  change  the  3D 
structure  of  the  biofilm.  Embedding  and  physical  sectioning  as 
applied  in  eukaryotic  systems  were  not  useful  because  they  gener¬ 
ated  samples  that  were  not  transparent,  were  autofluorescent,  or 
were  too  hard.  A  simple  and  effective  method  employing  acryl¬ 
amide  as  the  embedding  agent  is  described  below.  After  polymer¬ 
ization  of  the  acrylamide,  biofilm  samples  could  be  removed  from 


the  flowcells  as  intact  slabs  and  then  subjected  to  FISH  outside  the 
flowcell  with  subsequent  confocal  microscopy  (Fig.  51.7). 

General  Procedure  for  Embedding  of 
Flowcell-Grown  Biofilms  for  Fluorescence 
In  Situ  Hybridizatioin 

Details  of  this  procedure  have  been  published  (Christensen  et  al., 
1999).  Briefly,  the  biofilm  is  first  fixed  using  an  4%  aqueous  solu¬ 
tion  of  paraformaldehyde.  A  solution  of  standard  sequencing  grade 
acrylamide  (20%,  200  : 1  acrylamide  :  hw-acrylamide)  is  used,  to 
which  the  TEMED  ( N, A', N' . N'- te t ram e t h y leth y I e n e d i  a m i  n e )  and 
the  ammonium  persulfate  catalyst  are  added  just  prior  to  intro- 
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FIGURE  51.6.  Effect  of  antibiotic  (colistin)  treatment  on  Escherichia  coli  F+  biofilm.  GFP  expressing  E.  coli  cells  were  grown  in  a  flowcell  for  2  days.  Pro- 
pidium  iodide  (used  at  10|lM  final  concentration)  stains  dead  cells  red  (J.A.J.  Haagensen,  unpublished  results). 


duction  into  the  flowcell.  Catalyst  concentration  must  be  calibrated 
such  that  solidification  commences  just  after  the  flow  chamber 
becomes  filled  with  embedding  solution.  Also,  introduction  of  the 
solution  must  occur  slowly  enough  that  the  biofilm  is  not  damaged 
by  the  shear  force.  Bubbles  must  not  be  introduced  because  they 
can  both  destroy  the  biofilm’s  3D  structure  and  hinder  solidifica¬ 
tion.  Once  solidified,  the  polyacrylamide  gel  containing  the 
embedded  biofilm  is  reasonably  robust  and  the  coverglass  can  be 
removed.  Hybridization  can  be  done  on  several  samples  at  a  time 
using  standard  hybridization  slides  in  a  humidified  chamber.  Once 
hybridized,  samples  are  stable  for  up  to  several  months.  FISH  can 
also  be  performed  on  samples  containing  GFP  or  other  fluorescent 
proteins.  The  fixation  procedure  kills  the  cells  but  Aequorea  GFP 
and  derivatives  have  very  long  half  lives,  and  the  FISH  procedure 
will  allow  the  GFP  to  be  preserved.  However,  as  SDS  (a  perme- 
ableizing  agent)  will  reduce  GFP  lifetime  (unpublished  results),  it 
should  be  reduced  or  omitted  from  the  hybridization  in  FP- 
containing  samples.  FISH  efficiency  may  suffer,  but  the  FPs  will 
better  retain  their  fluorescence. 

Antibodies 

Antibody-labeled  cells  can  be  detected  microscopically  by  two 
routes.  In  primary  immunofluorescence,  also  known  as  direct 


immunofluorescence,  fluorophore  molecules  are  directly  conju¬ 
gated  to  the  antigen-targeting  IgG  molecules.  When  enough  of 
these  labeled  IgG  molecules  bind  to  a  cell,  fluorescence  is 
detectable.  Typically,  as  a  single  IgG  molecule  can  only  be  coupled 
to  between  one  and  four  fluorophore  molecules,  many  labeled  IgG 
molecules  must  bind  to  the  target  to  reach  the  fluorophore  con¬ 
centration  required  for  easy  detection  by  cameras  or  PMTs. 

In  secondary  or  indirect  immunofluorescence,  the  unlabeled 
IgG  is  detected  by  using  a  second  fluorescently  labeled  antibody 
made  against  the  animal  in  which  the  primary  antibody  was  made. 
Thus,  a  primary  antibody  made  in  a  rabbit  would  be  detected  using 
anti-rabbit  secondary  antibody  made  in,  for  example,  a  goat.  One 
advantage  of  this  approach  is  that  several  secondary  antibodies, 
each  bearing  several  fluorophores,  react  with  each  primary  anti¬ 
body.  Typically  at  least  a  3-fold  amplification  of  the  signal  is 
achieved  in  comparison  with  that  obtained  through  primary 
immunofluorescence.  Another  advantage  is  that  a  wide  range  of 
secondary  antibodies  (different  fluorophores,  different  production 
animals)  is  commercially  available.  A  third  advantage  is  that  com¬ 
mercially  prepared  secondary  antibodies  are  available  as  Fab  frag¬ 
ments,  that  is,  they  lack  the  Fc  region  that  is  recognized  by  Fc 
receptors  present  on  certain  bacteria. 

One  major  disadvantage  of  secondary  immunofluorescence  is 
that  the  use  of  more  than  one  primary  antibody  in  an  experiment 
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FIGURE  51 .7.  Fluorescence  in  situ  hybridization  (FISH)  in  combination  with  expression  of  fluorescent  protein.  CLSM  micrographs  showing  the  structural  rela¬ 
tionships  between  Acinetobacter  C6  and  P.  putida  Rl,  with  different  activity,  in  a  mixed  biofilm  consortium.  Three  days  after  inoculation,  the  biofilm  was  em¬ 
bedded  and  hybridized.  P.  putida  Rl  was  targeted  with  a  CY5-labeled  FISH  probe  (blue),  and  Acinetobacter  C6  with  a  CY3-labeled  FISH  probe  (red).  P.  putida 
Rl  cells  actively  making  ribosomes  were  monitored  via  a  degradable  GFP  expressed  from  the  rmBPl  promoter.  These  cells  are  cyan  (combination  of  green  GFP 
and  blue  FISH  probe),  xy- images  are  extended  focus  projections.  To  the  right  and  above  the  xy-images  are  virtual  vertical  sections  through  the  biofilm  collected 
at  the  positions  indicated  by  the  white  triangles.  (Reproduced  with  permission  from  Christensen  et  al.,  2002.) 


is  made  difficult.  Either  each  primary  antibody  must  be  made  in  a 
different  animal  (rabbit,  mouse,  chicken)  with  each  primary  anti¬ 
body  being  detected  with  a  different  secondary  antibody,  or  the 
sample  must  be  subjected  to  time-consuming  exhaustive  washing 
and  blocking  between  sequential  application  of  each  primary/ 
secondary  antibody  pair  to  assure  that  each  individual  primary 
antibody  is  detected  with  fidelity.  Even  meticulous  washing 
and  blocking  cannot  absolutely  exclude  migration  of  previously 
applied  secondary  antibodies  to  other  binding  sites. 

Preparation  of  Labeled  Primary  Antibodies 

For  studies  of  the  interactions  between  different  bacterial  species 
in  oral  biofilms  at  the  National  Institute  of  Dental  and  Craniofa¬ 
cial  Research  (NIDCR),  primary  immunofluorescence  is  used 
because  the  goal  is  to  identify  multiple  organisms  in  a  single 
sample:  frequently  a  complicated  undefined  natural  sample  such 
as  dental  plaque.  Polyclonal  antibodies  are  produced  commercially 
in  rabbits  from  bacterial  suspensions  we  provide.  The  suspensions 
are  phosphate-buffered-saline-washed  cells  from  overnight  culture 
and  are  standardized  at  1010  cells/mL.  The  suspension  is  mixed 
with  Freund’s  incomplete  adjuvant  and  injected  subcutaneously. 
A  booster  injection  is  given  every  week  until  a  total  of  three 
injections  have  been  made.  The  animal  is  exsanguinated  after 
about  2  months  and  the  serum  is  shipped  to  us  frozen.  The  IgG 
fraction  is  prepared  using  a  commercially  available  protein  A 


column  (Pierce);  typical  yields  range  from  1  to  3mgIgG/mL 
serum.  The  unlabeled  antibody  is  then  tested  for  cross-reactivity 
using  secondary  immunofluorescence.  Washed  cells  (about  5  x 
10s)  of  the  antigen  strain  are  used  to  determine  the  minimum  con¬ 
centration  of  primary  antibody  at  which  the  cells  are  easily  visible 
in  the  epi-fluorescence  microscope.  This  concentration  is  then  used 
to  check  for  cross-reactivity  with  other  bacterial  strains.  If  cross¬ 
reactivity  is  a  problem,  absorption  against  a  cross-reactive  strain 
is  performed  overnight  at  8°C  in  the  presence  of  protease 
inhibitors,  then  the  IgG  is  repurified  over  a  protein  A  column.  The 
final  antibody  is  then  labeled  with  Alexa  Fluor  protein  labeling  kits 
(Molecular  Probes).  Twice  the  recommended  amount  of  reactive 
dye  is  used  in  each  conjugation  because  we  have  not  often 
achieved  a  satisfactory  degree  of  labeling  using  the  standard  dye 
concentration.  In  our  conjugations,  typically  1  to  3  moles  of  Alexa 
Fluor  are  conjugated  per  mole  IgG.  Unlabeled  dye  is  removed 
using  the  column  supplied  in  the  kit,  and  typical  recovery  of 
labeled  IgG  is  90%.  The  labeled  primary  antibody  is  then  titrated 
to  determine  the  minimum  concentration  necessary  for  easy  detec¬ 
tion  of  the  antigen  strain.  With  these  Alexa  Fluor  polyclonal  con¬ 
jugates,  effective  concentrations  typically  range  between  1  and  5 
pg/mL.  Bovine  albumin  (0.1%)  is  used  as  a  blocking  agent  and  we 
have  yet  to  observe  non-specific  binding  of  antibody,  even  in 
natural  samples  from  the  oral  cavity  (Palmer  et  al.,  2003). 

In  certain  experimental  situations,  cross-reactivity  can  be 
advantageous.  If  one  wishes  to  distinguish  between  two  cell  types 
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in  a  dual-species  in  vitro  experiment,  then  it  is  only  important  that 
the  antibodies  employed  show  this  difference.  An  antibody  we 
have  prepared  to  Streptococcus  gordonii  that  cross-reacts  with 
Streptococcus  oralis  can  in  fact  be  made  specific  for  S.  gordonii 
through  absorption  against  S.  oralis.  However,  we  would  not  use 
the  specific  (absorbed)  antibody  in  an  in  vitro  experiment  con¬ 
taining  S.  gordonii  and  another  bacterium  that  is  unreactive  with 
the  S.  gordonii  antibody  because  the  less  specific  antibody  is  suf¬ 
ficient  under  these  circumstances. 

In  contrast,  when  one  examines  a  natural  population  such  as 
dental  plaque,  then  the  use  of  a  less  selective  antibody  will  temper 
claims  on  the  identity  of  antibody-reactive  organisms.  Nucleic  acid 
stains  can  be  used  to  reveal  cells  that  are  not  antibody  reactive 
(Fig.  51.8). 

Imaging  Bacteria  Without  Fluorescence 

In  certain  cases  it  may  be  necessary  to  image  unstained  bacteria 
on  the  confocal  microscope.  Many  instruments  are  equipped  for 
transmitted  light  differential  interference  contrast  (DIC)  imaging 
using  laser  light  and,  despite  the  fact  that  this  is  a  non-confocal 
technique  and  the  fact  that  many  flowcells  have  thick  bases,  good 
transmitted  light  images  can  often  still  be  obtained  [Fig.  51.9(A)]. 
Another  possibility  is  reflected  light  confocal  imaging,  more  prop¬ 
erly  known  as  backscattered  light  (BSL)  imaging.  In  this  approach, 
the  primary  dichroic  mirror  is  replaced  with  a  neutral  beam¬ 
splitter  that  transmits  the  laser  light  scattered  by  the  specimen  back 
to  the  detector.  Typically  the  488  nm  laser  line  is  used  but  other 
lines  work  equally  well.  Figure  51.9(B)  shows  a  maximum  pro¬ 
jection  of  a  30  [tm  thick  confocal  stack  through  a  two-species 
biofilm  containing  streptococci  (round  cells)  and  fusobacteria 
(long  thin  rods).  The  inset  shows  a  single  section  taken  20  pm  away 
from  the  coverslip.  BSL  imaging  is  an  approach  with  confocal 
resolution  applicable  to  thick  non-fluorescent  samples. 


IMAGING  EXTRACELLULAR  POLYMERIC 
SUBSTANCES  IN  BIOFILMS 

Microbial  biofilms  are  composed  not  only  of  cells  but  also  of  the 
extracellular  polymeric  substances  (EPS)  produced  by  the  microor¬ 
ganisms  (Wingender,  1999).  The  major  constituents  of  EPS  are  pro¬ 
teins,  polysaccharides,  nucleic  acids,  and  amphiphilic  polymeric 
compounds  such  as  lipopolysaccharides  and  emulsants.  A  multi¬ 
tude  of  structural  and  functional  roles  have  been  ascribed  to  EPS 
(Wolfaardt  et  al.,  1999)  but  its  characterization  is  difficult;  never¬ 
theless  two  major  approaches  have  been  taken.  The  first  approach 
involves  extraction  and  isolation  of  polymers  with  subsequent 
chemical  analysis  (Nielsen  and  Jahn,  1999).  Several  variations  of 
this  destructive  approach  exist  but  they  are  applicable  only  in  pure 
culture  studies  where  one  or  a  few  extracellular  polymers  are 
present.  However,  as  environmental  biofilm  systems  contain 
numerous  types  of  different  bacteria  and  therefore  also  contain 
numerous  types  of  polymers  (Staudt  et  al.,  2003),  chemical  analy¬ 
ses  are  often  presented  as  a  sum  parameter  (e.g.,  percentage  car¬ 
bohydrate,  protein,  etc.)  and  detailed  information  on  a  single 
polymer  class  or  individual  polymers  is  often  difficult  to  assess. 

The  second  approach  is  an  in  situ  staining  technique  employ¬ 
ing  specific  fluorescently  labeled  probes  in  combination  with 
laser-scanning  microscopy  (Lawrence  and  Neu,  1999;  Neu  and 
Lawrence,  2002).  The  approach  takes  advantage  of  the  carbohy¬ 
drate  specificity  of  lectins  that  bind  to  the  glycoconjugates  found 
in  the  biofilm  (Neu  and  Lawrence,  1999a,  1999b).  Lectin-binding 
analysis  of  biofilms  has  been  critically  assessed  (Neu  et  al.,  2001) 
and  70  commercially  available  lectins  have  been  tested  on  certain 
types  of  biofilms  (Staudt  et  al.,  2003).  Lectin  staining  may  be  com¬ 
bined  with  other  stains  such  as  general  nucleic  acid  stains,  specific 
gene  probes  (Bockelmann  et  al.,  2002)  or  probes  characterizing 
the  microhabitats  in  biofilms  (Lawrence  et  al.,  submitted). 


FIGURE  51.8.  Eight-hour-old  oral  biofilm  community  on  a  human  tooth  enamel  chip  retrieved  from  the  human  mouth.  Combination  of  primary  immunofluo¬ 
rescence  and  nucleic  acid  staining  demonstrate  interactions  between  members  of  this  undefined  natural  community.  Green  cells  are  reactive  with  AlexaFluor 
488-conjugated  anti -Streptococcus  gordonii.  Red  cells  are  reactive  with  AlexaFluor  546-conjugated  anti-streptococcal-receptor-polysaccharide.  Blue  cells  are 
stained  with  the  nucleic-acid-binding  SYTO  59. 
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FIGURE  51.9.  (A)  Transmitted  laser  light  image  collected  through  the  flow  chamber  shown  in  Figure  51.2.  (B)  Stereo-pair  maximum  projection  image  of  a 
30pm  thick  confocal  stack  collected  using  backscattered  light.  The  biofilm  is  composed  of  streptococci  (round  cells)  and  fusobacteria  (long  thin  rods).  (C)  shows 
a  magnified  image  from  the  upper  central  part  of  (B).  The  color  panel  is  an  anaglyph  stereo-pair  of  (B).  Use  red-green  glasses  to  view. 


Using  digital  image  analysis,  the  glycoconjugate  signal  can  be 
quantified  (Staudt  et  al.,  2004)  and,  after  object  definition,  the 
signals  of  cells  and  polymers  can  be  related  to  each  other  (Neu 
et  al.,  submitted).  In  contrast  to  immunological  techniques  requir¬ 
ing  the  isolation  of  polymers  and  subsequent  production  of  specific 
antibodies,  lectin-binding  analysis  can  be  applied  immediately  to 
environmental  biofilm  samples.  In  summary,  lectin-binding  analy¬ 
sis  allows  (1)  in  situ  visualization  and  quantification  of  glyco- 
conjugates  within  biofilms,  (2)  detection  of  cell-surface-bound, 
microcolony-related,  and  cell-free  glycoconjugates,  and  (3)  differ¬ 
entiation  of  various  types  of  glycoconjugates  in  microbial  biofilms. 
Clearly,  lectin-binding  analysis  serves  as  a  valuable  tool  for  exam¬ 
ining  the  polymer  matrix  of  microbial  biofilm  systems. 

Examples  of  lectin-binding  analysis  in  biofilms  developed 
from  activated  sludge  are  shown  in  Figure  51.10.  The  first  images 
(A-C)  show  the  principle  of  lectin-binding  analysis  in  a  chemo- 
autotrophic  biofilm.  The  biofilm  was  first  stained  with  Aleuria 
aurantia  —  AlexaFluor  488-conjugated  lectin  and  then  counter- 


stained  with  SYTO  60.  The  gray  scale  images  (A,B)  show  the 
single-channel  signal  of  each  stain;  however,  the  color  overlay  (C) 
reveals  the  true  architecture  of  the  biofilm  consisting  not  only  of 
cellular  signals  but  to  a  large  degree  of  glycoconjugates.  The 
sensitivity  of  lectin-binding  analysis  is  also  shown  in  a  series  of 
three  images  made  by  zooming  into  the  biofilm,  starting  with  an 
overview  (D),  via  the  microcolony  towards  the  single  bacterial  cell 
(E,F).  In  the  overview,  it  is  obvious  how  the  microbial  biofilm 
developed  in  the  direction  of  flow  (top  to  bottom  in  the  figure)  and 
how  it  is  embedded  in  a  thick  layer  of  glycoconjugates  (D).  This 
sample  was  stained  with  two  different  lectins  ( Solanum  tuberosum 
—  CY5,  Arachis  hypogaea  —  TRITC)  and  counterstained  with 
SYTO  9.  The  next  image  (E)  shows  a  microcolony  with  a  group 
of  bacteria  embedded  in  extracellular  glycoconjugates  forming  a 
capsular  sheath  around  the  cells.  The  biofilm  microcolony  was 
stained  with  AlexaFluor  488-conjugated  Solanum  tuberosum  lectin 
and  with  SYTO  60.  At  the  cellular  level,  it  is  possible  to  use  lectins 
to  distinguish  between  various  regions  on  the  bacterial  cell  surface 
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FIGURE  51.10.  Lectin-binding  analysis  in  bacterial 
biofilms.  (A-C)  Principle  of  glycoconjugate  (lectin) 
staining  in  combination  with  nucleic  acid  staining.  (A) 
Maximum  intensity  projection  of  a  biofilm  showing 
nucleic  acid  staining  (SYTO  60,  red).  (B)  AlexaFluor 
488-conjugated  Aleuria  lectin  staining  (green)  and  (C) 
the  resulting  overlay.  Single-photon  excitation/emis¬ 
sion:  488/500-550  nm  and  633/650-800  nm.  (D-F) 
Zooming  into  a  biofilm  sample  showing  different  pat¬ 
terns  of  lectin  staining.  (D)  Maximum  intensity  projec¬ 
tion  as  an  overview  with  two  lectins  (Cy5-conjugated 
Solatium  tuberosum ,  blue;  TRITC-conjugated  Arachis 
hypogaea,  red)  and  nucleic  acid  staining  (SYTO  9, 
green).  Colocalization  of  the  two  lectins  appears  pink. 
Single-photon  excitation/emission:  488/500-550nm, 
568/590-625,  and  633/650-800  nm.  (E)  Microcolony 
stained  with  AlexaFluor  488-conjugated  Solatium  lectin 
(green)  and  SYTO  60  (red).  Single-photon  excitation/ 
emission:  488/500-550  nm  and  633/650-800  nm.  Single 
bacterial  cells  showing  a  specific  glycoconjugate 
banding  pattern  after  lectin  staining  (green)  and  nucleic 
acid  staining  (SYTO  60,  red).  Excitation/emission  as 
in  (D). 


having  a  different  glycoconjugate  makeup  (F).  In  this  case,  the 
biofilm  sample  was  stained  with  AlexaFluor  488-conjugated 
Aleuria  aurantia  and  SYTO  60. 

In  the  natural  environment,  most  biofilms  are  routinely 
exposed  to  light.  These  phototrophic  biofilms  consist  of  hetero- 
trophic  and  chemo-autotrophic  bacteria  as  well  as  algae  and 
cyanobacteria.  The  latter  two  organisms  show  autofluorescence 
due  to  their  photosynthetic  pigments.  In  the  example  shown  as  an 
isosurface  projection  [Fig.  51.11(A)],  the  bacteria  are  stained  with 
SYTO  44  and  imaged  simultaneously  with  the  autofluorescence  in 
the  red  and  far  red  emission  channel.  The  different  makeup  of  the 
pigments  allows  a  separation  of  cyanobacterial  signal  (phycoery- 
thrins/phycocyanins  as  well  as  chlorophyll  a)  from  algal  signal 
(chlorophyll  a  only)  by  colocalization  (Neu  et  al.,  2004b).  Lectin¬ 


binding  analysis  may  also  be  applied  to  phototrophic  biofilms. 
Lectin-staining  using  AlexaFluor  488-conjugated  Aleuria  aurantia 
lectin  in  combination  with  autofluorescence  is  demonstrated  in  a 
three-channel  shadow  projection  [Fig.  51.11(B)].  The  image  shows 
the  distribution  of  algal  cells  (mainly  diatoms)  and  cyanobacterial 
cells.  Lectin  is  present  as  a  surface  coating  on  diatoms  as  well  as 
in  separate  glycoconjugate  clusters  in  between  the  phototrophic 
cells.  The  xyz-projection  [Fig.  51.11(C)]  demonstrates  a  four- 
channel  projection  of  reflection,  cyanobacterial,  and  algal  auto¬ 
fluorescences  in  the  red  and  far  red,  as  well  as  lectin  staining 
(AlexaFluor  488-conjugated  Iberis  amara  lectin).  Using  colocal¬ 
ization,  the  cyanobacterial  signal  in  the  red  channel  and 
far  red  channel  can  be  separated  from  the  algal  signal  that  is 
found  only  in  the  far  red  channel.  The  strong  lectin  signal  of 
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FIGURE  51.11.  Lectin-binding  analysis  in  phototrophic  biofilms.  (A)  Iso¬ 
surface  projection  of  a  thin  biofilm  (22  pm)  showing  algal  autofluorescence 
(blue),  cyanobacterial  autofluorescence  (pink),  and  SYTO  44  (green).  Grid  size 
=  10pm.  Single-photon  excitation/emission:  458/470-540 nm,  568/590-625, 
and  633/650-800  nm.  (B)  Shadow  projection  of  a  thick  biofilm  (220  pm)  after 
lectin  staining.  Cyanobacterial  autofluorescence  (pink),  algal  autofluorescence 
(blue),  AlexaFluor  488-conjugated  Aleuria  lectin  (green).  Single-photon 
excitation/emission:  488/500-550  nm,  568/585-625,  and  633/650-800 nm.  (C) 
Extended  xyz-projection  of  reflection  and  fluorescence  signals,  biofilm  thick¬ 
ness  =  ~45  pm.  Color  allocation:  cyanobacteria,  pink;  algae,  blue;  lberis  lectin, 
red.  Single-photon  excitation/emission:  488/480-495  nm,  568/590-625,  and 
633/650-800  nm. 


gly coconjugates  is  also  present  in  the  red  channel  due  to  the  cross¬ 
talk  between  the  green  and  red  channels. 


APPLICATION  OF  TWO-PHOTON 
LASER-SCANNING  MICROSCOPY  FOR 
BIOFILM  ANALYSIS 

Since  the  first  experimental  proof  of  two-photon  laser-scanning 
microscopy  (2PLSM)  by  Denk  and  colleagues  (Denk  et  al.,  1990) 
(see  also  Chapter  28,  this  volume)  and  the  many  applications  in 
cell  biology  (Konig,  2000),  2PLSM  has  found  its  way  into  several 
microbiological  laboratories.  Nevertheless,  few  published  exam¬ 
ples  of  2PLSM  biofilm  research  exist.  Benefits  of  2PLSM  for  the 
examination  of  biofilm  structures  are  obvious  as  microbial  biofilms 
form  extended  3D  structures  that  can  be  up  to  several  hundred 
micrometers  in  thickness.  As  a  result  of  microbial  activity,  biofilms 
show  gradients  of,  for  example,  oxygen  and  pH,  and  they  form 
layered  structures  in  which  different  cellular  and  polymeric  con¬ 
stituents  are  organized  in  three  dimensions. 

As  the  highly  complex  structure  of  environmental  biofilms 
makes  them  prone  to  scattering  (excitation  light  as  well  as  emis¬ 
sion  light),  the  longer  wavelengths  used  in  2PLSM  should  make 
this  technique  ideal  for  collecting  highly  resolved  images  at  deeper 
levels  within  microbial  biofilms.  The  first  application  of  2PLSM 
in  microbiology  (Vroom  et  al.,  1999)  demonstrated  this  capability 
deep  within  model  oral  biofilms. 

One  technical  issue  in  2PLSM  is  the  applicability  of  common 
fluorochromes  and  whether  multiple  fluors  can  be  excited  indi¬ 
vidually,  as  in  one-photon  laser-scanning  microscopy.  Neu  et  al. 
(2002)  addressed  2P  excitation  of  fluorochromes  specific  for  cel¬ 
lular  and  polymeric  biofilm  constituents  and  reported  the  optimal 
IR  excitation  wavelengths,  as  well  as  providing  the  first  multi¬ 
channel  2P  images  of  microbial  biofilm  systems.  In  this  study, 
fluorochromes  and/or  autofluorescence  was  excited  at  one  wave¬ 
length  and  the  emission  signals  detected  in  two  or  three  different 
channels.  This  approach  has  been  taken  one  step  further  in  a 
methodological  report  on  the  differentiation  of  photo-autotrophic 
biofilm  constituents,  such  as  cyanobacteria  and  algae,  using 
two-photon  excitation  (Neu  et  al.,  2004b).  Due  to  the  strong 
autofluorescence  signals  of  such  phototrophic  biofilms,  it  is  rela¬ 
tively  easy  to  collect  multi-channel  signals  from  deep  levels  within 
them. 

A  comparison  of  single-photon  and  two-photon  laser-scanning 
microscopy  of  biofilms  is  presented  in  Figure  51.12.  EDTA- 
degrading  biofilms  were  grown  on  carrier  material  (sand  and 
pumice)  in  a  fluidized-bed  reactor  under  shear  force.  The  extremely 
dense  biofilm  was  stained  with  the  SYTO  40.  The  xz-scan  [Fig. 
51.12(A)]  shows  blurred  signal  in  the  single-photon  image, 
whereas  the  two-photon  image  [Fig.  51.12(B)]  yields  higher  reso¬ 
lution  of  the  cells.  The  EDTA  biofilm  was  also  examined  using  the 
nucleic  acid  stain  SYBR  Green  (Molecular  Probes).  The  xy-images 
at  a  depth  of  14  pm  from  the  coverglass  side  of  the  biofilm  also 
demonstrate  the  lower  intensity  and  lower  resolution  of  the  single¬ 
photon  image  [Fig.  51.12(C)]  compared  to  the  two-photon  image 
[Fig.  51.1 2(D)] .  The  frame  indicates  the  cropped  section  which  was 
used  for  3D  projection.  The  3D  images  are  shown  as  a  comparison 
of  single-photon  [Fig.  51.12(E)]  and  two-photon  excitation  [Fig. 
51.12(F)],  and  again  demonstrate  the  higher  resolution  using  two- 
photon  excitation.  The  higher  signal  intensity  and  higher  resolution 
using  two-photon  excitation  may  be  due  to  the  higher  excitation 
photon  flux  resulting  in  increased  signal  from  the  fluorochrome, 
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FIGURE  51 .1 2.  Comparison  of  single-photon  and  two- 
photon  imaging  in  dense  biofilms  stained  with  SYTO  40. 
(A)  xz-scan  with  single-photon  excitation/emission: 
488/500-550  nm.  (B)  Two-photon  excitation/emission: 
840/400-750  nm.  (C,D)  xy-scans  after  staining  with 
SYBRGreen  at  the  same  depth  (14pm).  (C)  Single¬ 
photon  excitation/emission:  488/500-630  nm  (D)  two- 
photon  excitation/emission:  860/400-750  nm.  (E,F) 
Three-dimensional  projections  of  the  framed  area  in  C 
and  D.  (E)  Single-photon  excitation/emission:  488/500- 
630  nm.  (F)  Two-photon  excitation/emission:  860/400- 
750  nm.  Grid  size  is  5  pm.  See  also  3D  reconstructions 
on  Handbook  http://www.springer.com/0-387-25921-X. 


reduced  non-specific  signal  from  autofluorescence  away  from  the 
focal  plane  and  reduced  scattering. 

Due  to  the  wide  excitation  cross-section  of  most  fluorochromes 
for  two-photon  excitation,  it  is  possible  to  use  a  single  wavelength 
to  excite  two  fluorochromes  and  subsequently  record  the  emission 
in  two  channels.  Two  examples  showing  biofilms  stained  with  a 
lectin  and  with  a  nucleic-acid  stain  are  presented.  The  first  example 
shows  a  biofilm  stained  with  TRITC-conjugated  Canavalia  ensi- 
formis  lectin  and  SYTO  9  [Fig.  51.13(A)],  The  glycoconjugates  in 
this  image  are  mainly  located  in  large  clusters.  The  second 
example  shows  a  biofilm  stained  with  AlexaFluor  488-conjugated 
Aleuria  aurantia  lectin  and  SYTO  40  [Fig.  51.13(B)].  Flere  the 
lectins  are  mainly  staining  the  cell  surface  of  protozoa  and  the 
capsule  of  certain  bacteria.  The  third  example  is  a  phototrophic 
biofilm  in  which  the  autofluorescence  of  the  phototrophic  pigments 


from  algae  and  cyanobacteria  were  collected  in  the  red  and  far  red 
channel  [Fig.  51.13(C)].  One  advantage  of  using  a  single  beam  for 
exciting  two  fluorophores  is  that  one  can  be  sure  that,  regardless 
of  any  possible  effects  of  chromatic  aberration,  the  signal  detected 
in  both  channels  at  a  given  instant,  originated  from  the  same  loca¬ 
tion  in  the  cell. 

Because  it  uses  a  pulsed  laser,  2PLSM  can  also  be  employed 
to  measure  fluorescence  lifetimes  (see  Chapter  27,  this  volume). 
Two-photon  intensity  imaging  and  two-photon  lifetime  imaging 
were  discussed  recently  as  a  useful  tool  for  studying  the  microbial 
ecology  of  biofilm  systems  (Neu,  2004a).  One  example  of  fluo¬ 
rescence  lifetime  imaging  (FLIM)  is  the  measurement  of  pH  gra¬ 
dients  in  biofilms  made  possible  because  the  fluorescence  lifetime 
of  some  dyes  varies  with  pH  (Vroom  et  al.,  1999).  Another 
example  is  the  differentiation  of  starving,  slow-growing,  and  fast- 
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FIGURE  51.13.  Dual-channel  imaging  of  biofilms  after  two-photon  excita¬ 
tion,  presented  as  maximum  intensity  projection.  (A)  Staining  with  TRITC- 
conjugated  Canavalia  ensiformis  lectin  (red)  and  SYTO  9  (green).  Two-photon 
excitation/emission:  800/480-550  and  600-700  nm.  (B)  Staining  with  Alexa- 
Fluor  488-conjugated  Aleuria  aurantia  lectin  (green)  and  SYTO  40  (blue). 
Two-photon  excitation/emission:  800/425-500  and  500-600  nm.  (C)  Pho- 
totrophic  biofilm  showing  autofluorescence  of  pigments.  Cyanobacterial  pig¬ 
ments  (pink),  algal  pigments  (blue).  Two-photon  excitation/emission: 
800/580-640  and  650-800  nm. 


growing  bacteria  in  microbial  biofilms.  By  using  the  nucleic-acid 
stain  SYTO  13,  bacterial  activity  can  be  assessed  (Neu,  2004a). 
Although  FLIM  is  still  a  new  technique  in  biofilms  (Neu  and 
Lawrence,  2004),  two-photon  lifetime  imaging,  in  combination 
with  advanced  digital  image  analysis,  seems  about  to  open  up  new 
ways  of  studying  them. 


Limitations  of  Confocal  Laser  Scanning 
Microscopy  and  Two-Photon  Laser-Scanning 
Microscopy  in  Biofilm  Analysis 

Some  of  the  limitations  of  LSM  for  biofilm  examination  should  be 
discussed.  Microbial  biofilms  in  environmental  habitats  can  be 
several  millimeters  in  thickness  and  may  incorporate  non-cellular 
matter  such  as  detritus  and  sand  or  clay.  An  extreme  example 
are  microbial  mats  that  can  be  centimeters  thick  (Stal,  2000). 
Examination  of  biofilms  with  such  scattering  properties  and 
spatial  dimensions  are  a  challenge  for  both  CLSM  and  2PLSM.  In 
fact,  penetration  of  the  excitation  beam  becomes  the  limiting 
factor  (Barranguet  et  al.,  2004)  and  even  the  deeper  penetration 
of  infrared  (2PLSM)  does  not  help  for  scattering  samples. 
Consequently,  thick  biofilms  must  be  embedded,  physically  sec¬ 
tioned,  stained,  and  then  examined  (Yu  et  al.,  1994).  If  this 
approach  is  used,  specific  software  routines  must  be  developed  in 
order  to  analyze  the  distribution  of  the  fluorescence  signals  across 
all  the  sections  from  the  base  to  the  top  of  the  biofilm.  If  biofilms 
are  reasonably  solid  or  developed  under  high  shear  force,  they  can 
be  easily  embedded  in  OCT  medium  (Tissue-Tek,  Miles  Inc., 
USA)  frozen,  and  subsequently  cryosectioned  into  relatively 
thick  slices  (50  or  100|im).  After  sectioning,  the  biofilm  slices 
are  stained  and  may  be  mounted  on  slides  and  examined  as 
coverslipped  preparations.  The  same  procedure  is  applicable  to 
mobile  biofilm  systems  such  as  solid  sludge  granules  or  fungal 
pellets. 

Examples  of  sectioned  biofilms  appear  in  Figure  51.14.  In  the 
first  example,  the  biofilm  was  grown  on  pumice  in  a  fluidized  bed 
reactor  under  high  shear  force.  The  dense  biofilm  was  directly  sec¬ 
tioned  and  a  section  was  stained  with  AlexaFluor  488-conjugated 
Aleuria  aurantia  lectin  and  SYTO  60.  Due  to  the  thickness  of  the 
original  sample,  the  section  was  so  large  that  five  slightly  over¬ 
lapping,  tiled,  3D  data  stacks  were  collected  by  laser-scanning 
microscopy  and  assembled  in  Photoshop  to  show  the  full  thickness 
of  the  biofilm  [Fig.  51.14(A)],  The  second  example  of  a  photo- 
trophic  biofilm  was  prepared  by  cryosectioning  and  staining  with 
FITC-conjugated  Triticum  vulgaris  lectin.  As  the  brightness  of  five 
image  stacks  was  adjusted  to  view  the  bright  autofluorescence 
signal  of  cyanobacteria,  the  weaker  signal  of  the  green  algae  could 
not  be  visualized  [Fig.  51.14(B)],  Nevertheless,  the  distribution  of 
the  glyconjugates  forming  the  polymeric  matrix  of  the  pho- 
totrophic  biofilm  is  obvious  in  both  images. 

Another  two-photon  limitation  is  the  strong  absorption  of 
infrared  (IR)  light  by  some  plastic  materials.  As  discussed  else¬ 
where  in  this  chapter,  in  vitro  biofilm  research  frequently  uses 
plastic  materials  as  a  substratum  for  bacterial  attachment  and 
growth.  The  materials  include  polystyrene,  polycarbonate,  Teflon, 
and  silicone  rubber.  Regardless  of  the  reason  for  using  plastics,  if 
biofilms  developed  on  plastic  materials  are  examined  close  to  the 
substratum,  then  the  strong  IR  absorption  results  in  local  heating 
effects  with  subsequent  melting  of  the  plastic  material  and  destruc¬ 
tion  of  the  biofilm  (Neu  and  Fawrence,  2004).  Using  substrates 
that  do  not  absorb  in  the  IR,  such  as  glass,  solves  the  problem. 
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FIGURE  51.14.  Thick  environmental  biofilms  imaged  after  sectioning.  (A) 
Bacterial  biofilms,  developed  on  carrier  material  and  stained  with  AlexaFluor 
488-conj ugated  Aleuria  aurantia  lectin  (green)  and  SYTO  60  (red)  were  man¬ 
ually  sectioned  with  a  scalpel.  The  side  of  the  biofilm  attached  to  the  substra¬ 
tum  is  at  the  bottom,  the  interface  of  the  biofilm  with  the  bulk  liquid  is  at  the 
top.  Five  tiled  3D  stacks  were  recorded  using  single-photon  excitation/ 
emission:  488/500-550 nm  and  633/650-800  nm,  then  projected,  and  assem¬ 
bled  in  Photoshop.  (B)  Phototrophic  biofilm  sectioned  on  a  cryotome.  Differ¬ 
ences  in  autofluorescence  intensity  allowed  the  detector  to  be  adjusted  to  image 
cyanobacteria  only;  algal  signal  is  not  visible.  The  side  of  the  biofilm  attached 
to  the  substratum  is  at  the  bottom  of  the  image,  the  biofilm  water/medium 
interface  is  at  the  top  of  the  image.  Cyanobacterial  autofluorescence  (pink), 
FITC-conjugated  Triticum  vulgaris  lectin  (green).  Single-photon  excitation/ 
emission:  488/500-550 nm,  568/585-625  and  633/650-800 nm. 


TEMPORAL  EXPERIMENTS 

Because  biofilms  are  not  static  entities,  imaging  approaches  that 
maintain  a  temporal  perspective  are  preferred  to  those  that  deliver 
only  single-timepoint  data.  In  temporal  studies,  one  can  see  single 
cells  attach  to  the  substratum,  develop  into  microcolonies,  and  then 
into  mature  biofilms.  Even  in  so-called  mature  biofilms,  a  dynamic 
turnover  occurs. 

Although  experimental  biofilm  biology  does  not  yet  empha¬ 
size  measurements  of  rapid  processes,  that  is,  those  operating  on 
time  scales  of  milliseconds  or  seconds;  high-speed,  point-scanning 
and  disk-scanning  confocal  microscopy  have  yet  to  be  appropri¬ 
ately  applied  to  biofilm  research.  The  most  rapid  events  typically 
measured  in  a  biofilm  occur  over  the  course  of  minutes  and  thus 
are  well  within  the  range  measurable  using  even  the  slowest  single¬ 
beam  confocal  microscope.  The  most  commonly  measured  tem¬ 
poral  parameters  are  changes  in  overall  architecture  and  changes 
in  the  positioning  of  cells  relative  to  one  another.  Two  strategies 
are  used  to  address  these  temporal  measurements.  One  strategy  is 
to  align  flow  chambers  in  series  and  then  to  remove  them  sequen¬ 
tially  for  examination.  In  this  strategy,  it  is  crucial  that  all  the 
chambers  in  a  series  receive  the  identical  inoculum  at  the  same 
time;  this  is  accomplished  simply  by  introducing  a  volume  that  is 
twice  that  of  the  chambers  plus  any  intervening  tubing.  Also,  one 
must  be  confident  that  the  metabolic  activity  in  downstream  cham¬ 
bers  is  similar  to  that  in  upstream  chambers,  that  is,  that  the  quan¬ 
tity  and  quality  of  nutrients  in  the  most  downstream  chamber  is 
equivalent  to  that  in  the  upstream  chamber. 

A  series  approach  is  normally  employed  when  the  staining 
methods  employ  chemicals,  such  as  Acridine  Orange  or  DAPI, 
thought  of  as  toxic  to  the  cells.  Prior  to  staining  with  such  chem¬ 
icals,  the  flowcell  is  removed  from  the  system  because  the  biofilm 
is  not  expected  to  continue  normal  growth. 

A  second  strategy  is  to  use  non-toxic  biological  macromole¬ 
cules  produced  by  the  bacterial  cells  as  fluorescent  labels;  mole¬ 
cules  such  as  fluorescent  proteins.  In  this  approach,  it  is  assumed 
that  imaging  of,  for  example,  GFP-labeled  bacteria  does  not  harm 
the  cells  to  the  extent  that  they  no  longer  grow  normally.  In  this 
case,  a  single  flow  chamber  can  be  imaged  repeatedly  over  the 
course  of  an  experiment  and  can  even  be  left  connected  to  the 
pump  and  medium  reservoir  while  being  imaged.  Imaging 
conducted  in  this  manner  is  truly  temporal,  whereas  the  series 
approach  relies  on  replicate  samples  being  sacrificed  at  different 
timepoints.  Another  truly  temporal  imaging  approach  is  video 
microscopy;  a  technique  that  has  been  used  in  a  time-lapse  and  in 
a  real-time  approach  to  record  rapid  processes  such  as  cell  migra¬ 
tion  along  the  substratum  and  motion  of  biofilm  streamers 
(Klapper  el  al.,  2002).  Heretofore,  video  techniques  have  been 
dependent  upon  transmitted  light  microscopy  and  therefore  com¬ 
patible  only  with  some  flowcell  designs.  It  remains  to  be  seen  what 
can  be  learned  when  biofilms  growing  in  flow  chambers  are 
viewed  in  real-time  with  the  emerging  generation  of  high-frame- 
rate  confocal  microscopes. 

Time-Lapse  Confocal  Imaging 

Modern  confocal  microscopes  have  the  ability  to  automatically 
take  images  or  stacks  of  images  at  set  time  intervals.  The  most 
advanced  systems  include  a  motorized  computer-controlled  xy- 
stage  to  position  the  sample.  In  a  complex  setup,  a  time-lapse 
experiment  can  collect  3D  data  at  several  xy-positions,  using 
varied  settings  of  light  sources,  filter  sets,  number  of  slices,  and 
time  interval  between  samplings.  Some  systems  even  have  soft- 
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ware  auto-focus.  Special  features  such  as  regional  bleaching  can 
typically  also  be  controlled  from  the  software  while  performing 
time-lapse  experiments.  During  lengthy  time-lapse  experiments, 
temperature  control  is  an  important  consideration  and  microscopes 
can  be  fitted  with  thermostat-controlled  sample  holders  and  objec¬ 
tive  heaters  to  avoid  temperature- variation-induced  focus  shifts  as 
well  as  deleterious  biological  effects. 

Confocal  time-lapse  recordings  were  used  to  investigate 
the  spatiotemporal  development  of  flowcell-grown,  monospecies 
biofilms  as  well  as  of  communities  that  contain  mutants  introduced 
to  investigate  single-gene  influences  on  structure  over  time.  In  one 
example,  structural  development  and  organization  of  a  P.  aerugi¬ 
nosa  biofilm  was  followed  using  time-lapse  recording.  The 
end-stage  biofilm  is  shown  in  Figure  51.15,  and  the  development 
stages  are  shown  in  Figure  51.16.  A  movie  made  from  this  data 
will  be  available  on  the  Handbook  http://www.springer.com/ 
0-387-25921-X.  After  inoculation,  the  flow  cell  was  mounted 
on  the  xy-table  on  the  confocal  microscope.  The  focus  plane  at  the 
substratum,  where  the  cells  had  attached  to  the  glass  surface,  was 
determined  and  read  into  the  software.  Care  was  taken  to  allow  for 
mechanical  drift  over  the  duration  of  the  experiment  by  setting  the 
“near”  endpoint  a  bit  outside  below  the  substratum  plane.  The  far 
endpoint  is  set  to  allow  for  the  growth  of  the  sample  (this  point  can 
sometimes  be  adjusted  during  the  time  series).  Finally  the  number 
of  image  slices  of  the  defined  area  is  set,  together  with  number  of 
repetitions  and  time  intervals  for  the  time-lapse  recording. 


FIGURE  51.15.  Mushroom-shaped  multi-cellular  biohlm  structures  with 
yellow  caps  and  cyan  stalks  after  4  days  of  development.  (Reproduced  with 
permission  from  Klausen  et  al.,  2003.) 


FIGURE  51.16.  Time-lapse  recording  of  microcolony  formation.  (A)  Confocal  micrographs  showing  biofilm  development  initiated  with  a  1:1  mixture  of  YFP- 
expressing  P.  aeruginosa  PAOl  wild-type  and  a  non-motile  pilA  mutant  of  PAOl  expressing  CFP.  See  Klausen  and  colleagues  (2003)  for  experimental  details. 
Images  are  3D  projections  from  a  time-lapse  recording  during  the  first  64h  after  inoculation.  The  pilA  mutant  (blue)  was  preferentially  located  in  the  stalk  of 
the  characteristic  mushroom-shaped  structures  formed  by  this  Pseudomonas  strain.  Over  time,  the  motile  wild-type  mounted  the  pilA  mutant  structure,  forming 
a  cap  composed  of  the  migratory  wild-type  cells.  Timepoints  measured  in  hours  from  inoculation.  Top,  left  to  right:  10,  14,  17,  and  27  h.  Bottom:  35,  42,  59, 
and  64  hours. 
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SUMMARY  AND  FUTURE  DIRECTIONS 

•  Laser- scanning  microscopy  is  the  only  way  to  obtain  spa- 
tiotemporally  resolved  information  on  community  structure 
and  interactions  in  complex  microbial  communities. 

•  Fluorescence  in  situ  hybridization  (FISH)  and  variations  such 
as  FISH-MAR,  CARD-FISH,  or  combined  mRNA/rRNA 
FISH  (Pernthaler  and  Amann,  2004)  provide  new  directions 
for  examination  of  bacterial  identity  and  activity  within  micro¬ 
bial  communities. 

•  A  variety  of  reporter  systems  exist,  especially  fluorescent  pro¬ 
teins  and  their  degradable  variants,  which  are  useful  for  studies 
of  gene  expression  under  controlled  conditions. 

•  Lectin-binding  analysis  represents  the  only  in  situ  approach 
that  allows  simultaneous  visualization,  quantification,  and 
especially  differentiation  of  EPS  glycoconjugates  in  microbial 
biofilms. 

•  Two-photon  laser-scanning  microscopy  is  an  emerging  tech¬ 
nique  in  biofilm  research  with  potential  for  intensity  and 
lifetime  imaging. 

•  Limitations  of  laser-scanning  microscopy  (single-  or  multi¬ 
photon)  for  biofilm  examination  relate  only  to  the  sample 
parameters  (size,  thickness,  density).  These  are  the  most  com¬ 
mon  limitations  encountered  when  examining  natural  biofilms. 

•  New  strategies  in  probing  biofilm  systems  may  allow  us  to 
examine  matrix  components  in  greater  detail,  and  to  measure 
biofilm  metabolic  processes  in  real  time.  While  much  has  been 
accomplished  in  this  regard,  much  remains  to  be  done.  A  few 
pioneers  are  needed  to  begin  these  explorations.  Advanced 
microscopic  approaches  will  be  crucial  tools  in  advancing  our 
understanding  of  bacterial  biofilm  biology. 
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This  is  an  update  of  the  selected  bibliography,  as  of  early  1994,  of 
accessible  papers  on  confocal  microscopy.  In  the  10  years  since,  the 
number  of  papers  has  exploded,  in  part  because  many  of  us  publish 
in  little  increments  rather  than  a  single  summarizing  paper.  So  some 
of  the  papers  listed  are  nibbles  at  the  archive,  and  will  have  to  lead 
you  to  other  papers  on  almost  the  same  topic  by  the  same  author. 
Not  included  are  conference  reports  and  other  documents  not  likely 
to  be  available  in  most  technical  libraries.  I  have  tried  to  include  a 
note  as  to  the  content  —  often  taken  from  the  abstract. 

I  have  done  some  sorting  into  categories,  but  there  is  a  lot  of 
overlap.  CM  is  my  abbreviation  for  confocal  microscope. 
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Inoue,  S.,  1986,  Video  Microscopy,  Plenum  Press,  New  York.  A  very  complete 
basic  book. 
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Shotton,  D.,  ed.,  1993,  Electronic  Light  Microscopy  —  Techniques  in  Modem 
Biomedical  Microscopy,  Wiley-Liss,  New  York,  351  pp.  Many  of  the  usual 
suspects  have  contributed  chapters,  with  some  further  chapters  on  display, 
video,  and  sample  preparation.  A  good  companion  to  this  volume. 

Slater,  E.M.,  and  Slater,  H.S.,  1993,  Light  and  Electron  Microscopy,  Cambridge 
University  Press,  New  York.  A  fine  general  book  on  microscopes. 
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nonlinear  microscopy  and  techniques  for  temporal  and  spatial  calibration. 

Webb,  R.H.,  1991,  Confocal  microscopes.  Optics  Photonics  News  2:8-13.  A 
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Webb,  R.H.,  1996,  Confocal  optical  microscopy.  Rep.  Prog.  Physics 
59:427—471.  My  version  for  physicists,  with  some  (then)  new  material 
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Webb,  R.H.,  1999,  Theoretical  basis  of  confocal  microscopy,  Methods 
Enzymol.  Confocal  Microsc.  307:3-20.  My  version  for  biologists. 

White,  J.G.,  and  Amos,  W.B.,  1987,  Confocal  microscopy  comes  of  age.  Nature 
328:183.  General  article  by  two  of  the  early  users  of  the  CM. 

Wilson,  T.,  1980,  Imaging  properties  and  applications  of  scanning  optical 
microscopes,  Appl.  Phys  (Germany)  22:19-128.  A  review  paper,  with  57 
references. 

Wilson,  T.,  1985,  Scanning  optical  microscopy,  Scanning  7:79-87.  Summary 
article. 

Wilson,  T.,  ed.,  1990,  Confocal  Microscopy,  Academic  Press,  London.  Chap¬ 
ters  by  many  of  the  usual  folk.  Much  of  the  thinking  of  the  1980s 
summarized  and  updated. 

Wilson,  T.,  and  Sheppard,  C.J.R.,  1984,  Theory  and  Practice  of  Scanning 
Optical  Microscopy,  Academic  Press,  London.  Reprints  some  of  their  early 
papers. 

B.  HISTORICAL  INTEREST 

Some  of  the  original  articles,  mostly  now  superseded  by  later  work 
—  often  of  the  same  authors.  I  have  tried  to  avoid  the  “gee  whiz!” 
articles. 

Amos,  W.B.,  White,  J.G.,  and  Fordham,  M.,  Use  of  confocal  imaging  in  the 
study  of  biological  structures,  Appl.  Opt.  26:3239-3243.  A  good  general 
review,  and  an  early  description  of  the  MRC  (Bio-Rad)  microscope. 

Aslund,  N.,  Liljeborg,  A.,  Forsgren,  P.-O.,  and  Wahlsten,  S.,  1987,  Three- 
dimensional  digital  microscopy  using  the  PHOIBOS  scanner,  Scanning 
9:227-235.  Consecutive  optical  sections  to  generate  digital  three- 
dimensional  microscopy. 

Baer,  S.C.,  1970,  Optical  apparatus  providing  focal-plane  specific  information, 
US  Patent  No.  3,547,512.  An  early  CM  idea,  largely  ignored  for  17  years. 

Brakenhoff,  G.J.,  Blom,  P.,  and  Barends,  P,  1979,  Confocal  scanning  light 
microscopy  with  high  aperture  immersion  lenses,  J.  Microsc. 
1 17:219-232.  For  point  objects  the  theoretically  expected  factor  of  1 .4  can 
be  realized.  A  further  improvement  by  a  factor  of  1.25  after  apodization 
with  an  annular  aperture. 
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Brakenhoff,  G.J.,  van  der  Voort,  van  Spronsen,  E.A.,  Linnemans, 

W.A.M.,  and  Nanninga,  N.,  1985,  Three-dimensional  chromatin  distribu¬ 
tion  in  neuroblastoma  nuclei  shown  by  confocal  scanning  laser 
microscopy,  Nature  317:748-749.  An  early  use  of  optical  sectioning  and 
higher  resolution  for  studying  the  three-dimensional  morphology  of  bio¬ 
logical  structures. 

Chou,  C.-H.,  and  Kino,  G.S.,  1987,  The  evaluation  of  V(z)  in  a  type  II 
reflection  microscope,  IEEE  Trans.  Ultrason.  Ferroelectr.  Freq.  Control. 
34:341-345.  Theory:  V(z)  of  an  acoustic  microscope,  non-paraxial  and 
finite  pinhole  and  the  asymmetry  of  the  V(z)  curve  for  a  perfect  reflector. 
This  is  about  the  acoustic  microscope,  but  it  set  the  stage  for  a  lot  of  early 
CM  work. 

Davidovits,  P.,  and  Egger,  M.D.,  1971,  Scanning  laser  microscope  for  biolog¬ 
ical  investigations,  Appl.  Opt.  10:1615-1619.  Also:  Davidovits,  P.,  and 
Egger,  M.D.,  1969,  Scanning  laser  microscope,  Nature  223:831.  Two 
seminal  papers  of  historical  interest. 

Egger,  M.D.,  and  Petran,  M.,  1967,  New  reflected-light  microscope  for  viewing 
unstained  brain  and  ganglion  cells,  Science  157:305-307.  A  largely  unno¬ 
ticed  description  of  the  tandem  scanning  microscope,  but  it  was  all  there! 

Hamilton,  D.K.,  Wilson,  T.,  and  Sheppard,  C.J.R.,  1981,  Experimental  obser¬ 
vations  of  the  depth-discrimination  properties  of  scanning  microscopes, 
Opt.  Lett.  6:625-626.  Optical  sectioning. 

Maurice,  D.M.,  1973,  A  scanning  slit  optical  microscope.  Invest.  Ophthalmol. 
13:1033-1037.  A  pioneer  paper  describing  an  early  form  of  confocal 
microscopy  for  imaging  layers  in  the  cornea  of  the  eye.  This  system  used 
a  scanning  slit  3  pm  wide  to  give  depth  definition,  and  scanning  was 
carried  out  by  moving  a  photographic  film  and  the  specimen  in  opposite 
directions.  High-quality  images  of  the  cornea  were  obtained,  which  took 
about  20  min  to  form. 

Minsky,  M.,  1988,  Memoir  on  inventing  the  CSM,  Scanning  10:128-138.  A 
valuable  historical  document,  and  enjoyable  reading.  Minsky’s  patent  ran 
out  before  the  world  was  ready  for  the  idea,  but  his  early  ideas  have  all 
proved  out  well.  See  Minsky,  M.,  1961,  Microscopy  apparatus,  US  Patent 
No.  3,013,467. 

Petran,  M.,  and  Hadravsky,  M.,  1967,  Method  and  arrangement  for  improving 
the  resolving  power  and  contrast,  US  Patent  No.  3,517,980,  filed  4-12-67, 
granted  30-6-70. 

Petran,  M.,  and  Hadravsky,  M.,  1968,  Tandem-scanning  reflected-light  micro¬ 
scope,  J.  Opt.  Soc.  Am.  58:661-664.  Source  paper  on  TSM.  See  also  79. 
Petran,  M.,  Hadravsky,  M.,  Benes,  J.,  Kucera,  R.,  and  Boyde,  A.,  1985, 
The  tandem  scanning  reflected  light  microscope.  Part  1  —  The  principle, 
and  its  design,  Proc.  Roy.  Microsc.  Soc.  20:125-129. 

Ploem,  J.S.,  1987,  Laser  scanning  fluorescence  microscopy,  Appl.  Opt. 
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cal  and  heterodyne  microscopy  for  imaging  through  scattering  media, 
J.  Opt.  Soc.  Am.  A  13:46-52.  Applied  primarily  to  turbid  media. 

See,  C.W.,  and  Iravani,  M.V.,  1988,  Differential  amplitude  scanning  optical 
microscope:  Theory  and  applications,  Appl.  Opt.  27:2786-2792.  Differ¬ 
ential  means  two  adjacent  spots  to  catch  small  changes  of  height  or  index. 
This  is  a  dynamic  range  stretcher  for  those  quantities. 

Stelzer,  E.H.K.,  Marsman,  H.J.B.,  and  Wijnaendts  van  Resandt,  R.W.  1986,  A 
setup  for  a  confocal  scanning  laser  interference  microscope,  Optik 
73:30-33.  The  interference  mode  installed  on  a  CM  is  compared  with  that 
on  a  conventional  microscope  and  high-quality  images  are  presented. 

Torok,  P,  Laczik,  Z.,  and  Sheppard,  C.J.R.,  1996,  Effect  of  half-stop  lateral 
misalignment  on  imaging  of  dark-field  and  stereoscopic  confocal  micro¬ 
scopes,  Appl.  Opt.  35:6732-6739.  A  way  to  make  a  CM  yield  dark-field 
and/or  stereoscopic  imaging.  Extended  to  DPC  in  the  same  year:  Optik 
103:101-106.  As  usual,  there  are  a  few  other  papers  on  the  same  topic  at 
the  same  time,  by  the  same  authors. 

Wilson,  T.,  and  Carlini,  A.R.,  1988,  Effect  of  detector  displacement  in  confo¬ 
cal  imaging  systems,  Appl.  Opt.  27:3791-3799.  In  certain  cases  a  degree 
of  detector  offset  may  be  used  to  advantage  in  determining  the  position  of 
an  edge. 

Wilson,  T.,  Carlini,  A.R.,  and  Sheppard,  C.J.R.,  1985,  Phase  contrast 
microscopy  by  nearly  full  illumination,  Optik  70:166-169.  A  simple 
phase-contrast  technique  where  the  two  lenses  in  a  scanning  microscope 
are  not  equal  is  equally  applicable  to  conventional  and  CMs  and  is  demon¬ 
strated  experimentally. 

Wilson,  T.,  Juskaitis,  R.,  and  Tan,  J.B.,  1994,  Differential  imaging  in  confocal 
microscopy,  J.  Microsc.  175:1-9.  Uses  a  two-mode  optical  fiber  for  dif¬ 
ferential  amplitude  and  phase  contrast. 


H.  DISPLAY 

The  literature  on  three-dimensional  image  display  is  far  larger  than 
that  on  confocal  microscopy.  These  are  some  of  the  specialized 
papers. 

Carlsson,  K.,  Danielsson,  P.E.,  Lenz,  R.,  Liljeborg,  A.,  Majlof,  L.,  and  Aslund, 
N.,  1985,  Three-dimensional  microscopy  using  a  confocal  laser  scanning 
microscope,  Opt.  Lett.  10:53-55.  Visualizes  in  stereo  and  rotation  by 
making  look-through  projections  of  the  digital  data  in  different  directions. 
Contrast  enhanced  by  generating  gradient  volume  to  display  the  border 
surfaces  between  regions  in  opposing  senses  for  the  two  images. 

Conchello,  J.A.,  and  Hansen,  E.W.,  1990,  Enhanced  3-D  reconstruction  from 
CM  images.  I.  Deterministic  and  maximum  likelihood  reconstructions, 
Appl.  Opt.  29:3795-3804.  The  feasibility  of  obtaining  longitudinal  reso¬ 
lution  comparable  to  the  lateral  diffraction  limit  by  posterior  processing 
of  confocal  sections. 

Dilworth,  D.S.,  Leith,  E.N.,  and  Lopez,  J.L.,  1991,  Three-dimensional  confo¬ 
cal  imaging  of  objects  embedded  within  thick  diffusing  media,  Appl.  Opt. 
30:1796-1803.  Exfoliative  deconvolution  to  sharpen  a  volume  image  in 
which  the  blur  is  depth  variant. 

Hiraoka,  Y.,  Sedat,  J.W.,  and  Agard,  D.A.,  1990,  Determination  of  three- 
dimensional  imaging  properties  of  a  light  microscope  system.  Partial  con¬ 
focal  behavior  in  epifluorescence  microscopy,  Biophys.  J.  57:325-333. 
Through-focus  series  of  a  point  object  were  recorded  on  a  charge-coupled 
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device  image  detector.  From  these  images,  the  three-dimensional  PSF  and 
its  Fourier  transform,  the  optical  transfer  function,  were  derived  for  use  in 
processing  a  CM  image. 

Oldmixon,  E.H.,  and  Carlsson,  K.,  1993,  Methods  for  large  data  volumes  from 
confocal  scanning  laser  microscopy  of  lung,  J.  Microsc.  170:221-228. 
How  to  stitch  together  adjacent  small  fields  to  achieve  high  resolution  over 
a  large  area  (500  resels  per  diameter).  Intensity  variations  are  adjusted  for 
uniformity  of  measurement. 

Schormann,  T.,  and  Jovin,  T.M.,  1992,  Contrast  enhancement  and  depth  per¬ 
ception  in  three-dimensional  representations  of  DIC  and  CM  images, 
J.  Microsc.  166:155-168.  A  new  contrast  enhancement  transformation 
based  upon  local  statistics  and  a  gray-level  probability  density  function 
provides  improved  depth  perception,  increasing  the  number  of  usable 
optical  sections  by  up  to  five. 

I.  FIBER-OPTIC  CONFOCAL  MICROSCOPES 

In  this  variant,  the  fiber  is  an  essential  component  for  confocality, 
or  a  fiber  bundle  carries  the  object  plane  to  inaccessible  places. 

Balaji,  J.,  Garai,  K.,  Chakrabarti,  S.,  and  Maiti,  S.,  2003,  Axial  resolution  limit 
of  a  fiber-optic  fluorescence  probe,  Appl.  Opt.  42:3780-3784.  This  mostly 
theoretical  paper  has  no  scanning. 

Dabbs,  T.,  and  Glass,  M.,  1992,  Single-mode  fibers  used  as  confocal  micro¬ 
scope  pinholes,  Appl.  Opt.  31:705-706.  And,  Dabbs,  T.,  and  Glass, 
M.,  1992,  Fiber-optic  confocal  microscope:  FOCON,  Appl.  Opt. 
31:3030-3035.  The  beam-splitter  is  replaced  by  a  fiber-optic  splitter,  and 
the  core  of  a  single-mode  fiber  takes  the  place  of  both  the  source  and  detec¬ 
tor  pinholes.  Scanning  in  the  x-,  y-,  and  z-directions  is  by  moving  the  end 
of  the  optical  fiber. 

Gan,  X.,  Gu,  M.,  and  Sheppard,  C.J.R.,  1992,  Fluorescent  image  formation  in 
the  fibre-optical  confocal  scanning  microscope,  J.  Mod.  Opt.  39:825-834. 
Unlike  a  finite  circular  pinhole,  there  is  no  missing  cone  of  spatial  fre¬ 
quencies,  and  no  negative  tail  in  the  transfer  function. 

Ghiggino,  K.P.,  Harris,  M.R.,  and  Spizzirri,  P.G.,  1992,  Fluorescence  lifetime 
measurements  using  a  novel  fibre-optic  laser  scanning  confocal  micro¬ 
scope,  Rev.  Sci.  Instrum.  63:2999-3002.  A  fiber-optic  CM  with  a  mode- 
locked  dye  laser  excitation  source,  avalanche  photodiode  detector,  and 
time-correlated  photon-counting  electronics  allows  spatially  resolved  flu¬ 
orescence  decay  profiles  from  fluorescent  dyes  in  solution  and  polymer 
films. 

Giniunas,  L.,  Juskaitis,  R.,  and  Shatalin,  S.V.,  1991,  Scanning  fiber-optic 
microscope,  Electron.  Lett.  27:724-726.  A  phase-sensitive  scanning  fiber¬ 
optic  microscope  designed  to  be  used  as  an  endoscope.  Confocal  opera¬ 
tion  with  single-mode  fiber  yields  0.8  mm  lateral  resolution. 

Gobel,  W.,  Kerr  J.N.D.,  Nimmerjahn,  A.,  and  Helmchen,  F.,  2004,  Miniatur¬ 
ized  two-photon  microscope  based  on  a  flexible  coherent  fiber  bundle  and 
a  gradient-index  lens  objective,  Opt.  Lett.  29:2521-2523.  Typical  of  the 
fiber  bundle  relay  microscopes. 

Gu,  M.,  and  Bird,  D.,  2003,  Three-dimensional  optical-transfer-function  analy¬ 
sis  of  fiber-optical  two-photon  fluorescence  microscopy,  J.  Opt.  Soc.  Am. 
A  20:941-947.  The  latest  in  a  series  of  papers,  mostly  theory,  no  scanning. 

Gu,  M.,  and  Sheppard,  C.J.R.,  1991,  Signal  level  of  the  fiber-optical  CM, 
J.  Mod.  Opt.  38:1621-1630.  The  efficiency  of  total  power  transformation 
and  signal  level  of  the  fiber-optical  CM,  including  defocus  effects. 

Gu,  M.,  Sheppard,  C.J.R.,  and  Gan,  X.,  1991,  Image  formation  in  a  fiber-optical 
confocal  scanning  microscope,  J.  Opt.  Soc.  Am.  A  8:1755-1761.  Coherent 
transfer  functions  in  both  in-focus  and  defocused  cases  are  derived  and 
calculated. 

Jung  J.C.,  and  Schnitzer,  M.J.,  2003,  Multiphoton  endoscopy.  Opt.  Lett. 
28:902-904.  This  is  not  strictly  a  fiber,  but  a  long  rigid  GRIN  lens.  Pulse 
degradation  is  less  a  problem  than  in  a  fiber  bundle. 

Juskaitis,  R.,  and  Wilson,  T.,  Differential  CM  with  a  two-mode  optical  fiber, 
Appl.  Opt.  31:898-902.  Both  the  differential  amplitude  and  the  differen¬ 
tial  phase  images  can  be  obtained  by  adjusting  the  differential  phase  delay 
between  the  fiber  modes. 

Juskaitis,  R.,  and  Wilson,  T.,  1992,  Imaging  in  reciprocal  fiber-optic  based 
CMs,  Opt.  Commun.  92:315-325.  Single-  and  two-mode  fibers  in  a  recip¬ 


rocal  scheme.  Confocal,  differential  amplitude,  or  phase  contrast  imaging 
and  surface  profilometry  with  reduced  alignment  tolerances. 

Juskaitis,  R.,  and  Wilson,  T.,  1994,  Direct-view  fiberoptic  confocal  microscope, 
Opt.  Lett.  19:1906-1908.  A  moving  fiber  is  the  scan,  and  the  direct  view 
comes  from  a  synchronously  scanned  out-coupled  fiber. 

Juskaitis,  R.,  Reinholz,  F.,  and  Wilson,  T.,  1992,  Fiber-optic  based  confocal 
scanning  microscopy  with  semiconductor  laser  excitation  and  detection, 
Electron.  Lett.  28:986-988.  Collected  light  reenters  the  laser  and  the  image 
is  detected  as  a  modulation  of  the  laser  power. 

Juskaitis,  R.,  Wilson,  T.,  and  Watson,  T.F.,  1997,  Real-time  white  light  reflec¬ 
tion  confocal  microscopy  using  a  fiber-optic  bundle,  Scanning  19:15-19. 
White  light  to  avoid  speckle. 

Kimura,  S.,  and  Wilson,  T.,  Confocal  scanning  optical  microscope  using  single¬ 
mode  fiber  for  signal  detection,  Appl.  Opt.  30:2143-2150.  Always  coher¬ 
ent,  which  is  fundamentally  different  from  a  finite-sized  pinhole. 

Liang,  C.,  Sung,  K.B.,  Richards-Kortum,  R.R.,  and  Descour,  M.R.,  2002, 
Design  of  a  high-numerical-aperture  miniature  microscope  objective 
for  an  endoscopic  fiber  confocal  reflectance  microscope,  Appl.  Opt. 
41:4603^-610.  Another  fiber  bundle  relay,  this  one  with  specialized  optics 
and  real  images.  See  also:  Sung,  K.B.,  Liang,  C.,  Descour,  M.,  Collier,  T., 
Follen,  M.,  Malpica,  A.,  and  Richards-Kortum,  R.,  2002,  Near  real  time 
in  vivo  fiber  optic  confocal  microscopy:  Sub-cellular  structure  resolved, 
J.  Microsc.  207:137-145. 

Lin,  C.R,  and  Webb,  R.H.,  2000,  Fiber-coupled  multiplexed  confocal 
microscope,  Opt.  Lett.  25:954-956.  An  incoherent  fiber  bundle  with  a 
line-scanning  CM.  Like  most  Optics  Letters  papers,  this  one  hasn’t  gone 
anywhere  yet. 

Rouse,  A.R.,  Kano,  A.,  Udovich,  J.A.,  Kroto,  S.M.,  and  Gmitro,  A.F.,  2004, 
Design  and  demonstration  of  a  miniature  catheter  for  a  confocal  microen¬ 
doscope,  Appl.  Opt.  43:5763-5771.  Used  in  real  situations,  this  coherent 
fiber  bundle  relays  a  line-scanned  CM.  The  first  paper  from  this  group  was 
in  1999,  so  reality  is  tough! 

Zhou,  H.,  Sheppard,  C.J.R.,  and  Gu,  M.,  1996,  A  compact  confocal  interfer¬ 
ence  microscope  based  on  a  four-port  single-mode  fiber  coupler,  Optik 
103:45^-8.  Uses  a  four-port  coupler  and  heterodynes  against  an  internal 
reflection. 

J.  INDEX  MISMATCH 

Aberrations  due  to  index  differences  are  primarily  spherical. 

Arimoto,  R.,  and  Murray,  J.M.,  2004,  A  common  aberration  with  water- 
immersion  objective  lenses,  J.  Microsc.  216:49-51.  A  tilted  cover  slip 
introduces  off  axis  aberrations  with  a  water  immersion  objective. 

Booth,  M.J.,  Neil,  M.A.A.,  and  Wilson,  T.,  1998,  Aberration  correction  for 
confocal  imaging  in  refractive-index-mismatched  media,  J.  Microsc. 
192:90-98.  A  general  paper  on  the  subject  of  oil-immersion  objectives  and 
thick  “water”  samples. 

Diaspro,  A.,  Federici,  F.,  and  Robello,  M.,  2002,  Influence  of  refractive-index 
mismatch  in  high-resolution  three-dimensional  confocal  microscopy, 
Appl.  Opt.  41:685-690.  A  detailed  look  at  the  problem. 

Egner,  A.,  and  Hell,  S.W.,  1999,  Equivalence  of  the  Huygens-Fresnel  and 
Debye  approach  for  the  calculation  of  high  aperture  point-spread  functions 
in  the  presence  of  refractive  index  mismatch,  J.  Microsc.  193:244-249. 
Many  reference  to  earlier  work. 

Gu,  M.,  Day,  D.,  Nakamura,  O.,  and  Kawata,  S.,  2001,  Three-dimensional 
coherent  transfer  function  for  reflection  confocal  microscopy  in  the  pres¬ 
ence  of  refractive-index  mismatch.  J.  Opt.  Soc.  Am.  A  18:2002-2008.  Gu 
and  Sheppard  and  Torok  have  written  more  on  this,  some  of  which  is  found 
in  the  PSF  category. 

Haeberle,  O.,  Ammar,  M.,  Furukawa,  H.,  Tenjimbayashi,  K.,  and  Torok,  P., 
2003,  Point  spread  function  of  optical  microscopes  imaging  through  strat¬ 
ified  media,  Opt.  Express  1 1 : 2964-2969.  Stratified  media  exacerbate  the 
mismatch  problem. 

Jacobsen,  H.,  Hanninen,  R,  Soini,  E.,  and  Hell,  S.W.,  1994,  Refractive-index- 
induced  aberrations  in  2-photon  confocal  fluorescence  microscopy, 
J.  Microsc.  176:226-230.  The  mismatch-induced  aberrations  make  multi¬ 
photon  microscopy  more  difficult. 


894  Chapter  52  •  R.H.  Webb 


Kawata,  Y.,  Fujita,  K.,  Nakamura,  O.,  and  Kawata,  S.,  1996,  4Pi  confocal 
optical  system  with  phase  conjugation,  Opt.  Lett.  21:1415-1417.  Phase 
conjugation  is  said  to  relieve  the  mismatch  aberrations  in  4PI. 

Pawley,  J.B.,  2002,  Limitations  on  optical  sectioning  in  live-cell  confocal 
microscopy.  Scanning  24:241-246.  The  “optical  section”  surface  can  be 
profoundly  distorted  by  the  index  irregularities  associated  with  the  pres¬ 
ence  of  nuclei  and  other  subcellular  structures. 

Sheppard,  C.J.R.,  Connolly,  T.J.,  Lee,  J.,  and  Cogswell,  C.J.,  1994,  Confocal 
imaging  of  a  stratified  medium,  Appl.  Opt.  33:631-640. 

Sieracki,  C.K.,  Levey,  C.G.,  and  Hansen,  E.W.,  1995,  Simple  binary  optical- 
elements  for  aberration  correction  in  confocal  microscopy,  Opt.  Lett. 
20:1213-1215.  Pupil  engineering  to  alleviate  the  aberrations. 

Smithpeter,  C.L.,  Dunn,  A.K.,  Welch,  A.J.,  and  Richards-Kortum,  R.,  1998, 
Penetration  depth  limits  of  in  vivo  confocal  reflectance  imaging,  Appl.  Opt. 
37:2749-2754.  Index  mismatches  in  real  samples  limit  penetration  depth. 

Visser,  T.D.,  and  Oud,  J.L.,  1994,  Volume  measurements  in  3-dimensional 
microscopy,  Scanning  16:198-200.  The  refractive  index  mismatch 
between  oil  immersion  and  object  leads  to  a  severe  over-estimation  of  the 
object’s  size. 

Wan,  D.S.,  Rajadhyaksha,  M.,  and  Webb,  R.H.,  2000,  Analysis  of  spherical 
aberration  of  a  water  immersion  objective:  Application  to  specimens  with 
refractive  indices  1.33-1.40,  J.  Microsc.  197:274-284.  Spherical  aberra¬ 
tion  is  measured  by  an  interferometer  converted  from  a  confocal  micro¬ 
scope  for  samples  with  different  refractive  indices. 

K.  MULTIPLEX 

Including  tandem-scanning  (disk,  direct  view)  and  array 
microscopes. 

Andresen,  V.,  Egner,  A.,  and  Hell,  S.W.,  2001,  Time-multiplexed  multifocal 
multiphoton  microscope,  Opt.  Lett.  26:75-77 .  Adjacent  foci  illuminate  the 
sample  at  different  time  points.  More  details  appear  in  an  earlier  version: 
Bewersdorf,  J.,  Pick,  R.,  and  Hell,  S.W.,  1998,  Multifocal  multiphoton 
microscopy.  Opt.  Lett.  23:655-657. 

Boyde,  A.,  Xiao,  G.Q.,  Corle,  T.,  Watson,  T.F.,  and  Kino,  G.S.,  1990,  An 
evaluation  of  unilateral  TSM  for  biological  applications,  Scanning 
12:273-279.  Two  designs  of  tandem  scanning  reflected  light  microscopes, 
due  to  Petran  and  to  Kino. 

Brakenhoff,  G.J.,  and  Visscher,  K.,  1992,  Confocal  imaging  with  bilateral  scan¬ 
ning  and  array  detectors,  J.  Microsc.  165:139-146.  A  hybrid  CM/TSM 
using  an  array  detector  such  as  a  CCD  for  confocal  image  collection  and 
a  double-sided  scanning  mirror  to  scan  and  collect  data.  This  is  the  scheme 
implemented  by  Meridian. 

Brakenhoff,  G.J.,  and  Visscher,  K.,  1993,  Imaging  modes  for  bilateral  confo¬ 
cal  scanning  microscopy,  J.  Microsc.  171:17-26.  Details  of  the  double¬ 
sided  mirror  rescanning  system.  The  remitted  light  traverses  the  confocal 
pinhole  to  yet  another  scanning  mirror  (the  second  side  of  the  original  one) 
to  be  spread  again  into  an  image  viewable  by  eye  or  camera.  See  Maurice 
and  Koester  for  earlier  implementations.  This  is  the  scheme  implemented 
by  Meridian. 

Fewer,  D.T.,  Hewlett,  S.J.,  McCabe,  E.M.,  and  Hegarty,  J.,  1997,  Direct-view 
microscopy:  Experimental  investigation  of  the  dependence  of  the  optical 
sectioning  characteristics  on  pinhole-array  configuration,  J.  Microsc. 
187:54-61.  And  Fewer,  D.T.,  Hewlett,  S.J.,  and  McCabe,  E.M,  1998,  Laser 
sources  in  direct-view-scanning,  tandem-scanning,  or  Nipkow-disk- 
scanning  confocal  microscopy,  Appl.  Opt.  37:380-385.  Results  for  practi¬ 
cal  systems  as  regards  pinhole-array  design  and  fabrication. 

Juskaitis,  R.,  Wilson,  T.,  Neil,  M.A.A.,  and  Kozubek,  M.,  1996,  Efficient  real¬ 
time  confocal  microscopy  with  white  light  sources,  Nature  383:804-806. 
A  white-light,  multiple-point- source  method  which  can  in  principle 
produce  images  in  real  time,  with  light  efficiencies  as  high  as  50%. 

Neil,  M.A.A.,  Wilson,  T.,  and  Juskaitis,  R.,  1998,  A  light  efficient  optically 
sectioning  microscope,  J.  Microsc.  189:114-117.  A  one-dimensional  grid 
pattern,  on  a  spinning  disk,  produces  an  optically  sectioned  image  super¬ 
imposed  on  a  conventional  image.  A  blank  sector  on  the  disc  provides  the 
widefield  image,  which  is  then  subtracted  to  yield  the  optically  sectioned 


image  in  real  time.  See  also  Wilson,  T.,  Juskaitis,  R.,  Neil,  M.A.A.,  and 
Kozubek,  M.,  1996,  Confocal  microscopy  by  aperture  correlation,  Opt. 
Lett.  21:1879-1881. 

Tanaami,  T.,  Otsuki,  S.,  Tomosada,  N.,  Kosugi,  Y.,  Shimizu,  M.,  and  Ishida, 
H.,  2002,  High-speed  1-frame/ms  scanning  confocal  microscope  with  a 
microlens  and  Nipkow  disks,  Appl.  Opt.  41:4704^4708. 

Watson,  T.F.,  Juskaitis,  R.,  and  Wilson.  T.,  2002,  New  imaging  modes  for 
lenslet-array  tandem  scanning  microscopes,  J.  Microsc.  205:209-212.  A 
later  addition  to  this  series,  describing  the  now  common  disk  scanner  with 
microlenses. 

Xiao,  G.Q.,  Corle,  T.R.,  and  Kino,  G.S.,  1988,  Real-time  confocal  scanning 
optical  microscope,  Appl.  Phys.  Lett.  53:716-718.  This  describes  the  one¬ 
sided  tandem  scanning  microscope  from  Kino’s  lab. 

L.  NONLINEAR 

Including  multi-photon  and  third  harmonic  microscopes. 

Barad,  Y.,  Eisenberg,  H.,  Horowitz,  M.,  and  Silberberg,  Y.,  1997,  Nonlinear 
scanning  laser  microscopy  by  third  harmonic  generation,  Appl.  Phys.  Lett. 
70:922-924.  This  method  can  resolve  interfaces  and  inhomogeneities  with 
axial  resolution  comparable  to  the  confocal  length  of  the  beam. 

Booth,  M.J.,  and  Hell,  S.W.,  1998,  Continuous  wave  excitation  two-photon 
fluorescence  microscopy  exemplified  with  the  647-nm  ArKr  laser  line, 
J.  Microsc.  190:298-304.  NA  =  1.4  and  210mW. 

Brakenhoff,  G.J.,  Muller,  M.,  and  Ghauharali,  R.I.,  1996,  Analysis  of  efficiency 
of  two-photon  versus  single-photon  absorption  for  fluorescence  generation 
in  biological  objects,  J.  Microsc.  183:140-144.  Total  exposure  for  equiv¬ 
alent  fluorescence  by  single-photon  absorption  is  an  order  of  magnitude 
lower  than  by  two-photon  absorption. 

Brakenhoff,  G.J.,  Squier,  J.,  Norris,  T.,  Bliton,  A.C.,  Wade,  M.H.,  and  Athey, 
B.,  1996,  Real-time  two-photon  confocal  microscopy  using  a  femtosec¬ 
ond,  amplified  Ti:  sapphire  system,  J.  Microsc.  181:253-259.  With  the 
bilateral  line  scanning  approach  and  a  Ti :  Sa  laser. 

Egner,  A.,  Andresen,  V.,  and  Hell,  S.W.,  2002,  Comparison  of  the  axial  reso¬ 
lution  of  practical  Nipkow-disk  confocal  fluorescence  microscopy  with 
that  of  multifocal  multiphoton  microscopy:  Theory  and  experiment, 
J.  Microsc.  206:24-32.  Multi-photon  is  better. 

Schrader,  M.,  and  Hell,  S.W.,  1996,  4Pi-confocal  images  with  axial  superres¬ 
olution,  J.  Microsc.  183:189-193.  A  suitable  combination  of  aperture 
enlargement,  two-photon  excitation,  confocalization,  and  three-point 
deconvolution  yields  axial  resolution  of  145  nm. 

Schrader,  M.,  Bahlmann,  K.,  and  Hell,  S.W.,  1997,  Three-photon-excitation 
microscopy:  Theory,  experiment  and  applications,  Optik  104:116-124. 
Theory  and  experimental  results. 

Squier,  J.,  and  Muller,  M.,  2001,  High  resolution  nonlinear  microscopy:  A 
review  of  sources  and  methods  for  achieving  optimal  imaging,  Rev.  Sci. 
Instrum.  72:2855-2867.  Reviews  instrumentation  used  in  high  resolution 
nonlinear  microscopy  and  techniques  for  temporal  and  spatial  calibration. 

M.  POLARIZATION 

Polarization  always  matters  but  only  a  few  papers  deal  with  it  as 
a  separate  topic. 

Hansen,  E.W.,  Allen,  R.D.,  Strohbehn,  J.W.,  Chaffee,  MA.,  Farrington,  D.L., 
Murray,  W.J.,  Pillsbury,  T.A.,  and  Riley,  MF.,  1985,  Laser  scanning  phase 
modulation  microscope,  J.  Microsc.  140:371-381.  Quantitative  polarized 
light  imaging  with  a  phase-modulation  feedback  loop  for  precise  mea¬ 
surement  of  birefringence. 

Massoumian,  F.,  Juskaitis,  R.,  Neil,  M.A.A.,  and  Wilson,  T.,  2003,  Quantita¬ 
tive  polarized  light  microscopy,  J.  Microsc.  209: 13-22.  A  rotating  analyzer 
lock-in  detection. 

Nielsen,  P.M.F.,  Reinholz,  F.N.,  and  Charette,  P.G.,  1996,  Polarization- 
sensitive  scanned  fiber  confocal  microscope,  Opt.  Eng.  35:3084-3091.  A 
single-mode  fiber  with  polarization  rotator. 
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Wilson,  T.,  and  Juskaitis,  R.,  1995,  On  the  extinction  coefficient  in  confocal 
polarization  microscopy,  J.  Microsc.  179:238-240.  You  never  get  com¬ 
plete  extinction  in  a  CM. 


N.  PROFILOMETRY 

Many  profilometers  are  confocal  microscopes,  and  a  few  confocal 
microscopes  are  profilometers. 

Corle,  T.R.,  Fanton,  J.T.,  and  Kino,  G.S.,  1987,  Distance  measurements  by  dif¬ 
ferential  confocal  optical  ranging,  Appl.  Opt.  26:2416-2420.  Dithering 
generates  a  differential  measurement,  placing  a  zero-crossing  at  the  peak 
of  the  depth  response.  Sensitivities  to  surface  vibrations  of  0.01  nm  and 
thin  film  measurements  to  0.04  mm. 

Hamilton,  D.K.,  and  Matthews,  H.J.,  1985,  The  confocal  interference  micro¬ 
scope  as  a  surface  profilometer,  Optik  71:31-34.  A  non-contacting 
high-resolution  surface  profiling  technique,  with  feedback  to  maintain 
a  constant  phase  relationship  between  the  signal  and  the  reference  beams 
of  a  confocal  interference  microscope. 

Hamilton,  D.K.,  and  Wilson,  T.,  1982,  Three-dimensional  surface  measurement 
using  the  confocal  scanning  microscope,  Appl.  Phys.  B27:211-213.  0.1 
mm  axial  resolution  on  a  semiconductor. 

Lee,  B.S.,  and  Strand,  T.C.,  1990,  Profilometry  with  a  coherence  scanning 
microscope,  Appl.  Opt.  29:3784-3788.  Uses  coherence  effects  rather  than 
physical  apertures.  Longitudinal  resolution  is  decoupled  from  lateral. 

Matthews,  H.J.,  Hamilton,  D.K.,  and  Sheppard,  C.J.R.,  1986,  Surface  profiling 
by  phase-locked  interferometry,  Appl.  Opt.  25:237 2-237 4.  1-nm  height 
resolution.  A  feedback  arrangement  keeps  in  quadrature  the  two  arms  of 
a  confocal  interference  microscope  by  modulating  the  reference  beam 
using  an  electrooptic  phase  modulator. 

Toy,  D.A.,  1990,  Confocal  microscopy:  The  ups  and  downs  of  3-D  profiling, 
Semicond.  bit.  13:120-123.  Confocal  techniques  can  solve  the  problems 
of  depth  of  field  versus  resolution. 


O.  POINT  SPREAD  FUNCTION 

Including  pupillary  masking,  adaptive  optics,  problems  with  index 

mismatching,  and  modifications  of  the  PSF  by  selective  bleaching, 

4PI,  and  theta  microscopes  and  angled  objectives. 

Bahlmann,  K.,  and  Hell,  S.W.,  2000,  Polarization  effects  in  4Pi  confocal 
microscopy  studied  with  water-immersion  lenses,  Appl.  Opt.  39: 
1652-1658.  Refinements  of  the  4Pi  microscope. 

Barth,  M.,  and  Stelzer,  E.H.K.,  1994,  Boosting  the  optical  transfer-function 
with  a  spatially  resolving  detector  in  a  high  numerical  aperture  confocal 
reflection  microscope,  Optik  96:53-58.  A  CCD  view  of  the  PSF  at  the 
“pinhole”  position  allows  careful  analysis. 

Gu,  M.,  1998,  Image  of  a  sharp  edge  in  type-3  laser  scanning  microscopy,  Optik 
107:115-117.  A  way  to  measure  the  PSF. 

Haeberle,  O.,  Xu,  C.,  Dieterlen,  A.,  and  Jacquey,  S.,  2001,  Multiple-objective 
microscopy  with  three-dimensional  resolution  near  lOOnm  and  a  long 
working  distance.  Opt.  Lett.  26:1684-1686.  Confocal,  theta,  and  4Pi 
microscopes. 

Heintzmann,  R.,  Jovin,  T.M.,  and  Cremer,  C.,  2002,  Saturated  patterned  exci¬ 
tation  microscopy  —  A  concept  for  optical  resolution  improvement,  J.  Opt. 
Soc.  Am.  A  19:1599-1609.  The  post-acquisition  manipulation  of  the 
acquired  data  is  computationally  more  complex  than  in  STED  or  ground- 
state  depletion,  but  the  experimental  requirements  are  simple. 

Hell,  S.W.,  Lindek,  S.,  Cremer,  C.,  and  Stelzer,  E.H.K.,  1994,  Measurement  of 
the  4pi-confocal  point-spread  function  proves  75  nm  axial  resolution,  Appl. 
Phys.  Lett.  64:1335-1337.  The  ultimate  in  theta  or  angled  CMs.  Here  the 
two  objectives  look  at  each  other  and  thus  accept  light  from  4  pi  steradi- 
ans.  See  also:  Hell,  S.W.,  and  Stelzer,  E.H.K.,  1992,  J.  Opt.  Soc.  Am.  A 
9:2159;  Hell,  S.W.,  Stelzer,  E.H.K.,  Lindek,  S.,  and  Cremer,  C.,  1994, 
Confocal  Microscopy  with  an  increased  detection  aperture  —  type-b  4pi 


confocal  microscopy.  Opt.  Lett.  19:222-224.  Hell  (and  many  others)  have 
written  extensively  on  the  4Pi  microscope  with  many  variations. 

Juskaitis,  R.,  and  Wilson,  T.,  1998,  The  measurement  of  the  amplitude  point 
spread  function  of  microscope  objective  lenses,  J.  Microsc.  189:8-11. 
Using  an  optical  fiber  interferometer. 

Klar,  T.A.,  Dyba,  M.,  and  Hell,  S.W.,  2001,  Stimulated  emission  depletion 
microscopy  with  an  offset  depleting  beam,  Appl.  Phys.  Lett.  78:393-395. 
Stimulated  emission  depletion  (STED)  reduces  the  axial  extent  of  a  con¬ 
focal  spot  from  490  to  175  nm,  and  simultaneously  from  183  to  70  nm 
along  the  direction  of  the  offset. 

Klar,  T.A.,  Engel,  E.,  and  Hell,  S.W.,  2001,  Breaking  Abbe’s  diffraction 
resolution  limit  in  fluorescence  microscopy  with  stimulated  emission 
depletion  beams  of  various  shapes,  Phys.  Rev.  E  64:6.  Particles  that  are 
only  65 -nm  apart  are  resolved  with  focused  light. 

Kuypers,  L.C.,  Dorckx,  J.J.J.,  and  Decraemer,  W.F.,  2004,  A  simple  method  for 
checking  the  illumination  profile  in  a  laser  scanning  microscope  and  the 
dependence  of  resolution  on  the  profile.  Scanning  26:256-258.  Pointing 
out  that  most  real  CMs  are  somewhat  apodized  by  a  Gaussian  profile  at 
the  pupil. 

Lindek,  S.,  Stefany,  T.,  and  Stelzer,  E.H.K.,  1997,  Single-lens  theta  micros¬ 
copy  —  A  new  implementation  of  confocal  theta  microscopy,  J.  Microsc. 
188:280-284.  A  somewhat  randomly  chosen  member  of  a  series  on  theta 
microscopy  by  some  of  its  originators. 

Matthews,  H.J.,  Hamilton,  D.K.,  and  Sheppard,  C.J.R.,  1989,  Aberration  mea¬ 
surement  by  confocal  interferometry,  J.  Mod.  Opt.  36:233-280.  Aberra¬ 
tions  and  apodization  of  microscope  objectives  measured  from  the  defocus 
signal  in  a  confocal  interference  microscope  system.  Phase  distortions  can 
be  measured  to  approximately  1/100,  and  quantitative  information  is  given 
about  the  imaging  performance  of  the  lenses  in  situ  in  the  optical  system. 

Nagorni,  M.,  and  Hell,  S.W.,  2001,  Coherent  use  of  opposing  lenses  for  axial 
resolution  increase  in  fluorescence  microscopy.  I.  Comparative  study 
of  concepts,  J.  Opt.  Soc.  Am.  A  18:36 — 48.  and  II.  Power  and  limitation  of 
nonlinear  image  restoration,  J.  Opt.  Soc.  Am.  A  18:49-54.  Comparison  of 
PSF  and  OTF  for  many  variants,  including  the  standing  wave  microscope, 
the  incoherent  illumination  interference  image  interference  microscope 
[(IM)-M-5],  and  the  4Pi  confocal  microscope  for  one  and  two  photons. 

Nieman,  L.,  Myakov,  A.,  Aaron,  J.,  and  Sokolov,  K.,  2004,  Optical  sectioning 
using  a  fiber  probe  with  an  angled  illumination-collection  geometry:  Eval¬ 
uation  in  engineered  tissue  phantoms,  Appl.  Opt.  43:1308-1319.  I  have 
my  doubts  about  how  real  any  phantoms  are  compared  to  skin.  They  also 
used  oral  mucosa,  and  that’s  real  and  easy. 

Schrader,  M.,  and  Hell,  S.W.,  1996,  Wavefronts  in  the  focus  of  a  light  micro¬ 
scope,  J.  Microsc.  184:143-148.  Scanning  a  scattering  probe  through  the 
focal  region  of  a  lens  illuminated  by  plane  waves. 

Schrader,  M.,  Hofmann,  U.G.,  and  Hell,  S.W.,  1998,  Ultrathin  fluorescent 
layers  for  monitoring  the  axial  resolution  in  confocal  and  two-photon  flu¬ 
orescence  microscopy,  J.  Microsc.  191:135-140.  A  how-to  paper. 

Shaw,  P.J.,  and  Rawlins,  D.J.,  1991,  The  point-spread  function  of  a  confocal 
microscope:  Its  measurement  and  use  in  deconvolution  of  3-D  data, 
J.  Microsc.  163:151-165.  PSF  for  an  MRC-500  CM  using  subresolution 
fluorescent  beads.  The  resulting  optical  transfer  functions  were  used  in  an 
iterative,  constrained  deconvolution  procedure  to  get  three-dimensional 
confocal  data  sets  from  a  biological  specimen. 

Sheppard,  C.J.R.,  and  Choudhury,  A.,  2004,  Annular  pupils,  radial  polarization, 
and  superresolution,  Appl.  Opt.  43:4322-4327.  Bessel  beam  and  radial 
polarization  gets  to  lOOnm  PSF. 

Sheppard,  C.J.R.,  and  Gu,  M.,  1991,  Aberration  compensation  in  confocal 
microscopy,  Appl.  Opt.  30:3563-3568.  Spherical  aberration  due  to  focus¬ 
ing  deep  within  the  specimen  can  be  compensated  by  altering  the  effec¬ 
tive  tube  length.  Sheppard  and  Gu  have  contributed  many  papers  on  the 
general  theme  of  pupil  masking  to  improve  the  PSF.  This  is  an  early  one. 
A  recent  member  of  the  class  is: 

Sheppard,  C.J.R.,  and  Gu,  M.,  1992,  Axial  imaging  through  an  aberrating  layer 
of  water  in  confocal  microscopy.  Opt.  Commun.  88:180-190.  The  axial 
response  modeled  by  a  series  of  signals  reflected  from  different  interfaces. 
Altering  the  effective  tube  length  for  an  optimum  axial  response,  experi¬ 
mental  results  are  well  in  agreement  with  theoretical  predictions. 

Sheppard,  C.J.R.,  Hamilton,  D.K.,  and  Cox,  I.J.,  1982,  Optical  microscopy  with 
extended  depth  of  field  observation  of  optical  signatures  of  materials,  Appl. 
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Phys.  Lett.  41:604—606.  Depth  of  field  may  be  extended  while  high- 
resolution,  diffraction-limited  imaging  is  retained.  The  technique  is  similar 
to  one  already  used  in  acoustic  microscopy. 

Stark,  P.R.H.,  Rinko,  L.J.,  and  Larson,  D.N.,  2003,  Fluorescent  resolution  target 
for  super-resolution  microscopy,  J.  Microsc.  212:307-310.  The  authors 
claim  this  is  a  better  resolution  probe  than  beads. 

Stelzer,  E.H.K.,  1998,  Contrast,  resolution,  pixelation,  dynamic  range  and 
signal-to-noise  ratio:  Fundamental  limits  to  resolution  in  fluorescence 
light  microscopy,  J.  Microsc.  189:15-24.  “In  conclusion:  (a)  real  optical 
systems  will  never  be  able  to  achieve  the  theoretical  resolution,  (b)  wide- 
field  fluorescence  microscopy  will  often  provide  a  better  resolution  than 
confocal  fluorescence  microscopy,  (c)  decreasing  the  observed  volume 
does  not  necessarily  increase  the  resolution  and  (d)  using  multiple  fluo- 
rophores  can  improve  the  accuracy  with  which  distances  are  measured.” 

Stelzer,  E.H.K.,  Lindek,  S.,  Albrecht,  S.,  Pick,  R.,  Ritter,  G.,  Salmon,  N.J.,  and 
Strieker,  R.,  1995,  A  new  tool  for  the  observation  of  embryos  and  other 
large  specimens  —  Confocal  theta-fluorescence  microscopy,  J.  Microsc. 
179:1-10.  Details  with  real  samples  (and  with  phantoms). 

Tapang,  G.,  and  Saloma,  C.,  2002,  Behavior  of  the  point-spread  function  in 
photon-limited  confocal  microscopy,  Appl.  Opt.  41:1534-1540. 

Wang,  T.D.,  Contag,  C.H.,  Mandella,  M.J.,  Chan,  N.Y.,  and  Kino,  G.S., 
2003,  Dual-axes  confocal  microscopy  with  post-objective  scanning  and 
low-coherence  heterodyne  detection,  Opt.  Lett.  28:1915-1917.  This 
architecture  can  be  scaled  down  to  millimeter  dimensions  with  micro- 
electromechanical  systems. 

Young,  M.R.,  Jiang,  S.H.,  Davies,  R.E.,  Walker,  J.G.,  Pike,  E.R.,  and  Bertero, 
M.,  1992,  Experimental  confirmation  of  super-resolution  in  coherent  con¬ 
focal  scanning  microscopy  using  optical  masks,  J.  Microsc.  165:131-138. 
Super-resolution  in  a  coherent  scanning  microscope  with  a  special  optical 
mask,  a  Fourier  lens,  and  detector  pin-hole  to  carry  out  optical  processing 
of  the  image.  The  form  of  the  special  mask  was  calculated  using  the  theory 
of  singular  systems. 

P.  PUPIL  ENGINEERING 

Pupil  masks  and  modifications  to  improve  the  PSF. 

Boyer,  G.,  2002,  New  class  of  axially  apodizing  filters  for  confocal  scanning 
microscopy,  J.  Opt.  Soc.  Am.  A  19:584-589.  Typical  of  a  large  group  of 
papers  on  pupil  engineering  to  enhance  the  PSF. 

Davis,  B.J.,  Karl,  W.C.,  Swan,  A.K.,  Unlu,  M.S.,  and  Goldberg,  B.B.,  2004, 
Capabilities  and  limitations  of  pupil-plane  filters  for  superresolution  and 
image  enhancement.  Opt.  Express  12:4150-4156.  Resolution  cannot  be 
improved  solely  by  adding  pupil-plane  filters  but  signal-to-noise  can  be. 

de  Juana,  D.M,  Oti,  J.E.,  Canales,  V.F.,  and  Cagigal,  M.P.,  2003,  Transverse  or 
axial  superresolution  in  a  4Pi-confocal  microscope  by  phase-only  filters, 
J.  Opt.  Soc.  Am.  A  20:2172-2178.  Resolution-improving  phase  functions 
differ  for  transverse  and  axial. 

Grochmalicki,  J.,  and  Pike,  R.,  2000,  Superresolution  for  digital  versatile  discs 
(DVD’s),  Appl.  Opt.  39:6341-6349.  It’s  good  to  be  reminded  that  most  of 
the  CMs  in  the  world  are  in  a  consumer  product. 

Gu,  M.,  and  Sheppard,  C.J.R.,  1994,  3-Dimensional  transfer-functions  in  4pi 
confocal  microscopes,  J.  Opt.  Soc.  Am  A  11:1619-1627.  The  effects  of 
apodization  are  taken  into  account. 

Gu,  M.  1996,  Effect  of  apodization  on  axial  resolution  in  confocal  microscopy 
of  high  aperture,  Optik  102:120-124.  Gu  and  Sheppard  have  a  series  of 
papers  on  pupil  engineering. 

Gustafsson,  M.G.L.,  2000,  Surpassing  the  lateral  resolution  limit  by  a  factor  of 
two  using  structured  illumination  microscopy,  J.  Microsc.  198:82-87.  A 
very  complex  system. 

Lin,  L.,  Wang,  X.H.,  Wang,  Z.Q.,  and  Mu,  G.G.,  2003,  Binary  phase-only  pupil 
filters  application  to  high-aperture  confocal  microscopy,  Optik  114: 
329-332. 

Lindek,  S.,  Cremer,  C.,  and  Stelzer,  E.H.K.,  1996,  Confocal  theta  fluorescence 
microscopy  with  annular  apertures,  Appl.  Opt.  35:126-130.  An  annular 
aperture  is  the  simplest  pupil  mask. 

Liu,  L.,  Deng,  X.Q.,  Yang,  L.S.,  Wang,  G.Y.,  and  Xu,  Z.Z.,  2000,  Effect  of  an 
annular  pupil  filter  on  differential  confocal  microscopy,  Opt.  Lett. 


25:1711-1713.  Extends  the  axial  dynamic  range  and  improves  transverse 
resolution  by  sacrificing  axial  resolution. 

Martinez-Corral,  M.,  Kowalczyk,  M.,  Zapata-Rodriguez,  C.J.,  and  Andres,  R, 
1998,  Tunable  optical  sectioning  in  confocal  microscopy  by  use  of 
symmetrical  defocusing  and  apodization,  Appl.  Opt.  37:6914-6921.  Very 
simple  to  implement. 

Martinez-Corral,  M.,  Pons,  A.,  and  Caballero,  M.T.,  2002,  Axial  apodization 
in  4Pi-confocal  microscopy  by  annular  binary  filters,  J.  Opt.  Soc.  Am.  A 
19:1532-1536.  Proper  pupil  engineering  (masking)  improves  the  PSF, 
even  in  4Pi. 

Neil,  M.A.A.,  Juskaitis,  R.,  Wilson,  T.,  Laczik,  Z.J.,  and  Sarafis,  V.,  2000,  Opti¬ 
mized  pupil-plane  filters  for  confocal  microscope  point-spread  function 
engineering.  Opt.  Lett.  25:245-247.  Using  reconfigurable  binary  optical 
elements. 

Schilders,  S.P,  Gan,  X.S.,  and  Gu,  M.,  1998,  Microscopic  imaging  through  a 
turbid  medium  by  use  of  annular  objectives  for  angle  gating,  Appl.  Opt. 
37:5320-5326.  To  exclude  multiply  scattered  photons. 

Sheppard,  C.J.R.,  Sharma,  M.D.,  and  Arbouet,  A.,  2000,  Axial  apodizing  filters 
for  confocal  imaging,  Optik  111:347-354.  A  more  recent  contribution  to 
the  series. 

Q.  THICKNESS 

Distinguishing  between  index  and  thickness  allows  measurement 
of  both. 

Fukano,  T.,  and  Yamaguchi,  I.,  1996,  Simultaneous  measurement  of  thicknesses 
and  refractive  indices  of  multiple  layers  by  a  low-coherence  confocal  inter¬ 
ference  microscope,  Opt.  Lett.  21:1942-1944.  Combining  with  OCT, 
multiple  layers  can  be  sorted  out. 

Fukano,  T.,  and  Yamaguchi,  I.,  1999,  Separation  of  measurement  of  the  refrac¬ 
tive  index  and  the  geometrical  thickness  by  use  of  a  wavelength-scanning 
interferometer  with  a  confocal  microscope,  Appl.  Opt.  38:4065^-073. 
A  wavelength-scanning  heterodyne  interferometer  works  better.  Later 
papers,  Appl.  Opt.  41:4497-4502  and  Appl.  Opt.  41:2414-2419,  elaborate 
on  this. 

Ilev,  I.K.,  Waynant,  R.W.,  Byrnes,  K.R.,  and  Anders,  J.J.,  2002,  Dual-confocal 
fiber-optic  method  for  absolute  measurement  of  refractive  index  and  thick¬ 
ness  of  optically  transparent  media.  Opt.  Lett.  27:1693-1695.  Two  inde¬ 
pendent  identical  apertureless  fiber-optic  CMs  with  a  single  2x2  fiber 
coupler. 

Jester,  J.V.,  Petroll,  W.M.,  and  Cavanagh,  H.D.,  1999,  Measurement  of  tissue 
thickness  using  confocal  microscopy,  Methods  Enzymol.  307:230-245.  A 
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Practical  Tips  for  Two-Photon  Microscopy 
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INTRODUCTION 

As  is  clear  from  a  number  of  the  chapters  in  this  volume,  2-photon 
microscopy  offers  many  advantages,  especially  for  living-cell 
studies  of  thick  specimens  such  as  brain  slices  and  embryos. 
However,  these  advantages  must  be  balanced  against  the  fact  that 
commercial  multiphoton  instrumentation  is  much  more  costly  than 
the  equipment  used  for  confocal  or  widefield/deconvolution.  Given 
these  two  facts,  it  is  not  surprising  that,  to  an  extent  much  greater 
than  is  true  of  confocal,  many  researchers  have  decided  to  add  a 
femtosecond  (fs)  pulsed  near-IR  laser  to  a  scanner  and  a  micro¬ 
scope  to  make  their  own  system  (Soeller  and  Cannell,  1996;  Tsai 
et  al.,  2002;  Potter,  2005).  Even  those  who  purchase  a  commercial 
multiphoton  system  find  that  it  helps  to  understand  a  bit  more  about 
how  to  optimize  the  performance  of  the  fs  laser  system.1  This 
Appendix  has  been  added  to  the  Handbook  to  provide  the  basic 
alignment  and  operating  information  that  such  people  need. 

First,  the  safety  announcement .  .  . 

LASER  SAFETY 

Light  sources  for  multiphoton  microscopy  are  almost  without 
exception  very  powerful  pulsed  lasers  (laser  class  IV).  It  is  vital 
that  any  personnel  who  perform  alignment  or  other  operations  that 
carry  a  risk  of  beam  exposure  are  familiar  with  and  follow  laser 
safety  regulations.  During  routine  operation  one  MUST  ensure 
that  accidental  exposure  to  the  pulsed  laser  beam  is  prevented  by 
providing  proper  shielding  and  interlocks. 

During  alignment,  protective  eyewear  is  not  an  option  —  it  is 
essential! 

See  http://www.osha.gov/SLTC/laserhazards/  for  US 
guidelines. 

LASER  ALIGNMENT 

Just  as  in  any  other  type  of  microscopy,  correct  optical  alignment 
is  crucial  for  achieving  optimal,  diffraction-limited  performance  in 
2-photon  microscopy.  The  alignment  of  external  lasers  such  as  the 
Ti :  S  or  similar  2-photon  sources  into  a  laser  scanning  microscope 
can  be  simplified  if  a  well-aligned  “internal”  or  reference  laser  is 
available.  In  commercial  confocal  microscopes,  typical  candidate 
lasers  include  Argon-ion  or  green  HeNe  lasers  or,  more  recently. 


1  The  Multiphoton  Users  Group  e-mail  network  at  (mplsm-users@ 
yahoogroups.com),  operated  by  Steve  Potter  at  Georgia  Tech,  enrolled  its 
500th  member  in  2003. 


blue  and  green  diode  lasers.  To  provide  an  alignment  beam  to 
which  the  external  laser  can  be  aligned,  light  from  this  reference 
laser  needs  to  be  bounced  back  through  the  microscope  optical 
train  and  out  through  the  external  coupling  port: 

CAUTION:  Before  you  switch  on  the  reference  laser  in  this 
configuration  make  sure  that  all  PMTs  are  protected  and/or 
turned  off. 

Place  a  front-surface  mirror  on  the  stage  of  the  microscope  and 
focus  onto  the  reflective  surface  using  an  air  objective  for  conve¬ 
nience  (at  sharp  focus,  you  should  be  able  to  see  scratches  or  other 
mirror  defects  through  the  eyepieces).  The  idea  of  this  method  is 
to  cause  the  reference  laser  beam  to  bounce  back  through  the 
optical  train  and  emerge  from  the  other  laser  port.  To  do  this,  select 
filter  settings  that  will  allow  some  of  the  light  from  the  internal 
laser  to  exit  the  chosen  coupling  port.  In  order  to  bring  two  laser 
beams  to  co-linearity,  a  beam-steering  device  is  essential.  A  single¬ 
mirror  beam  steerer  provides  angular  control  while  changing  the 
separation  between  the  mirrors  of  a  2-mirror  steerer  provides  beam 
translation  (Fig.  Al.l). 

It  is  also  possible  to  achieve  beam  translation  with  a  second 
angular  control  mirror.  After  adjusting  the  incoming  near-IR  beam 
to  an  intensity  where  it  can  be  viewed  without  totally  overwhelm¬ 
ing  the  reference  beam,2  adjust  one  mirror  to  make  both  laser  spots 
merge  at  the  surface  of  the  other  (angle-adjustable)  mirror.  Then 
that  mirror  is  adjusted  to  bring  the  beams  to  co-linearity.  We  find 
it  useful  to  use  a  piece  of  light-blue  paper  as  this  shows  the  dimmed 
infrared  beam  well.  If  the  laser  has  been  tuned  to  the  far  part  of 
the  spectrum,  you  may  have  to  use  an  IR  viewer  or  viewer  card  to 
visualize  the  beam. 


TESTING  ALIGNMENT  AND 
SYSTEM  PERFORMANCE 

On  a  regular  basis  and  particularly  subsequent  to  laser  alignment, 
the  performance  of  the  multiphoton  microscope  should  be  tested. 
The  prime  indicator  of  proper  alignment  of  an  imaging  system  is 
its  point-spread  function,  as  measured  by  using  a  sample  contain¬ 
ing  sub-resolution  fluorescent  beads.  A  test  slide  can  be  prepared 
by  letting  a  drop  of  diluted  beads  dry  onto  a  coverslip.  The  beads 
are  then  embedded  in  a  drop  of  Sylgard  elastomer  (Dow  Corning, 
USA)  with  a  microscope  slide  placed  on  top.  We  usually  use 
0.2pm  beads  from  Molecular  Probes  (Eugene,  OR).  These  are 
available  in  a  range  of  colors  suitable  for  2-photon  microscopy.  It 


2  As  is  explained  below,  this  can  be  achieved  by  over-closing  the  slit  and/or 
reducing  pump  power,  because  mode-locking  is  not  required.  We  typically 
use  <20mw  @  800nm  and  <10mW  at  720nm. 
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FIGURE  A1 .1 .  (A)  2D  simplification  of  the  beam  alignment  process  using  a  conventional  beam-steerer.  A  vertical  translation  of  a  tilted  mirror  is  used  to  bring 
the  two  beams  to  a  common  point  on  a  second,  tiltable  mirror.  (B)  Rotation  of  the  second  mirror  at  the  point  of  the  common  spot  makes  the  two  beams  co-linear. 
(C)  The  co-linear  beams  after  alignment. 


takes  only  about  30  minutes  to  prepare  10  slides  in  this  way.  Once 
the  elastomer  has  set,  these  slides  will  last  for  months  if  kept  in  a 
dark  drawer.  As  a  result,  they  provide  a  good  standard  to  check  the 
microscope  sensitivity  and  resolution  provided  you  have  recorded 
microscope  and  laser  settings  (including  center  wavelength,  laser 
power  and  bandwidth/pulse  length)  with  each  reference  image. 

With  proper  alignment,  the  beads  should  blur  approximately 
evenly  as  you  focus  above  and  below  them.  Asymmetric  blurring 
above-and-below  best  focus  indicates  spherical  aberration  while 
motion  of  the  centroid  of  intensity  means  that  the  objective 
aperture  is  filled  asymmetrically.  The  spatial  resolution  (without  a 
pinhole)  should  be  similar  to  confocal  performance,  values 
between  0.2-0. 4  pm  in  plane,  full-width  at  half  maximum 
(FWHM)  and  0.5-0. 8  pm  out  of  plane  (in  the  z  direction)  should 
be  attainable  when  using  a  high-numerical-aperture  (NA  ~1.3) 
objective. 

A  very  weak  and  noisy  signal  can  have  a  number  of  causes.  If 
there  is  no  problem  with  the  detectors  or  emission  filters  (most  of 
which  would  also  be  apparent  when  operating  the  microscope  with 
conventional  [1 -photon]  laser  excitation),  check  that  the  laser 
beam  fills  the  objective  rear  aperture  fully  and  evenly  by  rotating 
the  objective  turret  to  an  empty  position,  placing  a  lens  tissue  over 
the  opening  and  inspecting  the  pattern  of  illumination  (using  an  IR 
viewer  if  necessary).  The  beam  should  be  accurately  centered  in 
the  empty  socket  and  should  form  a  uniform  circle  of  light  that 
will  cover  the  rear  aperture  (~8-10mm  wide)  of  a  typical  objec¬ 
tive  lens.  If  the  light  intensity  at  the  rear  aperture  is  low  (<10mW) 
make  sure  that  no  IR-opaque  optical  items  are  obstructing  the  illu¬ 
mination  path.1 * 3  It  is  also  possible  that  the  beam  is  so  badly  mis¬ 
aligned  that  only  scattered  light  is  being  observed.  You  can  check 
for  this  by  ensuring  that  adjustments  of  the  alignment  mirrors  have 
the  expected  effects  on  the  spot  in  the  BFR 

If  the  microscope  is  a  combined  confocal/multiphoton  system, 
the  bead  slide  is  also  a  useful  tool  to  disclose  alignment  offsets 
between  the  2-photon  laser  system  and  any  other  lasers.  In  partic¬ 
ular  you  should  check  for  any  axial  offsets  (i.e.,  focus  shifts),  par¬ 


1 1f  little  light  is  coming  out  of  the  objective,  it  may  be  the  anti-reflection  coat¬ 

ings  that  are  at  fault.  Coatings  used  to  reduce  reflection  losses  in  the  visible 

may  become  mirrors  in  the  near-IR.  See  the  transmission  tables  in  Chapter  7 
and  its  Appendix. 


ticularly  if  the  system  is  to  be  used  for  2-photon  flash  photolysis 
or  combined  confocal  and  multiphoton  co-localization  studies. 

In  our  laboratory  we  perform  a  basic  system  test  with  a  pre¬ 
pared  bead  sample  on  a  daily  basis.  This  check  (usually  conducted 
following  system  startup)  is  well  worth  the  -5  minutes  it  takes, 
especially  if  it  helps  avoid  debugging  signal  problems  later  when 
a  precious  biological  sample  is  on  the  stage. 

LASER  SETTINGS  AND  OPERATION 

Historically,  the  mode-locked  lasers  used  for  2-photon  imaging 
could  be  quite  temperamental  and  ensuring  that  proper  laser  oper¬ 
ation  was  a  large  part  of  the  challenge  of  running  a  multiphoton 
microscope.  With  the  advent  of  fully  computer-controlled  turn-key 
laser  systems,  this  has  become  less  of  an  issue.  In  any  case,  as  the 
most  versatile  source  for  2-photon  imaging  is  still  the  tunable 
Ti :  S  laser  in  the  femtosecond  configuration,  we  will  focus  on  it 
here.  Regardless  of  whether  you  are  using  a  fully  automated  or  a 
manually  adjusted  Ti :  S  system,  it  is  important  to  monitor  and 
optimize  the  laser  output  before  imaging. 

The  choice  of  center  wavelength  is  generally  determined  by 
the  fluorochromes  to  be  excited.  As  a  general  rule  of  thumb  you 
should  try  to  use  the  longest  wavelength  compatible  with  the  dyes 
in  your  sample  as  this  will  help  minimize  photodamage  and  also 
reduce  scattering  of  the  excitation  light.  Data  on  excitation  spectra 
is  now  available  from  many  sources  in  the  literature  and,  if  in 
doubt,  there  are  mailing  lists  where  one  can  ask  other  researchers 
for  advice  (see  http://groups.yahoo.com/group/mplsm-users/  and 
http://listserv.acsu.buffalo.edu/archives/confocal.html). 

MONITORING  LASER  PERFORMANCE 

During  tuning  and  imaging,  laser  operation  can  be  very  conve¬ 
niently  monitored  using  a  spectrum  analyzer.  We  use  a  system 
made  by  Rees  Instruments  (currently  available  models  include  the 
Rees  E200  series  laser  spectrum  analyzers  by  Imaging  and  Sensing 
Technology  Ltd.,  Alton,  UK)  to  monitor  a  secondary  beam  con¬ 
taining  only  a  small  fraction  of  the  total  output  power.  During  laser 
tuning,  this  device  allows  one  to  measure  the  center  wavelength 
and,  more  importantly,  the  width  of  the  spectrum.  The  spectral 
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width  of  the  beam,  as  displayed  by  the  analyzer,  provides  the  feed¬ 
back  for  optimizing  the  slit  width  and  position  to  obtain  mode- 
locked  operation  (with  manually  tuned  laser  systems).  The  start  of 
mode-locked  operation  is  indicated  by  the  change  of  the  spectral 
shape  from  one  or  a  small  number  of  sharply  defined  lines  which 
indicate  continuous  wave  (CW)  operation,  see  Figure  A  1.2A,  to  an 


approximately  Gaussian- shaped  output  spectrum  which  may  have 
a  spike  (Fig.  A1.2B)  indicating  CW  breakthrough.  Optimal  closure 
of  the  slit  leads  to  a  smooth  Gaussian-like  spectrum  (Fig.  A1.2C) 
which,  in  this  case,  is  ~5nm  wide  (FWHM).  At  750  nm  this  spec¬ 
tral  width  implies  a  120fs  pulse.  Closing  the  slit  further  can  lead 
to  an  oscillation  of  pulse  amplitude  (Q-switching),  which  is  shown 


FIGURE  A1.2.  (A-F)  show  spectrum  analyzer  output 
during  Ti :  S  tuning.  The  small  gradations  at  the  bottom 
indicate  lnm.  (A)  Before  mode-locking,  the  spectrum 
consists  of  a  few  narrow  spikes.  (B)  With  mode-locking 
underway,  the  spectrum  increases  in  width,  but  the  spike 
indicates  CW  breakthrough.  To  cure  this,  the  slit  needs 
to  be  closed  more.  (C)  Optimal  operation,  the  slit  has 
been  closed  just  enough  to  stop  CW  but  at  the  same  time 
not  so  closed  that  Q-switching  starts,  a  mode  of  behav¬ 
ior  shown  in  (D,E).  To  stop  Q  switching,  more  prism 
must  be  inserted  into  the  beam  path  (which  will  increase 
system  bandwidth)  and/or  the  slit  needs  to  be  opened 
(or  even  a  reduction  in  pump  power).  (F)  shows  the  short¬ 
est  pulse  that  can  be  readily  achieved  with  our  Coherent 
MIRA  900F  system.  The  FWHM  of  the  spectrum  is 
~14nm  at  a  center  wavelength  of  ~750nm.  (G)  shows  the 
relationship  between  the  FWHM  of  the  spectrum  and  the 
pulse  width  for  a  transform-limited  sec2  pulse,  with 
center  wavelength  indicated  next  to  each  curve.  In  this 
case,  a  14nm  bandwidth  from  our  laser  (F)  implies  a 
very  short  pulse  width  of  40  fs.  Generally,  we  use 
longer  pulses  (~120fs  e.g.,  C)  than  this  in  imaging 
experiments. 
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in  the  spectrum  as  oscillations  (Figs.  A1.2D,  A1.2E)  and  should  be 
removed  by  re-opening  the  slit  or  increasing  the  intra-cavity  group 
velocity  dispersion  by  moving  the  intra-cavity  prism  further  in.  By 
suitable  adjustment  of  the  slit  and  the  intra-cavity  group  velocity 
dispersion,  the  pulse  may  be  shortened  and  this  will  be  reflected 
in  an  increase  the  width  of  the  output  spectrum  (Fig.  A1.2F). 
With  our  laser,  a  14nm  FWFIM  bandwidth  can  be  achieved 
corresponding  to  a  ~40fs  pulse  at  750  nm.  During  imaging,  Q- 
switching  manifests  itself  as  a  sudden  increase  in  image  noise  due 
to  aliasing  between  laser  excitation  and  the  pixel  clock.  A  quick 
look  at  the  spectrum  should  indicate  if  the  laser  needs  tuning  to 
remove  this  source  of  image  noise. 

Typically,  pulses  leaving  a  commercial  Ti :  S  laser,  as  used  for 
2-photon  microscopy,  are  ~100fs  long.  Pulse  length  is  an  impor¬ 
tant  variable  that  is  most  accurately  determined  with  an  optical 
autocorrelator.  However,  from  a  practical  point  of  view,  a  spec¬ 
trum  analyzer  is  easier  to  use  than  an  autocorrelator  and  gives  suf¬ 
ficient  information  on  laser  performance.  The  length  of  the  laser 
pulse  is  inversely  proportional  to  the  spectral  FWHM  during 
mode-locked  operation.  Figure  A1.2G  shows  this  relationship  for 
various  center  wavelengths. 

POWER  LEVELS  AND  TROUBLE-SHOOTING 

In  our  experience  illumination  power  levels  at  the  sample  should 
be  kept  <20  mW  in  living  cells  to  minimize  the  risk  of  cell  damage, 
although  that  figure  is  dependent  on  the  nature  of  the  experiment, 
the  2-photon  absorber,  the  objective  NA,  and  sample  scattering. 

Problems  with  mode-locked  lasers  in  3D  microscopic  imaging 
most  often  arise  from: 

1.  Pump  laser  noise  (amplitude  noise  or  beam-pointing 
instability). 

2.  Pump  laser  alignment. 

3.  Dirt  on  mirrors. 

4.  Poor  alignment  within  the  cavity. 

5.  Stray  reflections  from  surfaces  that  reflect  energy  back  into 
the  cavity. 

6.  Poorly  trained  personnel  changing  the  alignment  between  the 
pump  and  the  prisms  of  the  Ti :  S  cavity  over  time. 

7.  UFM  (unidentified  fingerprints  on  mirrors!). 

8.  Air  currents  that  affect  beam-pointing  stability. 

9.  Loss  of  alignment  of  laser  to  microscope. 

10.  Poor  matching  of  laser  beam  profile  to  microscope  aperture. 

To  address  problems  1 — 4,  the  manufacturer  generally  provides 
troubleshooting  advice  that  should  be  consulted.  Problem  5  can  be 
avoided  by  using  an  optical  isolator,  i.e.,  a  device  which  allows 
light  to  pass  only  in  the  forward  direction  but  blocks  back  reflec¬ 
tions.  A  simpler  workaround  (that  has  worked  well  in  our  hands) 
involves  slightly  tilting  strongly  reflecting  surfaces  (e.g.,  neutral 
density  filters  —  see  below)  with  respect  to  the  optical  axis.  For 
laser  safety  you  should  provide  an  appropriate  beam  dump  for  any 
strong  reflections  off  the  optical  axis.  Problems  6  and  7  should  be 
resolved  by  the  system  manager.  Problem  8  can  be  reduced  by  sur¬ 
rounding  all  beams  with  plastic  tubes.  Problem  9  can  be  ascer¬ 
tained  using  a  reference  laser,  especially  a  laser  built  into  the 
microscope  itself.  Problem  10  arises  from  the  laser  beam  being  too 
small  to  fully  fill  the  objective  rear  aperture  (so  a  loss  of  resolu¬ 
tion  occurs)  or  too  large,  in  which  case  there  is  a  loss  of  intensity 
at  the  sample.  In  both  cases,  the  problem  can  be  fixed  using  laser 
beam  expansion  (or  compression)  with  a  telescope  (Galilean  beam 
expander).  In  our  microscope  we  use  ~4x  expansion  of  the  Ti :  S 


laser  beam  as  a  reasonable  compromise  between  filling  the  real 
aperture  adequately  and  throughput.  (We  built  a  simple  expander 
from  a  plano-convex  and  a  plano-concave  lens  which  were  single¬ 
layer  antireflection  coated.)  In  addition,  by  focusing  the  beam 
expander  carefully,  it  is  possible  to  minimize  the  axial  shift  of  focal 
plane  between  visible  light  and  the  IR. 

CHOICE  OF  PULSE  LENGTH 

The  dispersion  of  the  pulse  by  the  microscope  optics  is  typically 
>2000  fs2  at  800  nm.  This  suggests  that  the  shortest  pulse  width  that 
can  be  delivered  to  the  sample  would  be  >100fs  unless  group 
velocity  dispersion  compensation  is  performed  to  “prechirp”  the 
pulse  (Soeller  and  Cannell,  1996).  Shorter  pulses  increase  the  ratio 
of  3-  to  2-photon  excitation  and,  since  3-photon  excitation  at 
800  nm  would  correspond  to  hard  UV,  such  excitation  is  generally 
undesirable.  We  therefore  suggest  that  for  routine  operation 
~120fs  pulses  are  probably  optimal.  Perhaps  paradoxically,  in  the 
absence  of  GVD  compensation,  a  shorter  pulse  at  the  laser  is  trans¬ 
lated  to  a  much  longer  pulse  at  the  sample.  As  it  is  hard  to  run  a 
conventional  Ti:S  laser  with  pulses  longer  than  ~150fs,  longer 
pulses  at  the  sample  may  be  produced  by  making  very  short  pulses 
(e.g.,  40 fs)  at  the  laser.  See  Chapters  5  and  28  for  further  discus¬ 
sion  on  pulse  broadening. 

CONTROLLING  LASER  POWER 

Being  able  to  control  laser  power  electronically  is  useful  because 
it  permits  rapid  suppression  of  the  beam  at  the  end  of  each  scan 
line  where  the  beam  slows  and  stops  before  retracing  its  path.  This 
slow  movement  subjects  the  parts  of  the  specimen  at  either  side  of 
the  raster  to  very  high  integrated  excitation  which  is  very  damag¬ 
ing.  Unfortunately,  the  acousto-optic  modulators  (AOM),  which 
are  commonly  used  for  this  purpose  in  visible  light  microscopes, 
are  less  suitable  for  2-photon  because  heating  and  birefringent 
effects  in  the  crystal  reduce  beam  intensity  stability.  The  square- 
law  dependence  of  2-photon  excitation  on  input  power  amplifies 
this  instability  at  the  sample.  Additionally,  because  the  mode- 
locked  laser  beam  has  significant  bandwidth  (compared  to  a  CW 
laser)  the  beam  will  be  dispersed  if  it  is  diffracted  in  the  AOM.  As 
the  rear  aperture  of  the  objective  must  be  overfilled,  this  disper¬ 
sion  results  in  a  loss  of  bandwidth  and  therefore  a  longer  and  mis¬ 
shapen  pulse.  This  effect  can  be  avoided  if  one  uses  the  zero-order 
(i.e.,  undiffracted)  beam  of  the  AOM  for  microscopy  and  the  first 
order  beam  is  used  simply  to  extract  energy  from  it.  However,  as 
only  about  75%  of  the  beam  can  be  diffracted  out,  this  approach 
only  reduces  the  beam  to  25%  of  the  input  power. 

A  better  alternative  is  to  use  a  Pockels  cell.  While  more  expen¬ 
sive,  these  devices  are  much  faster  and  more  controllable  than  an 
AOM,  but  they  also  suffer  from  some  problems: 

1 .  The  Pockels  cell  has  a  limited  lifetime  that  is  dependent  on  the 
time  spent  in  the  energized  state. 

2.  Alignment  is  critical:  the  full  power  of  the  beam  must  pass 
cleanly  through  the  free  aperture  and  not  touch  the  interior  of 
the  cell  under  any  circumstance  or  damage  will  result. 

3.  High  voltages  are  present. 

It  should  be  noted  that,  for  ~120fs  pulses,  dispersive  broadening 
by  the  Pockels  is  generally  small  and  should  therefore  be  of  no 
concern  when  it  is  used  in  a  2-photon  imaging  setup. 

If  rapid  beam  modulation  is  not  needed,  laser  power  can  be 
controlled  by  neutral  density  filters  or  a  polarizer.  Such  neutral 
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density  filters  need  to  be  of  the  reflecting  type  as  high  powers 
destroy  absorbing  filters.  The  beam  reflected  from  the  filter  needs 
to  be  absorbed  by  something  for  safety  and  we  use  a  “beam  dump” 
made  of  black  anodized  aluminum  with  a  machined  recess  so  it  is 
hard  to  see  the  dumped  beam.  Since  the  output  beam  of  the  laser 
is  polarized,  beam  intensity  may  be  modulated  by  rotating  a  polar¬ 
izer  in  front  of  it.  Glass  Glan-Thompson  polarizers  can  be  used  but 
plastic  polarizers  are  quite  unsuitable  for  typical  power  levels  as 
they  melt  (see  also  Attenuation  of  Laser  beams  in  Chapter  5,  this 
volume. )! 


AM  I  SEEING  TWO-PHOTON  EXCITED 
FLUORESCENCE  OR  . . . 

Sometimes  it  is  unclear  if  a  detected  signal  is  due  to  multiphoton- 
excited  fluorescence  or  if  it  is  due  to  optical  bleed-through  of  the 
(much  more  intense)  near-IR  excitation  light.  Such  bleed-through 
can  occur,  for  example,  if  one  uses  filters  with  an  unknown 
response  in  the  near-IR  region.  A  simple  test  to  distinguish  between 
these  possibilities  can  be  made  by  taking  a  control  image  with  the 
multiphoton  laser  source  running  in  CW  mode  (at  similar  power). 
When  using  a  mode-locked  Ti :  S  laser  with  manually  operated  slit 
this  can  easily  be  achieved  by  over-closing  the  slit  until  mode¬ 
locking  is  lost  and  then  reopening  the  slit  with  the  starter  mecha¬ 
nism  disabled.  If  the  signal  in  question  disappears  when  using  CW 
illumination,  it  must  be  due  to  some  sort  of  multiphoton  excitation 
(2-  or  3 -photon  fluorescence,  or  second-  or  third-harmonic  gener¬ 
ation).  However,  this  simple  test  does  not  replace  the  more  complex 
illumination-power  vs.  signal-intensity  measurements  needed  to 
fully  characterize  each  of  these  high-order  excitation  processes. 


STRAY  LIGHT  AND  NON-DESCANNED 
DETECTION 

One  of  the  attractions  of  2-photon  microscopy  resides  in  the 
improved  penetration  depth  obtained  when  imaging  in  strongly 
scattering  biological  samples  such  as  brain  slices  (see  Soeller  and 
Cannell,  1999).  Central  to  this  advantage  is  the  need  to  collect 
emitted  photons  that  are  also  scattered  and  so  may  not  be  focused 
by  the  microscope  optics  and  are  therefore  lost  at  intermediate 
apertures.  This  problem  can  be  overcome  by  using  a  photomulti¬ 
plier  tube  that  is  mounted  close  to  the  sample  (so  that  the  emitted 
light  does  not  pass  through  the  scanning  system)  to  create  a  “non- 
descanned  detector.”  Such  detectors  are  arranged  so  that  any 
photons  of  the  right  color,  regardless  of  where  they  originate,  are 
directed  onto  the  photocathode.  As  a  result,  non-descanned 
detection  is  also  far  more  likely  to  pick  up  stray  light  from  the 
microscope  surroundings  than  conventional  confocal  optics.  For 
example,  in  a  normal  laboratory,  light  from  computer  screens  and 
equipment  LEDs  can  cause  a  strong  background  signal  even  when 
the  room  lights  are  turned  off.  To  shield  your  setup  from  this  stray 
light,  you  may  need  to  fabricate  suitable  shields  around  the  sample 
from  black  material.  Alternatively,  you  may  shield  the  whole 
microscope  from  the  surroundings  by  enclosing  it  in  a  completely 
light  tight  box.  This  can  be  conveniently  combined  with  electrical 
shielding  by  providing  a  Faraday  cage  around  3  sides  and  the  top 
of  the  instrument  and  fully  closing  it  during  imaging  by  drawing 
a  black  curtain  or  blind  across  the  fourth  side.  For  safety  reasons, 
this  cage  and  any  blinds  or  curtain  should  be  made  from  fire-proof 
materials. 


LASER  POWER  ADJUSTMENT  FOR  IMAGING 
AT  DEPTH 

Although  2-photon  excitation  penetrates  deeper  into  scattering 
samples  (such  as  brain),  the  loss  of  peak  excitation  power  at  the 
focus  caused  by  scattering  and  spherical  aberration  still  leads  to  a 
loss  of  signal  at  depth.  The  solution  to  this  problem  is  to  alter  the 
illumination  power  as  a  function  of  depth  and  this  is  where  the 
intensity  modulation  provided  by  the  Pockels  cell  may  be  used 
to  advantage.  There  are  alternative  ways  to  achieve  changes  in 
illumination  power  but  all  assume  that  the  maximum  power 
available  from  the  2-photon  laser  is  higher  than  is  needed  for 
normal  operation.  Thus,  a  wheel  of  reflective  neutral  density  (ND) 
filters  may  be  placed  in  the  beam  path,  providing  intensity  control 
to  quantized  levels  appropriate  for  different  imaging  depths. 
A  second  option  is  to  use  a  continuously  variable  reflective 
neutral  density  filter,  which  allows  more  precise  control  over  laser 
power,  but  requires  either  manual  rotation  during  imaging  or  a 
motorized  filter  wheel.  We  suggest  that  the  ideal  solution  is  to 
automatically  attenuate  the  laser  beam,  using  a  Pockels  cell 
supplied  with  a  varying  drive  voltage  controlled  by  the  focus 
position. 

In  our  experience,  the  laser  power  needs  to  increase  (roughly) 
exponentially  with  depth  (e.g.,  see  Fig.  A1.4  in  Soeller  and 
Cannell,  1999)  but  the  exponential  factor  is  highly  dependent  on 
the  sample.  Thus  a  control  experiment  may  be  needed  where  a 
similar  sample  is  labeled  with  fluorescent  beads  (~2  |im  in  diame¬ 
ter).  For  brain  slices,  or  other  tissues  which  can  be  perfused,  this 
can  be  achieved  by  injecting  the  beads  into  a  blood  vessel  before 
slicing.  By  imaging  the  beads  at  different  depths,  the  depth  depen¬ 
dence  of  the  excitation  may  be  determined  and  used  in  subsequent 
experiments.  (Using  beads  will  give  more  reliable  results  than 
simply  staining  the  entire  specimen  with  a  dye  as  this  avoids  prob¬ 
lems  arising  from  non-uniform  staining.)  It  is  important  to  note 
that  not  all  the  signal  loss  is  due  to  reduced  excitation  as  emitted 
light  is  also  lost  by  scattering  and  adsorption.  Thus,  even  if  com¬ 
plete  compensation  of  signal  loss  with  depth  can  be  achieved  by 
raising  excitation  power,  it  is  better  to  err  on  the  side  of  caution  as 
delivering  too  much  power  into  the  preparation  at  any  depth  may 
lead  to  other  concerns  —  for  example,  heating  and  other  higher- 
order  effects  (see  Chapter  38,  this  volume).  If  lOOmW  were  deliv¬ 
ered  and  (eventually)  absorbed  within  10mm3  of  tissue,  the 
average  rate  of  rise  of  temperature  would  be  2.5°C/s.  Although  this 
power  is  close  to  the  maximum  that  may  be  achieved  by  typical 
2-photon  microscopes,  it  is  clearly  in  the  range  where  heating 
effects  could  become  a  serious  problem. 


SIMULTANEOUS  IMAGING  OF 
MULTIPLE  LABELS 

Another  advantage  of  2-photon  excitation  is  that  the  2-photon  exci¬ 
tation  spectra  of  fluorochromes  are  wider  than  their  1 -photon  coun¬ 
terparts.  Multiple  labels  may  therefore  be  imaged  simultaneously 
by  using  a  single  excitation  wavelength  and  multiple  detectors  with 
appropriate  optics  to  isolate  each  different  emitted  wavelength. 
This  approach  has  several  benefits:  (1)  Removal  of  offset  problems 
caused  by  non-confocality  of  different  lasers;  (2)  reduction  in 
imaging  time  (which  may  be  important  for  imaging  of  live-cell 
processes);  (3)  reduction  in  the  total  amount  of  laser  exposure  to 
the  tissue  and  (4)  avoidance  of  chromatic  aberrations.  Care  must 
be  taken  to  ensure  that  bleed-through  from  one  channel  to  another 


Practical  Tips  for  Two-Photon  Microscopy  •  Appendix  1  905 


is  minimized  by  the  use  of  the  optimal  beam-splitters  (see  also 
Chapter  3,  this  volume).  If  some  bleed-through  is  unavoidable  then 
an  accurate  measurement  of  the  amount  of  bleed-through  can  be 
made  by  imaging,  in  all  channels,  control  slides  that  contain  the 
individual  fluorochromes.  From  these  measurements,  contribu¬ 
tions  from  bleed-through  from  one  channel  to  another  can  be  esti¬ 
mated  and  removed  by  subtraction  during  post-imaging  analysis 
(so-called  “spectral  unmixing”). 

MINIMIZE  EXPOSURE  DURING  ORIENTATION 
AND  PARAMETER  SETTING 

In  most  applications,  imaging  parameters  need  to  be  established 
by  trial  and  observation  prior  to  the  commencement  of  image 
acquisition.  Common  examples  are  scanning  across  tissue  looking 
for  “that  cover  image’’  and  then  establishing  the  upper  and  lower 
limits  of  a  volume  of  interest.  While  the  use  of  2-photon  excita¬ 
tion  prevents  photobleaching  above  and  below  the  focal  plane,  in¬ 
plane  photobleaching  can  be  severe  and  care  must  be  taken  to 
avoid  over-exposure  of  samples  to  illumination  light  during  these 
adjustment  procedures. 

The  key  is  to  think  before  imaging.  For  example,  if  the  sample 
needs  to  be  located  in  focus,  is  full  power  really  necessary  or  will 
the  detection  of  just  a  few  photons  be  sufficient?  It  follows  that 
during  setup,  the  detector  gain  should  always  be  set  high  and  laser 
power  as  low  as  possible.  Single  scans  should  be  used  in  prefer¬ 
ence  to  continuous  scanning.  Can  the  sample  be  moved  to  an  unex¬ 
posed  region  once  the  acquisition  parameters  are  set?  Once  the 
correct  settings  have  been  determined,  then  laser  power  can  be 
increased  for  actual  imaging  and  focal-plane  bleaching  indicates 
that  the  maximum  amount  of  information  available  has  been 
extracted  from  the  dye  in  the  sample. 

ULTRAVIOLET-EXCITED  FLUOROCHROMES 

The  use  of  ultraviolet  (UV)  excited  dyes  in  1 -photon  imaging  is 
restricted  by  the  opacity  of  conventional  optical  components  at  UV 
wavelengths  as  well  as  by  chromatic  aberrations  and  by  the  cost 


and  size  of  UV  lasers.  2-photon  excitation  of  UV  dyes  does  not 
suffer  from  these  problems  because  the  excitation  wavelengths  are 
near-infrared,  in  a  range  that  is  compatible  with  normal  optics. 
The  ability  to  use  UV  dyes  allows  more  labels,  and  colors,  to  be 
used  in  multiple-labeling  experiments.  In  addition,  combining  a 
UV-excited  probe  emitting  in  the  blue  part  of  the  spectrum 
allows  greater  spectral  separation  from  a  yellow-red  label.  UV 
dyes,  in  general,  may  be  excited  by  2-photons  at  wavelengths  <750 
nm.  For  example,  the  AlexaFluor  350  fluorochromes  (Molecular 
Probes,  Eugene,  OR)  come  in  a  range  of  forms.  The  near- 
UV-excited  nucleic-acid  probes  DAPI  and  Hoechst  are  often  so 
well  excited  using  2-photon  illumination  that  it  is  necessary  to 
use  very  low  concentrations  to  prevent  bleed-through  into  other 
channels. 
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Appendix  2 

Light  Paths  of  the  Current  Commercial  Confocal  Light 
Microscopes  Used  in  Biology 

James  B.  Pawley 


INTRODUCTION 

Since  biologists  became  aware  of  the  confocal  microscope  in  the 
late  1980s,  numerous  optical  designs  have  been  introduced  by 
manufacturers  to  try  to  meet  the  often-contradictory  requirements 
of  the  biological  microscopist.  Although  many  of  these  designs  are 
discussed  at  greater  length  in  other  chapters  of  the  Handbook,  it 
was  thought  that  it  might  be  both  useful  to  the  reader,  and  fairer 
to  those  designs  not  discussed  elsewhere,  to  provide  the  reader 
with  a  concise  compilation  of  all  the  designs  now  available. 

To  that  end  I  requested  optical  diagrams  and  tabular  informa¬ 
tion  from  all  of  the  major  suppliers  of  the  instruments  used  by  biol¬ 
ogists  for  3D  microscopy1  and  the  items  that  they  provided  make 
up  the  bulk  of  this  Appendix.  Often  manufacturers  were  hesitant 
to  provide  specific  information  about  details  such  as  PMTs  or  scan¬ 
ning  speeds  etc.,  because  they  realized  that  there  was  a  good 


1  We  have  neglected  to  include  any  information  on  the  systems  for  widefield/ 
deconvolution  only  because  the  optical  paths  of  such  systems  are  fairly 
straightforward,  and  not  in  need  of  explanation. 


chance  that  such  data  would  go  out  of  date  with  their  next  product 
announcement.  However,  I  tried  to  apply  the  same  criteria  to  all 
the  contributors  and  this  is  as  good  a  place  as  any  to  thank  the  man¬ 
ufacturers  for  their  splendid  cooperation. 

To  assist  the  reader,  some  of  the  optical  information  consid¬ 
ered  most  relevant  to  the  optical  performance  of  these  instruments 
has  been  collected  in  Table  A2.1.  Although  such  a  table  cannot 
contain  all  of  the  relevant  information  about  such  complex  instru¬ 
ments,  the  headings  have  been  chosen  to  reflect  those  specifica¬ 
tions  indicated  to  be  of  prime  importance  in  the  other  chapters  of 
the  Handbook.  Abbreviations  are  explained  in  the  footnote. 

Of  course,  the  manufacturers  are  correct  about  this  informa¬ 
tion  going  out  of  date.  Fortunately  the  WWW  is  now  there  to  bring 
you  up  to  date.  Even  when  the  models  are  all  different,  we  hope 
that  the  you  find  the  column  headings  in  the  table  of  optical  param¬ 
eters  useful  as  the  basis  of  questions  you  might  ask  about  future 
models. 

There  has  been  no  effort  to  compare  the  computer  operating 
systems  used  to  control  these  instruments.  I  wish  to  emphasize  that 
this  is  not  because  I  think  such  details  unimportant,  but  rather 
because  software  systems  tend  to  change  with  great  speed  and, 
in  addition,  operating  systems  are  probably  best  assessed  in  person. 
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FIGURE  A2.1.  Schematic  of  the  BD-CARV  II 
light  path.  The  variable  intensity  light  from  a 
Hg/metal  halide  light  source  passes  through  an  exci¬ 
tation  filter  before  being  defieted  by  a  dichroic  mirror 
towards  the  sample.  The  excitation  light  passes 
through  a  Nipkow  spinning-disk  containing  multiple 
sets  of  spirally-arranged  pinholes  placed  in  the 
intermediate-image  plane  of  the  objective  lens.  The 
column  of  excitation  light  is  projected  through  1000 
pinholes  to  simultaneously  scan  the  entire  field  once 
every  millisecond,  thereby  creating  a  full  image  of 
the  focal  plane  in  real-time.  The  emitted  light  passes 
through  the  dichroic  mirror  and  the  emission  filter 
before  either  entering  the  CCD  camera  or  the  binoc¬ 
ular  eye-piece.  The  pinhole  disk  can  be  moved  in  and 
out  of  the  light  path  to  produce  a  confocal  or  a 
widefield  fluorescence  image.  A  variable  slit  at  the 
image  plane  can  be  used  to  selectively  illuminate  an 
area  of  the  sample  allowing  Fluorescence  Recovery 
After  Photobleaching  (FRAP)  to  be  performed.  All 
movable  parts  including  the  filter  wheels,  spinning- 
disk  shutters,  and  mirrors  are  automated  and  are  con¬ 
trolled  via  touchpad  or  third-party  software.  Figure 
kindly  provided  by  BD-Biosciences,  (Rockville, 
MS). 
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FIGURE  A2.2.  Schematic  of  the  LaVision-BioTec  TriM-Scope  light  path.  Multifocal  multiphoton  microscopy  using  a  beamsplitter  built  with  flat  optics.  Light 
from  a  fs,  near-IR,  pulsed  laser  first  passes  a  polarizing  attenuator  and  a  beam-expander  before  entering  a  pre-chirp  compensator.  It  is  then  formed  into  as  many 
as  64  beams  of  equal  intensity  and  spacing  by  being  reflected  from  an  array  of  sliding,  planar,  optical  elements.  The  linear  array  of  beams  is  then  deflected  by 
2,  closely- spaced  galvanometer  mirrors  and  fed  into  the  microscope  by  being  reflected  off  a  high-pass  beam-splitter.  Two-photon-excited  fluorescence  from  any 
dye  located  at  the  focus  plane  of  the  objective  passes  through  the  short-pass  dichroic,  and  barrier  filters  to  a  CCD  camera  or  other  photodetector.2  Because  of 
the  large  number  of  parallel  beams  and  the  high-QE  of  the  CCD  camera,  it  is  possible  to  obtain  useful,  optical-section  images  at  up  to  3.5  k  frames/second  and, 
because  the  system  relies  on  2-photon  excitation,  bleaching  is  restricted  to  the  focal  plane.  For  more  discussion  see  Chapter  29,  this  volume. 


T.  Nielsen,  M.  Fricke,  D.  Hellweg,  R  Andresen,  (2001),  High  efficiency  beam  splitter  for  multifocal  multiphoton  microscopy,  J.  Microsc.,  201:368-376. 
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Table  A2.1.  Optical  Parameters  of  Current  Commercial  Confocal  Microscopes 


Company 

Model 

Lasers/Arc 

Retrace 

protect/Laser 

atten 

Fiber 

optics 

Pre-optics 

Beam 

expander 

Scanner 

Fastest 
line  scan, 
Hz 

Pixel 

times 

Scanned 
field, 
diam.  int 
im.  plane 

Largest 

Raster 

Pixel 

BD- 

Biosciences 

CARV  II 

X-Cite  120 
Hg/halide 
arc,  8-place 
filter  wheel 

NA/intensity 
controlled  by 
aperture 

Liquid- 
filled  light 
guide 

NA 

NA 

Single-sided 
Petran  disk 

1  k  fps 

5  k  rpm 

~1  JIS 

21  mm 

CCD 

Lavision- 

Biotech 

TriMScope1 2 3 

Ti-sapph 
750-100  nm, 
100  fs  pulse 

NA/ 

Attenuator 

0.1-100% 

Laser  is 
coupled 
directly 

NA 

Yes 

1-64  beams 
scanned  by  2 
galvos 

3.5  k  fps 

>  500  ns 

20  mm, 
adapted  to 
CCD  used 

CCD 

Leica 

TCS  SP2  AOBS 

Many, 

35 1-633  nm 

Yes/3 

AOTFs 

SM-PP 

Laser-merge 

Adjustable 

Rotatable  k-scan 

1.4  k  or 

2.8  k  in 
bi-direct 

>500  ns 

22  mm 

4096  x  4096 

MP  RS 

Ti-Sapph 

Yes,  EOM 

SM-PP 

Laser-merge 

Adjustable 

Rotatable  k-scan 

4  k,  or  8k 
in  bi- 
direct 

>500  ns 

22  mm 

4096  x  4096 

Nikon 

Cl -plus 

Up  to  3, 
408-638  nm 

AOM  (opt) 

SM-PP 

Laser-merge 

Fixed 

2  close  galvos 

500,  lk 
in  bi- 
direct 

>1.68us 
@512  x 
512 

17  mm 

2048  x  2048 

Clsi 

Up  to  3, 
408-638  nm 

AOM 

SM-PP 
laser  input 
fiber.  MM 
emission 
fiber 

Laser-merge 

Fixed 

2  close  galvos 

500,  lk 
in  bi- 
direct 

From 

4.08us 

at 

512x512 

in 

spectral 

mode 

17  mm 

w/scan 

rotation 

Up  to 

512  x 

512  in 

spectral 

mode 

Olympus 

FV  300 

Many, 

405-633  nm 
IR  port 

Yes,  AOTF 

SM-PP 

Laser-merge 

Fixed 

2  close  galvos 

1  k  or  2  k 
bi-direct 

>2  (is 

20  mm 

2048  x  2048 

FV  1000 

Many, 
active 
stabilizer, 
351-633  nm, 
IR  port 

Yes,  AOTF 

SM-PP 

Laser-merge 

Fixed 

2  pair  of  close 
galvos,  separate 
image/bleach 
scanners, 

circular  bleach 

2k  or  4k 
bi-direct 

>2  |is 

18  mm 

4096  x  4096 

DSU 

Hg  arc 

NA/intensity 
controlled  by 
aperture 

NA 

NA 

NA 

Interchangeable 
single-sided  slit- 
pattern  disk, 

3  krpm 

3  krpm, 

15  fps 

>1  (IS 

18  mm 

CCD 

Visitech 

VT-infinity 

VT-Eye 

Many, 

405-647  njm 

Many, 

351-647  nm 

Yes,  AOTF 

Yes,  AOTF 

SM-PP 

SM-PP 

Laser-merge 

(opt) 

Laser-merge 

(opt) 

NA 

Fixed 

Single  galvo 
scans  an  array 
of  point  sources 

1  galvo,  1  AOD 

2  kHz 

50  kHz 

2|is 

20-125  ns 

17 

CCD 

1024  x  1024 

Yokogawa5 

CSU  10 

2  or  3  lines6 

NA/AOTF 

SM 

3.5  pm 
core 

Laser-merge 

Fixed 

Double  Petran 

Disk  w/micro- 
lenses 

1800  rpm, 
360  fps 

~1  (IS 

13  x  9.5  mm 

CCD 

CSU  22 

3  or  4  laser 
lines 

NA/AOTF 

SM 

3.5  pm 

core 

Laser-merge 

Fixed 

Double  Petran 
Disk  w/micro- 
lenses 

Variable, 
to  5  k 
rpm,  1  k 
fps 

~1  ps 

13  x  9.5  mm  CCD 

Zeiss 

LSM510META 

Many, 

351-633  nm 
(Ti :  Sapph) 

Yes,  AOTF 
0.05-100% 
(AOM) 

SM-PP 

Laser-merge 

Adjustable 

2  close  galvos 

1.3  k  or 

2.6  k  in 
bi-direct 

640  ns- 
2.3  ms 

18  mm 

2048  x 

2048 

LSM  5  Pascal 

Many, 

405-633  nm 

No/ 

Mechanical 

attenuator 

0.05-100% 

SM-PP 

Laser-merge 

Adjustable 

2  close  galvos 

1.3  k  or 

2.6  k  in 
bi-direct 

640  ns- 
2.3  ms 

18  mm 

2048  x  2048 

LSM5-LIVE 

Many, 

405-635  nm 

Yes,  AOTF 
0.05-100% 

SM-PP 

Laser-merge 

Adjustable 

Cylindrical 

1  galvo 
line-scan 

(>60  k) 

120  fps, 
512x512, 
1010  fps, 
512x50 

16  ps- 
20  ms 

18  mm 

1024  x  1024 

1  Record  transmitted  light  through  disk. 

2  As  the  TriMScope  is  actually  a  multi-focus  multiphoton  fluorescence  illuminator  with  widefield  detection  onto  a  CCD,  its  performance  depends  a  great  deal  on  the  per¬ 
formance  of  this  device. 

3  These  numbers  assume  that  the  tube  mag  is  lx. 
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Table  A2.1.  ( Continued ) 


Zoom 

range 

Tube 

mag 

Beam 

dump 

Beam¬ 

splitter 

Pinhole 

alignment 

Pinhole  range 

Spectral 

selection 

Photodetector 

Channels 

Reflected/ 

transmitted 

Digitizer 

z-motion 

NA 

l.lx 

NA 

5-place 

dichroic 

wheel 

Self¬ 

aligning 

Fixed  70  pm, 

180  p  spacing 

8-place  filter 
wheel 

CCD  or  EM- 
CCD 

2-camera  port 

No/yes1 

CCD 

Piezo 
±100  nm 
(opt) 

ROI  for 

CCD  mode 

lx 

NA 

Short-pass 

dichroic 

NA 

(multiphoton 

excited 

only) 

8-place  filter 
wheel, 
spectrometer 
(opt) 

CCD  or  EM- 
CCD,  1-32 
PMT  array 

3 -camera  port 

Yes/no 

CCD, 

(opt  ion, 

PMT, 

12-bit) 

Stepper 

motor, 

(peizo 

opt) 

32  :  1 

Yes 

Acousto- 

Optic 

Preset 

Common 
pinhole,  adjust 
20-800  pm 

Prism, 

motorized 

mirrors 

4  PMT, 
cooling 
option, 

APD  option 

8 

Yes/yes 

12-bit 

Galvo, 

±40  nm 

32  :  1 

Yes 

Acousto¬ 

optic 

Fixed 

Common 
pinhole,  adjust 
20-800  pm 

Prism, 

motorized 

mirrors 

4  PMT, 
cooling 
option,  +2 
non- 

descanned 

8 

Yes/yes 

12-bit 

Galvo, 

±40  nm 

infinite 

infinite 

3.8 

Yes 

Dichroic, 
changes 
with  cube 

Dichroic, 
changes 
with  cube 

Fixed 

w/focusable, 
alignable 
pinhole  lens 
Fixed 

w/focusable, 
alignable 
pinhole  lens 

Common 

pinhole, 

30,  60,  100, 

150  pm3 
Common 
pinhole, 

30,  60,  100, 

150  pm4 5 6 

Replaceable 
filter  cubes 

3  diffraction 
gratings  for 

2.5  nm,  5  nm, 
and  10  nm 
channel  width 

3  side-window 

fiber-coupled 

PMTs 

32  element 
multianode 

PMT 

4 

32  acquired 
simultaneously 

Yes/yes 

Yes/yes 

12-bit 

12-bit 

Stepper, 

±50  nm 

Stepper, 

50  nm 
increments 

10:  1 
(infinity) 

3.42x 

Yes 

Dichroic, 
cubes,  2 
positions 

Common 

pinhole, 

alignable 

5  sizes 

Dichroic  filter 
cube 

3  PMTs,  2 
fluor,  1  trans 

3,  2  fl,  1  trans 

Yes/yes 

12-bit 

Stepper, 

±10  nm 

50:  1 
(infinity) 

3.82x 

Yes 

Dichroic 
wheel,  6 
positions 

Common 

pinhole, 

alignable 

adjust 

50-800  or 
50-300  on 
spectral 

2  diff-grating 
channels, 
motorized 
slits 

5  PMTs,  2 
spectral,  1 
trans,  Photon 
counting  mode 

5,  4  fl,  1  trans 

Yes/yes 

12-bit 

Stepper, 

±10  nm 

NA 

lx 

No 

Filter  cube 

Self¬ 

aligning 

Vert  &  horiz 
slits, 

5  sizes 

Dichroic  filter 
cube 

CCD  or 
EM-CCD 

1 

No/yes 

CCD 

Stepper, 

±10  nm 

NA 

No 

Dichroic, 

4  positions 

preset,  rect. 

array 

adjustable 

~1  k  fixed, 

50  pm 

Dichroics/ 

filters 

CCD 

CCD 

Yes/no 

CCD 

Piezo, 

±100  nm 

50:  1 

No 

Dichroic, 

6  positions 

preset/ 

adjustable 

5  slits, 

10-100  pm 

Dichroics/ 

filters 

4  hi-QE  PMTs 

4 

Yes/yes 

10  bits 

Piezo, 

±100  nm 

NA 

lx 

NA 

1  dichroic, 
exchangeable 
by  user 

Self¬ 

aligning 

50  pm,  20  k  on 
disk,  ~1  k 
/FOV 

Dichroics/ 

Filters, 

3  emisson, 

3  barrier 

CCD  or 
EM-CCD 

2-camera  port 

No/no* 

CCD 

NA 

lx 

NA 

Dichroic, 

3  positions 

Self¬ 

aligning 

50  pm,  20  k  on 
disk,  ~1  k 
/FOV 

Dichroics/ 

Filters, 

3  emisson, 

3  barrier 

CCD  or 
EM-CCD 

2-camera  port 

No/no* 

CCD 

0.7-40x 

0.84x 

Yes 

Dichroic, 

4  positions 

4  x,y,(z), 

diameter 

adjustable 

3,  200  steps, 
0.1-13  Airy 

Units  10-1  k 
pm 

3  dichroics, 

6  positions,  + 
spectral 

detector,  10  nm 
/channel 

3/4  filtered 
PMTs;&/or 
diff.  Grating 
w/32  pPMTs 
array,7 

8 

Yes/yes 

8-12-bit 

Microscope 

10  nm, 

Piezo, 

±5  nm 

0.7-40X 

0.84x 

Yes 

Dichroic, 

2  positions 

2  x,y, 

diameter 

adjustable 

1,  200  steps, 
0.1-13  Airy 

Units 

10-1  k  pm 

2  dichroics, 

6  positions 

2  filtered 

PMTs, 
trans  PMT 

4 

No/yes 

8-12-bit 

Microscope 

10  nm, 

Piezo, 

±5  nm 

0.5-2x 

1.18x 

Yes 

Achrogate, 
line-mirror 
on  clear 
blank 

2 

adjustable 

17  slits,  0.5-10 
Airy  units 

detector 

dichroic, 

12  positions, 

8  position 
barriers 

512  x  1  linear 
CCD 

2 

No/no 

8- 12-bit 

Microscope 

10  nm, 

Piezo,  ±5  nm 

4 These  numbers  assume  that  the  tube  mag  is  lx. 

5  Yokogawa  scanners  are  manufactured  by  Yokogawa  Electric  (Tokyo,  Japan),  but  retailed  by  a  number  of  companies  including,  Andor  Technologies  (Belfast,  UK),  Solamere 
Technology  (Salt  Lake  City,  UT),  PerkinElmer  (Downer  Grove,  II),  Visitech  (Sunderland,  UK). 

6  It  is  possible  to  use  4  lasers  with  a  quad,  dichroic  beamsplitter. 

7  Transmission  PMT  and  4-channel  non-descanned  PMT  detector  also  available. 


)  \^\° 


^  /o' 
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1 .  Detection  channels  with  stepless  tunable  bandpass  and  PMT 

2.  Beam  splitter  or  mirror  for  auxiliary  emission  outlet  (optional) 

3.  Emission  filter  and  polarization  filter  (rotatable)  (optional) 

4.  Excitation  pinholes  (excpt.  IR) 

5.  Merge  module.  Combination  of  up  to  4  visible  lasers 

6.  a)  Multiline  Ar-Laser  (457  -  476  -  488  -  496  -  5 14) 
c)  HeNe  Laser  594 
e)  HeNe  Laser  633 

g)  HeCd  Laser  442 
i)  Ar  Laser  35 1  -  364 

7.  EOM  for  intensity  control  of  IR  Laser 

8.  AOTF  for  intensity  control  on  VIS  and  UV  Lasers 

9.  Variable  adaptation  optics  for  UV  /  405nm  illumination 

10.  K-Scanning  module  for  optically  correct  scanning  method  and  field  rotation 

1 1 .  Scan  lens 

12.  Beam  splitter  for  non-descanned  reflected  light  mode  (optional) 

1 3 .  Objective  optics 

14.  Sample 

15.  Condensor  optics 

1 6.  Detectors  for  non-descanned  transmitted  light  (optional) 

1 7.  Secondary  beam  splitter  for  NDD  transmitted  light  (optional) 

1 8.  Secondary  beam  splitter  for  NDD  reflected  light  (optional) 

19.  Detectors  for  non-descanned  reflected  light  (optional) 

20.  Beam  splitter  for  UV  illumination  (optional) 

2 1 .  Variable  beam  expander  optics 

22.  Beam  splitter  for  IR  or  violet  illumination  (optional) 

23.  Acousto  Optical  Beam  Splitter  (AOBS™) 

24.  Pinhole  optics 

25.  Detection  pinhole 

26.  Spectral  detector  prism 


b)  HeNe  Laser  543 
d)  Kr  Laser  568 
f)  IR  Laser  TiS  for 
Multiphoton  excitation 
h)  Solid  state  Laser  430 
j)  Diode  Laser  405 


FIGURE  A2.3.  Schematic  diagram  of  Leica  TCS  SP2  AOBS.  The  Leica  TCS  SP2  AOBS  is  an 
advanced  confocal  microscope  in  which  all  filtering  and  beam-splitting  functions  are  performed 
by  either  liquid-crystal  or  acousto-optical  components.  This  makes  the  system  extremely  flexible 
in  terms  of  being  able  to  add  new  lasers  or  adapt  to  new  emission  bands.  The  acousto-optical 
beam-splitter  (AOBS)  is  essentially  transparent  except  at  exactly  the  laser  wavelengths  (see  Fig. 
3.23).  The  K-scan  galvanometer  mirror  arrangement  is  capable  of  being  rotated  around  the  optical 
axis  to  change  scan  directions.  There  is  one  adjustable  pinhole  for  all  4  prism/moving-mirror  spec¬ 
tral-detection  channels.  Leica  also  makes  the  TCS  SP5,  which  is  similar  but  employes  a  tandem 
scan  system  which  permits  one  to  switch  between  a  scanner  employing  a  normal,  analog  gal¬ 
vanometer  and  one  employing  resonant  galvanometer  for  high-speed,  bi-directional  scanning  at 
up  to  16  k  lines/s. 


1 .  TiS  Laser  (pulsed  IR) 

2.  EOM  for  intensity  control  of  IR  Laser 

3.  K-Scanning  Module  for  optically  correct  scanning 
method  and  field  rotation 

4.  Scan  optics 

5.  Beam  splitter  for  non-descanned  reflected  light  mode 
(optional) 

6.  Objective  lens 

7.  Sample 

8.  Condensor  optics 

9.  Detectors  for  non-descanned  transmitted  light 

10.  Secondary  beam  splitter  for  NDD  transmitted  light 

1 1 .  Secondary  beam  splitter  for  NDD  reflected  light 
(optional) 

12.  Detectors  for  non-descanned  reflected  light  (optional) 

13.  Variable  Beam  expander  optics 


FIGURE  A2.4.  Schematic  of  the  Leica  MP  RS  Multiphoton  Fluorescence  Microscope.  The  Leica 
MP  RS  is  a  single-beam  scanning  fluorescence  microscope  that  uses  a  ps  near-IR  laser  light  source  to 
produce  optical-section  images  of  suitable  specimens.  It  is  designed  for  viewing  living  cells  and  incor¬ 
porates  a  variety  of  non-descanned  detectors  to  record  both  transmitted  and  backscattered  fluorescence 
signal.  This  instrument  uses  a  fs-pulsed,  near-IR  laser  multiphoton  excitation  and  a  high  speed  gal¬ 
vanometer  to  provide  fast  imaging.  Figures  kindly  provided  by  Leica  Inc.  (Heidelberg,  Germany). 
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SMA  Connector 


3-COLOR 
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FIGURE  A2.5.  Schematic  of  the  Nikon  Clsi  light  path.  The  Cl -Plus  is  a  3-channel  fluorescence  plus  transmission,  single-beam,  galvanometer-scanned,  con- 
focal  microscope.  Because  both  the  lasers  and  the  PMTs  are  located  externally  and  coupled  through  fibers,  the  Cl  scan  head  is  extremely  compact  and  is  very 
easy  to  move  from  one  microscope  to  another.  The  standard  unit  includes  laser  module,  in  which  a  wide  variety  of  gas  and  solid-state  lasers  can  be  installed,  a 
scan-head  and  a  DU-3  three-PMT  detector  module  containing  the  collimating  and  focusing  lenses,  and  photomultiplier  tubes.  The  “si”  version  includes  an  addi¬ 
tional  sophisticated  spectral  detector  that  is  also  coupled  to  the  scan  head  through  a  multi-mode  fiber.  The  detector  itself  incorporates  a  Diffraction  Efficiency 
Enhancement  System  (DEES)  in  which  a  polarized  beam  splitter  separates  the  unpolarized  signal  beam  into  two  parts  (red  and  blue  lines).  One  part  passes 
through  a  prism  polarization  rotator  so  that  all  the  light  strikes  the  diffraction  grating  with  the  optimal  (s-plane)  polarization  to  be  diffracted  with  maximum  effi¬ 
ciency  by  one  of  3  gratings  (2.5,  5  and  lOnm/channel).  Both  ray  bundles  are  then  focused  onto  a  32  channel  micro  PMT  by  a  pair  of  reflecting  lenses  (A  and 
B).  Simultaneous  readout  is  possible  from  all  channels.  The  digitization  system  uses  2  sample-and-hold  circuits  to  optimize  signal  integration.  Figures  kindly 
provided  by  Nikon  Inc.  (Tokyo,  Japan). 
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FIGURE  A2.6.  (A)  Schematic  of  the  Olympus  Fluoview  1000.  The  Fluoview  1000  is  the  most  recent  single-beam  laser-scanning  confocal  fluorescence  micro¬ 
scope  introduced  by  Olympus.  It  offers  4  separate  fluorescence  detection  channels,  two  of  which  incorporate  diffraction  gratings  and  adjustable  slits  to  tune  the 
passband.  Besides  the  normal  scanning  mirrors  there  is  a  second  independent  SIM-scanning  arrangement  (not  shown  in  the  figure)  to  control  lasers  used  for 
photo-uncaging  or  for  intentionally  bleaching  the  specimen.  To  keep  the  signal  up  when  the  light  dose  to  the  specimen  must  be  kept  low,  this  new  scanner  not 
only  incorporates  dichroic  elements  employing  “hard’  coatings  to  ensure  the  highest  transmission,  it  also  offers  a  photon-counting  option  to  reduce  PMT  multi¬ 
plicative  noise.  Figures  kindly  provided  by  Olympus  Corp.  (Tokyo,  Japan). 
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FIGURE  A2.6.  ( Continued)  (B,  C)  Schematic  of  the  Olympus  DSU  disk-scanner.  The  Olympus  DSU  is  a  disk-scanning  confocal  fluorescence  microscope 
that  uses  a  mercury  arc  for  excitation.  The  optical  system  is  identical  to  that  used  for  normal  epi-fluorescence  with  the  exception  that  an  opaque  disk  is  located 
in  the  intermediate  image  plane.  Slits  in  this  coating  on  this  disk  allow  light  to  reach  the  focus  plane  and  prevents  light  from  this  plane  from  reaching  the  CCD 
camera.  To  keep  the  light  dose  to  the  specimen  low,  this  new  scanner  not  only  incorporates  “hard”  coatings  to  ensure  the  highest  transmission  of  the  dichroic 
elements,  it  also  offers  a  photon-counting  option  to  reduce  PMT  multiplicative  noise.  (C)  Layout  of  one  of  several  interchangeable  scanning  disks  used  in  the 
Olympus  DSU  disk-scanner.  The  thickness  and  spacing  of  the  slits  varies  on  the  5  available  disks  have  each  been  optimized  for  use  with  a  particular  objective. 
Figures  kindly  provided  by  Olympus  Corp.  (Tokyo,  Japan). 
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FIGURE  A2.7A.  Schematic  diagram  of  Visitech  VT  Infinity.  The  optical  path  starts  with  a  stationary  micro-lens  array  illuminated  by  an  expanded  laser  beam. 
A  galvanometer  mirror  (x)  incorporating  a  piezoelectric  micro-deflector  (y)  scans  the  array  to  cover  the  sample  and  then  de-scans  the  returning  fluorescence 
signal.  This  light  is  separated  from  the  illuminating  beam  by  a  dichroic  mirror,  and  passes  through  a  stationary  pinhole  array  to  create  confocal  data.  This  data 
is  re-scanned,  in  perfect  synchronization,  by  being  reflected  off  the  reverse  side  of  the  galvanometer  mirror  onto  a  sensitive  CCD  camera.  The  galvanometer 
scanner  is  readily  synchronized  to  the  camera  capture  parameters,  both  exposure  time  and  frame  capture  rate.  Either  multiple-line  lasers  or  multiple  lasers  in  any 
combination  can  be  coupled  through  an  AOTF  that  provides  high  speed  (~ps)  laser-line  selection  and  intensity  control.  Laser  excitation  can  be  coupled  in  either 
by  optical  fiber  or  by  direct  coupling.  Motor-driven  filters  change  dichroic  and  detection  bandpass.  This  system  couples  the  advantages  of  high-brightness,  laser 
illumination  with  multipoint  scanning  to  keep  the  instantaneous  intensity  down  while  providing  a  data  rate  high  enough  for  fast  image  detection,  using  a  high- 
quantum-efficiency  CCD  camera.  Figure  kindly  provided  by  Visitech  Inc.  (Sunderland,  UK). 
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FIGURE  A2.7B.  Schematic  diagram  of  Visitech  VT-eye.  The  VT-eye  incor¬ 
porates  a  novel  acousto-optical  deflector  (AOD)  scanner,  that  combines  ultra¬ 
fast  horizontal  scanning  to  provide  high-resolution  confocal  imaging  for 
real-time,  living-cell  confocal  microscopy.  The  AOD  scans  the  X  axis  at  up  to 
50,000  lines/s  or  400  frames/s,  fast  enough  to  capture  clear  images  of  dynamic 
events  such  as  Ca**  puffs,  sparks  and  waves.  Multi-wavelength  imaging  for 
multi-labeled  specimens  from  UV  through  the  visible  to  the  near  infrared  is 
achieved  by  using  a  selection  of  motorized,  primary  multi-band  dichroics.  The 
system  operates  with  almost  any  laser,  or  combination  of  lasers,  and  uses  AOTF 
technology  to  provide  fast  laser-line  selection.  The  VTeye  comes  with  up  to  4 
high-QE  PMTs.  The  piezoelectric  focusing  system  is  capable  of  changing  focus 
positions  at  up  to  100  slices  per  second.  Although  high-speed  acquisition 
creates  vast  quantities  of  data  in  a  very  short  time,  hours  of  experiments  may 
be  recorded  at  the  maximum  capture  rates  on  a  range  of  parallel,  hard-disk 
modules.  Figure  kindly  provided  by  Visitech  Inc.  (Sunderland,  UK). 
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FIGURE  A2.8.  Schematic  of  the  Yokogawa  CSU  22.  The  Yokogawa  scanner  was  the  first  disk  scanner  to  offer  both  laser  illumination  and  multibeam  excita¬ 
tion.  The  mircolenses  increase  the  efficiency  of  the  illumination  path  from  the  2-10%  common  to  ordinary  disks  to  almost  60%.  (A)  Laser  light  enters  the  scan 
head  through  a  single-mode  optical  fiber,  reflects  off  a  mirror  and  through  one  of  3  exciter  filters.  After  passing  through  an  ND  filter,  and  a  beam  expander,  it 
illuminates  the  microlens  array  on  the  top  disk  of  the  rotating  scanning  assembly.  The  lenses  focus  the  light  through  a  short-pass  dichroic  and  onto  the  array  of 
pinholes  in  the  lower  disk.  As  this  disk  is  in  an  image  plane,  the  light  passing  each  pinhole  is  focused  into  a  point  at  the  focus  plane  of  the  objective.  Fluores¬ 
cent  light  returning  from  the  focus  plane  passes  up  through  the  pinholes,  and  reflects  off  one  of  3  dichroic  mirrors  located  between  the  two  disks  and  into  the 
detection  path.  After  passing  through  one  of  3  barrier  filters,  a  selection  mirror  sends  this  light  either  to  the  camera  port  or  to  the  eyepiece.  (B)  Simplified  ray 
optical  diagram  of  the  CSU-22.  The  pinhole  disk  resides  in  an  image  plane  and  the  signal  passing  the  pinholes  is  first  made  parallel  by  a  relay  lens,  then  passed 
through  the  emission  filter  before  being  focused  onto  the  CCD  chip  by  a  second  relay  lens.  Other  details  shown  in  Figure  10.9.  Figures  kindly  provided  by 
PerkinElmer  Corp.  (Shelton,  CT). 
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FIGURE  A2.9A.  Schematic  diagram  of  Zeiss 
LSM-5-LIVE  Fast  Slit  Scanner.  The  LSM-5-Live  is 
a  line-scanning  confocal  microscope  using  line  illu¬ 
mination  and  a  linear  detector.  Because  it  illuminates 
about  lOOx  more  points  than  does  a  single-beam 
instrument,  the  LSM-5  Live  can  acquire  data  at  a 
much  higher  speed  while  still  keeping  the  peak  light 
intensity  low  enough  to  avoid  singlet-state  saturation. 
In  addition,  the  quantum  efficiency  of  the  linear  CCD 
is  about  lOx  greater  than  that  of  most  PMTs.  Laser 
light  enters  the  scan  head  through  optical  fibers  (1) 
where  it  is  combined  by  a  series  of  mirrors  (2,  3)  and 
then  passes  to  beam  shaper  (an  expander  and  a  cylin¬ 
drical  lens  that  converts  the  collimated  Gaussian 
beam  into  laser  light  with  a  rectangular  cross-section) 
(4)  and  also  focuses  it  precisely  onto  the  AchroGate 
beam  splitter  (5),  reflects  all  wavelengths  but  only 
along  a  reflective  line  across  its  center.  As  a  result,  no 
matter  what  the  wavelength,  it  reflects  100%  of  the 
laser  light  but  passes  >95%  of  the  signal  light  to  the 
detectors.  The  size  of  the  raster  on  the  specimen  is 
controlled  by  a  0.5-2x  zoom  optic  (6),  that  feeds  the 
light  to  the  y-scanning  mirror  (7),  through  the  scan 
lens  (8),  the  objective  lens  (9)  and  on  to  the  speci¬ 
men,  (10).  Returning  signal  follows  the  same  path  but 
mostly  misses  the  reflective  strip  in  the  Achrogate  and 
proceeds  through  a  wheel  of  secondary  dichroic 
beam-splitters  (11)  to  one  of  2  tube-lenses  (12)  that 
each  focuses  the  line  illuminated  in  the  specimen  onto 
a  17-position,  slit  aperture  plate  (13).  Light  passing 
the  slits  is  first  filtered  by  emission  filters  (14)  and 
then  detected  by  a  1  x  512  linear  CCD  detector  (15) 
(see  also  Fig.  9.6). 
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FIGURE  A2.9B.  Optical  beam  path  of  the  Zeiss  LSM  510  META.  A  unique  scanning  module  is  the  core  of  the  LSM  510  META.  It  contains  motorized 
dichroic  mirrors  and  barrier  filters,  adjustable  collimators,  individually  adjustable  and  alignable  pinholes  for  each  of  3  (or  even  4)  detection  channels,  as  well  as 
scanning  mirrors,  and  highly  sensitive  PMT  detectors  including  the  32  micro-PMTs  of  the  META  spectral  detector.  All  these  components  are  arranged  to  ensure 
optimum  specimen  illumination  and  efficient  collection  of  reflected  or  emitted  light.  The  highly  optimized  optical  diffraction  grating  in  the  META  detector  pro¬ 
vides  an  innovative  way  of  separating  the  fluorescence  emission  spectrum  to  strike  32  separate,  micro-PMTs,  each  of  which  covers  a  bandwidth  of  ~10  nm. 
Thus,  a  spectral  signature  is  acquired  at  each  pixel  of  the  scanned  image.  Such  a  dataset  can  subsequently  be  digitally  “unmixed”  to  separate  signals  from  dyes 
with  overlapping  emission  spectra.  The  Beam  Path:  (1)  Optical  Fibers,  (2)  Motorized  collimators,  (3)  Beam  combiner,  (4)  Main  dichroic  beamsplitter,  (5)  Scan¬ 
ning  mirrors,  (6)  Scanning  lens,  (7)  Objective  lens,  (8)  Specimen,  (9)  Secondary  dichroic  beamsplitter,  (10)  Confocal  pinhole,  (11)  Emission  filters,  (12)  Pho¬ 
tomultiplier,  (13)  META  detector,  (14)  Neutral  density  filter,  (15)  Monitor  diode,  (16)  Fiber  out. 
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FIGURE  A2.9C.  ( Continued)  Schematic  diagram  of  Zeiss  LSM  FCS  showing  how  the  fluorescence-correlation  spectroscopy  (FCS)  unit  is  attached  via  the 
base  port  of  the  Axiovert  200M  microscope  while  the  LSM  510  META  is  attached  to  the  side  camera  port.  All  figures  kindly  provided  by  Carl  Zeiss  Inc.  (Jena, 
Germany). 
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INTRODUCTION 

The  electronic  structure  of  crystalline  Si  is  such  that  electromag¬ 
netic  waves  having  the  energy  of  light  photons  (1.75-3.0  electron 
volts)  can  be  absorbed  to  produce  one  free  or  “conduction”  elec¬ 
tron.  If  an  image  is  focused  onto  a  Si  surface,  the  number  of  the 
photoelectrons  (PE)  produced  at  each  location  over  the  surface  is 
proportional  to  the  local  light  intensity.  Clearly,  all  that  is  needed 
to  create  an  image  sensor  is  a  method  for  rapidly  converting  the 
local  PE  concentration  into  an  electronic  signal.  After  almost  40 
years  of  NASA  and  DOD  funding,  the  slow-scan,  scientific-grade, 
charge-coupled  device  (CCD)  camera  is  now  an  almost  perfect 
solution  to  this  problem. 

Success  in  modern  biological  light  microscopy  depends  to  an 
ever-increasing  extent  on  the  performance  of  CCD  cameras. 
Because  such  cameras  differ  widely  in  their  capabilities  and  are 
also  items  that  most  biologists  buy  separately,  rather  than  as  part 
of  a  system,  some  knowledge  of  their  operation  may  be  useful  to 
those  practicing  biologists  who  have  not  yet  found  it  necessary  to 
be  particularly  interested  in  “electronics.”  Although  the  basics  of 
CCD  operation  are  described  in  many  other  chapters  (particularly, 
Chapters  4,  10  and  12)  this  Appendix  describes  the  operating 
principles  of  these  devices  in  greater  detail  and  also  discusses  the 
ways  that  they  “don’t  work  as  planned.”  It  then  covers  the  opera¬ 
tion  of  the  electron-multiplier  CCD  (EM-CCD),  a  new  variant 
that  reduces  the  read  noise  almost  to  zero,  although  at  the  cost 
of  reduced  effective  quantum  efficiency  (QE^).1  The  second 
section,  How  to  choose  a  CCD ,  is  a  review  of  CCD  specifications 
with  comments  on  the  relevance  of  each  in  fluorescence 
microscopy. 

PART  I:  HOW  CHARGE-COUPLED 
DEVICES  WORK 

The  first  step  is  to  imagine  a  rectangular  area  of  the  Si  surface  as 
being  divided  into  rows  and  columns,  or  more  usually,  lines  and 
pixels.  Each  pixel  is  between  4  x  4  pm  and  about  24  x  24  pm  in 
size  and  the  location  of  any  pixel  of  the  surface  can  be  defined  in 
terms  of  it  being  x  pixels  from  the  left  side,  on  line  y. 

To  construct  an  actual  system  like  this,  start  with  a  smooth  Si 
surface;  cover  it  with  a  thin,  transparent,  insulating  layer  of  Si02; 
deposit  onto  the  Si02,  a  pattern  of  horizontal  strips,  made  out  of  a 
transparent  conductor  called  amorphous  silicon  (or  poly-silicon), 
so  that  the  strips  cover  the  entire  image  sensor  area.  Although, 
viewed  from  the  top,  these  strips  partially  overlap  each  other,  they 


1  This  loss  can  be  avoided  if  the  system  is  used  in  photon-conting  mode. 


are  kept  electrically  separate  from  their  neighbors  by  additional 
layers  of  Si02.  Every  third  stripe  is  connected  together  to  form 
three  sets  of  interdigitating  strips  that  we  will  refer  to  as  Phases  1 , 
2  and  3  (<|p,  (|)2,  0 3,  Fig.  A3.1).  Taken  together,  all  these  phases  con¬ 
stitute  the  vertical  register  (VR)  and,  after  the  assembly  has  been 
exposed  to  a  pattern  of  light,  they  are  used  to  transfer  the  photo- 
induced  charge  pattern  downwards,  one  line  at  a  time.  The  pixels 
along  each  line  are  separated  from  each  other  by  vertical  strips  of 
positively  doped  material  injected  into  the  Si.  These  positive 
“channel  blocks”  create  fields  that  prevent  charge  from  diffusing 
sideways  without  reducing  the  active  area  of  the  sensor. 

Any  photon  that  passes  through  the  stripes  and  the  Si02,  is 
absorbed  in  the  Si,  producing  a  PE.  If  a  small  positive  voltage  (~15 
volts)  is  applied  to  the  dp  electrodes,  any  PE  produced  nearby  will 
be  attracted  to  a  location  just  below  the  nearest  <|p  strip  (Fig.  A3. 2). 
As  additional  PEs  are  produced,  they  form  a  small  cloud  of  PEs 
referred  to  as  a  charge  packet.  The  number  of  PEs  in  the  packet  is 
proportional  to  the  local  light  intensity  times  the  exposure  period 
and  the  problem  now  is  to  convey  this  packet  to  some  location 
where  its  size  can  be  measured,  and  to  do  this  without  changing  it 
or  losing  track  of  the  location  from  which  it  was  collected.  This 
will  be  achieved  by  using  the  overlying  electrodes  to  drag  the 
charge  packet  around  in  an  orderly  way  until  it  is  deposited  at  the 
readout  node  of  the  charge  amplifier. 

Charge  Coupling 

The  dragging  mechanism  operates  in  the  following  way:  First  (j)2 
is  also  made  positive  so  that  the  cloud  diffuses  to  fill  the  area 
underneath  both  <|p  and  (J)2-  Then  0!  is  made  zero,  forcing  the  packet 
to  concentrate  under  (|)2  alone  (Fig  A3. 2). 

So  far,  these  3  steps  have  succeeded  in  moving  the  charge 
packets  that  were  originally  under  each  of  the  <|p  electrodes  down¬ 
wards  by  one  phase  or  1/3  of  a  “line”  in  the  x-y  raster.  If  this 
sequence  is  now  repeated,  but  between  (|)2  and  (|p  and  then  again 
between  and  the  (|p  belonging  to  the  next  triplet  of  strips,  packets 
will  have  moved  down  by  the  one  entire  raster  line.  PEs  created 
within  a  particular  pixel  of  each  horizontal  stripe  remain  confined 
by  the  channel  stops  as  they  are  transferred  to  the  next  line  below. 

A  pixel  of  the  image  is  therefore  defined  as  the  area  under  a 
triplet  of  overlying,  vertical  charge-transfer  electrodes  and 
between  two  neighboring  channel  blocks.  The  pixels  on  scientific 
CCDs,  are  usually  square,  4  to  30  pm  on  a  side  while  those  on 
commercial,  video  CCDs  are  likely  to  be  wider  than  they  are  high, 
to  conform  with  the  reduced  horizontal  resolution  of  commercial 
video  standards.  Only  square  pixels  can  be  conveniently  displayed 
in  a  truly  digital  manner.  Larger  pixels  have  more  leakage  current 
(dark-current),  but  are  also  able  to  store  more  charge  per  pixel  (see 
Blooming,  below). 
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FIGURE  A3.1.  Layout  of  CCD  array,  viewed  en  face. 
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FIGURE  A3. 2.  Charge  coupling:  Three  stages  in  the 
process  of  moving  a  charge  packet  initially  beneath 
phase  1  (A),  so  that  it  first  spreads  to  be  also  under  phase 
2  (B)  and  finally  is  confined  to  entirely  under  phase  2 
(C).  These  3  steps  must  be  repeated  3  times  before  the 
charge  packet  has  been  moved  downwards  (or  in  the 
diagram,  to  the  right)  by  one  line  of  the  CCD  array. 
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At  the  bottom  of  the  sensor,  an  entire  line  of  charge  packets  is 
simultaneously  transferred  to  the  adjacent  pixels  of  the  horizontal 
register  (HR,  also  sometimes  called  a  shift  register).  Like  the  VR, 
the  HR  is  composed  of  a  system  of  overlying  poly-Si  electrodes 
and  channel  stops.  Each  column  of  pixels  in  the  VR  is  eventually 
transferred  directly  into  the  same  specific  pixel  on  the  HR.  The 
three  phases  of  the  HR  <t>5,  §e)  work  exactly  like  those  in  the 
VR,  except  that  they  must  cycle  at  a  much  faster  rate  because  the 
entire  HR  must  be  emptied  before  the  next  line  of  packets  is  trans¬ 
ferred  down  from  the  bottom  line  of  the  VR.  In  other  words,  in  the 
time  between  one  complete  line-transfer  cycle  of  the  VR  and  the 
next,  the  horizontal  register  must  cycle  as  many  times  as  there  are 
pixels  in  each  line. 

At  the  right-hand  end  of  the  HR  is  a  charge  amplifier  that  mea¬ 
sures  the  charge  in  each  packet  as  it  is  transferred  into  it  from  the 
last  pixel  of  the  HR.  The  first  pixel  to  be  read  out  is  that  on  the 
extreme  right-hand  side  of  the  bottom  line.  The  last  pixel  will  be 
that  on  the  left  side  of  the  top  line.2 

The  entire  charge-transfer  process  has  the  effect  of  coding 
position  as  time.  If  we  digitize  the  signals  from  the  charge  ampli¬ 
fier,  and  store  the  resulting  numbers  in  a  video  memory,  we  will 
be  able  to  see  a  representation  of  the  light  intensity  pattern  strik¬ 
ing  the  sensor  on  any  monitor  attached  to  this  video  memory.  Alter¬ 
natively,  as  long  as  the  dimensions  of  the  CCD  array  match  those 
of  some  video  standard,  such  as  NTSC  or  PAL,  the  time  sequence 
of  charge-packet  readout  voltages  can  be  smoothed  and,  with  the 
addition  of  synch  pulses,  turned  into  an  analog  video  signal.  While 
this  latter  process  is  often  convenient,  it  is  a  poor  plan  if  the  analog 
signal  must  then  be  re-digitized.  The  necessity  to  digitize  twice  can 
reduce  the  effective  horizontal  resolution  of  the  CCD  sensor  by 
about  a  factor  of  2  and  because  the  process  is  AC  coupled,  photo¬ 
metric  accuracy  is  severely  compromised. 

It  is  important  to  understand  the  relationship  between  the 
charge-transfer  electrodes  and  the  charge  packet.  The  electrodes 
do  not  somehow  “connect  to”  the  charge  packet,  and  “conduct”  it 
to  the  amplifier.  Such  a  process  would  be  subject  to  resistive  losses, 
charge  would  be  lost  and  a  lot  of  “wires”  would  be  needed.  The 
charge-coupling  process  is  better  thought  of  in  terms  of  a  ball 
bearing  “dragged”  over  the  surface  of  a  loose  blanket  by  moving 
a  cooking  pot  around  underneath  the  blanket.  The  weight  of  the 
ball  and  the  lip  of  the  pot  create  a  dimple  and  gravity  keeps  the 
ball  in  the  dimple  as  the  pot  is  moved.  The  voltage  on  the  charge- 
transfer  electrode  creates  an  electronic  “dimple.”  Changing  the 
voltages  on  nearby  electrodes  moves  the  dimple.  In  this  way, 
groups  of  charged  particles  (electrons)  can  be  pushed  around 
without  actually  “touching”  or  losing  them. 

Readout  Methods 

There  are  three  distinct  methods  for  reading  out  the  charge  pattern 
of  a  CCD:  full-frame,  full-frame  transfer  and  interline  transfer 
(Fig.  A3.4).  Most  early  scientific  CCDs  used  the  first  method, 
which  operates  as  has  just  been  described.  Although  full-frame 
readout  provides  the  largest  sensitive  area  for  a  given  area  of 
silicon,  the  lowest  level  of  readout  noise  and  the  greatest  photo¬ 
metric  accuracy,  it  also  has  some  disadvantages.  One  cannot  both 
collect  and  read  out  signal  at  the  same  time.  Unless  some  sort  of 
shutter  is  used  to  prevent  light  from  striking  the  sensor  during  ver¬ 
tical  transfer,  signal  will  be  added  to  any  packets  that  are  trans- 


2  This  may  seem  backwards  until  one  remembers  that  any  image  of  the  real 
world  is  usually  focused  onto  the  CCD  by  a  single,  converging  lens,  a  process 
that  always  inverts  the  image. 


ferred  past  bright  features  in  the  image,  producing  vertical  streak¬ 
ing.  This  problem  is  more  important  when  the  exposure  time  is 
short  relative  to  the  readout  time. 

In  frame  transfer  readout,  at  the  end  of  the  exposure,  the 
entire  charge  pattern  is  rapidly  (0.1-3  ms)  transferred  by  charge¬ 
coupling  to  a  second  2D  storage  array.  The  storage  array  is  the 
same  size  as  the  sensor  array  and  is  located  next  to  it  but  it  is  phys¬ 
ically  masked  with  evaporated  metal  to  shield  it  from  light.  The 
charge  pattern  is  then  read  out  from  the  storage  array  while  the 
sensor  array  collects  a  new  image.  Because  vertical  transfer  can  be 
much  faster  if  the  charge  packets  do  not  have  to  be  read  out,  this 
system  reduces  streaking  by  up  to  lOOOx  but  does  not  eliminate  it 
and  the  need  for  a  storage  register  reduces  the  fraction  of  the  Si 
surface  area  that  can  be  used  for  sensing  by  50%. 

In  interline  transfer,  the  masked  storage  cells  are  interlaced 
between  the  sensor  cells  (i.e.,  each  pixel  is  divided  into  sense  and 
read  areas).  After  exposure,  all  charge  packets  can  be  moved  to  the 
readout  array  in  less  than  a  microsecond.  This  ability  can  be  used 
as  an  electronic  shutter  to  eliminate  vertical  smearing  but,  because 
at  least  half  of  the  area  of  each  sensor  must  be  masked,  and  any 
light  striking  a  masked  area  is  lost,  the  “fill  factor”  of  the  sensor  is 
reduced,  proportionately  decreasing  QEefl.  A  solution  to  the  “fill- 
factor”  dilemma  is  to  incorporate  an  array  of  microlenses,  aligned 
so  that  there  is  one  above  every  pixel.  With  such  a  system,  most  of 
the  light  striking  any  pixel  will  be  focused  onto  the  unmasked  area.3 
Although  microlenses  restore  the  QEeff  somewhat,  the  full-well 
signal  possible  is  still  limited  by  the  smaller  sensitive  area. 

WHAT  COULD  GO  WRONG? 

When  I  first  heard  the  CCD  story,  it  struck  me  as  pretty  prepos¬ 
terous!  How  could  you  get  all  the  correct  voltages  (9  different 
voltage  combinations  per  pixel  shift,  ~3.6  million  for  each  TV 
frame,  108  million/s  for  video  rate!)  to  the  right  charge-transfer 
electrodes  at  the  correct  times?  How  could  you  get  all  of  the  charge 
in  a  packet  to  stay  together  during  a  transfer?  Wouldn’t  Poisson 
statistics  apply,  making  even  one  transfer  imprecise  and  the  2000 
transfers  needed  to  read  out  the  top,  right  pixel  of  a  1000  x  1000 
pixel  array  impossibly  inaccurate?  How  long  would  the  PEs  stay 
free  to  be  dragged  around  the  lattice?  Wouldn’t  the  charge  packets 
decay  with  time? 

In  fact,  many  of  these  problems  did  occur,  but  remedies  to  most 
have  now  been  devised.  The  difference  between  a  $300  commer¬ 
cial  CCD  camera  and  a  $65,000,  top-of-the-line  scientific  CCD  can 
often  be  measured  in  terms  of  how  many  of  these  remedies  have 
been  implemented.  Therefore,  it  is  worthwhile  trying  to  understand 
some  of  them  so  that  one  can  buy  what  one  needs.  The  following 
discussion  will  define  and  discuss  some  of  the  more  important 
CCD  technical  specifications. 

Quantum  Efficiency 

Quantum  efficiency  is  the  ratio  of  the  number  of  impinging 
photons  to  the  number  of  PEs  produced.4  Any  photon  with  energy 
in  the  range  of  1-100  eV  striking  crystalline  Si  has  a  very  high 
probability  of  producing  a  PE.  However,  reflections  and  absorp¬ 
tion  by  the  overlying  poly  silicon  electrodes,5  reduce  the  QE  of 


3  This  occurs  only  as  long  as  the  initial  angle  of  incidence  is  near  to  normal,  a 
condition  met  when  CCDs  are  used  for  light  microscopy. 

4  In  the  visible  range,  each  absorbed  photon  makes  only  one  PE. 

5  Kodak  had  pioneered  the  use  of  charge  transfer  electrodes  made  out  of  In  and 
Sn  oxides  that  scatter  less  light  than  do  those  made  of  poly-Si. 
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FIGURE  A3.3.  Four  CCD  readout  patterns:  Full-frame, 
frame-transfer,  interline  transfer  and  gain  register  (EM- 
CCD). 


Full  frame 


Full  frame 
transfer 


4>4 


Interline  transfer: 


<(>2  =  variable  voltage 


front-illuminated  CCDs  especially  in  the  blue  end  of  the  spectrum. 
To  reduce  this  effect,  some  UV-enhanced  sensors  are  coated  with 
fluorescent  plastics,  which  absorb  in  the  blue  and  emit  at  longer 
wavelengths.  Others  have  their  backs  etched  away  and  are  turned 
over  to  permit  the  illumination  to  reach  the  light-sensitive  area 
from  the  back  side.6  Figure  A3.4  shows  the  intrinsic  QE  of  differ¬ 
ent  types  of  CCD  (not  Qeff,  which  would  take  into  account  the  light 
lost  if  some  of  the  sensor  is  covered  by  charge  storage  areas).  The 
effective  QE  can  usually  only  be  determined  by  actual  measure¬ 
ment  or  by  very  careful  evaluation  of  the  published  specifications 
(QEeffective  QEintrinsic  X  fill  factor). 

Edge  Effects 

In  early  CCDs,  PEs  were  often  “lost”  in  the  crystalline  imperfec¬ 
tions  that  are  always  present  at  the  Si/SiCF  junction.  To  avoid  this, 
ion  implantation  is  now  used  to  make  an  N-doped,  sub-surface 
layer  called  the  buried  channel  about  1  pm  below  this  surface  (Fig. 
A3.2).  This  channel  attracts  the  free  PEs,  keeping  them  away  from 
the  edge  of  the  Si  crystal.  Any  serious  CCDs  will  have  a  buried 
channel  but  the  need  for  ion-implantation  keeps  CCD  chip  prices 
high!  Figure  A3. 5  shows  the  readout  noise,  in  root-mean  squared 
(RMS)  electrons/pixel,  for  surface  and  buried-channel  CCDs 
having  two  different  pixel  sizes.  From  this  you  can  see  that  small 
pixels  (here  ~5.5  x  5.5pm)  have  lower  read  noise  than  larger  ones 
(—17  x  17  pm),  mostly  because  the  larger  ones  have  higher  capac¬ 
itance  and  capacitance  is  the  most  important  parameter  of  read- 


6  Back-illuminated  CCDs  have  to  be  thinned  to  7-10|J.m  so  that  conduction 
electrons  created  near  what  would  have  been  the  back  surface  can  respond  to 
the  fields  created  by  the  buried  channel  and  the  CC  electrodes.  Thinning 
increases  cost  and  also  reduces  QE  at  longer  wavelength  where  the  absorp¬ 
tion  distance  of  the  photons  becomes  comparable  with  the  actual  thickness. 
Back-illuminated  CCDs  are  also  more  expensive  because  it  is  difficult  to 
create  electrical  contacts  with  electrodes,  etc.,  that  are  now  on  the  bottom  side 
of  the  chip. 


amplifier  noise.  One  can  also  see  that  at  readout  speeds  higher  than 
lMFIz  (or  1  second  to  read  out  a  1024  X  1024  CCD),  the  read 
noise  increases  with  the  square  root  of  the  read  speed. 

Charge  Loss 

The  lifetime  of  a  PE  (before  it  drops  back  into  the  ground  state) 
depends  on  the  purity  and  crystalline  perfection  of  the  Si  and  on 
other  factors  such  as  temperature.  Generally  it  is  long  enough  that 
little  charge  is  lost  during  the  exposure  times  commonly  used  in 
fluorescence  microscopy.  If  necessary,  it  can  be  increased  by 
cooling  the  detector,  something  often  done  to  reduce  dark  charge. 

Leakage  or  "Dark  Charge" 

Dark  charge  is  the  charge  that  leaks  into  a  pixel  during  the  expo¬ 
sure  time  in  the  absence  of  light.  It  can  be  thought  of  as  the  dark 
current7  deposited  into  one  pixel.  Many  processes  other  than 
photon  absorption  can  add  PE  to  the  charge  packet.  The  magni¬ 
tude  of  this  dark  charge  depends  on  the  length  of  the  exposure,  and 
is  substantially  reduced  by  cooling.  The  rule  of  thumb  is  that  for 
every  8°C  of  cooling,  the  dark  charge  is  halved.  As  noted  above, 
dark  charge  is  principally  a  problem  because  it  produces  Poisson 
noise  equal  to  the  square  root  of  its  magnitude,  and  if  this  is 
left  unchecked,  it  can  significantly  increase  the  noise  floor  of  the 
CCD. 

Since  -1987,  a  process  called  multipinned  phasing  (MPP)  has 
been  available  to  reduce  dark  charge  build-up  by  about  a  factor  of 
1000,  making  it  immeasurable  in  exposures  up  to  a  minute  or  so. 
This  feature  should  be  specified  if  one  expects  to  use  exposures 
longer  than  a  few  seconds  without  deep-cooling. 


7  A  current  is  a  flow  of  charge  measured  in  charge/time.  The  unit  of  charge  is 
the  Coulomb  (c).  The  unit  of  current  is  the  Ampere  (A).  One  Amp  represents 
a  flow  of  one  Coulomb/s  or  6.16  X  1018  electrons/s. 
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Quantum  Efficiency  vs.  Wavelength  (@  room  temp) 
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FIGURE  A3.4.  Intrinsic  QE  as  a  function  of  wave¬ 
length  for  a  front-illuminated  CCD  (blue),  a  visible- 
enhanced,  back-illuminated  CCD  (green)  and  a 
UV-enhanced  CCD  (red). 


Wavelength,  nm 


It  should  also  be  remembered  that,  while  dark  charge  is  never 
good,  its  average  value  can  be  measured  and  subtracted  on  a  pixel- 
by-pixel  basis,  by  subtracting  a  “dark  image”  from  each  recorded 
image  as  part  of  flat-fielding.  However,  because,  by  definition 
“dark”  images  contain  very  few  photons/pixel,  they  have  relatively 
high  Poisson  noise  and  low  S/N.  Therefore,  a  number  of  such 
images  must  be  averaged  to  produce  a  correction  mask  that  is  sta¬ 
tistically  defined  well  enough  that  subtracting  it  from  the  data  does 
not  substantially  increase  the  noise  present  in  the  final,  corrected 
image. 

This  is  not  a  problem  when  there  are  many  counts  in  each  pixel 
because  the  subtractive  process  of  dark-charge  normalization 
involves  a  change  that  is  small  compared  with  the  intrinsic  noise 
present  in  a  large  signal.  It  can  be  a  problem  when  the  black  mask 
image  is  subtracted  from  a  faint  image  that  also  contains  only  a 
few  counts/pixel. 

What  cannot  be  removed  by  flat-fielding  is  the  Poisson  noise 
associated  with  the  dark  charge.  This  is  equal  to  the  square  root  of 


the  number  of  electrons/pixel  it  represents.  CCDs  should  always 
be  operated  such  that  the  noise  on  the  dark  charge  is  less  than  the 
readout  noise.  On  conventional  CCDs  this  condition  can  usually 
be  met  quite  easily  by  slightly  cooling  the  sensor  (0°C  or  about 
-20°C  from  ambient).  The  use  of  lower  temperatures  is  compli¬ 
cated  by  the  risk  of  condensing  atmospheric  water,  a  process  that 
can  be  avoided  only  by  enclosing  the  sensor  in  a  vacuum  chamber. 
Generally,  a  vacuum-hermetic  enclosure,  combined  with  good 
outgassing  prevention,  carries  with  it  the  significant  benefits  of 
more  effective  cooling,  long-term  protection  of  the  sensor  from 
moisture  and  other  degrading  organic  condensates  as  well  as 
the  prevention  of  front-window  fogging.  At  video  rate,  where 
exposures  are  short,  dark  charge  is  only  a  problem  when  the 
readout  noise  is  reduced  to  <1  e/pixel,  as  it  is  when  an  “electron- 
multiplier”  (EM)  charge  amplifier  is  used  (see  below  and  also 
Chapters  4  and  10).  In  EM-CCDs  the  read  noise  is  so  low  that  dark 
current  becomes  the  main  source  of  noise  and  cooling  to  -80°C 
becomes  necessary. 


Readout  noise,  electrons,  rms 
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FIGURE  A3.5.  CCD  field  effect  transistor  (FET) 
noise  as  a  function  of  pixel  dwell  time  for  large  and 
small  pixels  and  when  using  buried  channel  vs.  surface 
channels.  Smaller  pixels  have  less  read  noise  because 
they  have  less  capacitance.  Buried  channels  have  almost 
lOx  less  read  noise  than  surface  channels. 
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TABLE  A3.1.  Typical  Performance  of  Various  Types  of  CCD  Cameras.  The  "Sensitivity"  Column  Is  a  Reasonable  Estimate  of 
the  Relative  Suitability  of  the  Camera  for  Detecting  Very  Faint  Signals.  It  Spans  a  Very  Large  Range  of  Performance! 


All  CCDs  are  not  equal 


Type 

Grade 

QE  %  (effective) 

Noise  (e/pix) 

Sensitivity  (relative) 

Bit  depth 

Dynamic  Range 

Video 

commercial  color 

10 

200 

i 

10 

1,000 

monochrome 

20 

200 

2 

10 

1,000 

Digital 

1  Mhz,  color 

15 

50 

12 

12 

4,000 

1  Mhz,  mono 

30 

50 

24 

12 

4,000 

Back.  Ilium/ 

slow-scan 

90 

5 

720 

15 

40,000 

LLL-CCD 

(EMCCD) 

45 

0.1 

(18,000)* 

9* 

200,000 

*  Because  the  gain  of  the  electron  multiplier  amplifier  is  unknown  and  large,  it  is  not  simple  to 

measure,  or  even  define,  the  sensitivity  and  bit  depth  of  the  EM-CCDs. 

Blooming 

As  more  photons  are  absorbed,  the  charge  packet  clustered  around 
the  buried  channel  grows  and  mutual  repulsion  between  these 
electrons  renders  the  field  imposed  by  the  charge-transfer  electrode 
ever  less  successful  in  keeping  the  packet  together.  The  maximum 
charge  packet  that  can  be  stored  without  it  overflowing  into  nearby 
pixels  can  be  estimated  by  multiplying  the  pixel  area  (in  square 
micrometers)  by  600PE/pixel  (i.e.,  27kPEs  for  a  6.7  x  6.7pm 
pixel,  540kPEs  for  a  30  x  30pm  pixel).  This  overflow  problem  is 
referred  to  as  “blooming”  and,  in  CCDs  for  the  home-video  market 
it  is  limited  by  the  presence  of  an  ;z- layer,  deeper  in  the  Si.  When 
the  charge  packet  gets  too  big,  mutual  repulsion  between  the  PEs 
forces  some  of  them  into  this  overflow  layer,  through  which  they 
are  conducted  to  ground. 

While  this  anti-blooming  feature  is  convenient  for  removing 
the  effects  of  the  specular  reflections  found  in  images  from 
everyday  life,8  it  is  not  incorporated  into  many  full-frame  or  frame- 
transfer  scientific  CCDs  because  it  reduces  QE  for  long- 
wavelength  light.  As  this  light  penetrates  farther  into  the  Si  crystal 
before  being  absorbed,  much  of  it  reaches  the  overflow  layer  where 
any  PEs  produced  are  lost. 

Incomplete  Charge  Transfer 

Sometimes,  an  imperfection  in  the  Si  will  produce  a  pixel  that 
“leaks’"  charge.  Charge  deposited  into,  or  transferred  through,  this 
pixel  will  be  lost,  producing  a  dark  vertical  line  above  it.  In  addi¬ 
tion,  if  one  pushes  the  pixel  clock  too  fast,  some  PE  in  the  packet 
will  not  move  fast  enough  and  they  will  be  left  behind.  In  general 
however,  on  a  slow-scanned,  scientific  CCD,  fewer  than  5  PEs  out 
of  a  million  are  lost  (or  gained)  in  each,  slower,  vertical  transfer 
and  only  50  (0.005%)  are  lost  during  each,  faster  horizontal  trans¬ 
fer.  In  such  devices,  the  main  noise  term  is  Poisson  noise  for  any 
signal  level  above  ~20  PE/pixel,9  and  it  seems  hard  to  imagine 
doing  much  better  than  this  except  for  signal  levels  <16  PE/pixel. 

On  the  other  hand,  it  is  also  true  that  the  vast  majority  of  CCDs 
made  (those  for  camcorders,  surveillance  cameras  and  even  many 


8  Features  such  as  the  image  of  the  sun  reflecting  off  a  shiny  automobile  can 
be  over  l,000x  brighter  than  the  rest  of  the  scene.  Fortunately,  such  extremely 
bright  features  are  seldom  found  in  microscopic  images  unless  a  crystal  of 
fluorescent  dye  occurs  in  the  field  of  view. 

9  This  calculation  assumes  that  the  read  noise  is  4  e/pixel,  and  this  will  be  less 

than  the  Poisson  Noise  for  any  signal  >16  PE.  However,  as  many  CCDs  used 
in  microscopy  have  >4  e/pixel  of  noise,  this  cut-off  point  should  not  be  con¬ 
sidered  inflexible. 


scientific  applications),  operate  with  much  (lOOx?)  less  perfection. 
In  microscopy  today,  we  find  CCDs  that  span  this  range  of  per¬ 
formance  (Table  A3.1). 

All  CCDs  are  not  equal!! 

CHARGE  AMPLIFIERS 

So  far,  I  have  described  an  image  sensor  in  which  up  to  90%  of 
the  impinging  photons  make  free  PEs  and  explained  how  the 
charge  packets  that  result  from  many  photons  hitting  a  given  pixel 
can  be  conveyed  to  the  charge  amplifier,  in  a  time-labeled  manner 
and  almost  without  change.  Clearly  the  performance  of  the  entire 
image  detector  will  depend  crucially  on  the  capabilities  of  this 
amplifier. 

What  Is  a  Charge  Amplifier? 

Although  most  scientists  have  had  some  exposure  to  electronic  cir¬ 
cuits  that  amplify  an  input  voltage  or  current,  they  may  be  less 
familiar  with  the  operation  of  the  type  of  charge  amplifier  found 
in  a  CCD.  The  following  outline  is  presented  to  enable  the  reader 
to  understand  enough  about  the  process  to  appreciate  some  of  the 
important  differences  between  the  various  types  of  CCD. 

Because  of  the  pulsatile  nature  of  the  CCD  charge  delivery 
system,  the  optimal  way  to  measure  charge  packet  size  is  to  deposit 
it  into  a  (very)  small  capacitor  (  the  “read  node”)  and  then  measure 
the  voltage  on  this  capacitor  with  a  high  impedance  amplifier.  As 
a  field-effect  transistor  (FET)  has  an  almost-infinite  input  imped¬ 
ance,  it  is  ideal  for  this  purpose  and  in  fact,  its  existence  makes 
charge-amplification  possible. 

There  are  two  basic  types  of  conventional  CCD  readout 
amplifier,  non-destructive  and  destructive.10  Both  employ  FET 
amplifiers. 

Non-destructive  (“skipper”)  amplifiers  use  an  FET  with  a 
“floating  gate”  to  sense  the  size  of  a  charge  packet  by  responding 
to  the  moving  field  that  is  produced  as  the  packet  is  transferred 
along  a  charge-coupled  register.  Because  the  charge  packet  itself 
is  not  affected  by  this  process,  the  process  can  be  repeated  hun¬ 
dreds  or  even  thousands  of  times.  If  the  results  of  all  these  mea¬ 
surements  are  averaged,  very  low  readout  noise  levels  (>+l 
electron/pixel)  can  be  obtained,  but  at  the  cost  of  a  substantial 


10  The  “electron-multiplier”  amplifiers  mentioned  previously,  act  essentially  as 
pre-amps  to  the  conventional  FET  amps  described  here.  They  will  be  covered 
later  in  this  Appendix. 
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would,  itself,  produce  a  random  electronic  noise  signal  larger  than 
this,  and  electronic  noise  increases  with  readout  speed,  read-node 
capacitance  and,  to  a  lesser  extent,  temperature. 

The  success  of  the  CCD  in  overcoming  this  limitation  depends 
on  two  factors: 

•  The  extremely  small  capacitance  of  the  read  node  compared 
to  that  of  any  other  photosensor  such  as  a  photodiode. 

•  Special  measurement  techniques  such  as  correlated  double¬ 
sampling 

Clearly  there  are  a  lot  of  tricks  to  making  the  perfect  CCD 
amplifier  and  not  all  CCDs  employ  them.  Table  A3.1  lists  typical 
performance  for  a  variety  of  common  camera  types. 


NOISE  SOURCES  IN  THE 
CHARGE-COUPLED  DEVICE 

Fixed  Pattern  Noise 


FIGURE  A3. 6.  Destructive  read-out  amplifier  for  a  CCD  chip. 


(xlOO  or  x  1,000)  increase  in  readout  time  and  logical  complexity. 
This  approach  might  make  sense  on  a  Mars  probe  but  it  has  not 
been  used  in  microscopy  to  my  knowledge. 

Destructive  readout  amplifiers  are  more  common,  probably 
because  they  can  operate  more  rapidly  (Fig.  A3. 6).  As  imple¬ 
mented  in  a  scientific  CCD,  the  charge  amplifier  consists  of  the 
following  components: 

•  overlying  charge  transfer  electrodes  to  drag  the  next  charge 
packet  into  the  “read  node” 

•  the  read  node  itself:  a  0.03-0. 1  picofarad  capacitor 

•  the  sense  FET 

•  the  reset  FET 

In  operation,  fields  from  the  overlying  charge-coupling  elec¬ 
trodes  force  a  charge  packet  into  the  read-node  capacitor,  creating 
a  voltage,  Vc,  that  is  proportional  to  the  amount  of  charge  in  the 
packet.  This  voltage  is  sensed  by  the  sense  FET  and  the  output  is 
passed,  via  additional  amplifiers,  to  the  analog-to-digital  converter 
(ADC)  where  the  signal  is  converted  into  a  digital  number.  Finally, 
just  before  the  next  charge  packet  is  coupled  into  the  read  node,  a 
reset  FET  discharges  the  capacitor,  forcing  Vc  to  zero,  and  allow¬ 
ing  the  read-FET  to  sense  it  again. 

FET  Amplifier  Performance 

The  signal  current  (signal  charge/s)  coming  from  a  CCD  sensor  is 
very  small.  Suppose  that  there  were,  on  average,  400  PE  in  every 
pixel  of  a  512  x  512  pixel  sensor.11  Reading  this  out  in  one  second 
would  constitute  a  current  of  only  10~u  Amps.  The  current  through 
the  bulb  in  a  home  flashlight  is  1010  times  more.  A  very  good  con¬ 
ventional  electronic  amplifier  designed  to  amplify  this  current 


11  Although  this  number  may  seem  small,  it  is  actually  quite  high  compared  to 
some  uses  in  biological  confocal  microscopy.  Many  authors  have  found  that 
in  normal”  use.  a  single-beam  confocal  microscope  used  to  image  a  fairly 
faint  stain  will  count  4-8  PE/pixel  in  bright  areas  of  the  image.  Allowing  that 
the  effective  QR  of  a  good  CCD  will  be  ~10x  higher  than  that  of  the  photo¬ 
multiplier  tube  used  in  the  confocal  microscope,  this  makes  the  expected 
peak  CCD  signal  in  an  image  from  a  disk-scanning  confocal  microscope  only 
40-80  PE/pixel. 


When  exposed  to  a  uniform  level  of  illumination,  some  pixels  in 
a  CCD  array  will  collect  more  charge  than  others  because  of  small 
differences  in  their  geometry  or  their  electrical  properties.  Conse¬ 
quently,  it  can  be  necessary  to  use  stored  measurements  of  the  rel¬ 
ative  sensitivity  of  each  pixel  to  normalize,  or  “flat  field,”  the  final 
dataset  on  a  pixel-by-pixel  basis.  This  is  accomplished  by  first 
recording  an  image  of  a  featureless  “white”  field.  This  is  often 
approximated  by  a  brightfield  transmission  image  with  no  speci¬ 
men,  a  process  that  will  also  record  “inhomogeneity,”  or  mottle, 
in  the  optical  system.  Differences  in  gain  between  pixels  are 
evident  as  visible  as  nonuniformities  in  the  digital  signal  stored  in 
the  memory  and  these  are  used  to  derive  multiplicative  correction 
coefficients.12 

Unfortunately,  one  can  only  preserve  the  high  precision  of  the 
CCD  output  if  the  coefficient  used  to  normalize  each  pixel  is 
equally  precise.  In  any  event,  these  correction  coefficients  vary 
with  both  the  photon  wavelength  and  the  angle  at  which  the  light 
passes  through  the  polysilicon  electrodes  on  its  way  to  the  buried 
channel.  This,  in  turn,  depends  on  the  details  of  the  precise  optical 
path  in  operation  when  an  image  is  recorded  and  may  even  change 
with  microscope  focus!  As  the  intrinsic  noise  of  a  pixel  holding 
360  k,  PEs  is  only  ±600  electrons  or  0.16%,  pixel-to-pixel  nor¬ 
malization  for  changes  in  sensor  gain  is  seldom  perfectly  effective 
and  consequently  there  is  usually  some  level  of  “Fixed-pattern 
noise”  superimposed  on  the  final  data. 

In  addition,  the  “white”  image  that  must  be  used  for  pixel-level 
sensitivity  normalization  is  itself  subject  to  intrinsic  noise  (±600 
electrons  for  a  signal  from  a  pixel  with  a  full  well  charge  of  360  k 
electrons)  and  so  multiplicative  normalization  may  actually  add 
some  noise  to  the  raw,  uncorrected  signal!  Fortunately,  if  the  white 
image  can  be  defined  by  a  multi-frame  averages  of  several,  nearly 
full-well  “white”  images,  this  normalization  noise  should  only  be 
noticeable  when  the  image  data  to  be  corrected  is  similarly  noise 
free.  Without  details  of  the  signal  levels  present  or  the  optical 
system  in  use,  it  is  difficult  to  estimate  the  magnitude  of  normal¬ 
ization  noise  but  it  will  be  comparatively  less  important  for  images 
of  faint  objects  containing  few  counts/pixel  because  these  mea¬ 
surements  are  themselves  less  precise. 


12  These  correction  coefficients  are  small  and  only  needed  when  operating  on 
images  involving  large  numbers  of  photons  (and  consequently  having  rela¬ 
tively  low  Poisson  noise  and  good  S/N). 
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It  should  be  also  noted  that  the  vignetting  and  “mottle”  visible 
in  images  characteristic  of  video-enhanced  contrast  microscopy 
will  produce  small  intensity  errors  in  the  data  obtained  by  both 
widefield  and  confocal.  However,  this  noise  term  will  be  more 
noticeable  in  widefield  where  more  photons  are  used  and  hence  the 
precision  of  the  data  is  greater.  Mottle  is  produced  by  dirt  and 
surface  imperfections  on  any  optical  components  that  are  not 
located  exactly  at  aperture  planes,  as  well  as  by  non-uniformities 
in  the  image  sensor.  Fortunately,  to  the  extent  that  it  is  stable  with 
time,  mottle  will  be  removed  by  the  flat-field  correction  for  CCD 
sensitivity  just  discussed. 

What  will  not  be  removed  is  any  change  in  signal  caused  by 
stray  light  (room  light,  light  that  goes  through  filters  designed  to 
remove  it,  etc.).  The  simplest  test  of  any  CCD  set-up  is  to  record 
an  image  of  “nothing”  (i.e.,  room  dark,  no  excitation,  no  specimen 
etc.).  Then  do  the  same  with  lOOx  longer  exposure  time  with  the 
room  lights  at  your  normal  operating  level.  Now  adjust  the  display 
look-up  tables  so  that  you  can  “see  the  noise”  in  both  the  images 
on  the  screen.  Although  the  only  difference  between  the  two 
images  should  be  increased  dark  noise  in  the  image  with  the  longer 
exposure,  this  is  seldom  the  case. 

Noise  from  the  Charge  Amplifier 

Noise  is  generated  by  both  the  readout  and  the  reset  FETs  in  the 
charge  amplifier.  Noise  generated  in  the  readout  FET  reaches  the 
ADC  directly.  If  thermal  noise  in  the  reset  FET  prevents  it  from 
completely  discharging  the  read-node  capacitance,  it  produces  a 
random  offset  at  each  pixel  (i.e.,  the  read-node  voltage  is  not  reset 
exactly  to  zero).  This  is  referred  to  as  Reset  Noise  and  has  the 
effect  that  the  dark  charge  seems  to  vary  from  pixel  to  pixel.  For¬ 
tunately,  Reset  noise  can  be  almost  eliminated  by  employing  the 
technique  of  Correlated  Double-sampling  (CDS)  in  the  readout 
amplifier.  In  CDS,  the  circuitry  of  the  charge-to-voltage  amplifier 
is  modified  so  that  the  output  is  proportional  to  the  difference 
between  the  value  of  Vc  just  after  the  reset  pulse  and  its  value  after 
the  next  charge  packet  has  been  inserted. 

Although  CDS  essentially  eliminates  the  effect  of  reset  noise, 
it  also  distorts  the  noise  spectrum.  On  the  one  hand,  this  distortion 
has  the  beneficial  effect  of  converting  the  low  frequency,  1/F  noise 
from  the  FET  into  broadband  noise  which  is  more  easily  treated 
theoretically  and  which  is  less  visually  distracting  than  the  short, 
horizontal  flashes  characteristic  of  1/F  noise.13  On  the  other  hand, 
it  means  that  the  input  to  the  ADC  must  be  carefully  frequency- 
filtered.  This  filtering  can  be  implemented  either  by  employing  RC 
circuits  or  by  using  dual-slope  integration  (DSI)  in  the  ADC  itself. 
If  there  are  large  intensity  variations  between  neighboring  pixels, 
the  use  of  RC  circuits  will  effectively  compromise  the  large 
dynamic  range  of  the  CCD.  Therefore,  ADCs  using  DSI  are 
employed  on  most  slow-scan  scientific,  cooled-CCDs. 

The  fact  that  CDS  and,  in  particular,  DSI  work  best  at  low 
readout  speeds  is  a  final  reason  why  most  scientific  CCDs  operate 
best  at  relatively  low  readout  speeds  (Fig.  A3. 5).  The  other  two 
reasons  are  improved  charge  transfer  efficiency  and  the  reduction 
in  broadband  electronic  noise  from  the  FETs  (noted  above.) 


13  In  a  CCD  without  CDS,  noise  features  will  seem  to  be  smeared  sideways, 
while  in  one  with  CDS,  they  will  appear  as  one-pixel- wide  stipple  with  no 
directionality. 


Where  Is  Zero? 

A  final  important  feature  of  the  CCD  readout  is  that,  compared  to 
the  photomultiplier  tube  (PMT),  it  is  relatively  difficult  to  deter¬ 
mine  the  exact  output  signal  level  that  corresponds  to  a  zero-light 
signal.  A  properly  operated  PMT  never  records  negative  counts. 
However,  as  the  electronic  readout  noise  of  a  cooled-CCD  is  an 
RMS  function  with  both  positive  and  negative  excursions,  there 
will  be  some  pixels  that  measure  lower  than  the  mean  value  of  the 
zero-light  pixel  intensity  distribution. 

To  ensure  that  no  data  is  “lost,”  scientific  CCDs  are  usually  set 
up  so  that  the  zero-light  signal  is  stored  to  be  a  few  tens  of  digital 
units  (ADU)  above  zero.  A  histogram  of  numbers  stored  from  a 
“black”  image  will  show  a  Gaussian-like  peak  centered  at  the 
offset  and  with  a  half-width  equal  to  2x  the  RMS  read  noise  (see 
Fig.  4.20).  This  offset  makes  it  more  difficult  to  apply  the  gain  and 
offset  normalization  procedures  to  images  that  record  only  a  few 
detected  photons  in  each  pixel,  a  factor  that  will  become  more 
important  as  CCDs  are  increasingly  used  to  image  living  cells  that 
cannot  tolerate  intense  illumination  and  which  therefore  produce 
substantially  lower  signal  levels. 


A  NEW  IDEA:  THE  GAIN  REGISTER  AMPLIFIER!! 

Early  in  2002,  a  new  type  of  readout  amplifier  was  introduced  by 
Texas  Instruments  (Houston,  TX)  and  E2V  Technologies  (Chelms¬ 
ford,  UK).  As  only  E2V  makes  back-illuminated  sensors,  I  will 
describe  their  system  but  both  work  along  similar  lines.  E2V  orig¬ 
inally  referred  to  their  device  as  the  “gain  register”  and  its  purpose 
is  to  amplify  the  size  of  the  charge  packet  before  it  arrives  at  the 
read  node.  Although  the  term  gain  register  has  recently  been 
replaced  by  the  term  “electron  multiplier”,  it  is  important  to 
remember  that  these  new  detectors  work  on  a  completely  different 
principle  from  that  employed  in  intensified-CCDs. 

The  gain  register  superficially  resembles  an  additional  HR, 
with  two  important  differences: 

•  There  are  4  phases  rather  than  the  usual  3  and  the  new 
phase  consists  of  a  grounded  electrode  located  between  (I)! 
and  02. 

•  The  charge  transfer  voltage  on  (f»2,  is  now  variable,  between 
+35  and  +40  volts  rather  than  the  usual  +15  volts. 

As  a  result,  when  (|)2  is  excited,  there  exists  a  high  electric  field 
between  it  and  the  grounded  electrode.  The  high  field  accelerates 
the  electrons  in  the  charge  packets  more  rapidly  as  they  pass  from 
0!  to  02  with  the  result  that  each  PE  has  a  small  (but  finite;  usually 
in  the  range  of  0.5%  to  1.5%)  chance  of  colliding  with  a  lattice 
electron  and  knocking  it  into  the  conduction  band  (Fig.  A3. 7). 
Assuming  the  1%  gain  figure,  this  means  that  for  every  100  PE  in 
the  packet,  on  average  one  of  these  will  become  two  electrons 
before  it  reaches  the  space  under  02-  Although  this  seems  like  a 
trivial  improvement,  after  it  has  been  repeated  as  part  of  the  400 
to  590  transfers  in  the  gain  register,  a  total  average  gain  of  hun¬ 
dreds  or  even  thousands  is  possible.  If  the  voltage  on  02  is  reduced 
to  normal  levels,  the  sensor  operates  as  a  normal  CCD. 

As  a  result,  a  single  PE  can  be  amplified  sufficiently  to  be 
safely  above  the  noise  of  the  FET  amplifier,  even  when  it  is  oper¬ 
ating  at  speeds  considerably  higher  than  video  rate  (35-50  MHz, 
vs.  13  MHz  for  video).  As  the  amount  of  gain  depends  exponen¬ 
tially  on  the  exact  voltage  on  02,  it  is  possible  to  “dial  in”  the 
amount  of  gain  needed  to  keep  the  signal  level  well  above  the  noise 
of  the  FET  amplifier.  However,  it  is  important  to  remember  that 
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FIGURE  A3. 7.  Energy  diagram  of  an  electron-multiplier  CCD  amplifier.  The  high  held  region  that  occurs  between  4>2  and  (|)DC  when  (]>2  goes  strongly  posi¬ 
tive  (right)  causes  about  1%  of  the  electrons  passing  this  region  to  collide  with  a  lattice  electron  with  sufficient  energy  to  boost  it  to  the  conduction  band.  Repeated 
over  hundreds  of  transfers,  this  process  is  capable  of  providing  an  average  amplification  of  hundreds  or  even  thousands  of  times. 


the  use  of  high  EM  gain  will  tend  to  saturate  the  “full-well”  capac¬ 
ity  of  later  pixels  in  the  gain  register,  reducing  intra-scene  dynamic 
range.14  Although  this  effect  can  be  reduced  to  some  extent  by 
making  each  pixel  in  the  gain  register  (and  the  read  node)  larger, 
this  approach  is  limited  by  the  fact  that  one  triplet  of  electrodes 
can  control  a  band  of  silicon  only  -18  pm  wide  and  because  a 
larger  read-node  capacitance  increases  the  read  noise  of  the  FET 
amplifier. 

In  sum,  the  gain-register  CCD  works  like  a  normal  fast-scan 
CCD  with  no  read  noise.  The  high  scan  speed  makes  focusing  and 
sample  scanning  quick  and  easy  and  the  device  preserves  the  full 
spatial  resolution  of  the  CCD  because  the  charge  packet  from  one 
pixel  is  always  handled  as  a  discrete  entity  (unlike  in  an  intensi- 
fied-CCD).  Of  course,  with  fast  readout,  there  is  less  time  to  accu¬ 
mulate  much  signal  and  the  resulting  image  may  have  considerable 
Poisson  noise.  But  this  is  not  the  camera’s  fault! 

Alternatively,  the  output  of  many  frames  can  be  summed  to 
reduce  Poisson  noise  or,  if  the  signal  is  bright,  one  can  turn  off  the 
EM  gain  and  have  a  fully  functional  scientific  CCD.15 

If  the  gain-register  CCD  is  read  out  fast,  there  is  so  little  time 
for  dark  charge  to  accumulate  that  cooling  would  seem  unneces¬ 
sary  until  one  remembers  that  one  can  now  “see”  even  one  PE  of 
dark-charge  above  the  read  noise.  Because  multi-pinned  phasing 
(MPP)  is  less  effective  during  the  readout  clockings,  significant 
dark  charge  can  be  generated  during  readout.  If  the  exposures  are 
short,  this  source  of  dark  charge  becomes  significant,  and  in  an 
EM-CCD,  even  one  electron  is  significant!  In  practice,  the  best 
performance  is  obtained  when  the  EM-CCD  is  cooled  to  between 
-80°  and  -100°C. 


14  If  a  register  designed  with  enough  pixel  area  to  hold  a  normal  full-well  charge 
of  30,000  electrons,  is  used  with  a  gain  of  lOOx,  then  the  pixels  near  the  end 
of  the  gain  register  will  become  full  whenever  the  original  charge  packet  has 
>300  electrons. 

15  Because,  as  noted  above,  the  read  node  of  the  FET  amplifier  at  the  end  of 
the  gain  register  in  an  EM-CCD  has  a  relatively  large  capacitance,  E2V  offers 
two  separate  FET  readout  amps.  The  one  mounted  at  the  end  of  the  gain  reg¬ 
ister  is  optimized  for  fast  readout.  The  other  is  mounted  at  the  end  of  the  HR 
not  connected  to  the  gain-register,  has  low  input  capacitance  and  is  optimized 
to  read  out  slowly  with  low  noise.  Signal  is  sent  to  the  latter  by  reversing  the 
charge  transfer  sequence  applied  to  the  HR. 


EM-CCDs  have  one  other  important  form  of  “dark  noise” 
called  Clock  Induced  Charge  (CIC,  also  known  as  spurious  noise). 
CIC  typically  consists  of  the  single-electron  events  that  are  present 
in  any  CCD,  and  are  generated  by  the  vertical  clocking  of  charge 
during  the  sensor  readout.  The  process  involved  is  actually  the 
same  impact  ionization  that  produces  multiplication  in  the  gain 
register;  however,  levels  are  much  lower  because  lower  voltages 
are  involved.  In  conventional  CCDs,  CIC  is  rarely  an  issue  as 
single-electron  events  are  lost  in  the  read  noise.  However,  in  EM- 
CCDs  where  the  read  noise  is  essentially  zero  and  dark  charge  has 
been  eliminated  through  effective  cooling,  CIC  is  the  remaining 
source  of  single-electron,  EM-amplified  noise.  If  left  unchecked, 
it  can  be  as  high  as  1  event  in  every  7  pixels.  Fortunately,  it  can 
be  minimized  by  careful  control  of  clocking  voltages  and  by  opti¬ 
mizing  the  readout  process  to  cope  with  faster  vertical  clock  speeds 
(down  to  0.4  ja,s/shift).  This  leaves  a  detector  with  less  than  one 
noise  pulse  in  every  250  pixels:  a  detector  extremely  well  adapted 
for  measuring  zero! 

Of  Course,  There  Is  One  Snag! 

The  charge  amplification  process  is  not  quite  noise  free  because 
the  exact  amount  by  which  each  electron  in  the  packet  is  ampli¬ 
fied  varies  in  a  stochastic  manner  (i.e.,  some  electrons  are  “more 
equal”  than  others.).  The  statistical  arguments  are  discussed  in  a 
paper  found  at  the  URL  listed  below  and  in  Chapter  4.  In  summary: 
as  the  multiplicative  noise  inherent  in  the  charge  multiplication 
process  creates  noise  that  has  a  form  very  similar  to  that  produced 
by  Poisson  statistics,  the  easiest  way  to  think  of  its  effect  is  to 
assume  that  the  amplifier  has  no  noise  at  all  but  that  the  signal 
being  fed  into  it  is  half  as  big  as  it  really  is.  In  other  words,  the 
camera  will  work  perfectly  but  it  will  work  as  though  it  has  a  QE 
that  is  only  half  of  what  it  really  is.  Back-illuminated  sensors  are 
now  available  with  an  intrinsic  QE  of  about  -90%  or  -45%  when 
used  in  the  gain-register  mode.  This  is  5-1  Ox  better  than  the  per¬ 
formance  available  from  most  PMTs  especially  in  the  red  end  of 
the  spectrum. 

It  is  worth  noting  that  one  can  use  electron  multiplication  and 
still  maintain  the  full  QE  by  using  the  detector  in  photon-counting 
mode,  as  is  now  being  done  by  many  astronomers.  Photon  count¬ 
ing  is  only  possible  when  one  is  able  to  confidently  see  a  single- 
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photon  event  as  different  from  any  dark  event  and  when  the 
number  of  photons  collected  during  an  exposure  is  so  low  that 
there  is  little  probability  of  >2  photons  falling  into  a  pixel.  To 
implement  photon-counting,  one  records  a  sequence  of  short  expo¬ 
sures  containing  “binary”-type  single-photon  data,  and  combines 
them  to  generate  an  image  that  is  free  of  multiplication  noise.16  To 
be  useful  for  recording  dynamic  events  in  living  cells,  an  extremely 
fast  frame  rate  would  be  needed.  This  may  be  more  possible  with 
some  future  EMCCD  sensors. 

(More  info  on  EM-CCDs  at  http://www.emccd.com  and 
http://www.marconitech.com/ccds/lllccd/technology.html) 

PART  II:  EVALUATING  A 
CHARGE-COUPLED  DEVICE 

A.  Important  Charge-Coupled  Device  Specs  for 
Live-Cell  Stuff! 

Although  in  Part  I,  much  time  was  spent  discussing  cooled,  slow- 
scan,  scientific  CCDs,  in  fact,  these  have  not  been  much  used  in 
biological  microscopy  since  Sony  introduced  the  ICX085,  1  k  x 
1.3  k,  micro-lens-coupled,  interline-transfer  chip  in  the  late  1990s. 
Although  initially  developed  not  for  the  scientific  market  but  to 
meet  the  needs  of  the  Japanese  high-definition  TV  standard,  these 
chips  offered  a  set  of  practical  advantages  that  biologists  found 
very  appealing: 

—  As  an  interline  transfer  chip,  it  needed  no  mechanical  shutter 
and  could  be  run  so  as  to  produce  a  continuous  stream  of 
images. 

—  The  high  readout  speed  (up  to  20Mhz)  allowed  real-time  im¬ 
aging  compared  with  the  5-10  s/frame  readout  then  common. 

—  The  6.45  x  6.45  pm  pixels  were  small  enough  to  sample  the 
image  produced  with  high-NA  40x  and  60x  objectives. 

—  The  lk  x  1.3k  raster  size  was  both  sufficient  for  most  bio¬ 
logical  microscopy  and  significantly  higher  than  that  of  the 
other  scientific  chips  then  available. 

—  Mass  production  allowed  the  development  of  a  micro-lens 
array  that  increased  the  QEeff  to  an  acceptable  level  for  a  front- 
illuminated,  interline  chip  and  did  so  at  a  price  biologists 
could  afford. 

As  a  result,  the  majority  of  CCDs  sold  for  use  in  biological 
microscopy  today  use  this  chip  or  its  higher-QE  cousin,  the 
ICX285.  Although  mass  production  made  quality  CCDs  available 
to  many  who  formerly  could  not  have  obtained  one,  it  is  impor¬ 
tant  to  remember  that  the  read  noise  of  ±8-24  electrons/pixel 
(depending  on  read  speed)  is  substantially  higher  than  ±2-3  elec¬ 
trons/pixel  that  characterized  the  best,  slow-scan,  scientific  CCDs. 
Although,  as  noted  below,  the  difference  is  only  important  if  the 
dimmest  pixel  records  fewer  than  ~50  electrons,  and  this  seldom 
occurs  in  widefield  fluorescence  microscopy,  the  disk-scanning 
confocal  fluorescence  microscopes  now  available  do  provide  an 
image  in  which  this  difference  is  significant  (Chapter  10). 

1.  Quantum  Efficiency  (QE): 

QE  is  the  ratio  of  photons  striking  the  chip  to  electrons  kicked  into 
the  conduction  band  in  the  sensor.  It  should  be  at  least  40%  and 


16  There  is  no  multiplicative  noise  because  any  spike  above  the  FET  noise  floor 
counts  as  one  electron,  no  matter  how  much  it  has  been  amplified. 


on  back-illuminated  chips,  it  can  reach  90%  (with  somewhat 
higher  fixed-pattern  noise). 

The  fill-factor  is  the  fraction  of  the  sensor  surface  actually  sen¬ 
sitive  to  light.  On  the  best  frame-transfer  CCDs,  it  can  be  almost 
100%.  On  interline  transfer  CCDs  it  may  be  only  40%.  Light  not 
absorbed  in  a  sensitive  area  is  lost,  reducing  the  QEeff  of  the  sensor 
proportionally. 

Factors  affecting  QE: 

Front-illuminated  chips 

•  Light  is  scattered  by  the  transparent,  polysilicon  charge- 
transfer  electrodes  that  overlie  the  photosensitive  silicon 
surface. 

•  This  scattering  is  more  severe  at  shorter  wavelengths.  Light 
that  is  scattered  is  not  detected. 

•  As  blue  light  is  absorbed  nearer  to  the  surface  than  red  light, 
and  “deep  electrons”  may  go  to  the  wrong  pixel,  CCD  resolu¬ 
tion  may  be  a  bit  lower  than  the  pixel  count  at  longer  wave¬ 
lengths,  especially  on  chips  with  small  pixels. 

•  Best  QE:  -20%  blue,  -35%  red/green 

•  Two  efforts  to  improve  the  QE  of  front-illuminated  chips 
include  “Virtual  Phase”  (one  phase  “open,”  Texas  Instruments, 
Houston,  TX)  and  the  use  of  indium  oxides  for  the  transfer 
electrodes  (Kodak,  Rochester,  NY).  These  have  increased  peak 
QE  to  the  range  of  55%. 

Micro-lens  array  chips 

•  Sony  has  pioneered  a  process  in  which  a  micro-lens  is  mounted 
above  each  pixel  of  a  front-illuminated,  interline-transfer 
CCD.  The  lenses  focus  most  of  the  impinging  light  onto  a  part 
of  the  CCD  where  reflection  losses  are  least,  pushing  the  QE 
to  65%  in  the  green,  less  in  the  red  (because  of  losses  to  the 
overflow  drain)  and  purple. 

Back-illuminated  chips 

•  Made  by  thinning  the  silicon  and  then  turning  it  over  so  that 
the  light  approaches  the  pixel  from  what  would  have  been  the 
back  side.  This  avoids  scattering  in  the  transfer  electrodes  and 
increases  the  QE  to  about  90%  in  the  green  and  >70%  over  the 
visible  range. 

•  More  expensive  because  of  the  extra  fabrication. 

•  Slightly  less  resolution  and  more  fixed-pattern  noise,  caused 
by  imperfections  in  the  thinning  operation,  and  the  presence 
of  two  sets  of  surface  states. 

Color  Chips 

•  One-chip  color  sensors  employ  a  pattern  of  colored  filters,  one 
over  each  pixel.  Light  stopped  by  any  such  filter  cannot  be 
detected  and  is  therefore  lost.  The  QE  of  such  sensors  is  there¬ 
fore  at  least  3x  lower  than  for  an  otherwise  comparable  mono¬ 
chrome  chip. 

•  3-chip  color  sensors  use  dichroic  mirrors  to  separate  the 
“white”  light  into  three  color  bands,  each  of  which  is  directed 
to  a  separate  monochrome  CCD  sensor.  While  this  would  seem 
to  ensure  that  “all  photons  were  counted  somewhere,”  because 
such  systems  seldom  employ  microlenses,  their  effective  QE 
is  not  much  better  than  the  1-chip  color  sensors  and  alignment 
of  the  signal  light  is  important.17 


17  While  the  QE  is  not  much  better,  the  resolution  of  the  3-chip  camera  is  the 
same  as  that  of  each  chip,  without  the  interpolation  needed  to  disentangle  the 
3  colored  images  from  the  output  of  a  1-chip  color  sensor. 
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•  Color  can  be  detected  by  making  sequential  exposures  of  a 
monochrome  chip  through  colored  glass  or  LCD  filters.  This 
produces  the  same  QE  losses  as  the  patterned  filter  but  has 
the  advantage  that  it  can  be  removed  when  higher  sensitivity 
is  needed.  This  design  is  not  suited  for  imaging  moving 
objects. 

2.  Readout  Noise: 

This  spec  is  a  measure  of  the  size  of  the  pixels  and  the  quality  of 
the  circuitry  used  for  measuring  the  charge  packet  in  each  pixel. 
It  is  measured  in  "±RMS  electrons  of  noise”  (i.e.,  67%  of  a  series 
of  “dark”  readings  will  be  ±  this  much). 

•  A  good  scientific  CCD  camera  should  have  a  noise  level  of 
<±5  electrons  at  a  readout  speed  of  1 M  pixels/second. 

•  The  readout  noise  increases  with  the  square  root  of  the  readout 
speed  (see  Table  A3. 2). 

•  NO  Free  Lunch!  A  chip  that  has  ±5e  RMS  of  noise  when 
readout  at  100  k  pixels/sec  (or  10  seconds  to  read  out  a  1024 
X  1024  chip),  should  produce  ±50e  RMS  of  noise  if  read  out 
at  10M  pixels/sec  (or  0.1  sec  to  read  the  same  chip). 

What  Is  "Good  Enough"? 

Very  low  readout  noise  is  only  essential  when  viewing  very  dim 
specimens:  luminescence,  or  low  level  fluorescence.  Read  noise  is 
only  a  limitation  when  it  is  more  than  the  statistical  noise  on  the 
photon  signal  in  the  dimmest  pixel  (i.e.,  >sqrt  of  the  number  of 
detected  photons  =  sqrt  #  electrons). 

Consider  the  signal  levels  that  you  plan  to  use.  Will  the  darkest 
important  part  of  your  image  have  zero  signal  or  do  you  expect 
some  background  signal  from  diffuse  staining  or  out  of  focus  light? 
If  the  dimmest  pixel  in  your  image  represents  ~100  electrons,  then 
the  Poisson  or  statistical  noise  on  this  background  signal  will  be 
±10  electrons.  “Adding”  an  additional  ±10  electrons  of  readout 
noise  will  not  make  much  difference  to  a  measurement  of  this 
background  signal  and  it  will  be  even  less  significant  when  added 
to  the  even  greater  Poisson  noise  present  in  pixels  where  the 
stained  parts  of  the  image  are  recorded. 

This  is  especially  true  because  RMS  noise  signals  add  as  the 
“sqrt  of  the  sum  of  the  squares”  (i.e.,  the  total  noise  from  ±10  elec¬ 
trons  of  readout  noise  and  ±10  electrons  of  Poisson  noise  is  only 
sqrt  (100  +  100)  =  ±14  electrons). 

On  the  other  hand,  if  you  are  really  trying  to  keep  those  cells 
alive  and  you  find  that  2,000  electrons  in  the  bright  areas  is  enough, 
the  dark  areas  may  now  be  only  50  electrons.  As  the  sqrt  of  50  is 
about  ±7,  an  additional  ±10  electrons  of  readout  noise  may  no 
longer  be  acceptable,  but  only  if  you  have  to  make  measurements 
in  the  dark  areas  on  your  image.  In  this  case,  the  obvious  choice 
is  a  slower,  quieter  CCD  or  an  EM-CCD. 

While  in  widefield  fluoresecence,  the  background  stain  level 
is  seldom  so  low  that  the  sqrt  of  the  signal  recorded  is  lower  than 
the  read  noise,  the  disk-scanner  does  provide  such  an  image 
(Chapter  10).  As  one  of  the  main  advantages  of  disk-scanning  is 
that  one  can  scan  an  entire  image  plane  very  rapidly,  the  fact  that 
one  can  read  out  the  EM-CCD  very  rapidly  without  increasing  the 
read  noise  makes  it  the  ideal  detector  for  this  type  of  scanner  (or, 
indeed  for  high-speed  line  scanning  confocal  microscopes). 

3.  Pixel  Size: 

•  Nyquist  sampling:  The  size  of  a  pixel  on  the  CCD  is,  in  itself, 
not  very  important  BUT  one  must  satisfy  the  Nyquist  crite¬ 
rion:  The  pixels  on  the  chip  must  be  ~4+  smaller  than  the 
smallest  features  focused  onto  it18  (see  Chapter  4):  Pixel  size 


TABLE  A3. 2.  Dynamic  Range  and  Pixel  Size 


12-bit  camera 
w/small  pixels 

14-bit  camera 
w/large  pixels 

Pixel  Size 

6.7  x  6.7 pm 

24x24 

Full  Well 

27,000 

345,000 

Least  significant  bit  = 

6.5  electrons 

21  electrons 

Implied  noise  level 

+13  electrons 

+42  electrons 

on  the  chip  determines  the  total  specimen-to-ehip  magnifi¬ 
cation  needed! 

Two  examples: 

a.  1.4  NA  lOOx  objective  and  a  lx  phototube. 

•  The  Abbe  Criterion  resolution  @  400  nm  is  about 
0.22pm.  Magnified  by  a  total  magnification  of  lOOx, 
this  becomes  22  pm  at  the  CCD. 

•  A  CCD  having  8x8  pm  pixels  samples  such  an  image 
adequately  (~2.8  pixels/resolution  element). 

b.  1.3  NA  40x  objective  and  a  lx  phototube. 

•  The  Abbe  Criterion  resolution  @  400  nm  is  now 
0.25  pm.  Magnified  40x  this  becomes,  10  pm. 

•  A  CCD  having  8  X  8  pm  pixels  is  inadequate  to  sample 
this  lower-mag,  high-resolution  image. 

If  you  must  use  this  objective,  you  need  either  a  higher  mag 
phototube  (2.5x)  or  a  chip  with  3  X  3  pm  pixels  or  (as  CCD 
pixels  are  seldom  this  small),  some  combination. 

•  Saturation  signal  level:  The  maximum  amount  of  signal  that 
can  be  stored  in  a  pixel  is  fixed  by  its  area.  The  proportion  is 
600  electrons/square  pm,  so  a  10pm  X  10pm  pixel  can  store 
a  maximum  of  60,000  electrons  before  they  start  to  bleed  into 
neighboring  pixels.  In  practice,  as  fluorescent  micrographs  of 
living  cells  seldom  produce  signals  this  large,  large  pixels  are 
usually  unnecessary. 

However,  the  saturation  level  also  represents  the  top  end  of 
another  spec,  the  dynamic  range.  This  is  usually  quoted  as 
12-bit  (4000: 1)  or  14-bit  (16,000: 1)  etc.,  and  represents  the 
ratio  between  the  full-well  saturation  level  and  the  readout 
noise.  Therefore,  a  camera  with  relatively  high  readout  noise 
can  still  look  good  in  terms  of  dynamic  range  if  it  has  large 
pixels  and  hence  a  high  full-well  capacity.  Conversely,  a  12-bit 
camera  with  small  pixels  can  have  less  actual  noise-per-pixel 
intensity  measurement  than  a  14-bit  camera  with  large  pixels. 

In  this  case,  the  noise  level  of  the  14-bit  camera  is  >3x  that 
of  the  12-bit  camera.  Your  signal/pixel  would  have  to  be 
3x  larger  in  order  to  be  “seen”  when  using  this  particular 
14-camera. 

4.  Array  Size:19  The  argument  for  small 

•  Assuming  0.1pm  pixels  (referred  to  the  object  plane),  a  512 
X  512  pixel  chip  will  image  an  area  of  the  specimen  that  is 
about  51  X  51  microns.  If  this  is  enough  to  cover  the  objects 
you  need  to  see,  this  small  chip  has  a  lot  of  advantages  over 
chips  that  are  1024  X  1024,  or  larger. 

•  Lower  cost 


18  Of  two  times  smaller  than  the  '‘resolution,”  as  defined  by  Rayleigh,  or  Abbe. 

19  The  array  size  refers  to  the  number  of  lines  and  pixels  in  the  sensor,  not  to 
its  total  area. 
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•  4x  fewer  pixels  to  read  out,  meaning  either: 

—  4x  slower  readout  clock,  giving  2x  lower  readout  noise. 
—  Same  clock  speed  and  noise  level  but  4x  faster  frame  time. 
(Easier  to  scan  specimen  to  find  the  interesting  part!  Time 
is  money!) 

•  4x  less  storage  space  needed  to  record  data. 

The  argument  for  big: 

•  Manufacturing  improvements  are  reducing  readout  noise 
levels  at  all  readout  speeds,  and  CCDs  with  more  pixels  often 
also  have  smaller  pixels  which  can  lead  to  lower  read  noise.  If 
your  labels  are  bright,  having  a  larger  chip  allows  you  to  see 
more  cells  in  one  image  (as  long  as  they  are  confluent!). 
Assuming  that  Nyquist  is  met  in  both  cases,  a  large  print  of  an 
image  recorded  from  a  larger  sensor  always  looks  sharper  than 
one  from  a  smaller  array. 

•  Binning:  Binning  refers  to  the  process  of  summing  the  charge 
from  neighboring  pixels  before  it  is  read  out.  This  increases 
the  size  of  each  charge  packet  read  (making  it  look  brighter) 
and  reduces  the  number  of  pixels.  For  example:  2x2  binning 
allows  the  owner  of  a  1024  X  1024  chip  to  obtain  the 
speed/noise  performance  similar  to  the  smaller  chip  (512  X 
512)  and  to  do  so  in  a  reversible  manner.  However,  the  optical 
magnification  may  need  to  be  increased  to  preserve  Nyquist 
sampling. 

Before  deciding  that  you  need  a  larger  chip,  compare  what  you 
would  get  if  the  same  money  were  spent  on  another 
scope/CCD/graduate  student! 

Bottom  line: 

•  If  more  pixels  means  smaller  pixels,  they  will  each  catch  fewer 
photons  unless  the  magnification  is  reduced  proportionally. 
More  pixels  at  the  same  frame20  rate  mean  somewhat  higher 
read  noise  because  the  pixel  clock  must  go  faster. 

5.  Readout  Speed: 

Although  readout  speed  has  been  discussed  above,  we  haven’t 
mentioned  that  some  good  CCD  cameras  have  variable  speed  read¬ 
outs  and  the  new  EM-CCDs  impose  no  high  read  speed  penalty 
(Table  A3. 3). 

It  is  convenient  to  be  able  to  read  out  the  chip  faster  when 
searching  and  focusing  as  long  as  one  can  then  slow  things  down 
to  obtain  a  lower  read  noise  in  the  image  that  is  finally  recorded. 
However,  the  read  speed  is  only  one  limitation  on  the  frame  rate: 


TABLE  A3. 3.  CCD  Specifications 


Array  size 

Pixel  Clock  Rate 

Noise  level* 

Frame  time 

Frame  rate/s 

640  x  480 

13  MHz 

200  e/pixel** 

0.033 

30 

(video  rate) 

512  x  512 

100  kHz 

5  e/pixel 

2.5  sec 

0.4 

1  MHz 

15  e/pixel 

0.25  sec 

4 

5  MHz 

35  e/pixel 

0.05  sec 

20 

1024  x  1024 

100  kHz 

5  e/pixel 

10  sec 

0.1 

1  MHz 

15  e/pixel 

1  sec 

1 

5  MHz 

35  e/pixel 

0.2  sec 

5 

*  Assumes  conventional  FET  circuits.  **The  readout  noise  is  relatively  higher  at 
video  rate  because  the  higher  speed  often  precludes  the  use  of  various  techniques, 
such  as  correlated  double  sampling,  that  reduce  readout  noise. 


20  The  readout  speed  of  a  2  X  2  binned  1024  X  1024  is  a  bit  slower  than  an 
actual  512  X  512  because  twice  as  many  vertical  clock  cycles  are  needed, 
and  one  still  needs  to  read  out  pixel  by  pixel  in  the  horizontal  direction. 


if  the  signal  level  is  so  low  that  1  s/frame  is  required  to  accumu¬ 
late  enough  signal  to  be  worth  reading  out,  then  reducing  the  read 
time  much  below  0.1  s  loses  some  of  its  appeal. 

Faster  readout  speeds  are  particularly  important  for  moving 
specimens,  especially  when  doing  widefield/deconvolution  or 
when  following  rapid  intracellular  processes,  such  as  vesicle  track¬ 
ing  or  ion  fluxes. 

6.  Shutter  Stability: 

Though  not  strictly  a  CCD  spec,  electronic  (LCD)  or  mechanical 
shutters  are  often  built  into  modern  CCD  cameras.21  The  latter  have 
the  disadvantages  of  producing  vibration  and  having  a  limited  life¬ 
time  but  the  advantage  that  they  transmit  all  of  the  light  when  they 
are  open  (even  an  “open”  LCD  can  absorb  >50%  of  the  light,  other 
electronic  shutters  may  be  better). 

There  seems  little  point  in  having  a  camera  capable  of  record¬ 
ing  (say)  40,000  electrons/pixel  with  an  accuracy  of  ±200  e  (or 
0.5%!)  if  the  shutter  opening  time  is  only  accurate,  or  even  repro¬ 
ducible,  to  ±10%.  If  one  shutters  the  light  source  instead  of  the 
camera,  similar  limitations  apply. 

7.  User-friendliness: 

State-of-the-art  cameras  often  seem  to  have  been  designed  to  make 
sure  that  no  one  unwilling  to  become  a  devotee  of  “CCD  Opera¬ 
tion”  can  possibly  use  them  efficiently!  Start  off  by  asking  to  see 
an  image  on  the  screen,  updated  and  flat-fielded  at  the  frame-scan 
rate  and  showing  as  “white”  on  the  display  screen,  a  recorded 
intensity  that  is  only  ~5%  of  the  full-well  signal.  This  is  where  you 
should  do  most  living-cell  work.  Then  ask  the  salesman  to  help 
you  to  save  time-series  of  this  image.  Increase  the  display  con¬ 
trast  until  you  see  the  noise  level  of  the  image,  both  before  and 
after  “flat-fielding."  Put  a  cursor  on  one  pixel  in  the  top  frame  of 
the  stack  and  plot  its  intensity  over  the  series. 

8.  “The  Clincher"  (Well,  at  least  sometimes  . . .): 

Ask  him/her  what  the  intensity  number  stored  in  the  computer  for 
some  specific  pixel  means,  in  terms  of  the  number  of  photons  that 
were  recorded  at  that  location,  while  the  shutter  was  open.  To 
answer,  the  salesperson  will  have  to  know  the  QE,  the  fill  factor 
and  the  conversion  factor  between  the  number  of  electrons  in  a 
pixel  and  the  number  stored  in  the  computer  memory  (sometimes 
called  the  gain-setting).  To  help  them  out,  any  “real”  scientific 
CCD  camera  has  the  latter  number  written,  by  hand,  in  the  front 
of  the  certification  document  (usually  a  number  between  3  and  6). 
If  the  salesman  doesn’t  understand  the  importance  of  this  funda¬ 
mental  number,  what  hope  is  there  for  you?  (Hint:  It  is  important 
because  the  Poisson  noise  is  the  sqrt  of  the  number  of  electrons  in 
the  well,  not  the  sqrt  of  some  arbitrarily  proportional  number 
stored  in  your  computer.) 

B.  Things  That  Are  (Almost!)  Irrelevant  When 
Choosing  a  Charge-Coupled  Device  for 
Live-Cell  Microscopy 

1.  Dynamic  Range: 

This  is  the  ratio  of  the  “noise  level”  to  the  “full-well”  (or 
maximum)  signal.  Although  16-bit  may  sound  a  lot  better  than  12- 
bit,  you  need  to  think  before  you  are  impressed. 

The  noise  level  should  not  be  more  than  5  electrons/measure¬ 
ment.  Period! 


21  Often  the  same  advantage  can  be  gained  by  shuttering  the  light  source.  This 
may  become  more  common  as  pulsed  laser  or  light-emitting-diode  light 
sources  are  introduced  (see  Chapters  5  and  6). 
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Twelve  bits  is  4,000  levels.  If  the  first  level  represents  5  elec¬ 
trons  (in  fact,  it  should  represent  half  the  noise  or  2.5  e),  then  the 
4,000th  represents  20,000  electrons  or  (assuming  a  QE  of  50%), 
about  40,000  photons/pixel/measurement. 

How  often  do  you  expect  to  be  able  to  collect  this  much  signal 
from  an  area  of  a  living  cell  only  100  x  lOOnm  in  size?  You  should 
be  able  to  get  a  good,  8-bit  image  using  only  6%  of  the  dynamic 
range  of  a  12-bit  CCD  (Fig.  A3.8).22 

As  the  “full-well”  signal  is  only  proportional  to  the  area  of  the 
pixel  on  the  chip  (area  in  sq.  pm  x  600),  the  dynamic  range  is  only 
really  impressive  if  it  is  high  AND  a  chip  has  small  pixels.  Then 
it  means  that  the  readout  noise  is  low.  A  test  for  actual  dynamic 
range  is  described  below. 

Bottom  line:  For  disk-scanning  confocal  microscopy,  a  large 
dynamic  range  is  only  important  if  it  reflects  a  low  readout  noise 
level. 

Easier  to  just  check  the  readout  noise! 


3.  "Imaging  Range"  "Sensitivity"  (or  anything 
measured  in  LUX): 

Stick  to  something  you  (and  I?)  understand:  Photons/pixel  or  elec¬ 
trons/pixel.  The  other  conversions  are  not  straightforward. 

4.  "Neat  Results": 

Unless  you  know  how  well  stained  the  specimen  is,  you  cannot 
evaluate  an  image  of  it  in  a  quantitative  manner.  (Though  you 
may  not  want  to  admit  this!)  By  all  means,  view  your  own  speci¬ 
mens,  but  viewing  “test  specimens”  that  are  not  expected  to  fade 
and  have  a  known  structure  (fluorescent  beads  in  some  stable 
mounting  medium?)  facilitates  A/B  comparison.  If  you  do  use  your 
own  test  specimens  to  compare  cameras,  be  sure  to  view  them  on 
the  same  scope,  and  with  the  same  conditions  of  pixel  size  and 
readout  time  etc. 

Better  still .  .  . 


2.  High  Maximum  Signal  (high,  full-well 
number,  because  of  large  pixels): 

On  living  cells,  you  will  probably  never  have  enough  light  to  reach 
a  full-well  limit  of  even  20,000  electrons.  Even  if  you  do,  there 
are  better  ways  to  use  it  (more  lower-dose  images  to  show  time 
course?). 


If  16k  electrons  (full  well)  =  12  bits,  each  digital  level  =  4  electrons. 


—  1 2  bits 


16k  -i 


electrons 


If  the  read  noise  is  ±8  electrons,  or  2 
gray  levels,  one  can  obtain  a 
useful, “8-bit”  (256  levels)  image  by 
using  only  6%  of  its  dynamic  range. 


FIGURE  A3. 8.  Not  using  the  full  dynamic  range  of  a  CCD.  As  most  scien¬ 
tific  CCDs  have  more  dynamic  range  than  one  “needs”  in  live-cell  fluorescence 
microscopy,  the  excitation  dose  to  the  specimen  can  be  reduced  if  one  sets  up 
the  CCD  control  program  to  display  an  8-bit  image  using  only  the  bottom  1 ,024 
levels  of  a  12-bit  image.  Such  an  image  is  more  than  adequate  for  many  func¬ 
tions  in  live-cell  biological  microscopy  (particularly  when  other  factors  such 
as  dye-loading  etc.,  may  cause  larger  errors)  and  will  require  only  6%  as  much 
signal  as  would  a  “full-well”  image. 


22  Remember,  given  optical  and  geometrical  losses,  you  can  collect  no  more 
than  about  3-10%  of  the  photons  produced,  and,  each  fluoroscein  molecule 
will  only  produce  perhaps  30,000  excitations  before  “dying.” 


C.  A  Test  You  Can  Do  Yourself!!! 

Set  up  each  camera  that  you  want  to  evaluate  on  a  tripod,  add  a 
C-mount  lens,  and  an  ND  3  or  ND  4  filter.  Hook  up  a  monitor  or 
computer  and  view  some  scene  in  your  laboratory  under  ordinary 
illumination  (avoid  light  from  windows  which  may  vary  from  day 
to  day). 

Close  the  lens  aperture  down  until  you  can  no  longer  discern 
the  image  (see  Fig.  4.20).  This  is  the  “noise-equivalent  light  level”: 
the  signal  level  at  which  the  electron  signal  (i.e.,  photons/pixel  x 
QE)  just  equals  the  total  noise  level.  Your  measure  is  the  aperture 
at  which  the  image  disappears.23  Because  it  is  sensitive  to  both  QE 
and  readout  noise  level,  this  is  a  very  useful  measure  of  what  we 
all  think  of  as  the  “sensitivity.”  Of  course,  the  signal  level  depends 
not  only  on  the  light  intensity  but  also  on  the  exposure  time  and 
the  pixel  area,  so  make  sure  to  keep  the  former  constant  and  make 
allowances  for  the  latter. 

If  you  do  not  have  even  these  meager  facilities  (a  C-mount 
lens,  an  ND  filter,  a  tripod  and  some  time),  take  an  image  of 
nothing.  Look  at  “no  light”  for  one  second,  and  for  100  seconds. 
Ask  to  see  a  short  line  profile  that  plots  intensity  vs.  position  along 
a  line  short  enough  that  one  can  see  the  intensity  of  each  individ¬ 
ual  pixel.  The  difference  in  the  average  intensity  between  the  short 
and  long  exposure  is  a  measure  of  the  leakage.24  With  a  little  cal¬ 
ibration  from  the  published  full-well  specs  (a  spec  less  open  to 
“interpretation”  than  “noise”),  you  can  even  get  a  direct  measure 
of  the  read  noise  level  from  these  dark  images.  (It  should  be  the 
standard  deviation  of  the  values  as  long  as  they  are  counted  in  elec¬ 
trons,  not  “magic  computer  units”  and  as  long  as  fixed-pattern 
noise  is  not  a  factor.)  And  just  trying  to  work  it  all  out  will  give 
you  some  idea  if  the  salesman  knows  anything  .  .  . 


D.  Intensified  Charge-Coupled  Devices 

Intensified  CCDs  (ICCDs)  are  just  that:  the  mating  of  an  “image 
intensifier”  to  a  CCD.  The  idea  is  that  the  photon  gain  of  the  inten- 
sifier  (can  be  200-2000x)  will  increase  the  signal  from  even  a 


23  If  the  lens  doesn’t  have  a  calibrated  aperture  ring,  you  can  open  the  aperture 
all  the  way  and  reach  the  “threshold”  exposure  level  by  reducing  the  expo¬ 
sure  time  and  adding  ND  filters.  Remember  to  also  correct  for  pixel  area. 
Larger  pixels  intercept  more  photons. 

24  With  a  good  EM-CCD,  this  measurement  can  be  done  using  a  short 
exposure  and  high  EM  gain,  then  counting  the  number  of  amplified  dark 
charge/CIC  spikes  across  a  typical  line  of  the  raster. 
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single  photoelectron  above  the  read  noise  of  the  CCD.  This  occurs, 
and  can  be  particularly  useful  where  fast  readout  is  needed  such 
as  when  measuring  ion  transients.  Finally,  pulsing  the  voltage  on 
the  intensifier  section  makes  it  possible  to  shutter  (“gate”)  the 
camera  on  the  ns  time  scale,  making  the  ICCD  useful  for  making 
fluorescence  lifetime  measurements  (Chapter  27,  this  volume ). 

However,  ICCDs  do  not  have  the  photometric  accuracy  of 
normal  CCDs  for  a  number  of  reasons: 

•  The  relationship  between  number  stored  in  memory  and 
the  number  of  photons  detected  is  generally  unknown  and 
variable. 

•  The  intensifier  photocathode  has  low  QE25  (compared  to  that 
of  a  back-illuminated  CCD). 

•  The  “resolution”  is  generally  only  dimly  related  to  CCD  array 
size  because  of  blooming  in  the  intensifier.  To  check  this, 
reduce  light  intensity  until  you  can  see  the  individual  flashes 
produced  by  single  photoelectrons.  See  how  many  lines  wide 
they  are.  (They  should  be  one  line  wide.) 

•  They  have  additional  noise  sources:  phosphor  noise,  ions  in 
intensifier  section  create  flashes,  high  multiplicative  noise  in 
the  intensifier  section  greatly  decreases  QEeff,  etc. 


25  And  the  GaAsP  photocathode  with  better  QE,  have  to  be  cooled,  making  the 
assembly  very  expensive. 


•  Photocathode  resistivity  can  produce  “dose-rate”  effects:  non- 
linearities  in  which  the  recorded  intensity  of  the  brightest  areas 
may  depend  on  (and  affect)  the  brightness  of  nearby  features. 

Because  I  expect  that  EM-CCDs  such  as  those  mentioned 
above  will  soon  supplant  ICCDs  except  where  fast  gating  is 
needed,  I  have  not  gone  into  more  detail  here.  For  more  info,  go 
to:  http://www.stanfordphotonics.com/ 
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2D  imaging,  blind  deconvolution  approach, 
476-477. 

2D-time  vs.  3D-time,  embryo,  762-764. 

2D  pixel  display  space,  291. 

2DCHO  dataset,  818. 

2DHeLa  dataset,  818. 

2-photon,  (2PE).  See  Two-photon  excitation. 

3D  Constructor,  282. 

3D  imaging,  alternative  approaches, 
475-476,  607-624.  See  also. 
Confocal  topics;  Multidimensional 
microscopy  topics, 
episcopic  fluorescence  image  capture 
(EFTC),  607-608 

light  sheet  microscopy  (SPIM),  613 
magnetic  resonance  microscopy  (MRM), 
618-624 

amplitude  modulation  of  RF  carrier, 
620 

applications,  623-624 
basic  principles,  618-619 
botanical  imaging,  624 
developmental  biology,  624 
Fourier  transform,  image  formation, 

620 

future  developments,  624 
hardware  configuration,  621,  622 
histology,  623,  624 
image  contrast,  622-623 
image  formation,  619-621 
Larmor  frequency,  620 
phenotyping,  623 
schematic,  619 
strengths/limitations,  622 
micro-computerized  tomography 
(Micro-CT),  614-618 
contrast/dose,  614-615 
CT  scanning  systems,  615-618 


dose  i's.  resolution,  616 
layout,  614 

living  mouse,  615,  617 
mouse  femur,  616 
operating  principle,  614 
tumor-bearing  mouse  image,  617 
optical  coherence  tomography  (OCT), 

609- 610 

human  retina  609 
schematic,  610 
Xenopus  laevis  embryo,  610 
objectives  on  a  tandem  scanner,  154, 

304 

optical  projection  tomography  (OPT), 

610- 613 

lamprey  larva,  612 
mouse  embryo,  612 
plants,  774-775 
setup,  611 

real-time  stereo  imaging  using  LLCD 
related  methods,  607-625 
selective  plane  illumination  microscopy 
(SPIM),  613 
Medaka  heart,  614 
surface  imaging  microscopy  (SIM), 
607-608 

3D  Scanning  Light  Macrography,  672. 

3D  for  LSM,  282. 

3D  methods  compared,  448-451,  644-647. 
table,  647 

3D  multi-channel  time-lapse  imaging 
(4D/5D).  See  also,  Time-lapse 
imaging, 
table,  384. 

3D3T3  high-content  screening  dataset,  820, 

821. 

3DHeLa  high-content  screening  dataset, 
820,  821. 

3PE.  See  Three-photon  excitation. 


4D  imaging.  See  Four  dimensional  imaging. 

4Pi  microscopy,  561-570. 

4Pi-PSF,  570 
axial  resolution,  563 
I5M,  561,  569-570 
OTF,  569-570 

living  mammalian  cell  imaging,  564—565 
Golgi  apparatus,  image,  566 
lobe-suppression  techniques,  561 

interference  of  excitation  and  detection, 
561 

confocal  detection,  561 
two-photon  excitation  (2PE),  561 
MMM-4Pi  microscopy,  554,  556, 

563-564 
basics,  565 
scheme,  563 

optical  transfer  function  (OTF),  562,  563 
outlook,  568-569 

point  spread  function  (PSF),  562-563 
signal-to-noise  ratio,  561 
space  invariance  of  PSF,  457,  490,  564 
theoretical  background,  562-563 
type  C,  with  Leica  TCS,  4Pi,  565-568 
imaging  of  living  cells,  568 
lateral  scanning,  567 
mitochondrial  network,  image,  568 
optical  transfer  function  (OTF),  567 
resolution,  567 
sketch,  566 

thermal  fluctuations  minimized,  567 
z-response,  563 

5D  image  space,  display,  291-294. 

2D  pixel  display  space,  291 
animations,  292-293 
color  display  space,  291 
efficient  use,  292 

image/view  display  options  overview, 
table,  293 
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5D  image  space,  display  ( cont .) 
multiple  channel  color  display,  292 
optimal  use,  293-294 
pseudo-color,  173-175,  190,  291 
stereoscopic  display,  293 
true  color,  291 

A 

Abbe,  Ernst,  1,5. 

Abbe  refractometer,  377. 

Abbe  resolution  criterion,  36,  37,  60,  61, 

65-68,  574,  575,  631-636,  928.  See 
also,  Rayleigh  criterion, 
breaking  the  Abbe  limit,  573 
calculation,  65-66 

individual  point  features  separated  by, 

68 

pixel  size,  62,  65,  634-635,  784,  928 

Abbe  sine  condition,  151,  239. 

Abbreviations,  list.  125. 

Aberrations,  109,  146-156,  241,  411-412, 
471,  480-481,  542,  629,  640-641, 
654,  655,  657-659,  747.  See  also, 
Chromatic  aberrations;  Refractive 
index  mismatch;  Spherical 
aberration. 

astigmatism,  145,  151-152,  245-247, 

249,  483, 

axial,  242,  505,  542,  630 
chromatic,  152-156,  160,  177-178,  209, 
242-243,  641,  659 
in  2-photon  disk  scanning,  542,  550, 
554 

of  AODs,  56 

axial  chromatic  registration,  287,  658 
chromatic  registration,  657-658 
of  collector  lenses,  657-658 
intentional,  for  height  measurement, 

224 

magnification  error,  155,  287,  331,  493, 
542,  641,  657,  883,  904 
measurement  of,  243-244,  654,  659 
multi-photon  microscopy,  542 
of  optical  fibers,  504,  507 
signal  loss,  in  confocal,  156,  178,  542, 
641 

standards,  table,  157 

coma,  145,  151-152,  245-246,  249,  483, 
630 

detecting,  241 

monochromatic,  147-152,  542.  See  also, 
Spherical  aberration 
optical,  avoiding  with  thin  disk  lasers, 

109 

of  refractive  systems,  146-156 
signal  loss,  156,  178,  542,  641 
spherical,  15,  34,  147-149,  151,  160,  192, 
208,  241,  244,  247,  330,  395, 
404-413,  454-455,  463,  466,  480, 
542,  629,  640,  654-655,  657,  658, 
728,  772,  774.  See  also.  Spherical 
aberration;  Mismatch,  refractive 
index 


blind  deconvolution  to  remove, 

480^181 

cause  of  signal  loss,  330,  389,  395, 

413,  457,  542,  661 
chapter,  404^-13 
correction  for  refractive  index 

mismatch,  192,  287,  411-412,  542 
corrections  for,  145,  411-412,  654-655 
corrector  optics,  192,  395,  398,  411, 
477,  640,  655,  657 

deconvolution,  463,  466,  468,  469,  471, 
480,  498^199,  784 
generated  by  specimen,  192,  418, 
454^155,  654,  658,  747,  772,  775 
of  GRIN  lens,  108 
for  IR  wavelengths,  160 
measurement  using  small  pinholes, 

145,  407 

monochromatic.  147-151 
multi-photon  excitation,  542,  407-410 
PSF,  148,  407,  455,  471,  481,  492,  657 
secondary,  247,  249 
in  thick  embryo  imaging,  747 
Zernicke  coefficients,  247,  248 
wave-front,  measuring  performance,  145 
Ablation,  2-photon,  107,  764-765. 

Absorber,  saturable-crystal.  107,  111,  112. 
to  cover  gap  in  titaniurmsapphire  lasers, 
112 

indium-gallium  arsenide,  InGaAs,  111 
Absorption,  25,  163,  309-312,  338-339, 

341,  514-518,  542,  550,  613,  704. 
2-photon,  405,  535-536,  541,  545,  550, 
552,  705,  719.  764,  884 
caged  compounds,  543,  544 
CARS,  595-596,  599 
contrast,  162-165,  211,  595,  610,  613, 
770,  779 

cross-section,  189,  426 

energy  levels,  514,  517,  682,  697,  705, 

792 

excited  state,  544,  692 
fiber  optics,  501,  502 
filters,  552 

of  fluorescent  dyes,  table,  345 
fluorescent  excitation,  45,  88 
FRET,  184 

and  heating,  21,  218,  252,  539,  685 
of  incident  light,  163,  177,  427 
by  ink,  73 

and  laser  operation,  82,  108,  110,  116 
light  lost  by,  25,  166,  414-418,  457,  654 
lighting  models,  283,  285,  309-312 
molar  extinction,  80-81,  343,  353,  357, 

793 

nonlinear,  188,  416,  427,  680,  704, 
709-710 

of  optical  materials,  158 
and  photodamage,  22,  685-686,  690,  750 
in  photodetectors,  253 
photon,  550,  749 

quantum  dots,  221,  343,  357-358,  696, 
759,  801 


RESOLFT/STED,  573 
self-absorption,  490 
spectra,  217,  267,  338-339,  345,  355, 
390,  415-416,  421,  538-539, 
681-682,  706 
in  UV,  195 

Absorption  coefficient,  complex  specimen, 
164. 

Absorption  contrast,  164-167,  195,  427. 
equations,  164,  539 
heating,  539,  685 
Accuracy. 

biological  vs.  statistical,  24,  36-37,  68, 
312 

position,  39 — 41 

Acetoxymethyl  ester  indicators,  726. 
deposits  formaldehyde,  738 
derivatization,  738 
formula/reaction,  359,  738 
loading  method  (AM  ester),  358-359, 
361,  726,  738-739,  744 
painting  brain  slices  with,  726-737 
Achrogate  beam-splitter/scan  mirror,  50, 
212,  231-232,  916. 
operation,  50,  232,  916 
Zeiss  LSM5  line  scanner,  212,  231-232, 
916 

Achromat,  152,  153,  244. 
chromatic  correction  of,  153 
flatness  of  field  and  astigmatism,  152 
longitudinal  chromatic  correction,  153 
measurement,  244 

Acousto-optical  beam  splitters  (AOBS),  45, 
55-57,  88,  102,  211,  218,  395. 
to  select  wavelength  and  intensity,  88, 

102 

to  separate  illumination  and  emission,  45, 
218 

Acousto-optical  components,  43,  54-57. 
tellurium  oxide  crystal,  55 
thermal  stability,  56,  57,  219 
Acousto-optical  deflectors  (AOD),  25,  33, 

54- 56,  88,  447,  519,  543,  664,  762, 
908. 

as  beam-splitters,  to  reduce  loss,  33 
to  gate  light  source,  25.  See  also,  AOM 
group  velocity  dispersion  due  to,  88, 
540,  646 

multi-photon  excitation,  88,  540,  543, 

646 

multi-tracking,  664performance,  55 
problem  descanning  fluorescent  light,  56, 
447 

Acousto-optical  modulators  (AOM),  11, 

55- 57,  88,  231,  519,  540,  543. 
FRAP  experiments,  for  controlling  laser, 

56 

group  velocity  dispersion,  88 
Acousto-optical  tunable  filters  (AOTF),  43, 
55-56,  88,  102,  219,  237,  346,  543, 
651,  660,  673,  806,  908. 
for  selecting  CW  laser  lines,  88,  102 
blanking,  54,  55,  237,  389,  543,  628,  651 
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leakage,  660 

to  regulate  light  intensity,  43 
to  spectrally  filter  light,  55 
thermal  sensitivity,  56-57,  219 
Acridine  Orange,  23,  344,  531,  665-667, 
691,  774,  874. 
bleaching,  693-694 
Acronyms,  list,  125. 

Actin  filament,  7,  236,  372,  378,  383,  692, 
696,  714,  719,  748-749,  753,  756, 
759-760,  773,  781,  804,  819, 
824-825,  854,  856. 
widefield  source  suitability,  142 
Active  laser  medium,  defined,  8 1 . 

Active  mode-locked,  pulsed  laser,  111. 
Actual  focal  position  (  AFP)  defined,  405. 
Actuator,  galvanometer,  52. 

Acute  neocortical  slice  protocol,  723. 
Adams,  Ansel,  zone  system,  7 1-72. 
Adaptive  optics,  892. 

ADC.  See  Analog-to-digital  converter. 
Adipocyte  cells,  CARS  imaging,  604. 
Adjacent  fields,  automated  confocal 
imaging,  810. 

ADU,  analog  digital  units,  74-77,  630,  925. 
Advanced  Visual  System.  See  AVS. 
Aequorea  victoria,  biofilms,  348,  356,  736, 
794,  873-874,  877. 
variants,  table,  873,  874 
Aequorin,  Ca2+  reporter, 736-737,  739,  741, 
802. 

developmental  cellular  application,  736 
ion  binding  triggers  light  emission,  737 
Ca^  signal  detection,  737 
AFP.  See  Actual  focal  position. 

AIC.  See  Akaike  Information  Criterion. 

Airy  aperture,  optimum  for  NA,  28. 

Airy  disks,  4,  24,  65.  131,  145-146,  151, 
156,  210,  443,  444-449,  454-456, 
463-465,  474,  485,  492-493,  562, 
567,  630,  655-657. 

Abbe  criterion  resolution,  65-66,  225 
defined,  146,  444 
diameter  in  image  plane,  210,  225 
four-lobed,  from  astigmatism,  151 
image,  38,  146,  225 
intensity  ratios,  28,  145-146 
inverse,  11 
and  line  spacing,  24 
radius  and  pixel  size,  4,  24,  38,  39,  60, 
65-67,  227,  485 

Vi.  NA  and  wavelength,  1,  4,  146 
Airy  figure  image,  38,  75,  79,  146,  147, 
225,  479.  486-487,  562. 

FWHM  as  optimal  pinhole/slit  size,  28, 
36,  225,  232,  443,  454,  463-465, 
564,  567-568,  630-631,  633, 
655-657 

and  resolution,  65-67 
size,  and  Nyquist  criterion,  38,  39,  60 
Airy  unit,  28,  36,  41,  210,  222,  227,  232, 
274,  443-451,  632,  775,  779. 

Akaike  Information  Criterion  (AIC),  825. 


AlexaFluor  dyes,  81,  103,  184-185,  190, 
192,  236,  330,  342-344,  353-357, 
360,  363,  393,  395,  416,  533,  540, 
694,  726,  731,  749,  794,  799,  804, 
810,  814,  854,  878,  880,  905. 
fluorescence  excitation,  355 
living  cells  rapid  assessment,  table, 

360 

structure,  356 

Alexandrite  (Cr3+  in  BeAl204),  tunable  laser, 
109. 

Alga. 

autofluorescence,  357 
autofluorescent  image,  173,  175,  192, 
194-195,  438-439,  528,  585,  785, 
870,  881-885 
biofilm,  870,  881-885 
cell  chamber  for,  429 
in  laser  cooling  water,  116 

Aliasing,  38-39,  271,  291,  293,  448,  588, 
590-592,  640,  830,  833-834, 
836-839,  903. 

and  Nyquist  criterion,  38-39,  448 
temporal,  39,  41,  391,  836-837,  839 

Alignment,  25,  85,  134-135,  157,  505, 

629-631,  651. 

of  laser  systems,  to  reduce  instability, 

85 

of  optical  coherence  tomography,  610 
of  optical  system,  thermal  stress,  85 
importance,  25,  630 
and  PSF,  646 

of  source,  134-135,  629-631 

Alkali  vapor  lasers,  diode-pumped, 

103-105. 

Allium  cepa.  See  also.  Onion  epithelium. 

Alpha  blending,  302,  304. 

Alumina  (A1203)  ceramic  tubes  for  lasers, 

102. 

Amira,  282-283,  286,  296,  302,  308,  312, 
775-778. 

Amoeba  pseudopod,  detail,  168. 

Amplifier  rods,  maintenance,  116. 

Analog  digitization,  for  photon  counting,  29, 
33-37,  41,  65,  74,  78,  251,  254, 
258-261,  263-264,  404.  460,  495, 
522,  525-526,  542,  634,  766. 

Analog-digital  unit  (ADU),  to  calibrate 
CCDs,  74,  77,  630,  925. 

Analog-to-digital  converter  (ADC).  31-34, 
64-66,  70,  72,  74-75,  258-259,  261, 
263,  286,  521,  630-632,  924-925. 

Analyze  (software),  281-282,  288,  290, 
301-304,  312,  651. 

Analyzer,  in  pol-microscopy,  25,  157,  229. 

Analyzer,  spectrum,  901-902. 

Anemonea  majano,  sulcata,  874. 

Angular  deflection,  distortion,  211. 

Aniline  Blue  stain,  430-432,  435,  438,  774. 

Animations,  281,  283-285,  289-290, 

292-293,  295,  299,  308,  312,  764, 
829,  835-839,  841-844. 

Anisotropic  crystals,  114. 


Anisotropic  sampling,  287-288. 
when  resampling,  833-835 

Anisotropic  specimens,  163,  286,  320,  329, 
420,  623,  675,  678,  690,  710,  793. 

Anisotropy  analysis,  chimeric  proteins,  794. 

Anisotropy  of  fluorescence,  742,  794. 

Anisotropy  of  interference  filters,  49. 

Annular  aperture,  4,  9,  20,  211,  889. 

3D  pattern  of  point-source  from  lens, 
4-20 

in  specimen-scanning  confocal 
microscope,  9 

Anti-bleaching  agents,  36,  340,  363,  368, 
375,  499,  694. 

Antibody  stains,  292,  339,  342-343,  348, 
357-360,  375,  528,  576-578,  582, 
610,  612,  664,  696,  731,  748,  760, 
789,  802-804,  812,  852-855, 
877-880. 
artifacts,  664 
biofilms,  877-880 
FRET,  790-791 

high-content  screening,  812-815,  818 
in  situ,  612 
penetration,  387 

preparation,  369,  371-372,  375-377,  878 
and  TEM,  852-855 

Antifade  agent,  36,  340,  363,  368,  375,  499, 
694.  See  also,  Antioxidants. 

Antiflex  optics,  to  reduce  reflections,  158, 
171,  507,  513. 

Antioxidants,  living  cell  imaging,  341-342, 
363,  389,  390,  729,  794. 

Anti-reflection  (AR)  coatings,  1,  8-9,  25, 

49,  117,  139,  145,  151,  158-159, 
212,  505-506,  901. 
color  effect,  139 
of  optical  fibers,  506 

AOBS.  See  Acousto-optic  beam-splitter. 

AOTF.  See  Acousto-optic  tuning  filter. 

APD.  See  Avalanche  photodiode, 

Apochromat,  15,  147-148,  151,  153-155, 
158,  240-245,  409^110,  454^155, 
655,  659,  771. 
chromatic  correction,  153 
compared  with  fluorite  objective,  154 
longitudinal  chromatic  correction,  153 

Apodization.  high-NA  objective  lenses,  240, 
243,  249-250,  272,  567,  889. 

Applied  Precision  Instruments  (API),  131, 
137,  282,  388,  651. 

APSS  up-converting  dye,  saturation,  165. 

AR.  See  Anti-reflection. 

A.  thaliana,  169,  173,  174,  175,  193,  196, 
202,  416,  420-421,  423,  425,  426, 
427,  431,  771,  772,  773,  775,  778, 
779,  780. 

attenuation  spectra,  416 
birefringent  structures  in  cells,  420-421. 
See  also,  Anisotropic  specimens 
bleaching,  203 
double  imaging,  169 
fluorescence  spectra,  421,  423,  425 
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A.  thaliana  ( cont .) 

GFP  protein  fusion,  773 
limitations  for  imaging,  772 
mesophyll  protoplasts,  196,  426 
optical  sectioning,  772,  775 
protoplasts,  195-196,  203,  416,  421, 
425-427,  429-430  438^139,  693 
root  tip  fluorescence  spectra,  173-175 
seedling,  autofluorescent  image,  202 
three-dimensional  reconstruction,  190, 

193,  771,  775,  777-778,  781 
two-channel  confocal  images,  169,  175, 
193,  196,  203,  427,  431,  772 
two-photon  excitation,  advantages,  779, 
780 

two-photon  fluorescence  image,  427,  780 
two-photon  fluorescence  spectra,  425,  426 
Arc  lamps,  132,  136-138. 

current/stability  of  plasma,  138-139 
monitoring  during  exposure,  137 
radiance,  137-138 

sensitivity  to  environmental  variation,  136 
shape  of  discharge,  132 
shift  of  wavelength  with  temperature,  137 
stability  of,  t's.  filament  lamps,  137 
Area  of  interest.  See  also,  Region  of 
interest. 

identifying,  201-202 
Argon-ion  laser,  85-86,  90-102,  107, 

109-110,  112,  119,  124,  203,  338, 
341,  346,  353,  355,  375,  540-541, 
655,  657. 

CW,  90-103,  107,  109-110,  112,  119, 

124 

emission  stability,  86,  102 
references,  124 

Argon-krypton  mixed-gas  laser,  90,  92,  93, 
102,  108,  119,  203,  343,  375,  748, 
798,  811. 

Artificial  contrast,  vibration  and  ambient 
light,  201-204. 

Artificial  lighting,  image  display,  306-312 
Astigmatism,  145,  151-152,  245,  247,  249, 
483,  505,  542,  630. 
of  AOD,  914 
and  flatness  of  field,  152 
and  intensity  distribution,  152,  246,  630 
laser  optics,  89,  106-107,  505 
measuring  subresolution  pinholes,  145 
at  off-axis  points,  151,  245,  247,  249 
ATP-binding  cassette,  362. 

ATP-buffer,  802-803,  812. 

ATP-caged,  544. 

ATP-gated  cation  channels,  359. 

Attenuation  of  light. 

by  specimen,  164,  287,  298,  304,  320, 
321,  414-418,  428,  439,  538,  558, 
706,  779,  782 
plots.  415,  706 

of  laser  beams,  85,  87,  354,  415,  904 
modeling,  309,  311,  320-321,  330 
of  PSF,  456,  462^163,  466,  494 
x-ray,  614-615 


Atto  Bioscience  CARV  confocal 

microscope,  215,  229,  230,  907. 

Autofluorescence,  44,  81,  90,  173,  175,  195, 
202,  339-340,  360-361,  369-370, 
387,  414,  416,  421-434,  442-445, 
447-449,  451,  509-510,  528,  530, 
545,  607,  612,  663,  667-670,  678, 
682,  690,  698,  706,  710-711,  713, 
729,  742-743,  745,  764-765, 
769-773,  779,  781-782,  785,  798, 
815,  874,  876,  881-885. 
of  alga  chloroplast,  168,  172-176,  202, 
429^135,  556,  785 
A.  thaliana  seedling,  202,  303,  307, 

772 

bleaching,  202,  698,  729 
cell  wall,  303,  431,  438,  770 
emission  spectra  in  plants,  176,  421-423 
extracellular  matrix,  311 
fixation,  as  a  cause,  358,  369-370 
fluorescent  probes,  339-340,  360-361 
harmonic  signals.  See  Harmonic  signals 
lamprey  larvae,  612 
multi-photon  microscopy  (MPM)  See 
also,  harmonic  signals,  545 
optical  materials,  45,  158 
plants,  190,  193-195,  421^428,  770-772 
plots,  176,  421-423 
removal  using  spectral  unmixing,  192, 
382,  664-667 
examples,  665-666 

removal  on  basis  of  fluorescence  lifetime, 
345-346,  348,  349,  528 
UV  excitation,  347 

Automated  3D  image  analysis  methods, 
316-335.  See  also,  Automated 
interpretation  of  subcellular 
patterns. 

biological  objects,  319 
blob  segmentation  example,  322-324 
gradient-weighted  distance  transform, 
323 

model-based  object  merging,  323-325 
watershed  algorithm,  322-325,  777, 

822 

combined  blob/tube  segmentation, 
328-330 

data  collection  guidelines,  319-320 
defined,  316,  328 
future  directions,  334 
hypothesis  testing,  318 
illustrations,  317 
image  preprocessing,  320-321 
background  subtraction,  320 
morphological  filters,  320 
signal  attenuation-correction,  320-321 
v.s'.  manual,  316-317 
montage  synthesis,  282,  293,  312, 

328-332,  748,  753,  851-852,  855, 
858-859 

defined,  329-330 
examples,  330-332,  780-781 
neuron,  330 


scanning  electron  micrographs, 
851-852,  855 

TEM  implementation,  858-859 
neurobiology  example,  320 
quantitative  morphometry,  331 
rationale,  316 

registration  synthesis,  328-331 
defined,  328 

landmark-based,  328-329 
multi- view  deconvolution,  291,  330, 
675-677 

segmentation  methods,  321-322 
bottom-up,  321 

hybrid,  bottom-up/top-down,  322 
integrated,  322 
intensity  threshold-based,  321 
region-based,  321-322 
top-down,  322 

segmentation  testing  methods,  333-334 
manual  editing,  333-334 
specimen  preparation,  319-321 
imaging  artifacts,  320 
stereology,  316 

time  series  in  vivo  images,  319 
tube-like  object  segmentation  example, 
324-328 

mean/median  template  response,  328 
skeletonization  methods,  324-325 
vectorization  methods,  324,  326, 

327 

types,  318-319 

Automated  fluorescence  imaging,  814. 

endpoint  translocation  assays,  814 
Automated  interpretation  of  subcellular 
patterns,  818-828.  See  also, 
Automated  3D  image  analysis 
methods  2D  dataset  analysis, 
automated  2D  analysis  methods,  818 
2D  subcellular  location  features, 
819-820 

2DHeLa  dataset  images,  819 
CHO  cell  dataset,  818,  table,  820 
Haralick  features,  818-820 
HeLa  cells  2DHeLa  dataset,  818 
Zemike  moments,  818-820 
automated  3D  analysis  methods,  824 
classification  results,  824 
feature  normalization,  824 
feature  selection,  824 
automated  classification  of  location 
patterns,  824-825 
classification  accuracy,  826 
confusion  matrix  for  3DHeLa  images 
using  SLF10,  table,  824 
confusion  matrix  for  3DHeLa  images 
using  SLF17,  table,  825 
features  in  SLF17,  table,  825 
measured  classification  accuracy,  table, 
825 

clustering  of  location  patterns  with 
clustering  consistency,  table,  826 
exclusion  of  outliers,  825 
methods,  826 
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optimal  clustering  determination, 

825- 826 

optimal  consensus  tree,  827 
clustering  of  location  patterns,  825-826 
downsampled  images,  different  gray 
scales,  824-825 
future  directions,  827-828 
high-resolution  3D  datasets,  820-822 
3D3T3,  820 
3DHeLa,  820 

color  images  from  3DHeLa,  821 
image  acquisition  requirements, 
821-822 

images  from  3D3T3,  821 
image  database  systems,  827 
image  processing/analysis,  822-823 
3D  SLF,  822-823 
edge  features,  823 
feature  calculation  process,  822 
morphological  features,  823 
segmentation  of  multi-cell  images,  822 
texture  features,  823 
protein  subcellular  location,  818 
statistical  comparison  of  patterns, 

826- 827 

AutoMontage  software,  282,  293,  304. 
Avalanche  photodiode  (APD),  77,  233, 

252-255,  404,  527,  542,  558,  567, 
698. 

array,  for  multi-beam  sensing,  558 
noise  currents,  256 
pulse  pileup,  253,  527 
unsuitability  for  non-descanned  detection, 
542 

vacuum  ADP,  254-255 
Average  intensity,  66,  110.  516,  556,  668, 
684,  695,  747,  763-764,  816,  838, 
930. 

equation,  302,  309,  668 
AVS  (Advanced  Visual  System),  282-283, 
286,  300,  311-311,  862,  863. 

Axial  chromatic  aberration.  155,  658-659. 
Axial  chromatic  registration,  154,  658. 

Axial  contrast.  See  z-contrast. 

Axial  edge  response,  409^-10,  654. 
calculations  for  glycerol,  table,  409 
calculations  for  water,  table,  409 
Axial  illumination,  60-61,  134. 

Axial  laser  modes,  82,  110. 

Axial  minimum,  3D  diffraction  pattern.  4, 
147. 

Axial  rays,  spherical  aberration,  148. 

Axial  resolution,  3-4,  6,  172,  182,  209,  211, 
225-228,  230,  240-241,  243-244, 
320,  370,  395,  407-411,  413, 
444^146,  489,  493,  499,  511,  513, 
551-553,  559,  561-568,  571-577, 
610-611,  613,  649,  651,  654, 
656-657,  659,  674,  704,  747, 
750-751,  822. 

4Pi  microscopy,  561-568 
coding,  display,  305 
defined,  3-4,  240,  444-446 


focus  shift,  243,  407-410 
as  function  of  pinhole  diameter,  656 
magnification,  215 
measurement,  194,  656-657,  659 
multi-photon,  750 
multi  view,  678 

near  focal  plane,  slit-/point-scan  confocal 
microscopes,  225-228 
SHG,  704 

SPIM,  614,  674,  751 
STED,  571-577 

tandem- scanning  confocal  microscope,  6, 
225 

tomography,  610-611 
using  mitTor,  656-657 


Back-focal  plane  (BFP),  34,  50-51,  58, 

61-62,  84,  126-128,  166,  208-210, 
225,  239,  268,  487,  509,  627,  629, 
708. 

Background  light,  from  transmission 
illuminator,  201-202. 

Background  noise,  260-262,  275. 

Background  signal,  12,  26,  28,  37,  68-69, 
71-72,  88,  90,  112,  115,  158,  162, 
168,  172-173,  175,  184,  188, 
201-202,  221-225,  227,  232,  235, 
248,  251,  257,  266-275,  278-279, 
283,  287,-288,  290,  301-302,  305, 
312,  321,  326,  339-340,  343,  345, 
348,  360-362,  375,  421,  423, 
428^129,  432-^133,  442^151,  462, 
465,  All-All,  486,  493,  497,  506, 
509-510,  518-519,  535,  541,  543, 
553,  559,  582,  584-585,  595, 
598-600,  602,  604,  621,  633,  656, 
663-370,  676,  694,  697,  698,  707, 
713,  727,  733-734,  736,  747, 
755-757,  760,  798,  801,  803,  809, 
813,  815,  818,  822,  830,  836,  839, 
851. 

Background  subtraction,  284,  301,  320,  473, 
510. 

Back-illuminated  CCD,  31,  77,  222,  232, 
234,  754. 

Back-propagation  neural  network  (BPNN), 
818. 

Backscattered  light  (BSL),  22-23,  57, 

83-84,  130,  141,  145,  165,  169-170, 
180-182,  191,  196,  202,  212,  221, 

228,  240,  376,  378,  416,  430,  436, 
442,  631,  879. 

access  to,  antiflex  optics,  6,  57,  141,  212, 

229,  507,  513,  609,  631,  704,  707, 
854,  879,  990 

biofilm,  image,  880 
contrast,  effect  of  specimen  absorption, 
165 

effect  of  coherence  on,  130-131,  170 
images  made  using,  22-23,  154,  436-438, 
513,  638,  855,  880 
Amoeba  pseudopod,  168-170,  191 


cheek  cells,  22,  23 
diatom,  145,  438,  638-640,  881 
latex  bead,  182,  196,  197,  653 
transparent  ciliate  protozoa,  141 
LLLCD  objectives/3D  color-coded  BSL 
as  a  noise  signal,  663 
optical  coherence  tomography,  609 
practical  confocal  microscopy,  631 
front  specimen,  202 
unmixing,  192,  382,  664-667 
Back-thinned  CCD,  31,  77,  222,  232,  234, 
754. 

QE  plot,  29 
Bacteria.  See  Biofilms. 

Ballistic  microprojectile  delivery,  360,  726, 
803. 

Ballistic  photons,  418,  427,  538. 

Ballistic  scans,  40,  41. 

Balloon  model  segmentation  methods,  776. 
Bandpass,  optical  filters,  43-44,  46,  48,  49, 
51,  76,  87,  132.  141,  173,  204,  341, 
528,  708,  798. 
for  CARS,  598-599 
coupling  short  and  long-pass  filters,  46 
excitation  and  emission,  48,  141,  217, 

341,  708,  757,  798 
laser,  106-107 
liquid  crystal,  425 

to  select  range  of  wavelengths,  43-44 
spectral  detector,  203-204,  662-663, 
666-667 

Bandwidth,  32,  64,  69. 

3  dB  point,  definition,  59,  65 
of  AOBS,  57 

electronic/optical,  digitization,  32,  34,  70, 
238 

head  amplifier,  251 

limiting,  to  improve  reconstruction,  69 
Nyquist  reconstruction,  output,  64,  69,  70, 
238 

Bead,  fluorescence  emission,  181,  182,  196. 
fluorescent,  454.  477,  493,  499,  527,  652, 
653,  656,  659,  784,  900,  904,  930 
image,  656 
table,  653 

glass,  in  water,  181,  198-199 
latex,  fluorescence  image,  196,  407, 
455-457,  463,  471,  656 
in  water,  confocal  serial  sections,  182 
Beam  blanking,  54,  55,  237,  389,  543,  628, 
651. 

Beam  collimation,  728. 

for  fiber  delivery,  506 
Beam  delivery,  with  fiber  optic  coupling, 
85-88,  107,  216,  503,  506-508. 
Beam  deviation,  unintentional,  15-16. 

Beam  expander,  8,  84,  124,  208,  212-214, 
231,  650,  682,  708,  728,  907. 
advantages,  213 

Beam  pointing,  lasers,  85,  103,  107,  201, 
250. 

active  cavity  stabilization,  87 
Beam  quality,  of  diode  lasers,  107. 
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Beam  shift,  vignetting  due  to,  211. 

Beam-splitter,  33,  46-48,  50-51.  See 
Dichroic  mirrors. 

Achrogate,  50,  212,  231-232,  916 
AOBS,  56-57 
broadband,  346 

dichroic,  25,  33,  35,  43-51,  56-57, 

83-84,  88,  139,  132,  135,  143,  151, 
162,  203-204,  207-208,  211-214. 
217-218,  229,  231-232,  266,  339, 
341,  346,  375,  386,  424,  469, 
503-504,  552,  563-564,  599, 
630-632,  647,  650,  657-658,  664, 
667,  691,  707-708,  747,  771-772, 
810,  846,  879,  910,  907 
table,  799 

fiber-optic,  503-504 
forty-five  degree,  performance,  47 
fused-biconic  coupler,  503-504 
long-pass  cut-off,  43,  46,  51,  175,  204. 

564,  801,  875 
multi-photon,  540-541 
polarizing,  13,  50,  57,  85,  87,  100.  217, 
513,  631 

spectral  problems,  50-51 
triple  dichroic,  33,  46.  48,  217-218,  658, 
783 

losses  due  to,  33 
performance,  46-48 

Beam  scanning,  along  optical  axis,  215, 

555. 

Beam-scanning  confocal  microscope.  See 
Confocal  entries;  Flying  spot 
ultraviolet  (UV)  microscope, 
chromatic  correction,  177 

Beam-scanning  systems,  6,  7,  16,  132,  146. 
151,  156,  166,  177,  214-215,  218, 
381,  554,  562,  564,  567,  568,  599. 
coma  in,  151 

off-axis  aberrations  affecting,  156 

Before-bleach/after-bleach  ratio,  FRET,  794. 

Benchtop  fiber-optic  scanning  confocal 
microscopes,  507-508. 

Bertrand  lens,  61,  157,  412,  643. 

Beryllium  oxide  (BeO),  for  laser  tubes,  102. 

Beta  barium  borate  (BBO),  non-linear 

crystal  for  frequency  doubling,  100. 
109,  114-115,  125. 

BFR  See  Back-focal  plane. 

Bibliography,  annotated,  889-899. 
adaptive  optics,  892 
books  on  3D  light  microscopy,  889 
differential  phase  contrast,  892 
display  methods,  892-883 
fiber-optic  confocal  microscopes,  883 
general  interests,  891 
historical  interests,  889-890 
index  mismatch,  893-894 
multiplex,  894 
non-linear,  894 

point  spread  function,  895-896 
polarization,  894-895 
profilometry,  895 


pupil  engineering,  896 
review  articles,  889 
technical  interests,  891-892 
theory,  890-891 
thickness,  896 
turbidity,  896-897 
variants  on  main  theme,  897-899 
Binding  equation,  for  fluorescent  indicators, 
740. 

Biocytin,  730,  731. 

EM  imaging  of  brain  cells  labeled,  731 
protocol,  730 

Biofilms,  287,  688,  529,  530,  624,  779, 

870-887. 

2-photon  imaging,  530,  882-885 
dual-channel  imaging,  884 
limitations  of  CLSM  and  2-photon,  884 
single-photon/2  photon  comparison, 

883 

thick  environmental  biofilms  image, 

885 

autofluorescence,  545 
backscattered  light,  880 
fluorescent  proteins  for.  table,  874 
future  directions,  887 
GFP  variants  for,  table,  873 
imaging  extracellular  polymeric 
substances  (EPS),  879-882 
lectin-binding  analysis,  figures,  881, 

882 

lifetime  imaging,  530 
magnetic  resonance  microscopy,  624 
making  bacteria  fluorescent,  873-874 
pH  imaging,  530,  739-745 
sample  mounting,  870-873 

flow  chamber  system  setup,  872-873 
perfusion  chambers,  870-872 
pump  selection,  871 
upright  vx.  inverted  microscopes,  870, 
872 

water-immersible  lenses  149.  161,  209, 
411,  429,  568,  613,  727,  737,  870, 
872. 

stains  for,  874-879,  875 
Acridine  Orange,  23,  344,  531, 
665-667,  691,  774.  874 
antibodies,  877-878 
biofilm  community  on  tooth,  879 
DAPI.  874.  See  also,  DAPI 
effect  of  antibiotic  treatment,  877 
embedding  for  FISH,  876-877 
FISH  with  fluorescent  protein. 

875-876,  878 

imaging  bacteria,  backscattered  light, 
879 

live/dead  stain.  Streptococcus  gordonii, 
876 

nucleic  acid,  874—875 
preparing  labeled  primary  antibodies, 
878 

SYTO,  874-875 
temporal  experiments,  885-886 

multi-cellular  biofilm  structures,  886 


time-lapse  confocal  imaging,  885-886 
transmitted  laser  light  image,  880 

Bioimagers,  kinetics,  endpoint  analysis, 
816-817. 

Biolistic  transfection,  360.  724-726,  803. 

Biological  accuracy,  I's.  statistical  accuracy, 
24,  36-37,  68,  73,  312. 

Biological  reliability,  of  measurements,  24, 
36-37,  68,  73,  312. 

Biological  specimens,  6,  11,  12-13.  See 
also,  Plant  cell  imaging.  Biofilms, 
Specimen  preparation,  and  entries 
under  specific  equipment  and 
cell/tissue  type. 

backscattered  light  images,  22-23,  25, 
167-168,  170,  880 
CARS  imaging,  603-604 
adipocyte  cells,  604 
epithelial  cells,  603 
erythrocyte  ghosts,  603 
distortions  caused  refractive  index 
inhomogeneity,  40-41,  181,  182, 
198-199,  419 

tandem  scanning  systems  for,  6,  1 1 
Yokogawa  CU-10,  12-13 

Biophotonic  crystals.  188,  428. 

Bio-Rad,  25,  33,  35-36,  70,  113,  214,  260, 
630,  638-640,  657,  748-752,  757, 
759-762,  858,  889. 

1024ES.  710-711,  714,  718-719 
data  storage,  585 
using  white  light  source,  113 
MRC  1024,  photon  counting,  33 
photon  efficiency,  25,  32,  261, 
748-752 

MRC-600  scanner,  full-integration 
digitizer,  70 
PMT,  260-261 
Radiance-2100,  23,  185 
resolution,  657 

Biosensors,  fluorescent,  33-8348,  799,  805. 
See  also.  Dyes,  Fluorophores,  and 
Chapters  16  and  17. 
future,  805 

mitotic  clock  measurements,  799 

Birefringence,  6,  15,  54,  83,  103,  109,  113, 
116,  162-164,  188,  189,  414, 
420-421,  431,  434,  436.  438,  479, 
503,  710-711.  714,  717,  894. 
acousto-optics,  54,  55 
collagen  fibers,  164,  188,  717 
contrast.  15,  162-164.  188,  414-428, 
431-438,  710-711.  714,  717,  719, 
894 

deconvolution,  479-480 
defined,  163,  188 
in  fiber-optics,  503 

harmonic  generation  from,  428,  431-438 
images  of  Cymbopetalum  baillonii,  189 
in  laser  components,  85,  103,  109,  113, 
116 

quarter-waveplate,  6 
table.  715 
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Birefringent  crystals,  188,  420-421. 
optical  effects  of  acoustic  fields  on,  54, 

55 

Black-body  radiation,  44,  135-136. 
from  incandescent  lamps,  44,  126, 
135-136 
spectrum,  136 

Bleaching,  10,  12-13,  20,  24,  44,  63-64,  90, 
142,  186-187,  194,  202-203,  210, 
218,  220,  222,  340,  382-387,  442, 
539-540,  690-702,  797,  905,  907. 
2-photon  excitation,  539-540,  680-689, 
905 

acceleration,  341 
of  acceptor  in  FRET,  184-187 
anti-bleaching  agents,  36.  See  also,  Anti¬ 
bleaching  agents 

bleach  patterns,  3D,  538,  628,  693 
beam  blanking,  to  reduce,  53-54 
before/after  ratio,  for  donor/acceptor  pair, 
794 

chapter,  690-702 

combining  fluorescence  with  other, 
383-386 

in  dye  lasers,  103 

dynamics,  202-203 

fluorescence  correlation  spectroscopy, 

383,  801 

fluorescence  lifetime,  382-383 
fluorescence  recovery  after 

photobleaching,  51,  54.  56,  80,  90, 
187,  210,  218,  224,  229,  237,  362, 
382,  684,  390,  691,  759,  801,  805, 
850 

FRET,  186,  382,  794-798,  800 
fluorescence  speckle  microscopy,  383 
in  four-dimensional  imaging,  222 
improvement,  recent,  36 
laser  trapping,  383 
linear  unmixing,  192,  382,  664—667 
of  living  cells,  212,  220,  382,  797.  See 
also,  FRAP,  FLIP 
intensity  dependence,  341,  363 
mechanism,  222-223 
of  non-specific  fluorescence,  27,  44,  74 
optical  tweezers,  383,  385 
performance  limitations,  221,  224,  232, 
381,  448-450,  556,  693.  See  Chapter 
39 

photoactivation,  187,  224,  383,  385,  541, 
544-545,  693,  759 

photo-uncaging,  383.  See  also ,  Photo- 
uncaging  and  signal  per  pixel, 

63-64 

spectral  unmixing,  192,  382,  664-667 
table,  384-385 
techniques,  125 

temperature  as  a  variable,  696-698 
time-lapse  fluorescence,  382 

Bleedthrough  fluorescence,  185,  203,  664, 
904. 

multi-tracking,  reduces  bleed-through, 

664 


Blind  deconvolution,  190,  468-487.  See 
also.  Deconvolution. 

2D  approach,  476^-77 
3D  approach,  475^-76 
advantages/limitations,  468^172 
algorithms,  472^-74 
of  A.  thaliana  seedling  image,  190 
confocal  stack,  470 
data  collection  model,  472 
data  corrections.  All 
defined,  469 
DIC  schematic,  475 
DIC  stack  example,  470 
different  approaches,  475-477 
deblurring  algorithm,  476 
Gold's  ratio  method,  476 
inverse  filter  algorithm,  476 
iterative  constrained  algorithms, 
475^176 

Jansson-van  Cittert  algorithm,  476 
nearest-neighbor  algorithm,  476 
no-neighbor  algorithm,  476-477 
processing  times/memory  table,  476 
Richardson-Lucy,  497,  568 
TIRF  microscopy.  All 
differential  interference  contrast  (DIC), 
473-475 

examples,  469,  470,  481,  482,  483 
flowcharts,  473,  474 
future  directions,  483 
Gerchberg-Saxton  approach,  472 
hourglass  widefield  PSF,  474 
light  source/optics  alignment,  478 
maximum  likelihood  estimation  (MLE), 
All-All,  669-670 
new  developments,  478-480 
live  imaging,  480 
polarized  light  microscopy,  479 
subpixel  imaging,  478-479 
optical  sectioning  schematic,  469 
OTF  frequency  band,  474 
simulated  example,  481,  482 
speed,  482-483 

spherical  aberration  correction,  480-481, 
471 

spinning-disk  confocal  example,  481, 
482,  482 

transmitted  light,  bright-field  (TLB),  472, 
All 

two  photon  example.  481,  483 
widefield  simulated  example,  481,  469 
WWF  stack  example,  469 
Blind  spots,  due  to  sampling  with  large 
pixels,  38. 

Blue  Sky  Research,  ChromaLase  488,  107. 
Boar  sperm  cells,  557. 

BODIPY  dye,  142,  342-343,  353-356,  389, 
692,  749,  760-762. 

BODIPY  TR,  methyl  ester  dyes,  760-762. 
Bolus  injection  protocol,  360,  726,  728, 

731. 

Bone,  reflectance,  167. 

Books  on  3D  LM,  listing,  889. 


Borohydride,  to  reduce  glutaraldehyde 
autofluorescence,  374,  770. 

Botanical  specimens,  414-439,  624, 
784-785.  See  also.  Plant  cell 
imaging,  and  Chapters  21  and  44. 
birefringent  structures,  420-421.  See  also. 
Birefringence 
deconvolution,  784-785 
effect  of  fixation  on,  195,  428 
Equisetum,  774 

fluorescence  properties,  421-428 
emission  spectra,  421-423 
microspectroscopy,  421-426 
fluorescence  resonance  energy  transfer, 
425. 

See  FRET,  425 

harmonic  generation  properties,  428, 
711-715 

light  attenuation  in  plant  tissue,  414-418 
absorption  spectrum,  415 
A.  thaliana  example,  416 
maize  stem  attenuation  spectra,  417, 

418 

M.  quadrifolia  attenuation  spectra,  416 
M.  quadrifolia  optical  sections,  419 
Mie  scattering,  162-163,  167,  417-418 
nonlinear  absorption  in,  416-417 
Rayleigh  scattering,  162-163,  167,  417, 
703 

light-specimen  interaction,  425-428 
living  plant  cell,  429-439 

calcofluor  staining  procedure,  424,  438 
callus,  429 

cell  walls,  168-169,  188-189,  303, 

306,  416^117,  420-421,  428-431, 
435-136,  438,  439,  710-711, 
713-715,  769-776,  779-781 
chamber  slides,  use,  429 
culture  chamber,  429 
cuticle,  434-437,  715,  717,  779 
fungi,  438-439,  624,  782,  870 
hairs,  431,  434^136,  772 
meristem,  168,  420,  430,  770,  776-778, 
783 

microsporogenesis,  431-432 
mineral  deposits,  163,  420,  436-438, 
703 

pollen  germination,  420,  433-434,  781, 
783 

pollen  grains,  202,  305,  313,  420, 

431- 433,  553,  558,  781,  783 
protoplasts,  195-196,  203,  416,  421, 

423-427,  429-431,  438^139,  693 
root,  172,  174,  303,  307,  421,  429, 
430-431,  438,  464-465,  556, 
772-773,  775,  111,  779-783 
starch  granules,  202,  420^-21,  428, 

432- 433,  435,  703,  710-712,  715, 
719 

stem,  168,  172,  180,  417-419,  421, 

424,  429,  556,  707,  710-711, 
713-714 

storage  structures,  435-436 
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Botanical  specimens  ( cont .) 

suspension-cultured  cells,  189, 
429-430 

tapetum,  433-434,  779 
waxes,  420.  428,  434-435,  714-715 
point  spread  function  in,  784 
refractive  index  heterogeneity,  192, 
418-420 
maize  stem.  419 
Bovine  embryo,  750. 

Boyde,  Alan,  2,  6,  141,  154,  224.  See  also. 
Stereoscopic  images. 

BPNN.  See  Backpropagation  neural 
network. 

Bragg  grating,  tuning  diode,  107. 

Brain  slices,  392-398,  722-734.  686. 
beam  collimation,  728 
choice  of  objectives,  395,  727-728 
future  directions,  929 
image  processing  for,  732-734 
algorithms,  733 

alignment,  center  of  mass  in,  732-733 
alignment,  based  on  image  overlap, 
732 

automatic  detection  of  neurons, 
733-734 

drift/vibration  compensation,  396,  732 
image  de-noising  using  wavelets,  734 
image  processing/analysis,  330-331, 
395-396,  730-732 
biocytin  protocol,  730 
classified  using  cluster  analysis, 
731-732 

correlated  electron  microscopy,  731 
montaging,  331 

neuron  reconstruction,  330-331,  730 
protocol  for  PCA/CA,  731-732 
spectral  imaging,  382 
two-photon/neurolucida  system,  316 
image  production,  729 
2-photon  excitation,  727 
deep  imaging,  395 
living  neurons,  725 
maintaining  focus,  395,  732 
microglia,  397-398 
neuronal  ensembles,  726 
objective  lenses,  choice  of,  727-728 
second  harmonic  imaging,  729-730 
in  vivo  observations,  387 
preparation,  387 

labeling  cells,  394-396,  724-727 
biolistic  transfection,  724-725 
bolus  injection,  726 
calistics,  726 
choice  of  dyes,  729 
diolistics,  726 

dye  injection/patch  clamp,  726 
genetic  manipulation,  725-726 
GFP  transgenic  mice,  726 
Helios  Gene  Gun,  724 
live-dead  staining,  393 
painting  with  AM-ester  indicators, 
726-737 


photoactivation,  383 
slice  loading,  726 

linear  unmixing,  192,  382,  664-667 
making  brain  slices,  393,  722-724 
acute  slices,  722-723 
autofluorescence,  383 
cultured  slices,  724 
mouse  visual  cortex,  723 
primary  visual  cortex,  724 
protocols,  731 
thalamocortical  slice,  724 
photodamage,  729 
pulse  broadening,  728 
reducing  excitation  light,  390-391 
resolution,  729 

second  harmonic  imaging  (SHG), 
729-730 

silicon-intensified  target  (SIT)  camera 
use,  730 

slice  chamber,  394 
protocol,  727 
speckle  microscopy,  383 
useful  techniques,  table,  384-385 
time-lapse,  382 
two-photon  imaging,  727 
calcium  imaging,  729 
z-sectioning,  729 
Breakdown. 

electrical,  in  PMTs,  263,  660 
optical,  high  power  density,  198,  680, 
682,  685,  687,  703,  705 
Brewster  surfaces,  83. 

Brewster  windows,  83,  102-103,  115. 
Bright-field  microscopy,  6,  127,  130,  201, 
224,  229,  448,  468.  649,  728. 

CCD  for,  127,  483 
deconvolution,  468,  472-473 
depth  of  field,  4 

low  coherence  light  for,  130,  134-135, 
139-140 

optical  projection  tomography,  610-612 
Brightness,  source,  21,  26,  126-127, 
129-130,  141-142,  215. 
and  exposure  time,  141-142 
gray  levels,  71-73 

as  limitation  of  disk-scanners,  21,  215 
of  non-laser  light  sources,  126-127 
of  sun,  127,  135 

Brillouin  background,  in  glass  fibers,  88. 
Brillouin  effect,  reduction,  110. 

Brownian  motion,  microtubules,  11. 

BSL.  See  Backscattered  light. 

Buffering,  fluorescent  ion  measurement, 
740. 

Bulk  labeling,  in  living  embryos,  761. 

C 

C.  elegans,  746,  748,  766,  856,  857-858. 
cryopreparation,  857-858 
FRET  imaging,  766 
as  model  system,  746,  748 
TEM  images,  856,  857 
Ca2+  imaging,  see  Calcium  imaging. 


Ca2+  indicators,  346-347,  738,  742-743. 

See  also,  Ca2+  sparks,  737-738,  742. 
discovery,  737,  738 

Caenorhabditis  elegans.  see  C.  elegans. 
Caged  compounds,  759-760. 

multi-photon  excitation,  543-544 
Calcein  AM  dye,  355,  360,  362-363, 
426^127,  430,  685,  804,  812. 
Calcium  imaging,  529,  545,  584,  736-737, 
812. 

calibration,  742-743 
data  compression,  584 
intensity  image,  529 
introduction,  736 
multi-photon  excitation,  545 
ratiometric,  189 
signal-to-noise  ratio,  737 
single-cell  kinetic,  812 
TIRF  for  measuring,  1 80 
very  fast  imaging,  237 
Calcium  ion  dyes,  183.  189,  237,  736,  737, 
741-743.  See  also,  fura-2,  Fluo-3 
and  Indo-1. 

Fluo-3  and  Fura  Red  indicator  system  for 
determining,  183 

Fluo-3  indicator  system  for  determining, 
737 

fura-2  reactions,  741-742 
Indo-1  and  Fura-2  indicator  system  for 
Calcofluor,  424,  438. 

staining  procedure,  438 
Calibration,  34,  75-76,  742-745. 

Ca2+  sparks,  742 
of  CCD  to  measure  ISF,  75-76 
confocal  microscopy,  742 
errors  in.  744 

of  ion  concentrations,  742-745 
ion  interference,  745 
of  effective  pinhole  size,  34 
in  vitro,  742 
Calistics,  726. 

Callus,  429. 

Cambridge  Technology,  galvanometers, 

54. 

cAMP  indicators,  347. 

Canna,  422,  710. 

fluorescence  spectra.  422 
as  function  of  excitation  intensity,  165 
nonlinear  absorption,  710 
Carbon  arc  lamps.  136. 

CARS.  See  Coherent  anti-stokes  Raman 
scattering. 

CARS  correlation  spectroscopy  (CS-CARS), 

602. 

Raman  spectra,  602 

CARV  disk-scanning  confocal  microscope, 
215,  226,  229,  230,  907-908. 
diagram,  230.  907 

CAT.  See  Computed  axial  tomography. 
Cathode-ray  tube  (CRT),  5-6,  53,  67, 

72-73,  291,  293,  588-589. 
gamma,  compensation,  73 
Cavities,  of  dielectric  coatings,  46,  47. 
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Cavity-dumped  lasers,  111,  114. 

for  FLIM  imaging,  1 14 
CCD.  See  Charge-coupled  devices. 

CD.  See  Compact  disks. 
cDNA-GFP  fusion,  in  plants,  773. 

Cedara,  281-282,  288,  302,  308. 

Cell  adhesion  imaging  with  TIRF,  90. 

Cell  autofluorescence,  742. 

Cell  chambers,  11,  22,  191,  219,  370-371, 
386-387,  394,  429-430,  564, 
610-611. 

for  4Pi  confocal,  564 
for  biofilms,  870-873,  875,  877,  880,  885 
brain  slice,  394,  723,  727,  729 
for  epithelial  cells,  370-371,  377,  386 
finder  chamber,  683 
flow  chamber,  870-873,  875,  877,  880, 
885 

for  high-content  screening,  810 
for  optical  projection  tomography, 

610-611 
perfusion,  394 

for  plant  cells,  191,  429^-30 
simple,  22,  394 
for  SPIM,  613,  625,  673 
table  of  required  functions,  380 
table  of  suppliers,  388-389 
test  chamber/dye,  654,  661 
Cell  cycle,  790,  791. 

Cell  damage,  2-photonmicroscopy,  680-688 
See  also ,  Bleaching;  Photodamage, 
absorption  spectra  of  cellular  absorbers, 
681 

intracellular  chromosome  dissection,  688 
mitochondria,  686 
nanosurgery,  219,  686-687 
one-photon  vs.  multi-photon,  680-689 
by  optical  breakdown,  198,  680,  682,  685, 
687,  703,  705 
photochemical,  682-685 
absorbers/targets,  682 
beam  power  sensor,  683 
impact  on  reproduction,  686,  685 
laser  exposure  parameters,  682-683 
NIR-induced  DNA  strand  breaks, 
683-684 

NIR-induced  ROS  formation,  683 
photodynamic-induced,  684 
spectral  characteristics,  table,  682 
photothermal,  685 

reproductive  effect,  short  NIR  pulses, 

682,  686 

ultrastructure  modifications,  685-686 
Cell  microarray  (CMA),  815-816. 

Cell  motility,  757. 

Cell  nuclei,  optical  effects,  23. 

Cell  pellet,  three  dimensional,  815. 

Cell  surface  targeting  assays,  813. 

Cell  walls  of  plants,  168-169,  188-189, 

303,  306.  416-417,  420-421, 
428-431,  435-436,  438,  439, 
710-711,  713-715,  719,  769-776, 
779-782. 


labeling,  775 
viability,  780 

Cell-by-cell  analysis,  817. 

Cell-cell  signaling,  778. 

Cellular  structures,  optical  effects,  22-23. 

Center-of-mass  alignment  protocol,  733. 

Center  pivot/off-axis  pivot  mirrors,  1,  214. 

Cerium,  doping  of  quartz  lamp  envelope, 
116. 

CFP  and  YFP  molecules,  in  FRET  pair, 

798-800. 

Chambers  for  living  cell  imaging,  388-389. 
commercial  suppliers,  table,  388-389 

Charge  amplifiers,  923-924. 
defined,  923 
destructive  readout,  923 
FET  amplifier  performance,  923 
non-destructive  (skipper),  923 

Charge-coupled  device  (CCD),  26-28, 

30-31,  39,  61-62,  65,  70,  74-78,  88, 
127,  137,  142,  215,  233,  254, 
458^159,  460-461,  482,  552,  558, 
644,  754-755,  784,  918-931.  See 
also.  Electron-multiplier 

CCD. 

bit  depth,  75 
camera,  918-931 

advances  in,  for  speed,  754-755 
bright-field  imaging,  127 
for  disk  scanner  systems,  78,  205,  215, 
220,  233-235,  349,  459,  754-755 
pixel  size,  62,  65,  634-635,  784 
specifications,  table,  929 
time  for  sampling,  70 
choosing, 
color,  927 

computer-assisted  pulse  shaper,  88 
confocal  imaging,  458-459 
cooled,  advantages  and  limitations,  30-31 
quantum  efficiency,  26-28 
spatial  quantization  of  signal,  39 
digital  camera,  75 
digital  vi.  video  camera,  61-62 
electron  multiplier-CCD,  30-31,  76-77, 
233-235,  262,  459-461,  482, 
925-926 

multiplicative  noise,  77,  234,  257,  262, 
926 

result,  205,  234,  755 
table,  233,  459 
evaluating,  927-931 
array  size,  928-929 
"the  clincher,”  929 
comparison,  CCD/EM-CCD,  table, 

233,  459 

dynamic  range  vs.  pixel  size,  table,  928 

maximum  signal,  930 

quantum  efficiency,  927-928 

readout  noise,  928 

readout  speed,  928-929 

self  test,  930 

sensitivity,  930 

shutter  stability,  929 


specifications,  927,  table,  233,  929 
user-friendliness,  929 
gain-register,  76-78,  460-461 
intensified,  930-931.  See  also.  Intensified 
CCD 

monitoring  during  exposure,  137 
multi-focal  multi-photon  microscopy,  552, 
558 

noise  sources,  256,  924-925 
charge  amplifier,  925 
clock-induced  charge  (CIC),  234,  926 
fixed  pattern  noise,  924-925 
multiplicative  noise,  77,  234,  257,  262 
noise  vs.  pixel  dwell  time,  922 
table,  256 

operation,  254,  918-927 
blooming,  923 
charge  amplifiers,  923-924 
charge  coupling,  918-920 
charge  loss,  921 
dark  charge,  921-922 
destructive  readout  amplifiers,  924 
edge  effects,  921 
electron  multiplier,  926-927 
FET  amplifier  performance,  253,  922, 
924 

frame  transfer  readout,  920 
full-frame  readout,  920 
gain  register  amplifier,  925-926 
incomplete  charge  transfer,  923 
interline  transfer  readout,  920 
leakage,  921-922 

non-destructive  (skipper)  amplifiers, 
923-924 

possible  problems,  920 
quantum  efficiency  vs.  wavelength, 

922 

quantum  efficiency,  920-921 
readout  methods,  920 
signal  level  representing  zero  photons, 
"925 

storage  array,  920 
performance,  table,  256,  459,  923 
piezoelectric  dithering,  increases 
resolution,  70 

pixel  size,  62,  65,  634-635,  784,  928 
quantum  efficiency  and  noise,  29,  644, 
920,  922 

measuring,  74-76,  926 
sensors  size,  parallel  data  collection,  142 
snapshot  camera,  65 
specifications,  described,  927-930 
testing,  930 

Cheek  cells,  backscattered  light  image, 
22-23. 

Chemical  environment  probe,  517. 

Chimeric  fusion  proteins,  794,  801-802. 
anisotropy  analysis,  794 
cloning  for  FRET,  801-802 
overexpression,  802 

Chinese  hamster  ovary  cell,  197,  556.  684+, 
818. 

Chirp,  pre-compensation,  88,  111,  602,  907. 
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Chlorophylls,  autofluorescence,  A.  thaliana, 
175,  194,  203,  425-426,  528,  711, 
714,  779,  782,  881. 
bleaching,  203 
FLIM,  528 

Cholera  toxin  transport,  790-791,  796-797, 
802. 

FRET,  796-797,  802-803 

Chromatic  aberrations,  134,  152-156,  178, 
242-245,  657-658,  659. 
apparatus  for  measuring,  243 
axial  chromatic  registration,  243-345, 

658,  651-659 

of  incandescent  and  arc  lamps,  134 
intentional,  for  color/height  encoding,  154 
lateral  chromatic  registration,  657-658 
fluorescent  latex  bead  labeled,  178 
linear  longitudinal  chromatic  dispersion, 
154,  659,  664 
measuring,  242-245 

Chromatic  corrections,  157,  177. 
excitation/emission  wavelength,  177 
tube  length,  table,  157 

Chromatic  magnification  difference.  See 
Lateral  chromatic  aberration. 

Chromatin,  385,  390,  684,  693-695,  812. 

Chromophores,  338-348,  543-544, 

803-804.  See  also.  Dyes; 
Fluorophors;  Fluorescent  probes  etc. 
cellular  introduction  methods 
electroporation,  359-360,  803 
microinjection,  360-361,  388,  739, 

748,  755,  795,  803-804 
table,  344-345,  803 
transfection  reagents,  358,  360,  362, 
556,  682,  790-791,  795,  803 
multi-photon  excitation,  543-544 

CIC,  clock-induced  charge.  EM-CCDs,  234, 
926. 

Circular  exit  pinhole,  9. 

Circular  laser  beam,  corrective  optics,  106. 

Classification,  pattern.  See  Automated 

interpretation  of  subcellular  patterns. 

Clathrin-GFP  dynamics,  236. 

Clearing  agents.  See  also.  Mounting  media, 
optical  projection  tomography,  (OPT) 

610,  624 

plant  material,  166,  417-420,  439, 
774-775 

Clock,  role  in  digitizing  and  reconstructing 
analog  signal,  64. 

Clock-induced  charge,  in  EM-CCDs,  234, 
926. 

Closterium,  192-194. 
chloroplast  autofluorescence,  192-195 
signal  variation  with  depth,  194 

CLSM.  See  Confocal  laser-scanning 
microscopy. 

Cluster  analysis  (CA),  731-732,  826. 
neurons  classified  using,  731-732 
protocol  with  PCA  and,  731-732 
subcellular  patterns,  826 

CMA.  See  Cell  microarray. 


CNS,  (central  nervous  system),  392-393, 
395.  See  also.  Chapters  19  and  41. 

Codecs,  image  processing,  831,  836, 
840-841. 

Coefficient  of  variation,  660,  661. 

Cohen's  k  statistic,  826. 

Coherence  length,  7-8,  84. 
defined,  7-8,  84,  130-131 
reducing,  for  laser  light,  84 

Coherence  surface,  84. 

Coherence  volume,  84. 

Coherent  anti-stokes  Raman  scattering 
(CARS),  90,  204,  550,  595-605. 
advantages,  204,  596 
correlation  spectroscopy,  602-603 
defined,  595 
energy  diagram,  596 
epi-detected,  597-599 
forward/backward  detected,  597-599 
Hertzian  dipole  radiation  pattern,  598 
history,  595-596 

imaging  of  biological  samples,  603-604 
adipocyte  cells,  604 
artificial  myelin,  204 
epithelial  cells,  603 
erythrocyte  ghosts,  603 
intensity  distribution,  597 
mapping  intracellular  water,  90 
microscope  schematic,  599 
multiplex  CARS  microspectroscopy, 
601-602 

non-resonant  background  suppression, 
600-601 

energy  diagram  for  multiplex  CARS, 
601 

epi-detection,  600 

phase  control  of  excitation  pulses,  600 
picosecond  t's.  femtosecond  pulses, 

600 

polarization-sensitive  detection,  600 
time-resolved  CARS  detection,  600 
optimal  laser  sources,  599-600 

pumped  optical  parametric  oscillator 
(OPO) 
systems,  600 
perspectives  on,  604-605 
unique  features  under  tight-focusing, 
596-597 

Coherent  illumination,  1,  83-84. 
properties  of  laser  light,  83-84 
and  resolution,  1 

Collagen  fibers,  164,  188,  313,  361,  393, 
514,  703-704,  715. 
autofluorescence,  545 
birefringence,  164.  188,  717 
gels,  393 

polarization  microscopy,  164,  188 
second  harmonic  image  (SHG),  703-704, 
715 

Collector  optics,  elliptical  and  parabolic, 
129. 

Colliding-pulse,  mode-locked  laser  (CPM  ), 
540. 


Colloidal  gold  labels,  167,  241,  846-859. 
contrast,  167 

electron  microscope  markers,  846-857 
correlative,  850,  852,  855 
SEM,  850 
TEM 

FluoroNanoGold,  854 
GFP  related,  854-855,  857-858 
measuring  resolution,  241 
quenches  fluorescence,  854 
Rayleigh  scattering,  167 
Colocalization,  517,  650,  667-670,  794, 
813,  881. 

FRET,  FRET.  519 
erroneous,  581 
Color  display,  291,  292. 
display  space,  291 
multiple  channel  display,  292 
palette,  291 

pseudo,  173-175,  190,  291 
resolution,  291 
true,  291 

Color  centers,  in  optics,  avoidance,  116. 
Color  filters,  43-52.  See  also.  Filters, 
long-pass,  43-46,  175,  203-204,  212 
short-pass,  45,  46 
bandpass,  44,  45 
Color  print  images,  592. 

Color  reassignment,  173-175,  190,  291. 
Coma,  145,  151,  245,  247,  249,  483, 

630. 

distortion  away  from  optical  axis,  151 
observation  using  point  objects,  145, 

246 

Commelina  communis,  images,  712. 
Commercial  confocal  light  microscopes, 
906-917. 

BD-CARV  II,  230,  907 
La  Vision-BioTec  TriM-Scope,  907 
Leica,  TCS  SP2  AOBS,  910 
Leica  MP  RS  Multi-photon,  910 
Nikon  Cl  si,  911 
Olympus  DSU,  913 
Olympus  Fluoview-1000,  912 
optical  parameters  of  current,  table, 
908-909 

Visitech  VT  Infinity,  914 

Visitech  VT-eye,  914 

Yokogawa  CSU  22,  231,  915 

Zeiss  LSM  510  META  optical,  916-917 

Zeiss  LSM-5-LIVE  Fast  Slit  Scanner 

schematic,  232,  916 

Compact  disks  (CD)  for  data  storage,  499, 
586-587,  588,  731. 

Compact  flash  cards,  588. 

Components,  of  confocal  fluorescence 
microscopes,  43-58,  207-208. 
acousto-optical  devices,  54-57 
chapter,  43-58 

electroptical  modulators,  (Pockels  cells), 
25,  54,  57,  87,  116,  543,  701, 

903-904 

filters/beam-splitters,  44-51 
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mechanical  scanners  (galvanometers), 
51-54 

polarizing  elements,  58 
Computed  axial  tomography  (CAT), 
610-611. 

Compression,  data  see.  Data  compression. 
Condenser  lens,  size.  129. 

magnification,  128-129 
Configuration  of  pixels  in  image  plane,  62. 
ConfMat.  See  Confusion  matrix  based 
method. 

Confocal  disk-scanning  microscope.  See 
also.  Disk-scanning  confocal 
microscopy. 

Confocal  fluorescence  microscope,  73,  207, 
404-413.  See  also,  Confocal 
microscopy;  Confocal  laser-scanning 
microscopy. 

basic  optical  layout,  207 
limitations  due  few  photons,  73,  459 
refractive  index  mismatch,  404^-13  See 
also.  Refractive  index 
Confocal  imaging,  4-5,  232,  235-236,  737, 
738,  746-766,  809-817.  See  also, 
next  major  head  and 
Chapters  35  and  36. 

4Pi.  See  4Pi  microscopy 
automated 

for  cytomics  chapter,  809-817 
of  microarray  slide,  816 
platforms  used  for,  810 
real-time,  810 
temperature  control,  810 
types  of  assays  for,  811,  813-814 
workstations,  814 
of  biofilms,  Chapter  50 
deconvolution,  753.  See  Deconvolution 
by  disk-scanning  confocals,  232 
fast,  235-236 

of  fluo-3  loaded  cardiac  myocyte,  737 
fluorescent  indicators  for,  738 
high-resolution  datasets,  cell 
arrangements,  776 
of  living  cells,  813 
of  living  embryos,  chapter,  746-766 
methods  compared,  459,  644-647.  See 
Chapters,  22,  23,  and  24 
of  plants,  773.  See  also.  Chapters  21and 
43 

t'.v.  non-confocal,  746 
time-lapse.  See  Time-lapse  imaging 
Confocal  laser-scanning  microscopy 

(CLSM),  9-15,  32,  38,  81,  89.  118, 
222-224,  408,  518,  678,  690,  697, 
750-751,  754,  884-885.  See  also, 
next  major  head 

advantages  and  limitations,  11-12, 
222-223,  644-647,  884-885 
alternatives  to,  644-647,  754 
comparisons.  644-647 
disk-scanning  and  scanned  slit,  table,  224 
digitizer  employing  full  integration  for,  32 
edge  response,  408 


fluorescence  lifetime  imaging,  chapter. 

518 

laser  power  required.  8 1 
laser  requirements  for,  89 
v.s\  multi-photon  laser-scanning 

microscopy,  750-751.  See  Chapters 
22,  23,  24 

photobleaching,  690,  697 
Vi.  selective  plane  illumination 
microscopy,  (SPIM),  678 
stage-or  object-scanning,  13-15 
TEM  mode,  118 

zoom  magnification  and  number  of  pixels, 
38 

Confocal  microscopy,  90,  141,  265, 

381-399,  444^147,  453-467, 
650-670,  742,  770.  774,  779,  810, 
811,  815,  870-887.  See  also, 
preceding  major  head  and  Chapters 
35  and  36. 

art  of  imaging  by,  650 

automated,  platforms  used  for,  810 

balancing  multiple  parameters  for,  650 

of  biofilms,  870-887 

calibration  of,  742 

cell  microarray  and,  815 

colocalization,  667-670 

effect  of  MLE  and  threshold,  table,  669 
fluorogram  analysis,  669 
image  collection,  667-668 
nerve  fiber,  669 
quantifying,  668 
setting  thresholds,  668 
spatial  deconvolution  in  studies, 
668-670 

v.s\  deconvolution,  644-647,  453^-67.  See 
also,  Chapters  22,  23,  24 
CCD/confocal  imaging  combination, 
458^159 

deconvolving  confocal  data,  461-464, 
466,  488-500 

fluorescence  excitation.  459 
fluorescent  light  detection,  459-460 
gain  register  CCDs,  460-461 
image  sections,  figures,  455,  456,  462 
imaging  as  convolution,  453^157 
integration  of  fluorescence  intensity, 

459 

limits  to  linearity,  457 
model  specimens,  461 
noise,  459-463 
out-of-focus  light,  461 
point  spread  function,  453-457 
practical  differences,  458,  463-466 
resolution,  459^-63 
same  specimen  comparison,  465 
sensitivity,  459-463 
shift  invariance,  457,  490,  564 
single  point  imaged,  454 
summary  of  pros/cons,  table,  459 
temporal  resolution,  458 
focus  positioning,  651-652 
getting  a  good  confocal  image,  629-63 1 


alignment  of  optics,  629-630 
back-focal  plane  (BFP),  210,  509,  629, 
633 

focus,  629 
low  signal,  631 
mirror  test  specimen,  630 
no  signal,  631,  660 
simultaneous  BSL/fluorescence,  631 
high-content  screening  systems,  table,  811 
illumination  sources,  126-144,  650-651 
See  also.  Lasers;  Non-laser  sources 
acousto-optic  tuning  filter  (AOTF), 

651. 

laser  sources,  chapter,  80-125 
laser  stability,  65 1 
power  measurement,  650-651 
living  cells,  381-399.  See  also.  Living 
cells 

Minsky  first  confocal  design,  2,  4—6,  11, 
141,  216,  890 

monitoring  instrument  performance, 
650-663 

illumination  source,  650-651 
optical  performance,  652-660 
photon  efficiency,  14-15,  24 
scan  raster/focus  positioning,  651-652 
signal  detection,  660-663 
with  non-laser  light,  141 
objective  lens,  652-660.  See  Chapter  7 
optical  performance,  652-660.  See  also 
Chapters  7,11 

axial  chromatic  registration,  658-659 
axial  resolution  vs.  pinhole,  656-657. 

See  also.  Axial  resolution 
contrast  transfer  function,  656.  See 
CTF 

coverslip  thickness  and  RI,  table,  654 
field  illumination,  658 
flatness  of  field,  659 
Focal  Check™  beads,  657-659 
lateral  chromatic  registration,  657-658 
lateral  resolution,  655 
refractive  index,  654.  See  Chapter  20 
resolution  test  slides,  656 
self-lensing  artifacts,  659 
spherical  aberration,  correction,  654, 
655 

subresolution  beads,  655-656 
x-y  and  z  resolution  using  beads,  656 
optimizing  multi-labeling,  663-667 
bleed-through  between  channels,  663 
control  samples,  establishing  limits, 

663 

measuring  autofluorescence,  663 
multi-tracking,  reduces  bleed-through, 

664 

positively  labeled  sample,  664 
reflected  light  contribution,  663 
secondary  conjugate  contribution,  664 
photon  efficiency,  24,  26,  28,  30  33-34, 
36 

polarizing  elements,  57 
scan  raster,  65 1-652 
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Confocal  microscopy  (cont.) 

malfunctioning  system,  653 
phototoxicity  from  uneven  scan  speed, 
651 

sources  of  fluorescent  beads,  table,  653 
well-calibrated  system,  652 
x  and  y  galvanometers,  65 1-652 
z-drive  mechanism,  652 
z-positioning  calibration,  654 
z-positioning  stability,  652 
separating  signal  by  spectral  regions  for, 
664  " 

sequential  collection  reduces  bleed- 
trough,  664 

signal  detection  for,  660-663 
coefficient  of  variation,  660-661 
instrument  dark  noise,  660 
PMT  linearity,  661-662 
signal-to-noise  ratio,  660 
spectral  accuracy,  662 
spectral  detector  systems,  662 
spectral  resolution,  662-663 
wavelength  response,  663 
signal  level,  444^145 
signal-to-noise  ratio,  444^147 
spectral  analysis,  of  plants,  770 
spectral  unmixing,  192,  382,  664-667 
limitations  to,  667 
overlapping  fluorophores  separation, 
664-667 

removing  autofluorescence,  667 
stage-scanning,  9 
staining  plant  cells,  774 

structured-illumination  methods,  265 
vs.  two-photon  excitation,  779 
Confusion  matrix  based  method  (ConfMat), 
826. 

Constant  output  power  laser  stabilization, 

86. 

Continuous  wave  (CW)  laser,  87,  88, 

90-118. 

beam  intensity  stabilization,  86-87 
diode  (semiconductor).  105-110 
output  power/cooling,  108 
pumped  solid-state,  table,  94,  95 
dye  lasers,  86,  103,  112,  114,  124, 
540-541 

fiber  up-conversion,  109-110 
gas  lasers 

Argon-ion,  90,  102 
Krypton-ion,  102 
HeNe,  102 
HeCd,  103 

cesium  and  rubidium  vapor,  103-105 
table,  92-93 
titanium-sapphire,  109 
Contrast,  7,  11,  16,  37,  39,  49,  59-62,  68, 
159,  162-204,  248,  421,  473,  488, 
542,  599-600,  607,  622,  656,  657, 
675.  See  also.  Rose  criterion  and 
CTF. 

absorption,  equations,  164 
chapter,  162-206 


defined,  162 
flare,  649 

formation  of,  chapter,  162-206 
fluorescence.  See  Dyes,  and  Fluorophores 
as  function  of  feature  size,  16, 

61-62,  37,  634 
intrinsic,  633 
measuring,  16,  59 

polarization.  See  Polarization  microscopy 
second  harmonic  generation.  See  SHG 
and  statistics,  633 

third  harmonic  generation.  See  THG 
Contrast  medium,  and  laser  power,  80-81. 
Contrast  method,  defines  signal  required, 
126. 

Contrast  transfer  function  (CTF),  16,  35, 
37-39,  59-62,  656,  747. 
in  confocal  vi.  non-confocal  microscopy, 
16. 

See  Chapter  11 

as  function  of  grating  period,  16 
of  microscope  optical  system,  35 
relationship  with  objective  BFP,  61 
and  spatial  frequencies,  16,  37 
and  stages  of  imaging,  62 
Control,  of  non-laser  light  sources,  138-139. 
Convalaria  majalis,  425,  556. 
fluorescence  microscopy  of  rhizome,  425 
multi-focal  multi-photon  imaging,  556 
Conversion  techniques,  259-260. 
analog-to-digital,  259 
digital-to-analog,  259-260 
Convolution,  a  primer,  485-487. 

3D  blurring  function,  486 
Fourier  transforms.  487 
geometrical  optics,  487 
out-of-focus  light,  486^-87 
Cooling  water,  checking/maintaining. 
116-117. 

Cork  microstructure,  770. 

Correction  collar,  (spherical  aberration),  15, 
145-149,  158,  160-161,  178, 
241-242,  247,  377,  407.  410-412. 
471,  654-655,  657. 

adjustment,  377,  407,  471,  499,  654-655 
dry  objectives,  410 
multimedia,  640 
Correctors,  70.  147. 

spherical  aberration,  15,  151,  147,  192, 
411-412 

Intelligent  imaging  innovations,  78-79, 
151,  192,  395,  411,  654 
to  stored  data,  second  Nyquist  constraint, 
70 

Corrective  optics,  for  diode  lasers,  107-108. 
Correlational  light  microscopy/electron 
microscopy,  731,  434,  436^-37, 
846-860. 

BSL  image,  855 

brain  slices,  731then  CLSM,  856-857 
cryopreparation  of  C.  elegans,  857-858 
DIC  image  tracking,  849 
DIC  image/UV  fluorescence  image,  850 


different  requirement  of  LM/EM, 

846-850 

early  4D  microscopy,  846 
fluorescence/TEM  to  analyze 
cytoskeleton,  854 
fluorescent  micrographs,  851 
FluoroNanoGold  for  cryosections,  854 
GFP,  854.  See  also.  Green  fluorescent 
protein 

HVEM  stereo-pair,  848-849 
immuno-stained  bovine  aorta,  852 
LVSEM  of  FRAPed  microtubules,  849, 
850 

phalloidin  as  correlative  marker,  235-236, 
344,  376,  378,  694,  696,  756,  804, 

854-856 

phase-contrast  imaging,  85 1 
postembedding,  855 
quantum  dot  labeling,  853 
same  cell  structure  LM/SEM,  850-852 
same  cell  structure  LM/TEM,  852-856 
SEM  images  at  5kV  and  20kV,  847,  848 
TEM  cross-section  of  C.  elegans,  856 
TEM  longitudinal  section  of  C.  elegans, 
857 

tetracysteine  tag  labeling,  221,  348,  357, 
853 

tiled  montage  TEM  images,  858 
time-series  DIC  images,  847 

Correlative  LM/EM.  See  Correlational  light 
microscopy/electron  microscopy. 

Coumarin  dye,  114,  339,  344-345,  353,  355, 
654-655,  661,  693. 

Counting  statistics,  20,  30.  See  Poisson 
statistics. 

Cover  glass.  See  Coverslip. 

Coverslip,  and  spherical  aberration,  15, 
147-150,  201.  See  also.  Spherical 
aberration. 

CPM  laser.  See  Colliding  pulse  mode-locked 
laser. 

Crane  fly  spermatocyte,  metaphase  spindle, 
15. 

Creep,  in  piezoelectric  scanners,  57. 

Cr:Fosterite,  femtosecond  pulsed  laser,  109, 
114,  415,  541,  706-709,  712-714. 

Critical  angle,  for  reflection  of  incident  light 
surface  of  refracting  medium,  167, 
502. 

Critical  illumination  of  the  specimen, 
128-129. 

Crosslinking  fixatives,  369. 

Crosstalk. 

between  fluorescence  channels,  203,  424, 
882 

between  disk  pinholes,  227 
between  excitation  foci,  553-556, 
558-559,  564 

CRT.  See  Cathode-ray  tube. 

Crystal  Fiber  A/S,  HC-800-01  bandgap 
fiber,  88. 

CSU.  See  Confocal  scanning  unit. 

CTF.  See  Contrast  transfer  function. 
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Curtains,  laser,  safety,  118,  904. 

Cuticles,  plant,  434-437,  715,  717,  779. 
insect,  166 
maize,  436 

CW.  See  Lasers,  continuous  wave. 

Cyan  fluorescent  protein  (Cyan),  221-222. 

Cyanine  dyes,  339,  342,  344,  353-355, 

362-363,  374,  443,  540,  587,  760, 
854,  874. 

Cytomics,  810,  811. 

automated  confocal  imaging,  810 
automated  confocal  imaging,  table,  811 

Cytoskeletal  structures,  24,  188,  190, 

328-329,  368,  370,  372,  378,  383, 
461-462,  577,  703,  715,  719, 
773-774,  813,  846-848,  852,  854. 
LM-TEM  analysis,  846,  854 
stabilizing  buffer,  852 

Cytosolic  markers,  757. 

Cytotoxicity,  reducing,  36-37.  See  also, 
Bleaching;  Phototoxicity. 

D 

DAC.  See  Digital-to-analog  converter. 

Damage  threshold,  LED  sources,  139. 

DAPI,  140,  344-345,  355,  358,  376,  431. 
plants,  43 1 
use  of.  376 

Dark  current,  29,  76,  234. 
fixed-pattern  noise  due  to,  76 
of  photomultiplier  tube,  29,  660 
reducing,  234 

Dark  noise,  defined,  232. 

Darkfield  microscopy,  5,  7,  172,  474,  672. 
depth  of  field,  4 

Data,  11-12,  33,  64,  76,  237. 

conversion  from  ADU  to  electron  data,  76 

degradation  by  multiplicative  noise  and 

digitization,  33 

reconstructing,  64 

speed  of  acquisition,  11-12 

storage  of  volume  of  data,  237 

Data  collection  guidelines,  319-320. 

Data  collection  model,  blind  deconvolution, 
472. 

Data  compression,  288-289,  292-293,  295, 
319,  499,  580-585,  762,  764,  819, 
835-836. 
algorithms,  580 

discrete  cosine  transform  (DCT),  581 
Huffman  encoding,  580 
Lempel-Ziv- Welch  (LZW),  580 
run-length  encoding  (RLE),  580 
archiving  systems,  580 
gzip,  580 
PKzip,  580 
WinZip,  580 
calcium  imaging,  584 
color  images,  581 
different  techniques,  table,  581 
Dinophysis  image,  585 
effects  on  confocal  image,  583 
examples,  583-585,  592,  834-837 


file  formats  for,  580-588 

fractal  compression,  581-582 
GIF  (graphics  interchange  format), 

580 

JPEG  (Joint  Photographic  Experts 
Group),  581-584 
MPEG,  836-839,  840-841 
PNG  (portable  network  graphic),  581, 
584 

QuickTime,  829,  831,  836-837, 
840-844 

TIFF  (tagged  image  file  format),  580 
wavelet  compression,  581-584,  819 
movies,  836-842 
artifacts,  839 

compression  ratios,  842-843 
entrope,  841 

MPEG  formats,  840-841 
Up-sampling,  838 
pixel  intensity  histograms,  584 
testing,  830,  835 

time  required,  table,  581for  WWW  use, 
816 

useful  websites,  844-845 
Data  projectors,  590. 

Data  storage,  106.  See  also.  Mass  storage. 
Data  storage  systems,  287,  395.  580,  594, 
764. 

chapter,  580-594 

characteristics  of  3D  microscopical  data, 
287 

databases,  861-869.  See  Databases 
random  access 

CDR,  CDRW,  586-587 
DVD,  587 
Magnetic  disks,  586 
semiconductor,  FLASH  memory,  588 
for  remote  presentation,  842 
role  for  STED,  577 

Databases,  2D/3D  biology  images,  827, 
861-869. 
benefits,  863-864 

fast,  simple  machine  configuration,  863 
improved  analysis  and  access,  863 
performance,  863 
remote  monitoring,  863 
repeatability  of  experiments,  863 
submissions  to  other  databases,  863 
criteria/requirements,  866-867 
digital  rights  management,  867 
metadata  structure,  867 
query  by  content,  866-867 
user  interface,  866 
data/metadata  management,  861-862 
future  prospects,  867 
image  database  model,  864-865 
image  information  management,  862 
image  management  software,  table,  865, 
868 

instrument  database  model,  864 
laboratory  information  management 
systems  (LIMS),  862 
microscopy  data/metadata  life  cycle,  863 


modern  microscopes  design  aims, 
862-865 

projects,  865-866 
Bioimage,  865-866 
Biomedical  Image  Library  (BIL),  866 
Scientific  Image  DataBase  (SIDB),  866 
recent  developments,  861-862 
MPEG-7  format,  862 
relational  database  management 
systems  (RDBMS),  862 
TIFF  format,  861 
software  for,  868-869 
ACDSee,  868 
Aequitas,  868,  869 
Cumulus,  868 
Imatch,  868,  869 
iView,  868,  869 
Portfolio,  868 
price,  868 

Research  Assistant,  868 
ThumbsPlus,  868,  869 
system  requirements,  864 
DBR.  See  Distributed  Bragg  reflector. 

DCT.  See  Discrete  cosine  transform. 
Deblurring  algorithm,  476. 

Deconvolution,  7,  26-28,  39,  40,  66, 

189-190,  222-223,  278,  456^158, 
464,  468,  488-500,  542,  564,  736, 
746,  751-753,  778,  784-785,  828, 
864,  900,  929.  See  also.  Blind 
deconvolution, 
of  2-photon  images,  498 
and  3D  Gaussian  filtering,  70,  281,  285, 
323,  392,  395,  667.  See  also, 
Gaussian 

4Pi  lobe  removal,  562,  565 
advantages  and  limitations,  458,  475 
algorithms,  472-476,  490,  495-497,  751, 
778 

comparison,  497-498 
iterative  constrained  Tikhonov-Miller, 
497 

Jansson-van  Cittert,  496 
nearest  neighbor,  495-496 
non-linear  constrained  iterative, 
496-497 

Richardson-Lucy,  497,  568 
Weiner  filtering,  496 
background  history,  488-490 

blurring  process  contributions,  488 
equation  showing  restoration  possible, 
489 

image  formation,  489-490 
schematic  diagram  of  convolution,  489 
blind,  189-190,  431,  463,  469,  472-473, 
478,  486,  492,  496-497,  646 
chapter,  468-487 
maximum  likelihood  estimation, 
472-477,  483,  669-670 
blurring  process  contributions,  488 
confocal  data,  39,  40,  453-467,  488-500, 
753,  778.  See  also,  Confocal 
microscopy,  vs.  deconvolution. 
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Deconvolution  ( cont .) 

of  simulated  confocal  data,  40 
CARS  data  cannot  be  deconvolved,  397, 
399 

chapter,  453^-67 

colocalization,  668-670  comparison  of 
methods,  66,  453,  467,  475-477, 
497-499,  644-648 
convolution  primer,  485 — 478 
convolution  and  imaging,  490-491 

Fourier  transform  of  PSF,  489,  490 
linearity,  490 

optical  transfer  function.  490-491 
shift  invariance,  457,  490,  564 
and  data  compression,  584-585 
test  results,  401,  461,  464-466, 
481-482,  483 
defined,  189-190,  468 
display  of  data,  301,  830,  835-836 
examples,  40,  190,  392,  411,  462,  466, 
471,  488-498,  510 
4Pi,  468,  565 

botanical  specimens,  784-785 
brightfield,  411,  475,  478 
cardiac  t-system,  498 
confocal,  470 
DIC,  470 
polarized,  479 

of  simulated  confocal  data,  40 
STED,  574-576 
flatfielding  the  data,  477 
black  reference,  76 
white-reference,  76 
fluorescence  lifetime  imaging,  521 
four  dimensional  deconvolution,  391-392, 
752 

Fourier  transform  of  PSF,  489,  490 
future  directions,  483,  766 
and  image  formation,  490-492 
linearity  and  shift-invariance,  457,  564 
live  imaging,  480,  564,  751-754 
missing  cone  problem,  494 
model  specimens,  461,  464-466, 

481-482,  483 

multi-photon,  488-500,  542 
multi-view  montaging,  330,  677 
ion  imaging,  736 
noise,  495,  635 

and  Nyquist  reconstruction,  59,  65,  67, 

68,  222-223,  635 
suppressing  Poisson  noise,  39 
optical  sectioning,  752 
out-of-focus  light,  26-28,  431,  487,  644 
and  pinhole,  26,  487 
point-spread  function  (PSF),  223,  241, 
247,  453,  463,  471,  489^192,  635, 
655 

approximations,  493 
measuring  PSF,  492-494 
and  Poisson  noise  reduction,  320 
pre-filtering,  281,  497,  581 
problem  with  specimen  heterogeneity,  22, 
648 


purpose,  468 

requirements  and  limitations,  489-494 
diagram  demonstrating  convolution, 
489 

linearity,  490 

missing  cone  problem,  494 
noise,  495 

optical  transfer  function,  490-491 
point  spread  function,  489,  492-494 
shift  invariance,  457,  490,  564 
sampling  frequency,  635 
spherical  aberration,  471,  480 — 48 1 
stain  sparsity,  28 

structured  illumination,  comparison, 
265-279 

subpixel  refinement,  478-479 
temporal/spatial,  392,  458,  753 
transmitted  light  imaging,  472,  475, 

478 

of  wavelength  spectra,  382,  663-667, 
771-772 
limitations,  667 

Deconvolution  lite,  68-70. 

Deflector,  acousto-optical.  See  Acousto- 
optical  deflector. 

Defocusing,  size  and  intensity  distribution, 
146. 

Degree  of  modulation,  268-270. 
locally  calculated,  268-270 

absolute  magnitude  computation, 
268-269 
equations,  269 

homodyne  detection  scheme,  268-269 
max/min  measured  intensity  difference, 
268 

scaled  subtraction  approach,  269-270 
square-law  detection,  268-269 
synthetic  pinholes,  268 

Delamination,  and  interference  fringes, 
168-170. 

Delivery,  dye,  355,  357-360,  810. 

Deltavision,  132,  282. 

Demagnification,  and  numerical  aperture, 
127. 

Depth  discrimination,  in  LSCM.  See  Axial 
resolution. 

Depth  of  field,  4,  9,  13. 
extended-focus  images,  9 
fluorescence  microscopy,  4 
phase-dependent  imaging,  13 

Depth-weighting,  projection  images,  304, 
306. 

exponential,  304 
linear  or  recursive,  304 

Derived  contrast  (synthetic  contrast), 
188-201. 

Descanned  detection.  166,  208,  212,  537, 
540-542,  754,  904. 

Design  of  confocal  microscopes,  43,  145, 
166,  207-211,  237.  See  also. 
Commercial  confocal  light 
microscopes. 

4Pi,  563,  566 


of  confocal  fluorescence  microscope,  208 
efficiency  in,  43 

fast-scanning  confocal  instruments,  237 
intermediate  optical  system,  207-209 
of  microscope  optics,  145 
MMM,  552,  555 
practical  requirements,  210-211 
of  transmitted  confocal  microscope,  166 
Detection  efficiency,  34,  35,  210-211. 
measurement,  34-35 
practical  requirements,  210-211 
Detection  method,  multi-photon,  541,  542. 
descanned,  542 

Detectors,  9.  11,  25,  28,  251-264.  See  also. 
Photomultiplier  tube;  Charge- 
coupled  device,  etc. 
area  detectors.  See  Image  detectors 
assessment  of  devices,  260-262 
CCD,  254 

noise  vs.  pixel  dwell  time,  922 
comparison,  table,  255-256 
conversion  techniques,  259-260 
descanned,  208,  212,  537,  540-542,  774, 
904 

direct  effects,  252 
errors,  211-212 
evaluation,  211-217 
future  developments,  262-264 
history,  262-264 
image  dissector,  254-255 
image  intensifier,  13,  232-233,  519-520, 
524,  555-556 

gated,  233,  519-522,  524,  555-556 
intensified.  See  Intensified  CCD 
MCP-PMT.  See  MicroChannel  plate 
microchannel  plate,  232-233,  255,  262 
MCP-CCD,  262 
gated,  519,  523-524,  527,  532 
noise  internal,  256-259 
internal  detection,  256 
noise  currents,  table,  256 
photoemissive  devices,  256-257 
photon  flux,  257-258 
pixel  value  represented,  258-259 
non-descanned,  185,  201,  218,  381,  447, 
456,  507,  542,  552,  559,  643,  646, 
727,  750,  779,  904,  909-910 
phase-sensitive,  518-520,  619 
photoconductivity  effects,  252,  253 
photoemissive,  254 

photography.  See  Photographic  systems 
photomultiplier  tube,  9,  11.  See  also, 
PMT 

photovoltaic  effect,  252-253 
photon  interactions  in,  252-256 
point  detectors,  260-261 
quantal  nature  of  light,  251-252 
quantum  efficiency  (QE)  vs.  wavelength, 
25 

for  second  harmonic  detection,  table,  707 
silicon-intensified  target  (SIT)  vidicon, 
730 

spectral,  203-204,  662-663,  666-667 
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TCPSC,  518,  520-523,  526 
time-gated,  522 
thermal  effects,  252 
work  functions,  table,  252-253 
vacuum  avalanche  photodiode,  254,  255 
Developmental  biology,  545,  624. 

multi-photon  microscopy  (MPM),  545 
Dextran  labeling,  173-174,  292,  512,  757. 
DFB.  See  Distributed  feedback. 
4',6-diamidino-2-phenylindole,  140, 

344-345,  355,  358,  376,  431.  See 
DAPI. 
plants,  43 1 
use  of,  376 

Diatom,  438,  638-640,  881. 
as  standard  for  measuring  objectives,  145 
test  specimen,  638-640 
DIC.  See  Differential  interference  contrast. 
Dichroic  filters,  212. 
intensity  loss,  212 
transmission,  212 

Dichroic  mirrors  (beam-splitters),  44,  45, 
50-51,  129,  211,  217-218. 
coating  for  collection  mirrors,  129 
double  and  triple,  217-218 
effect  of  deflection  angle,  211 
separating  emission/excitation,  44-45, 
50-51 

Die,  of  light-emitting  diode,  133,  134. 
Dielectric  butterfly,  galvo  feedback,  54. 
Differential  interference  contrast  (DIC) 

imaging,  10,  14,  76,  127,  146,  171, 
453,  468,  473-475,  846. 
blind  deconvolution,  473-475 
converting  phase  shifts  to  amplitude,  171 
narrow  bandpass  filter  use,  76 
Nomarski  DIC  contrast,  2,  268,  746,  892 
photon  flux  reduction,  127 
schematic  for,  475 
three  dimensional,  470 
Wollaston  prism,  156,  468,  473,  475 
Diffraction,  61,  65. 

contrast  transfer  function,  16,  35,  37-39, 
59-62,  656,  747 

and  sharpness  of  recorded  data,  65 
Diffraction  limit,  210-211.  See  also , 
Rayleigh  criterion, 
defined,  210 

point-spread  function,  146 
practical  requirements  for,  210-211 
Digital  light  processor  (DLP),  projectors, 
590. 

Digital  memory  system,  64. 

Digital  microscopy,  optics/statistics/ 
digitizing,  79. 

Nyquist  sampling,  146 
Digital  printers,  591-593. 

Digital  processing,  in  disk-scanning 
confocal,  12. 

Digital  projectors,  590. 

Digital  rights  management  (DRM),  830, 
844. 

Digital  video  disks  (DVD),  587-588. 


Digital-to-analog  converter  (DAC),  64, 
259-260. 
operation,  64 

Digitization,  25,  31-32,  36,  38-39,  59, 

62-63,  66,  72,  75,  79,  259,  261,  286, 
460,  495,  639,911. 
aliasing.  See  Aliasing 
blind  spots,  38 
and  Nyquist  criterion,  38-39 
precision,  25 
and  pixels,  62-63 

of  voltage  output  of  photomultiplier  rube, 
31-32 

Dil  derivatives,  760. 

Dimethylsulfoxide  (DMSO),  697,  726-727, 
760,  875. 
handling,  739 

DIN  standard,  microscopes,  156. 

Dinophysis  image,  585. 

Diode  injection  lasers,  105-108. 

Diode  lasers,  86,  87,  107,  112,  116. 
distributed  feedback,  107 
emission  stability,  86 
intensity,  87 
maintenance,  116 
modulated,  112 
noise  sources,  86 
physical  dimensions,  106 
violet  and  deep  blue,  107 
visible  and  red,  107 
wavelength  stabilization,  87 

Diode-pumped  alkali  lasers  (DPAL), 
103-105. 

Diode-pumped  lamp  (DPL),  108-109. 

Diode-pumped  solid-state  lasers  (DPSS), 
108-109,  111,  112. 
kits,  companies  offering,  109 
passively  mode-locked,  1 1 1 
ultrafast,  112 

Diolistics,  ballistic  gene  transfer,  726. 

Dipping  objective,  149.  161,  209,  411,  429, 
568,  613,  727,  737,  870,  872. 

Direct  permeability,  358-359. 

Discrete  cosine  transform  (DCT),  581. 

Disk-scanning  confocal  microscopy, 
215-216,  224,  225,  228-229, 
234-235  754,  755. 
advantages  and  limitations,  223-224 
for  backscattered  light  imaging,  228-229 
chapter,  221-238 

commercial  instruments,  907,  913,  915 
comparing  single-  vs.  multi-beam,  224 
table,  226 

and  electron  multiplier  CCDs,  78,  205, 
215,  220,  233-235,  349,  459, 
754-755 
embryo,  754 

high-speed  image  acquisition,  216, 
222-224,  754 
image  contrast  in,  168-171 
microscopes,  table,  224 
optical  sectioning,  235 
types,  228-232 


Dispersion,  optical,  56,  88,  152,  154,  242, 
411,  542-543,  609,  683. 
in  acousto-optical  devices,  3.  15,  55-56, 
88 

CARS  signal  generation,  728 
compensation,  566-567 
defined,  152 

in  fiber  lasers,  ultra-fast  pulses,  88,  110, 
113 

by  filter  blank  material,  211 
generates  third  harmonic  signal,  704-705 
group  delay  dispersion,  537-538,  543 
group  velocity  dispersion,  88,  111,  210, 
537,  609,  903 

in  optical  coherence  tomography,  (OCT), 
609 

in  optical  fibers,  502,  504,  507 
and  temperature,  15,  411 
for  multi-channel  detection,  5 1 
using  to  correct  for  chromatic  aberration, 
153 

Display  software.  See  Presentation  software. 

Displays,  580,  588-590,  594,  892. 
cathode  ray  tube  (CRT),  5-6,  53,  67, 
72-73,  291,  293,  588-589 
data  projectors,  590 

digital  light  processor  (DLP),  590 
halftoning  vs.  dithering,  589 
international  television  standards,  589 
liquid  crystal  (LCD),  589-590 
supertwisted  nematic  (STN),  589 
thin-film  transistor  (TFT),  589 
monitors,  588-589 

Distortion,  39-41,  152. 

and  resolution,  practical,  39-41 

Distributed  Bragg  reflector  (DBR)  diode 
laser,  107. 

Distributed  feedback  (DFB)  diode  laser, 

107,  113. 
ultrafast,  113 

Dithering  vs.  halftoning  display,  589. 

DLP.  See  Digital  light  processor. 

DMSO,  697,  726-727,  760,  875. 
handling,  739 

DNA  damage,  390,  517,  539,  680,  682-684, 
812. 

DNA  probes,  273,  317,  339,  343,  354,  358, 
360,  362,  369,  393,  396,  459,  520, 
531-532,  539-540,  691-695,  774, 
779,  782,  812,  818-825,  828,  874. 
DAPI,  140,  344-345,  355,  358,  376, 

431 

DRAQ5,  343 

Hoechst,  DNA  dye,  136,  339,  344,  360, 
362,  520,  565-566,  683,  782 

DNA  sequencing,  constructs  for,  801-802. 

DNA  transfer,  724,  756,  760,  773,  790, 
802-804. 

Dominant-negative  effects,  755. 

Donor/acceptor  pair  (FRET),  790,  792-794, 
796-797.  See  also,  FRET, 
before  bleach/after  bleach  ratio,  794 
equations,  790,  792-794 
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Donor/acceptor  pair  (FRET)  (cont.) 
fluorescence,  796-797 
fluorophores,  794 
separation  in  nm,  table,  793 

Double-image,  diagram  and  example,  169. 

Double-label,  375. 

Down-conversion,  parametric,  114. 

DPAL.  See  Diode-pumped  alkali  lasers. 

DPL.  See  Diode-pumped  lamp. 

DPSS.  See  Diode-pumped  solid-state  lasers. 

Drift,  386-387,  652,  655,  732. 

CCD  read  amplifier,  76 
compensation  for,  392-393,  732-733, 

886 

focus,  16,  40,  115,  190,  219,  386,  489, 
567,  652,  720,  729,  886 
compensating,  396,  732 
lasers,  85-86,  115 

DRM.  See  Digital  rights  management. 

Drosophila ,  273,  675-676,  747-748, 

751-752,  754,  756,  759,  804,  810. 
living  embryo,  675-676,  752 
salivary  chromosomes,  273 
SPIM,  image,  675-676 

Duty  cycle,  laser,  defined,  110. 

DVD,  587-588. 

Dye  lasers,  86,  103,  112,  114,  124,  540-541 
in  cancer  treatment,  112 
colliding-pulse  mode-locked,  112 
with  intra-cavity  absorbers,  112 
noise  and  drift,  86 
references,  124 
as  wavelength  shifters,  103 

Dye-filling,  studying  micro-cavities, 
173-174. 

Dyes,  22-23,  36,  44,  90-102,  109,  116,  118, 
165,  173.  183,  212,  222,  342-346, 
353-358,  360,  430,  461,  462,  527, 
528,  575,  726,  736-738,  740-745, 
748,  749,  755,  759-760,  774,  775, 
782,  804.  See  also.  Green  fluorescent 
protein  (GFP);  Rhodamine  dyes; 
Fluorescein. 

affect  on  living  cells,  391,  748 
AlexaFluor,  353-355 
Aniline  Blue,  430^132,  435,  438,  774 
APSS  and  Carina  yellow,  non-linearity, 
165 

bandwidth  of  emission,  44 
BODIPY  TR,  methyl  ester,  760 
BOPIDY,  142,  342-343,  353-355,  389, 
692,  749,  760-762 
Calcein  AM,  355,  360,  362-363, 

426-427,  430,  685,  804,  812 
calcium  dyes,  346-347 
cAMP,  347 

characteristics  of  probes/specimen,  table, 
344-345,  354-355 

coumarin,  114,  339,  344-345,  353,  355, 
654-655,  661,  693 
cyanine,  339,  342,  344,  354-355, 

362-363,  374,  443,  540,  587,  760, 
854,  874 


dil  derivatives,  355,  362,  389,  726,  760 
donor  acceptor  pair,  794.  See  also,  FRET 
DNA  probes,  343-344,  531-532, 
818-825,  See  also,  DNA  probes 
DRAQ5,  343 

dyes  vi.  probes,  353,  for  embryos,  748, 
761 

exciting  efficiently,  44 
fade-resistant,  36  See  also.  Antifade; 
Bleaching 

for  fatty  acid,  347.  See  also,  FM4-64, 
below 

Feulgen-stained  DNA,  166,  200,  298, 

433,  437 

Fluo-3  and  Fura  Red,  for  calcium,  180, 
183,  345,  434 
Fluo-3  for  calcium,  737 
fluorescein,  353,  355.  See  Fluorescein; 
FITC 

fluorescence  lifetime,  517,  527-528 
FluoroNanoGold,  854 
FM4-64,  FM1-43.  lipophilic  dyes,  236, 
355,  359-360,  389,  556,  755, 

760-761 

fura-2,  103,  189,  234,  257,  345,  346,  348, 
358-359,  361,  529,  531,  726-727, 
730,  733,  741-743,  810,  812,  846, 
850 

Fura  Red,  180,  183,  345,  454 
future  developments,  348-349 
genetically  expressed,  348 
Glutathione,  342,  358,  545,  694,  779, 

782 

hazards  in  using,  116,  118 
for  ion  concentration,  346-347 
ion-sensitive  probes,  table,  531 
kinetics,  741-742 
lanthanum  chelates,  345-346 
laser/filter  configuration,  table,  799 
lineage  tracers,  461 

lipid  dyes,  236,  355,  359-360,  389,  556, 
755,  760-761 

living  cells,  rapid  assessment,  table,  360 
loading,  uniformity,  749.  See  also. 
Loading 

LysoTracker  Red  DND-99,  360 
membrane  labels,  344-345 
membrane  potential,  205,  346 
microinjection,  360-361,  388,  739,  748, 
755,  795,  803-804 

MitoTracker  Red  ,  142,  170,  353,  358, 
360,  430-431,692,  750 
multi-photon  excitation,  543-544 
nano-crystals,  343,  345.  See  also. 
Quantum  dots 

Nile  Red,  435,  528,  575,  774,  782 
organic,  342-343,  353-356 
oxygen  sensor,  347 
patch  clamp  loading,  360,  726,  734, 
738-740 

pH  indicator,  346,  739-745.  See  also, 
pH 
imaging 


photoactivatable,  187,  210,  224,  383,  385, 
541,  544-545,  693,  729,  759-760, 
912 

Kaede,  187,  383,  385 
Kindling,  574,  760 

PA-GFP,  187,  383,  385,  752,  759-760 
photodestruction,  340-34 1 .  See  also. 
Bleaching 
and  Chapter  39 

photophysical  problems,  338-340 
absorption  spectra,  339 
autofluorescence,  339-340 
contaminating  background,  339-340 
optimal  intensity,  340 
Rayleigh/Raman  scattering,  339 
singlet  state  saturation,  338-339.  See 
saturation,  below 
triplet  state  saturation,  339 
phycobiliproteins,  338,  341,  343, 

355-357,  693 

for  plants,  774—775.  See  Chapters  21  and 
44 

two-photon,  782 

propidium  iodide,  344,  355,  360,  426, 

651,  693-695,  773,  778-779,  782, 
812,  875,  877 

quantum  yield,  172,  180,  184,  338-845, 
347,  353-354,  360,  363,  383,  421, 
543-544,  574,  661,  683,  690-692, 
710,  737,  792,  794-795 
ratio  methods,  346-348,  742-743 
rhodamine,  353,  355.  See  also, 

Rhodamine 
excitation,  109 

saturation  ,  21-22.  41,  142,  222,  265, 

276,  338-340,  448,  643,  647,  899 
Schiff-reagent,  262,  369,  770,  774-775, 
778 

selection  criteria  for,  353-358 
signal  optimization  strategies  for, 

341-342 

SNARF,  345-346,  531,  739,  744-745 
specimen  damage,  340-341 
spectral  properties,  212,  342,  344—345 
spectral  unmixing.  192,  382,  664-667 
for  STED,  table,  575 

Dynamic  Image  Analysis  System  (DIAS), 
396-397,  783-784. 
living  cells  of  rodent  brain,  396 
of  plant  cells,  783-784 

Dynamic  range,  929-930. 

E 

e2v  Technologies,  EM-CCDs,  76-77, 

233-234,  237,  262,  460,  925-926. 

E-CARS.  See  Epi-detected  CARS. 

ECL.  See  Emitter-coupled  logic. 

Edge  detector  (software),  309,  322,  327, 

396,  823-826. 

Edge  effect,  self-shadowing,  172. 

Edge-emitting  diode  laser,  89,  106. 
corrective  optics  for,  89 
cross-section  through,  106 
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Efficiency,  laser.  102,  105-106.  See  also, 
Quantum  efficiency  (QE);  Photon 
efficiency. 

of  diode  injection  lasers,  105-106 
wall-plug,  of  argon-ion  lasers,  102 

EFIC.  See  Episcopic  fluorescence  image 
capture. 

EGS.  See  Ethylene  glycol-Ws-succinimidyl. 

E-h.  See  Electron-hole. 

Electro-magnetic  interference,  in  electro- 
optical  modulators,  57. 

Electron  microscopy.  167. 
brain  slices,  730-731 
chapter,  846-860 
cryo-techniques,  854 
fixation,  167,  368-369 
immuno-stained,  371-372,  852 
micrographs.  479,  847-853,  855-858 
tomography  (EMT),  610-611 

Electron-multiplier  CCD  (EM-CCD),  30-31, 
74-75,  78,  142,  233-235,  262, 
466-467,  482,  647,  678,  737, 
753-754,  784,  923-926. 
advantages  and  disadvantages,  30-31, 

220,  228,  233-235,  237,  459-460, 
647,  737,  909,  923-926 
CIC,  clock-induced  charge,  234,  926 
and  disk-scanners,  76,  205,  215,  220, 

270 

frame-transfer,  262,  234 
gain-register  amplifier,  76-77,  258,  753, 
925 

interline-transfer,  233-234 
mean-variance  curves,  78 
multiplicative  noise,  77 
noise  currents,  256 

parameters,  vs.  normal  CCD.  table,  233 

QE(effective),  78,  927 

readout  amplifier,  76-77,  258,  753-754, 

925 

results,  235,  237,  755 

Electron-beam-scanning  television,  6-7. 

Electron-hole  (e-h)  pairs  and  photon 
counting,  29. 

Electronic  bandwidth,  64-65.  See  also, 
Bandwidth. 

Electronic  noise,  defined,  232. 

Electronik  Laser  Systems  GmbH,  VersaDisc, 
109. 

Electrons,  interaction  with  light,  129-130. 

Electro-optical  modulators  (EOM),  25,  54, 
57.  87,  116,  543,  701,  903-904. 

Electroporation,  359-360,  795,  803. 
for  chromophores,  803 

Ellis,  Gordon,  2,  3,  7,  8,  13,  14,  84.  129, 

131,  478,  507. 

Embryo  imaging.  See  Living  embryo 
imaging. 

Embryos,  761-766. 

bulk  labeling,  with  dyes,  761 
depiction,  in  time  and  space,  762-764 
dyes,  for  multi-wavelength  analysis,  756 
FRET,  764-766 


labeled  proteins,  756 
photobleaching,  759 
transcriptional  reporters,  756 

EM-CCD.  See  Electron-multiplier  CCD. 

Emission  filter.  See  Filters. 

Emission  spectra,  of  arc  sources,  136,  176. 

Emission  spectra,  fluorophores,  1-  V5.  2- 
photon  excitation,  42 1 . 

Emitter-coupled  logic  (ECL),  259. 

EMT.  See  Electron  microscopy  tomography. 

Endomicroscopy,  511,  513,  514. 
distal  tip  for,  514 
fiber-optics,  513 
human  cervix  image,  513 
human  gastrointestinal  track  image,  514 
miniaturized  scanning  confocal,  511 

Endoplasmic  reticulum,  374,  770,  819. 
and  DiOC6,  390 
FLIP,  382 
FRET,  795 

genetic  fluorescent  probes,  771,  783 
and  harmonic  signal  generation,  703 
in  ion-imaging,  738 
and  phototoxicity,  685 
table,  363 

Endpoint  data  analysis,  816-817. 

Endpoint  translocation/redistribution  assays, 
814. 

Energy  diagram,  lasers,  102,  105,  106. 
argon-ion  laser,  102 
helium-cadmium  laser.  1 05 
helium-neon  laser,  105 
semiconductor  laser,  106 
titanium: sapphire  four-level  vibronic 
laser,  109 

Energy,  of  single  photon,  35,  127. 

Energy  transfer  rate,  for  FRET,  790,  792. 

Entrance  aperture.  See  Back-focal  plane. 

EOM.  See  Electro-optical  modulators. 

Epi-detected  CARS  (E-CARS),  597-599. 
erythrocyte  ghosts,  603 

Epi-fluorescence  microscopy.  See 

Fluorescence  microscopy,  44,  166, 
172-173,  195,  202,  235. 

Epi-illuminating  confocal  microscope,  9, 
166.  See  also,  Confocal  laser 
scanning  microscopy;  Confocal 
microscopy. 

Episcopic  fluorescence  image  capture 
(EFIC),  607-608. 
mouse  embryo  image,  608 

Epithelial  cells,  14-15,  603. 

CARS  image,  603 

oral,  optical  sections,  surface  ridges, 
14-15 

EPS.  See  Extracellular  polymeric 
substances. 

Erythrocyte  ghosts,  CARS  imaging,  603. 

Ester-loading  technique.  See  Acetoxymethyl 
esters  loading  method. 

Ethylene  glycol-W.s-succinimidyl  (EGS), 
369. 

Euphorbia  pulcherrima,  spectrum,  710. 


European  Molecular  Biology  Laboratory 
(EMBL),  53,  212. 
compact  confocal  camera,  212 

Evanescent  waves,  90,  177,  180,  245,  503, 
801. 

defined,  90,  180 
optical  fibers,  503 
resolution  measurement,  245 

Excess  light.  See  Stray  light. 

Excimer  lasers,  112,  116. 
maintenance,  116 
for  tissue  ablation.  112 

Excitation  efficiency,  multi-focal  multi¬ 
photon  microscopy,  552. 

Excitation  filter,  requirements,  44.  See  also, 
Filters. 

Excitation  source,  laser.  See  Lasers;  Non¬ 
laser  sources. 

Excitation  wavelength  change,  contrast,  173. 

Explants,  for  imaging  living  embryo, 
748-749. 

Exposure  time,  62,  65,  71-76,  81,  127,  137, 
141-142,  176,  212,  219,  224,  226, 
231-236,  267,  270,  276,  346,  363, 
392-393,  423,  427,  459-460,  477, 
495,  556,  613,  627-628,  651,  655, 
681-686,  692-697,  708,  746-747, 
753-755,  760-764,  783-784,  816, 
822,  850-851,  873. 
for  CCDs  and  EM-CCDs,  127,  137, 
141-142,  231-236,  267 
disk  scanners,  231-235 
laser,  safety,  117-118,  839,  900,  903-904 
reducing,  753-755 
and  source  brightness,  141-142 
total,  comparison  of  methods,  442,  449 
UV,  116 
x-ray,  614-616 

External  laser  optics,  maintenance,  117. 

External  photoeffect.  See  Photoemissive 
effect. 

External  Pockels  cell,  25,  54,  87,  116,  543, 
701,  903-904. 

External-beam  prism  method,  laser  control, 
90. 

Extracellular  polymeric  substances  (EPS), 
183,  311,  358,  376,  703-704,  717, 
760,  783,  870,  879-880.  See  also. 
Collagen, 
bleaching,  693 
damage,  685 
dye,  361 

lectin-binding  in  biofilms,  870,  879-880 

matrix,  760 

negative  contrast,  173 

in  optical  projection  tomography,  612 

plants,  438,  783 

preparation,  376 

Extrinsic  noise,  reduction,  21. 

F 

Fabry-Perot  interferometer,  optical  cavity, 
81-82. 
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Fast  Fourier  transform,  487. 

to  identify  interference  fringes,  202 

Fast  line  scanner,  231-232. 

Fatty  acid  indicator,  347. 

FBG.  See  Fiber  Bragg  Grating. 

FBR.  See  Fiber  Bragg  Reflector. 

FBTC.  See  Fused  biconical  taper  couplers. 

F-CARS.  See  Forward-detected  CARS. 

FCS.  See  Fluorescence  correlation 
spectroscopy. 

Feedback,  136,  139. 

for  control  of  light-emitting  diode,  139 
to  increase  source  stability,  136 

Femtosecond  pulsed  lasers.  See  Ultrafast 
lasers. 

Feulgen-staining,  DNA,  166,  200,  298,  433, 
437 

Fianium-New  Optics,  Ltd.,  FemtoMaster- 
1060 

fiber  laser,  113-114. 

Fiber  Bragg  Grating  (FBG),  laser 
stabilization,  87. 

Fiber  Bragg  Reflector  (FBR),  stabilizes 
laser,  87. 

Fiber  lasers,  85,  101,  109-110,  113-114, 
124. 

defined,  109-110 
temperature  sensitivity,  85 
tutorial  reference,  124 
ultrafast,  101,  113-114 

Fiber  optics.  See  Chapter  26. 
beam-splitters,  503-504 
Bow-tie,  pol-preserving  fiber,  503 
cable,  for  delivering  ultrafast  pulses,  88 
laser  output,  106 
pigtail,  106 

Fiber  optics  used  in  microscopy,  501-507. 
evanescent  waves  in  optical  fibers,  503 
fiber  image  transfer  bundles,  504-505 
fiber-optic  beam-splitters,  503-504 
fused  biconical  taper  couplers,  503-504 
glass  made  from  gas,  501 
gradient-index  optical  fibers,  501-502 
key  functions  of  fibers,  505-507 
delivering  light,  505-506 
detection  aperture,  506 
diffuse  illumination,  507 
for  femtosecond  laser  pulses,  507 
large-area  detection,  507 
large-core  fibers,  as  source/detection 
apertures,  507 

same  fiber  for  source  and  detection, 
506 

single-mode  fiber  launch,  505 
SMPP  optical  arrangement,  216 
managing  insertion  losses,  506 
angle  polishing  of  fiber  tips,  506 
anti-reflection  coating  of  fiber  tips,  506 
index  matching  of  fiber  tips,  506 
microstructure  fibers,  504 
modes  in  optical  fibers,  502 
polarization  effects  in  optical  fibers,  503 
polarization-maintaining  fibers,  503 


step-index  vs.  gradient  index,  502 
step-index  optical  fibers,  501-502 
transmission  losses  in  silica  glass,  502 

Fiber-optic  confocal  microscopy,  501-515, 
893. 

benchtop  scanning  microscopes,  507-508 
clinical  endomicroscopy,  513 
distal  tip,  514 
human  cervix  image,  513 
human  gastrointestinal  track  image, 

514 

image  transfer  bundles,  504-505 
managing  insertion  losses,  506 
miniaturized  scanning  confocal,  508-512 
bundle  imagers  for  in  vivo  studies,  509 
with  coherent  imaging  bundles, 
508-509 

imaging  heads,  508-512 
objective  lens  systems,  509 
optical  efficiency,  509 
optical  schema,  508 
resolution,  509 
rigid  endoscope,  511 
vibrating  lens  and  fiber,  510-511 
in  vivo  imaging  in  animals,  510-514 

Fiber-optic  interferometer,  240-241,  504, 
609. 

diagram,  241 

for  measuring  point  spread  function, 
240-241 

Fiber-optic  light  scrambler,  8,  13,  131-132, 
143. 

Fibroblasts,  292,  361,  691,  798,  803,  852. 

Field  diaphragm,  34-35,  127-128,  139,  461, 
627,  648-649. 

Field  effect  transistor  (FET)  CCD  amplifier, 
30-31,  77,  922-927,  929. 
noise  vs.  pixel  dwell  time,  922 

Filament-based  lamps,  34,  44,  126-132, 
135-138,  346,  507,  648,  663. 
fiber  optic,  507 

image,  100  W  halogen  bulb,  135 
size,  126-127 
spectrum,  44,  136 
stability,  34,  137 

File  formats,  multi-dimensional  images, 
288-289. 

Fill  factor. 

of  CCD,  920-921,  927,  929 
disk-scanning  microscopes,  224-228,  233, 
552 

Filtering,  digital,  281,  810.  See  also, 
Deconvolution. 

Gaussian,  41,  65.  See  also,  Gaussian 
filters 

multi-dimensional  microscopy  display, 

281 

nonlinear,  deconvolution,  190 
sets,  for  automated  confocal  imaging,  810 
smoothing,  effect  on  contrast,  59 
to  reduce  “noise”  features,  70 

Filters,  optical,  43-51,  70,  89,  162,  190, 

212,  753.  See  also,  Fleat  filters. 


conventional,  45 
hard  vs.  soft  coatings,  45-49 
intensity  loss,  212 
interference,  45-51 

conventional  and  hard  coatings,  46 
multi-channel  detection,  51 
ND  filters,  43,  89 
notch  and  edge,  50 

tuning  with  angular  dependence,  50 
to  select  image  contrast  features,  162 
short-pass,  interference  type,  46 
transmission  vs.  laser  line,  212 
types,  46 

wavelength  selective,  43-51 
FiRender,  281-282. 

First  or  front  intensity,  projection  rule,  302, 
304. 

FITC.  See  Fluorescein  isothiocyanate. 
Fixation,  specimen,  368,  378,  428,  852,  854, 
856. 

antibody  screening  with  glutaraldehyde 
fix,  377 

artifacts,  195,  369-373,  428,  624,  815, 
854,  857 

autofluorescence,  358,  663 
borohydride  to  reduce  autofluorescence, 
374,  770 
chapter  368-378 
characteristics,  368-370 
chemical  fixatives,  369 
crosslinking  fixatives,  369 
freeze  substitution,  369,  769,  854-856 
microwave  fixation,  369 
protein  coagulation,  369 
cryo-fixation,  854 

dehydration,  166,  368,  417^418,  481, 

611,  623-624,  815,  849,  854-855 
effect  on  plants,  428 
for  electron  microscopy,  167,  368-369, 
372,  479,  731,  851-860 
ethylene  glycol-Ws-succinimidyl,  369 
evaluation,  371-374 

cell  height  to  measure  shrinkage, 
371-373 

MDCK  cell  example,  372,  373 
formaldehyde,  369-370,  373 
general  notes,  374-378 
geometrical  distortion,  372-373,  815 
GFP,  854,  See  also.  Green  fluorescent 
protein 

arsenical  derivatives,  348 
glutaraldehyde,  369,  370 
high-content  screening,  815 
immunofluorescence  staining,  371,  372, 
852 

improper  mounting,  376 
microwave,  377-378 
mounting  methods,  370-374 
critical  evaluation,  371-374 
media  refractive  index,  table,  377 
technique,  371 

optical  properties  of  plants,  428 
pFl  shift/formaldehyde,  370-371,  373 
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plants.  See  also.  Botanical  specimens. 
Plant  cells,  428,  769-770, 

773-774 

refractive  index  of  mounting  media,  table, 
377 

optical  effects,  428 

refractive  index  of  tissue/organs,  table, 
377 

shrinkage,  369-373,  624,  815,  854 
staining,  370-371 
tissue  preparation,  376 

Fixed  wavelength  lasers,  table,  119-120. 

Fixed-pattern  noise,  74—76,  278,  924,  927, 
931. 

Flare,  out-of-focus  light.  6,  132,  157-158, 
172,  395.  456,  465-466,  469,  471, 
481,  649,  731. 

Flatness  of  field,  145,  151,  154,  418,  457, 
639. 

measurement/  small  pinholes,  145,  457, 
639 

objectives,  to  improve,  151-152 

Flat-fielding  CCD  data,  76,  477. 
black  reference,  76 
white-reference,  76 

Flexible  scanning,  51-52. 

FLIM.  See  Fluorescence  lifetime  imaging 
microscopy. 

FLIP.  See  Fluorescence  loss  in 
photobleaching. 

Flip  mirrors,  to  control  laser,  58. 

Floppy  disks,  586. 

Fluorescein.48,  80-81,  88,  203,  261, 

353-355,  375,  443,  582,  697,  781, 
794,  930. 

arsenical  derivatives,  348 
calculating  laser  power  needed,  80-81, 
443 

derivatization,  diagram,  354 
double-labeling.  375 
filters  for,  48 

photobleaching  quantum  yield,  363 
rhodamine  and,  FRET  between,  794 

Fluorescein  isothiocyanate,  88,  198,  203, 
261,  263,  335,  375,  394,  397-398, 
511-512,  527-528,  582-583, 
693-694,  781,  794,  799,  884,  885. 
See  also.  Fluorescein. 

2-photon,  781 

biofilms,  884-885 

dextran,  292,  512 

filter  sets,  48-49 

FRET,  794,  799 

lifetime,  527-528,  532 

photobleaching  quantum  yield,  363 

toxicity,  391,  693-694 

Fluorescence  anisotropy  measurements,  742. 

Fluorescence  contrast,  172-173. 

Fluorescence  correlation  spectroscopy 

(FCS),  5,  363,  383,  385,  602,  801, 
803,  805,  917. 
and  CARS.  602 
FRET,  801 


laser  requirements,  81 
table.  385 

Fluorescence  emission,  botanical  specimens, 
425-428. 

1-  vs.  2-photon  excitation,  421 
Fluorescence  imaging,  deconvolution  vi. 

confocal,  459-460,  644-648. 
Fluorescence  in  situ  hybridization  (FISH), 
316-317,  319,  323,  331,  333-334, 
343,  875-878. 
biofilms  stains,  875-878 
with  fluorescent  protein,  878 
Fluorescence  ion  measurement,  736-738, 

740-745.  See  also.  Calcium  imaging, 
pH,  etc. 

calcium  imaging,  736-737 
concentration  calibration,  742-745 
indicator  choice,  738 
interpretation,  740-741 
pH  imaging,  346.  739-745 
water-immersion  objectives,  737 
Fluorescence  lifetime  imaging  microscopy 
(FLIM),  108,  111,  114.  139,  204, 

233,  382-383,  385,  516-533, 
799-801. 

advantages,  766.  800 
alternatives  to,  766 
analysis,  251 

applications,  516-518,  527-532 
calcium  imaging,  529 
chemical  environment  probe,  517 
FRET,  517-518 

ion  concentration,  517,  528-530 
multi-labeling  with  dyes,  517,  527-528 
pH  imaging,  529-530 
probes,  517 
table,  530-532 

comparison  of  methods,  523-527 
acquisition  time,  525-526 
bleaching,  524 
cost,  526 

detector  properties,  526-527 
multi-exponential  lifetime,  523-524 
photon  economy,  524-525 
pile-up  effect  on  detection  efficiency, 
526 

shortest  lifetime,  523 
table,  526 

decay  process  of  excited  molecule,  table, 
518 

frequency  domain  methods,  518-520 
disk-scanning  implementations,  520 
phase  fluorometry  method,  518-519 
point-scanning  implementations,  520 
widefield,  spinning-disk.  519-520 
frequency-domain.  108 

reducing  repetition  rate,  111 
FRET.  799-801 
history,  516 

Jablonski  diagram,  516,  517,  697,  792 
with  light-emitting  diode  sources,  139 
limitations,  800 
living  cell  images,  204 


methods,  518-527 
comparison,  523-527 
frequency  domain,  518-520 
time  domain,  520-523 
multi-focal  multi-photon  microscopy, 
555-556 

quantitative  fluorescence,  517-518 
quantum  efficiency,  516 
spectroscopy,  516 
table,  385 

time  domain  detection  methods,  520-523 
point-scanning,  522 
streak  camera,  520 
TCSPC  FLIM,  522-523 
time-gated  FLIM.  523 
use  of  intensified  CCDs  for,  233 
Fluorescence  loss  in  photobleaching  (FLIP), 
187,  382,  384,  801. 

FRET,  affected  by,  801 
table,  384 

Fluorescence  microscopy,  4,  9.  13,  43^14, 
154,  166,  172-173,  195,  202,  235, 
251,  448-451,  809-810  See  also, 
Widefield  (WF)  fluorescence 
microscopy. 

chromatic  correction,  154 
compared  to  disk-scanning  microscopes, 
235 

vs.  confocal  imaging,  13 
depth  of  field.  4 

filters  for  selecting  wavelengths  for, 
43-44 

folded  optical  path,  166 
increase  contrast  with  less  intensity, 
172-173 

signal-to-noise  ratio  comparative, 

448-451 

bleaching-limited  performance, 
448-450 

configurations  of  microscope,  448,  449 
disk-scanning  microscope,  449 
line  illumination  microscope,  449 
saturation-limited  performance,  450 
scanning  speed  effects,  450-45 1 
S/N  ratios,  table,  450 
wide  field  (WF)  microscope.  450 
spectral  problems,  44 
Fluorescence,  quenched  by  colloidal  gold, 
854. 

Fluorescence  recovery  after  photobleaching 
(FRAP).  51,  54,  56,  90,  187,  210, 
218,  224,  229,  237,  362,  382,  384, 
390,  691,  759,  801,  805,  850. 
in  biofilms,  874 

damage  to  cellular  structure,  341, 

859-851 

damage  to  microtubules,  341,  850-851 
efficiency  of  illumination  light  path,  210 
related  to  TEM  of  same  specimen, 
850-851 

setups  for,  218,  907 
table,  384 

using  CARV2  disk-scanner,  229,  907 
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Fluorescence  resonance  energy  transfer 

(FRET),  26-28,  34,  184-187,  204, 
218,  221-222,  382,  384,  425, 
517-518,  556,  650,  691,  741-742, 
764-766,  788-806,  796-797. 
based  on  protein-protein  interactions,  800 
based  sensors,  798-799 
botanical  specimens,  425 
C.  elegans ,  766 
chapter,  778-806 

cloning  and  expression  of  fluorescent 
constructs  for,  801-804 
donor/acceptor  pair,  790,  792-794 
donor,  796-797 
efficiency,  792 

experimental  preparation,  795 
FCS  and,  801 
FLIM  and,  799-801 

between  fluorescein  and  rhodamine,  794 
fluorescence  lifetime  imaging,  517-518 
fluorescent  proteins,  794-795 
FRAP  and,  801.  See  also.  Fluorescence 
recovery  after  photobleaching  future 
perspectives,  805 

induced  by  cholera  toxin  transport,  797 
intramolecular,  765 
kinetics,  741-742 
in  living  cells,  195-186,  204 
chapter,  788-806 
in  living  embryos,  764-766 
MMM,  797-798 
nanobioscopy  of  protein-protein 
interactions 

acceptor  bleach  for,  797-798 
donor  fluorescence  for,  796-797 
measurement  methods  for,  795 
sensitized  emission  of  acceptor, 
795-796 

photobleaching,  691 
practical  measurements,  792 
probes,  221-222 
quantum  dots,  801 
setups,  218 

small  molecules,  794-795 
spatial  orientation  factor,  792-793 
spectrofluorimetry,  793 
spectroscopic  properties  used  for,  795 
standards  for,  34 
table,  384 
theory,  790-794 
TIRF  and,  801 

total,  measured  with  widelield,  26-28 
in  transgenic  animals,  765 
wavelength  depiction,  793 
Fluorescence  saturation,  singlet-state,  21-22, 
41,  142,  265,  276,  339,  448,  643, 
647,  899. 

Fluorescence  speckle  microscopy  (FSM). 

13,  383,  385,  889. 
table,  385 

Fluorescent  biosensor,  799,  805. 
future.  805 

mitotic  clock  measurements,  799 


Fluorescent  constructs  for  FRET,  801-802. 
cloning  of  fluorescent  chimeras,  801-802 
expression  and  over-expression,  802 
functional  activity  of  expressed,  802 

Fluorescent  dyes.  See  Dyes;  Fluorescent 
indicators;  Fluorescent  probes. 

Fluorescent  efficiency,  34. 

Fluorescent  emission,  incoherence,  130. 

Fluorescent  indicators,  346-348,  736-743. 
See  also.  Fluorescent  probes,  and 
particular  ions, 
binding  equation,  740-741 
buffering,  740 
calcium  imaging,  736-737 
calibration,  742-743 
indicators,  738 

cellular  introduction,  738-739.  See  also. 
Loading 

cellular  trapping,  738 
choice,  738 

concentration,  741-742 
dialysis,  740 
free  diffusion,  741 

genetically  expressed  intracellular,  348 
green  fluorescent  protein,  348 
ion  indicators,  348 
ligand-binding  modules,  348 
handling,  739-740 
inaccurate  measurements,  740-741 
intracellular  parameters  imaged,  346-348 
Ca2+,  346-347 
cAMP,  347 
fatty  acid,  347 
ion  concentrations,  346-347 
membrane  potentials,  346 
other  ratioing  forms,  347-348 
oxygen,  347 
pH,  346,  739-745 
wavelength  ratioing,  346 
positive  pressure,  740 
selectivity,  743 

Fluorescent  intensity  (IF),  TIRF,  180. 

Fluorescent  labels,  342-346,  530-532,  761, 
775.  See  also.  Dyes;  Fluorescent 
probes;  Chapters  16-17,  and  by 
name  of  dye. 

Fluorescent  probes.  353-364,  387-389,  517, 
530-532,  736-737,  739-740,  755, 
769,  771,  773,  783,  806,  810,  811. 
See  also.  Dyes,  Fluorescence 
indicators  and  by  name  of  dye. 
Chapters  16,  17. 
automatic  living  cell  assays,  811 
bound,  737 
care,  739-740 

characteristics,  table,  344-345,  354 
development,  736 
dye  criteria  for,  353-358 
AlexaFluor  dyes,  353-355 
BOPIDY  dyes,  353-355,  749,  760-762 
coumarin  dyes,  353,  355 
cyanine  dyes,  353,  374,  587,  760,  854, 
874 


dye  classes,  table,  355 
dye  vs.  probes,  353 
fluorescein,  353,  355.  See  also. 
Fluorescein  fluorescent  proteins, 
355-357 

GFP,  355-357.  See,  Green  fluorescent 
protein  indicators  of  intracellular 
sate,  346-348 
Ca2+  indicators,  346-347 
protein 

multi-photon  excitation,  357-358 
phycobiliproteins,  355-357 
probes/specimen  characteristics,  table, 
354 

quantum  dots,  357 
rhodamine,  342-345.  See  also, 
Rhodamine 

excitation,  737,  344-345 
for  fluorescence  lifetime  imaging,  517, 
530-532 

genetically  encoded,  for  plant  imaging, 
769,  771,  773,  783.  See  also. 
Transcriptional  reporters; 
Transfection  agents  for  high-content 
screening,  810 

high  specificity/high  sensitivity,  806 
living  cell  imaging,  387-389 
rapid  assessment  by,  table,  360 
loading  methods,  358-360.  See  also. 
Loading 

acetoxymethyl  esters,  359 
ATP-gated  cation  channels,  359 
ballistic  microprojectile  delivery,  360, 
724-725,  802-803 
direct  permeability,  358-359 
electroporation,  359-360,  795,  803 
microinjection,  360-361,  388,  739, 

748,  755,  795,  803-804 
osmotic  permeabilization,  359 
peptide-mediated  uptake,  359 
transient  permeabilization,  359 
whole-cell  patch  pipet  delivery,  360, 
726-727,  734,  738-740 
photoactivatable,  210,  224,  383,  385,  541, 
544-545,  693,  759-760,  912 
Kaede,  187,  383,  385 
Kindling,  574,  760 

PA-GFP,  187,  383,  385,  752,  759-760 
photobleaching,  362-363.  See  also. 
Bleaching 

phototoxicity,  363-364  See  also. 

Phototoxicity  factors  influencing, 
table,  363 

specimen  interactions,  361-362 
cytotoxicity,  362 
localization,  361-362 
metabolism,  361-362 
perturbation,  362 

target  abundance/autofluorescence, 
360-361 
tissues,  360 

Fluorescent  proteins,  187,  355-357,  739, 
794-795. 
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emission  change  after  photodamage, 

187 

FRET,  794-795 

genetically  engineered  variants,  739 
ion  binding  regions,  739 

Fluorescent  lights,  stray  signal,  201,  632, 
904. 

Fluorescent  staining,  371,  393,  438,  774. 

See  also ,  Dyes;  Staining, 
immunofluorescence,  371,  372,  852 
living  cells,  393 
microglia,  319-320,  393-398 
nuclei  of  living  or  dead  cells,  393 

Fluorite  (CaF2),  optical  to  reduce  chromatic 
aberration,  153. 

FluoroNanoGold,  cryosections,  854. 

Fluorophores,  44,  338-349,  543-544, 

664-667,  748,  794,  799.  See  also. 
Dyes,  Fluorescent  labels. 

Flying  spot  detector  for  measuring  photon 
efficiency,  34—35. 

Flying  spot  ultraviolet  (UV)  microscope, 
6-7. 

Fly’s-eye  lenses,  for  diode  lasers,  107-108. 

FM4-64,  FM1-43,  and  other  lipophilic 

membrane  dyes,  236,  355,  359,  360, 
389,  556,  775,  760-761. 

Focal  Check™  beads,  657-659. 

Focal-plane  array  detection,  2-photon, 

542. 

Focal  shift  for  mismatched  RI,  405, 
407-410,  553. 
defined,  405 
dependence,  410 
for  glycerol,  table,  409 
for  water,  table,  409 

Focus,  3-4,  13,  36,  197. 
for  confocal  microscope,  36 
displacement,  by  living  cell  specimen, 
22-23 

effect  of  coverslip,  197 
extended,  9 

in  phase-dependent  imaging,  13-14 
planes,  diagram,  27 

position,  confocal  microscopy,  65 1-652 

Focused  spot.  See  Point  spread  function. 

Folded  optics,  for  trans-illuminated  confocal 
microscopy,  166. 

Formaldehyde,  369-370,  373-377,  428, 

738. 

AM-loading  releases  formaldehyde, 

738 

fixation  protocol,  37 1 
permeabilization  agents  for,  375 
pH  shift  method,  370-371,  373 
for  plants,  428 
stock  solutions,  370-37 1 

Forster  distance,  defined,  184,  790,  792, 

793. 

Forster  equation,  184,  790,  793. 

Forster  resonance  energy  transfer.  See  also, 
Fluorescence  resonance  energy 
transfer. 


Forsterite  laser  (CrJ  in  MgSi04),  109, 

114,  415,  541,  706,  707-709, 
712-713. 

second/third  harmonic  generation,  114 
tunable,  109 

Forward-detected  CARS(F-CARS), 

597-599,  603. 
erythrocyte  ghosts,  603 
Foundations  of  confocal  LM,  chapter,  1-19. 
Four-dimensional  images,  746-749,  752, 
761-764. 

advantageous  techniques,  746-747 
automatic  image  analysis,  321 
deconvolution,  495 

embryogenesis  visualization  strategies, 
761-764 
living  cells,  393 
of  living  embryos 

cellular  viability,  747-748 
challenges,  762 

dataset  display  strategies,  393,  763-764 
deconvolution,  752 
for  large  thick  specimen,  746-747 
photobleaching  during,  747-748 
photodamage  during,  746 
required  datasets  for,  746-747 
multi-photon,  535 
structured  illumination,  482 
SPIM,  676 
Fourier  analysis. 

4Pi  microscope,  563,  576 

analogy  with  image  reconstruction,  69 

of  blind  deconvolution,  472-476,  478 

and  convolution,  485-487 

of  image  formation,  446,  454,  456-457 

MRM,  618-620 

of  periodic  test  specimen,  638-639 
of  short  laser  pulses,  88,  728 
SPIM  multiview  processing,  675-677 
STED,  574 

of  structured-illumination  images,  268, 

270-273 

and  wavelet  processing,  734 
Fourier  plane.  See  Back-focal  plane,  201, 
245,  509. 

Fourier  space,  270-27 1 . 

Fourier  transform,  201,  202,  271,  487,  489, 
490^192,  620. 

of  AC  interference  in  image,  201-202, 

651 

and  convolution,  487 

and  deconvolution,  487,  490-492 

for  detecting  stray  light  into  detector, 

201 

identifying  interference  fringes,  202 
of  microtubule  TIRF  image,  183 
missing  cone  problem,  494 
MRM  image  formation,  620 
of  point  spread  function,  489,  490 
Fractal  compression,  581-582. 

Frame  rate.  See  also ,  Speed 
in  confocal  microscopy,  1 1 
matching,  838-839 


FRAP.  See  Fluorescence  recovery  after 
photobleaching. 

Free  diffusion,  of  fluorescent  indicators, 

741. 

Free-ion  concentration,  742. 

Freeze  thawing,  731,  739. 

Frequency,  52,  65,  82. 

laser  vs.  pumping  power,  82 
of  resonant  galvanometer,  52 
of  sampling  clock,  64 

Frequency  doubling.  See  Second  harmonic 
generation. 

Frequency-resolved  optical  gating  (FROG) 
for  pulse  length  measurement,  115. 

FRET.  See  Fluorescence  resonance  energy 
transfer. 

Frustrated  total  internal  reflection,  defined, 
177. 

FSM.  See  Fluorescence  speckle  microscopy. 

Full-well  of  CCD  pixel,  defined,  75. 

Full-width  half  maximum  (FWHM) 
resolution. 

4Pi,  562,  567 

of  beams  in  scanning  disk,  554 
of  CARS,  597,  599 
of  confocal  performance,  656-657, 
661-662 

of  emission  wavelength 
LED,  136 
quantum  dots,  343 
of  interference  filters,  44 
laser  bandwidth,  93,  95,  100.  101 
laser  pulse  length,  109,  112,  507,  537, 
538,  902 

micro-surgery  precision,  219,  687 
multi-photon,  682-683,  901-902 
objective  resolution  (PSF),  149,  209,  225, 
444-445,  456,  492,  509,  552,  571 
PMT  rise  time,  225 
resolution,  with  spherical  aberration, 

407 

table,  409 
SPIM,  675 
STED.  572,  576-578 
z-resolution,  measured,  194 

Fundamental  limits,  chapter,  20-42. 

Fungi,  438^139,  624,  782,  870. 

Fura-2  [calcium  ion]  indicator  dye,  103, 

189,  234,  257,  345,  346.  348, 
358-359,  361,  529,  531,  726-727, 
730,  733,  741-743,  810,  812,  846, 
850. 

Fused  bi-conical  taper  couplers  (FBTC), 
503-504. 

Future,  143-144,  160,  192,  219-220,  234. 
of  EM-CCD  with  interline  transfer,  234 
of  laser-scanning  confocal  microscopes, 

219 

of  non-laser  light  sources,  143-144 
spherical-aberration  corrector,  15,  147, 
151,  192 

of  tunable  objective,  1 60 

FWHM.  See  Full-width  half  maximum. 
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Gain,  31,  232. 

of  image  intensifies  232 
photomultiplier  tube,  front  collisions  at 
first 

dynode,  diagram,  31 
Gain  register,  (EM-CCD)  76-78, 

233-234. 

CCD  (CCD),  76-78 
of  electron  multiplier-CCD,  233-234 
Gain  setting,  75,  115. 
defined,  75 

effect  of  bandwidth  on,  115 
GAL4  genes,  773. 

Gallium  arsenide  (GaAs). 
diode  laser,  107,  111 
InGaAs  photodiode,  707-708 
LEDs,  133,  138.  143 
PMT  photocathode,  4,  28-29,  232,  252, 
255,  263,  464,  527,  931 
Galvanometer,  11,  25,  36,  40,  51-54,  56,  57, 
63,  211,  215,  223,  231-232,  513, 

543,  552,  558,  599,  651-652,  753, 
806,  907,  910-911,  914,  931.  See 
also.  Linear  galvanometers, 
defined,  52-54 
distortion,  211 

electromechanical  properties,  40 
errors,  40 

in  fiber-optic  micro-confocal,  513 
figure,  63 

line-scanner,  231-232 
linear,  52,  53,  223 
measurement,  651-656 
multi-focal,  554 
multi-photon,  543 

resonant,  25,  52-54,  56-57,  223,  447, 

510,  539,  543,  552,  558,  910 
specifications  for,  214,  543 
ultra-precise,  211 

x-y  scanners,  213-215,  223,  651-654, 

806,  907,  910-911,  914 
Gamma,  brightness  non-linearity,  72-73, 

287,  832-833. 
data  projector,  590 
display,  582-583,  589,  832-833 
Gas  lasers,  86,  90-105.  See  also,  CW  lasers; 
Pulsed  lasers, 
continuous  wave,  90-105 
maintenance,  116 
noise  sources,  86 
pressure,  102 

Gating,  intensified  CCD,  25,  233,  262,  522, 
555. 

Gaussian  beam  profile,  lasers,  80-81,  83-84, 
108-109,  111,  113,  116,  231,  269, 
338,  456,  496,  502,  538-539,  554, 
891. 

in  CARS,  597 

converted  into  line,  231,  916 

fiber  optic,  502,  505,  506 

filling  back-focal  plane,  210,  509,  629, 

633 


“Gaussian-to-flat-wavefront”  converter, 
554 

Kerr  effect  produces  self-focusing.  111 
laser  beam  profile,  538-539,  554,  597, 
635-636 

noise,  473,  497,  925 
from  optical  fiber,  502,  505-506 
optical  tweezers,  89.  See  also.  Laser 
trapping  spatial  filter,  89,  729 

Gaussian  filters,  digital,  39,  41,  65,  70,  89, 
281,  285,  301,  323,  338,  391-392, 
399,  497,  499,  510,  650,  667-668, 
676,  729,  734,  753,  764,  830. 
of  3D  data  to  reduce  Poisson  noise,  39, 
41,  65,  69-70,  269,  281,  285,  323, 
391-392,  399,  499,  510,  635-636, 
650,  667-668,  676,  764,  830 
“Gaussian  blob,”  635-636 
and  Nyquist  reconstruction,  65 
in  presentation  displays,  830 
results,  285,  676,  733,  835-837 

Gaussian  laser  pulses,  536-536,  902. 

Gaussian  noise,  473,  497,  925. 

Gaussian  norm  statistical  tests,  830,  835, 
837. 

GDD.  See  Group  delay  dispersion. 

Gene  gun,  360,  724-724,  730. 

Geometric  contrast,  180-187. 

Geometric  distortion,  6,  23,  36,  39-41,  53, 
152,  211,  215-216,  265,  297,  329, 
372-373,  448,  480,  590,  641, 
653-654,  741,  835. 
kinetic,  741 
measurement,  651-656 
projector,  590 

of  specimen  preparation,  372-373,  815, 
872 

Gerchberg-Saxton  algorithm,  deconvolution, 
472. 

GFP  See  Green  fluorescent  protein. 

Ghost  images,  from  transmission 
illuminator,  201-202. 

GIF  (Graphics  interchange  format),  580. 

Gires-Tournois  interferometer  (GTI),  to 
reduce  GVD,  88. 

Glan-Taylor  polarizer,  85,  87,  100,  171. 
in  single-sided  confocal  microscope, 

171 

Glan-Thompson  polarizer,  attenuator,  85, 
904. 

Glutaraldehyde,  fixative,  369,  369-374, 
377-378,  428,  438,  731,  852. 
antibody  screening  with,  377 
autofluorescence  of,  374,  428,  770 
fixation  protocol,  370 
stock  solutions  for,  370 

Glutathione  (GSH),  342,  358,  545,  694,  779. 
visualization,  in  plant  cells,  782 

Glycerol,  immersion/mounting  medium, 

404,  407,  409-410,  435,  563,  654, 
698,  785. 
clearing,  198,  200 
diffusion  in,  698 


immersion  objective  lenses,  412,  563,  567 
example,  785 

mounting  media,  371,  373,  375,  377-378, 
420 

Rl-mismatch,  table,  409,  410 

Goggles,  laser,  for  eye  protection,  118. 

Gold’s  ratio  method,  476. 

Golgi  receptor,  374,  389,  556,  564-566, 

791. 

Golgi  stain,  107,  283,  298. 

Gourard  shading,  308,  309,  311. 

Gouy  phase  shift,  597. 

Graded  index  (GRIN)  lenses,  84. 
in  diode  lasers,  108 

Gradient  index  optical  fibers,  501-502. 

Gradient-weighted  distance  transform, 

323. 

Graphics  interchange  format.  See  GIF. 

Grating,  periodic. 

GVD  compensator,  88,  504,  538,  686 
laser  tuning,  90,  103,  106-107,  111 
minimum  spacing,  1,  16,  652 
OCT  phase-delay,  609 
pulse  compressor,  113 
spectral  detector,  87,  346,  422,  664, 

772 

structured  illumination,  266-267,  273 

Gray  levels,  71-76. 

intensity  spread  function,  74-76 
printer,  592 

Green  fluorescent  protein  (GFP),  90,  174, 
221-222,  348,  355-357,  429, 
478-479,  556,  568,  571,  612,  614, 
625,  675-676,  690,  692,  698-699, 
724-725,  727,  731,  741,  747-752, 
755,  756-763,  766,  769-773, 
781-785,  798-806,  812-815,  820, 
854-859,  862,  873-875,  877-879, 
885.  See  also.  Transfection  reagents; 
Transcriptional  reporters, 
biofilms  labeling,  873 
or  CFP  molecules,  as  FRET  pair,  798 
constructs,  in  embryos,  756 
EM  imaging,  brain  cells,  731,  854-859 
FRET,  793-795,  798-803 
image  contrast,  174 
limitations,  760 
membrane  localized,  749 
methods  with  Correlative  LM/EM,  854 
mice,  727 

photoactivatable,  187,  383,  385,  752, 
759-760 

photobleaching,  690,  692,  698 
for  plant  imaging,  424,  429-430, 

769-773,  781-785 
direct  visualization,  773 
genetic  fusions,  773,  783 
genetic  marking,  773 
two-photon  excitation,  782-783 
protein  fusions/cytoskeleton,  773-774, 

801 

tagged  proteins,  758 
TIRF,  90 
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FRET,  794 

Grey  levels,  71-76. 
printer,  592 

GRIN.  See  Graded  index. 

Ground  state  depletion  (GSD),  573. 

Group  delay  dispersion  (GDD),  537-538, 
543. 

Group  velocity  dispersion  (GVD),  88.  Ill, 
210,  537,  606,  609,  903. 
in  optical  coherence  tomography,  609 
pulse  broadening  due  to,  88,  111,  210, 
537-538,  543,  606,  609,  728, 

903 

GSD.  See  Ground  state  depletion. 

GTI.  See  Gires-Tournois  interferometer. 

Guinea-pig  bladder,  calcium  sparks,  image, 
237. 

GVD.  See  Group  velocity  dispersion. 

Gzip,  580. 

H 

Hairs,  plant,  431,  434-436,  772. 

Halftoning  vs.  dithering,  589. 

Halogen  lamps,  126-127,  132,  136-139, 
143,  159,  663. 

brightness  vs.  temperature,  136 
filaments,  132 
image,  135 
lifespan,  136 

power  available,  126-127 
stability  plot,  137 

Haralick  features,  818-820. 

Hard  coatings,  for  interference  filters,  45, 
48. 

Hard  copy,  580,  590-594. 
photographic  systems  for,  590-591 
printers,  591-593 
aliasing,  592 
color  images,  592 
digital,  591-593 
grey  levels,  592 
inkjet,  593 
laser,  593 
posterizing,  591 
scaling  techniques,  592 

Harmonic  signals,  2,  49,  80,  90,  100,  109, 
113-114.  162-163,  174,  179-180, 
188,  243,  361,  414,  428,  535,  545, 
550,  556,  577,  596-597,  682, 
703-704,  708-719,  722,  729,  734, 
894  See  also.  Second  harmonic 
generation;  Third  harmonic 
generation  see  Structured 
illumination, 
chapter,  703-721 
contrast,  179-180,  188 
descanned  detection,  56 
in  lasers,  109,  113,  114,  115 
plants,  428 

second  and  higher,  114 

Haze,  from  out-of-focus  light,  227. 

HBO-50  mercury-arc  bulb,  126. 

HCS.  See  High  content  screening. 


Heat,  84-85,  89-90,  109,  129,  133. 
filtering,  dichroic  filters,  43^14,  129, 

132 

heat  sink  for  LED  light  source,  133 
from  laser  cooling,  84-85,  109 
of  optical  trap,  89-90 
placing  system  components,  129 
Heat  filters,  to  exclude  IR  light,  43-44,  129, 
132. 

liquid,  132 

Heating.  See  also.  Thermal  variables, 
detectors,  252 

microwave  fixation,  377 
in  magnetic  resonance  imaging,  621-622 
multi-focal,  multi-photon,  551,  556,  685, 
903 

specimen,  by  the  chamber,  387-389,  394, 
732 

specimen,  by  the  illumination,  43,  89, 

132,  211,  218,  341,  536,  539,  544, 
556,  621-622,  681,  685,  884,  903 
calculation,  89,  685,  904 
stability,  652 

HeLa  cells,  391-392,  693,  799,  812,  814, 
820,  828,  854. 

Helios  Gene  Gun  System,  724. 
Helium-cadmium  (He-Cd)  laser,  83,  86,  90, 
93,  103,  105,  115. 
operational  lifetime,  115 
output  variation,  86 
transverse  electromagnetic  mode,  83 
Helium-neon  (He-Ne)  laser,  82,  84,  88-90, 
93,  102-103,  105,  107,  240,  241, 

376,  673,  680,  798,  799,  864,  875. 
four  state,  82,  105 
Heterectis  crispa,  874. 

Hidden-object  removal,  304-305. 

High  content  screening  (HCS),  809-817. 
for  cytomics  chapter,  809-817 
data  management/image  informatics, 
816-817 

fluorescence  analysis  of  cells,  table,  812 
multiple  fluorescent  probes,  810 
High  resolution  spatial  discrimination,  813. 
High  throughput  screening  (HTS),  809. 

High  voltage  electron  microscope  (HVEM), 
846. 

stereo  images  of  platelets,  848-849 
Hippocampal  brain  slices,  268,  316-317, 
393,  556-557,  722,  724-725,  727. 
calcium  imaging,  556-557 
culture  protocol,  724-725 
damage,  341 

at  neurons,  205,  268,  316-317,  393 
Histology,  623,  624. 

Historic  overview  of  biological  LM,  table, 
2-3. 

Hoechst,  DNA  dye,  136,  339,  344,  360,  362, 
520,  565-566,  683,  782,  812. 

4Pi,  image,  565-566 
FLIM  image,  521 
high-content  screening,  812,  814 
Holey  optical  fiber/non-linear  effects,  88. 


Holographic  diffusers,  to  reduce  coherence, 
84. 

Holography,  holomicrography,  7-8. 

Hooke,  Robert,  image  of  cork,  769-770, 

785. 

HTS.  See  High  throughput  screening. 

Huffman  encoding,  580-581. 

Human  endomicroscopy,  confocal. 
cervix,  513 

gastrointestinal  track,  514 

Human  retina,  viewed  with  OCT,  609. 

Huygens,  3D  software,  104,  413,  669,  778. 

Huygens-Fresnel  wavefront  construction, 
406. 

HVEM.  See  High-voltage  electron 
microscope. 

Hybrid  mode-locked  dye  laser,  540-541. 

Hymenocallis  speciosa,  fluorescence  spectra, 
422. 

Hysteresis. 

in  Piezoelectric  scanners,  57,  754 
temperature  cycling  of  lenses,  249 

I 

I5M,  (Incoherent  Illumination  Image 

Interference  Imaging),  275,  561, 
569-570,  672. 

optical  transfer  function  (OTF),  569-570 

ICNIRP.  See  International  Commission  of 
Non-Ionizing  Radiation  Protection. 

ICTM.  See  Iterative  constrained  Tikhonov- 
Miller  algorithm. 

IEC.  See  International  Electrotechnical 
Commission. 

IF.  See  Fluorescent  intensity. 

Illumination,  44,  210.  See  also.  Structured- 
illumination  microscopy,  and 
Chapter  6. 

brightness,  table,  140 

errors,  211-212 

evaluating,  211-217 

goal  in  confocal  microscopy,  210 

path,  211-212 

types  of  lamps,  44 

vignetting  caused  by  beam  shift,  211-212 

Image(s),  9,  11-12,  30-31,  38-39,  59,  145, 
192,  210,  219,  280,  286-290.  See 
also.  Multidimensional  microscopy 
images. 

contrast,  7,  11,  16,  39,  49,  60-62,  68, 

159,  162,  165,  167,  173-175,  180, 
189-190,  192,  201-204,  248,  421, 
473,  488,  542,  599-600,  607,  622, 
656,  657,  675 
chapter,  162 
flare,  649 
definition,  280 

degradation  of,  measuring,  145 
extended-focus,  9 

motion  between  specimen  and  objective, 
39 

multi-dimensional  microscopy,  286-290 
anisotropic  sampling,  287 
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Image(s)  ( cont .  ) 

calibrating  image  data,  286-288 
contrast  transfer  function  (CTF),  61. 

See  CTF 

data  type/precision  in  computations, 
288-289 

digitization,  defined,  62 
dimensions,  286-288 
display  devices,  non-linearity  of,  72-73 
file  formats,  table,  288-289 
processor  performance,  289-290 
Voxel  rendering  speed,  290 
real,  disk-  and  line-scanners,  30-31 
reconstructing,  and  noise  reduction, 
38-39.  See  also.  Reconstruction; 
Nyquist  reconstruction  sharpness  of 
its.  signal  intensity,  192 
of  source  and  detector  pinholes,  210 
speed  of  acquisition,  11-12.  See  also. 
Speed  as  sum  of  point  images,  59 
thermal  distortion,  219.  See  also.  Thermal 
variables 

Image  analysis.  See  Automated  3D  image 
analysis  methods;  Automated 
interpretation  of  subcellular  location 
pattern. 

Image  dissector,  254—255. 
in  trans-illumination  mode,  10 

Image  enhancement.  See  Deconvolution, 
488-499. 

Image  iconoscope,  for  television,  6-7. 

Image  intensifiers,  13,  232-233,  235,  255, 
460,  All,  519-520,  522,  524, 
555-556,  730,  737,  784,  801,  930. 

Image  Pro  Plus,  282,  290. 

Image  processing.  See  also.  Automated  3D 
analysis  methods,  and  Multi¬ 
dimensional  microscopy  display, 
for  display.  Chapter  14 
for  measurement.  Chapter  15 

Image  resolution,  8,  9.  See  also.  Resolution. 

Image  substrate,  automated  confocal,  810. 

ImageJ,  free  software,  282,  290,  395, 
732-733,  762-764,  795,  858. 

Imaging  system,  optics  characterized  by 
CTF,  61. 

Imaging  techniques,  382-386,  394-395. 
combining  fluorescence  with  other, 
383-386 

fluorescence  correlation  spectroscopy, 

383 

fluorescence  lifetime  (FLIM),  382, 
516-532 

fluorescence  loss  in  photobleaching 
(FLIP),  382 

fluorescence  recovery  after 
photobleaching,  382 
fluorescence  resonance  energy  transfer, 

382 

fluorescence  speckle  microscopy  (FSM), 

383 

laser  trapping,  383 

linear  unmixing.  192,  382,  664-667 


multi-channel  time-lapse  fluorescence, 

382 

optical  tweezers,  383 
photoactivation,  187,  224,  383,  385,  541, 
544-545,  693,  759 

photo-uncaging,  383.  See  also.  Photo- 
uncaging  physiological  fluorescence, 

383 

spectral,  382 
table,  384-385 
time-lapse  fluorescence,  382 
Imaris,  software.  193,  281-282,  284, 

287-288,  290-291,  299,  301-303, 
308,  311-312,  764,  795. 

In  vitro  fertilization,  mitotic  apparatus,  188. 
In  vitro  preparations. 

2D  mixed-cell,  assays,  813 
antifade  agents.  See  also.  Antifade,  342 
automated  analysis,  318-320 
backscattered  light  image,  513 
biofilms,  870,  872,  879,  884 
bleaching,  551,  851 

brain  slices.  See  Brain  slices,  392-393, 
725 

cell  maintenance,  387 
cytoskeleton,  368 
fertilization.  188 
GFP,  357 

high  content  screening,  809,  813-816 
high  speed  imaging,  11,  237,  809,  813, 
815-816 

ion  imaging,  calibration,  742 
living  cell  imaging,  387 
micro-CT,  614,  617 
micro-MRI,  618,  621,  623-625 
multi-photon,  535 

optical  coherence  tomography  image,  609 
photodamage,  684 
In  vivo  (intact  animal)  imaging,  112, 
368-377,  512,  545,  806. 

2-photon  microscopy  (MPM),  535,  543, 
545 

cell  preparations,  387 

comparison  with  fixed  material,  368-377 

FLIM  calibration,  517 

labeling,  372-373 

miniaturized  confocal,  504,  508,  511-513 
micro-CT,  614,  617 
micro  MRI,  618,  621,  623-625 
molecular  imaging,  806 
photodamage,  684,  693-694,  698 
“stick”  lenses,  806 

Incandescent  lamps,  34,  126,  133-137,  477, 
499  See  also.  Halogen  lamps, 
black-body  radiation  emitted  by,  135-136 
spectrum  its.  temperature,  137 
stability,  137,  477 
Incidence  angle,  49,  50. 
efficiency,  143 

interference  filters/transmission,  49 
reflectivity,  diagram,  50 
Incident  light  beam,  sample  interaction, 
162-163. 


Indo-1,  calcium  indicator,  103,  189,  257, 

345,  346,  348,  529,  531,  544,  693, 
697,  742-743. 

Infinity  corrected  optics,  155-157,  166,  239, 
405. 

advantages,  156-157,  166,  239,  405 

Infinity  PhotoOptical,  InFocus  spherical 
aberration  corrector,  15,  151. 

Infinity  space,  generating,  157. 

Information,  27,  60,  64.  73-74,  179,  235, 
241,  243,  268,  270-275,  278,  330, 
334,  353,  369,  382-383,  396,  398, 
443,  448,  459,  468,  475-476,  481, 
487,  488^190,  494,  496-499,  506, 
512-513,  517,  519,  522-524, 
543-544,  556,  559,  570,  580-587, 
596,  643,  650,  732,  715,  769,  774, 
776,  779,  782,  790,  794,  800. 
3-dimensional,  321,  378,  396,  747 
4Pi,  570 

and  bleaching,  222,  690-692,  705 
CARS,  597-598.  602 
colocalization,  668 
confocal,  461,  462 
contrast,  see  Chapter  8  and  Contrast 
crystal  orientation,  179,  188 
display  of,  280-281,  288-291,  293, 
295-297,  299-301,  304-305,  311 
efficiency,  336,  628,  631 
of  electronic  signal,  limitations  on,  64 
genetic,  756,  762-763 
lost  signal,  25-28 
matching  gray  levels  to,  73-74 
micro-CT,  615 
micro  MRI,  618 

and  Nyquist  sampling.  See  Nyquist 
sampling,  38,  39,  634-637 
chapter,  59-79 

optical  projection  tomography,  612 
out-of-focus  light,  27,  368,  458,  461,  746, 
784 

parallel  vi.  serial  acquisition,  223-224 
PSF,  245,  247,  250 

from  second  harmonic  generation  signal, 
179 

Shannon  theory,  443 
on  source  brightness,  137 
spectral,  665-667 
SPIM,  614,  675-378 
storage,  106 
chapter,  580-594 
theory,  4,  64,  443 

transmission,  contrast  transfer  function, 

37,  60 

Index  mismatch.  See  Spherical  aberration. 

Infrared  (IR)  lasers,  89,  383,  385.  See  also. 
Ultrashort  lasers;  Laser  tweezers, 
solid  state  lasers,  108-109 

Infrared  paper,  to  identify  infrared  beams  for 
safety  purposes,  118. 

Inkjet  printers,  593. 

Innova  Sabre/frequency-doubling  crystal, 

102. 
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Insect  cuticle,  transparency  to  NIR  light, 

166. 

Installation  requirements,  for  laser  sources, 
85. 

Instrument  dark  noise,  660.  See  also.  Noise 
Integrated  circuit  (IC)  chip,  9. 

Intelligent  imaging  innovations,  (III),  3D 

imaging  system  supplier,  78-79,  151, 
192,  395,  411,  654. 

Intensified  CCD,  13,  232-233,  460,  477, 
519-522,  524,  555,  556,  737,  784, 

930. 

Intensity,  light,  26,  37,  43,  58,  59,  61, 

71-72,  86,  87,  133,  136,  163,  165, 
180,  189,  192,  208,  217,  222,  228, 
258,  270,  391,  413,  426,  459,  461, 
487,  536,  538,  571-573,  633,  681, 
693,  705,  810,  901. 
of  excitation  light,  80,  222,  680-682 
laser  beam,  stability,  86 
losses 

detection  path,  table,  217 
illumination  path,  table,  217 
minimum  needed,  392 
on  optical  response  of  specimen,  165 
in  photons/second,  80 
regulating,  43,  88 

singlet-state  saturation.  See  Saturation 
and  visibility,  37 

Intensity  control. 

continuous  wave  laser,  88 
non-laser,  128 

Intensity  distribution,  146-154. 

of  Airy  disk,  65,  146.  See  also.  Airy  disks 
changes  with  focus,  147,  407,  455, 
463,  471 

effect  of  coverslip  thickness,  149 
effect  of  RI  mismatch,  148.  See  also. 
Spherical  aberration 
in  focal  spot,  plots,  147-154 
nonsymmetrical  change  with  focus, 

148 

unit  image,  147 

with  astigmatism,  152 

with  coma  present,  151 

with  spherical  aberration,  148-150, 

212 

Intensity  loss,  with  spherical  aberration  in 
detection  path,  148-150,  212. 

See  Spherical  aberration. 

Intensity  spread  function  (ISF),  74-78. 

CCDs  and  PMTs  compared,  table,  78 
defined,  75 

estimating  intensity  measurement  error, 

76 

and  gray  levels,  74-75 
measuring,  75 

Interference  contrast. 

differential  interference  contrast,  (DIC), 
10,  14,  76,  127,  146,  171,  453,  468, 
473-475,  846,  See  also.  Differential 
interference  contrast, 
deconvolution  of,  473-475 


phase-contrast,  9,  171,  368,  372,  453, 

506,  643,  649,  731,  851,  854,  890, 
892.  See  also.  Phase  contrast 
centering  the  phase  rings,  643.  See 
also,  Bertrand  lens  scanning,  9,  13 
using  fiber  optics,  506 
RI  inhomomogeneity  and  contrast,  22-23, 
41 

Interference  filters,  45-51,  102,  136,  212. 
in  argon-ion  laser  systems,  102 
continuously-graded,  137 
destructive  and  constructive  reflections, 

45 

transmission,  212 
types,  46-49 

Interference  fringes,  coverslip  surface,  168, 
170. 

Amoeba  plasma  membrane/coverslip,  170 
in  close  proximity,  168 

Interference  mirrors,  46. 

Interference  mode,  coherent  light,  130. 

Interference,  speckle  pattern,  8,  13,  84,  90, 
130-132,  144. 

in  backscattered  light  images,  448 
fluorescence  speckle  microscopy  (FSM), 
13,  383,  385,  889 

Interferometer. 

4Pi  microscopy,  561 
Fabry-Perot  (laser),  81-82 
fiber-optic,  for  testing  objectives, 

240-241 

Gires-Tournois,  88 

Mach-Zender,  to  measure  pupil  function, 
245 

optical  coherence  tomography,  (OCT), 
504,  609 

Twyman-Green,  239 

Inter-fluorophore  distance,  measurement, 
184.  See  also.  Fluorescence 
resonance  energy  transfer. 

Interfocal  crosstalk,  227-228. 
disk  scanners,  227-228,  444,  449 
time  multiplexing  as  solution  to, 

553-554 

Interlocks,  laser  safety,  118. 

Intermediate  optical  systems,  LSCMs, 
chapter,  207-220. 

Internal  focusing  elements,  in  objective, 

157,  511. 

International  Commission  of  Non-Ionizing 
Radiation  Protection  (ICNIRP),  117. 

International  Electrotechnical  Commission 
(IEC),  117. 

International  television  standards,  589. 

Internet  sources.  See  Links, 
lasers,  123,  124 

Intrinsic  noise,  21.  See  also,  Poisson  noise. 

Inverse  filter  algorithm,  476,  477. 

Ion-binding  in  Aequorin  emits  light,  737. 

Ion  concentrations,  346-347,  517,  528-530, 
741. 

chapter,  736-745 
determination,  517,  528-530 


Ion-concentration  imaging,  736-738, 

740-745.  See  also.  Calcium  imaging, 
pH,  etc. 

calcium  imaging,  736-737 
concentration  calibration,  742-745 
indicator  choice,  738 
interpretation,  740-741 
pH  imaging,  739-745 
water-immersion  objectives,  737 

Ion  sensitive  probes,  optical,  348,  737. 
table,  531-532 

IR.  See  Infrared;  Near  infrared. 

Irradiance,  arc  and  halogen  light  sources, 
130. 

table  comparing,  130 

ISO  standard,  microscope  dimensions,  156. 

Iso-intensity  surface,  or  arc  sources,  304. 

Iterative  constrained  algorithms,  475^176. 
See  also.  Deconvolution;  Nonlinear 
constrained  iterative  deconvolution 
algorithms. 

Iterative  constrained  Tikhonov-Miller 
algorithm  (ICTM),  497. 

J 

Jablonski  energy  diagrams,  516,  517,  697, 
792. 

Jansson-van  Cittert  algorithm,  476,  496. 

Jitter,  defined,  for  scanners,  54. 

JND.  See  Just  noticeable  difference. 

Joint  Photographic  Experts  Group.  See 
JPEG. 

JPEG  (Joint  Photographic  Experts  Group), 
581-584. 

Just  noticeable  difference  (JND),  ocular 
response,  72-73. 

K 

Kaede,  photoactivatable  fluorescent  protein, 
emission  change  after  photodamage, 
187,  383,  385. 
example  image,  187 

Kalman  averaging,  21,  39,  53,  304,  306, 

627,  638,  655,  750,  754,  781. 
comparison  with  deconvolution,  in 
reducing 
intensity,  39 

Kepler,  Johannes,  788. 

Kerr  cell,  516. 

mode-locking  (KLM),  111,  133 
of  titaniunnsapphire  lasing  rod,  113 

Kerr  effect,  defined.  111,  179. 

self-focusing  of  pulsed  laser  light,  1 1 1 

Kindling  proteins.  574,  760. 

Kinetics,  691,  694-698,  741-742,  774,  796, 
810-812,  816-817. 
bleaching,  691,  694-698 
and  endpoint  data  analysis,  816-817 
fluorescence,  262-263,  348,  383,  385, 

571,  578,  741-742.  See  also,  FLIM 
FRET,  796 

high  content  screening,  810-812, 

816-817 
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Kinetics  (cont.) 

ion  concentration  dyes,  741 
and  STED,  571,  578 

Kino,  Gordon,  confocal  design,  6. 

KLM.  See  Kerr  lens  mode-locking. 

Kohler  illumination,  34,  127-128,  131,  229, 
251,  627,  648-649. 
coherence  of  light,  131 
in  disk  scanner,  229 

field  diaphragm,  35,  127-129,  139,  461, 
627,  645,  648-649 

to  limit  non-uniformity  of  illumination, 
127-128 

to  measure  photon  efficiency,  34 

Krypton  laser,  102,  119,  346,  355. 
comparison  with  argon-ion  laser,  102 
wavelength,  102 

Krypton/argon  (Kr/Ar)  laser,  90,  92,  93, 

102,  108,  119,  203-204,  343,  375, 
748,  798,  811. 
stabilization,  88 

KTP.  See  Potassium  titanium  oxide 
phosphate. 

L 

Labeled  structures,  plants,  757,  761,  775. 
bulk  labeling,  living  embryos,  761 
cell  walls,  775 
selective  labeling,  757 

Label-free  microscopy,  noise,  114. 

Lamp  housing,  134. 

Lamprey. 

labeled  axons,  235,  236 
larva,  optical  projection  tomography 
image,  612 

Landmark-based  registration  synthesis 
method,  328-329. 

Lanthanide  chelates,  345-346. 

Large  mode  area  photonic  crystal  fiber 
(LMAPCF),  110. 

Larmor  frequency,  MRM  imaging, 

618-622. 

Laser(s),  7-9,  44,  80-83,  88,  90,  94, 

112-114,  119-120,  131,  540-543, 
599-600.  See  also.  Fiber  lasers; 
Mode-locked  lasers;  Multi-photon 
ultrafast  lasers;  Up-conversion  fiber 
lasers;  Ultrafast  lasers. 

Alexandrite,  109 
amplifier  rods,  116 

attenuation  of,  85,  87-88,  354,  415,  904 
axial  or  longitudinal  modes,  83 
basic  operation,  81-83,  116 
CARS  microscopy  requirements,  599-600 
chapter,  80-125,  table,  119-120 
coherence,  spatial  and  temporal,  83-84 
colliding-pulse  mode-locked  (CPM),  540 
for  confocal,  7,  9-10,  77-78,  280, 
535-545 

continuous-wave,  90-110 
control  of  power,  543 
Cr:Forsterite,  109,  114,  415,  541, 
706-709,  712-714 


excitation  wavelength  choice,  540-542. 
See  also,  Acousto-optical  devices, 
filters 

femtosecond  pulsed  laser,  44.  See  also. 
Ultrafast  lasers 

fiber-based  lasers,  109-111,  113-118 
table,  94 

ultrafast,  113-114 
up-conversion  fiber  lasers,  109-110 
fiber  light  delivery,  \Ql,See  also.  Fiber- 
optics 

GaAs,  107,  111 

gas,  90,  91-10.  See  also,  lasers  by  gas. 
alkali- vapor,  103 
Ar-ion,  90,  101-102 
Kr-ion,  102 
HeNe,  102-103 
HeCd,  103 
heat  removal,  84 

hybrid  mode-locked  dye  laser,  540-541 
important  properties  for  confocal,  80 
light  delivery,  87-89 
fiber-optic,  106 
mirrors,  88 

longitudinal  modes,  82-83 
maintenance,  115-116 

active  media  replacement,  115 
cooling  components,  116-117 
optical  resonator,  116 
metal  vapor,  112 
microscopical  uses 

nonlinear:  2-  3-photon,  90 
Raman  and  CARS,  90 
TIRF,  90 

tweezers,  89.  See  Laser  trapping 
multi-photon.  See  Multi-photon 
microscopy 
Nd:glass,  706-708 

Nd:YAG,  lasers,  88-89,  91,  95,  97,  103, 
107-109,  111,  113-115,  117,  218, 
245,  514,  680,  798 

Nd:YLF,  lasers,  89,  98,  100,  103,  109, 
112-114,  750,  760-761 
Nd:YV04,  lasers,  89,  95,  100,  103, 
107-109,  111,  113-114,  541 
NO  SMOKING,  116 
performance  tables,  91-101 
phase  randomization,  8,  13.  131-132,  143 
pointing  error,  87 

active  cavity  stabilization,  87 
polarization,  83,  88-89 
power  control,  543 
pulse  broadening/compensation,  88, 
901-904 

pulsed,  110-115.  See  also.  Titanium- 
sapphire,  Cr:Forsterite, 
Nd;glass,YAG/YLF/YV04,  etc. 
cavity  dumped,  111 
Kerr  lens  mode-locked.  111 
modulated  diode  lasers,  112 
pulse-length  measurement,  115, 
901-903 
purpose,  110 


saturable  Bragg  reflector.  111 
ultrafast,  DPSS  lasers,  112 
ultrafast,  fiber  lasers,  113 
white-light  continuum  lasers,  113 
why  are  they  useful?,  110 
pumping  power  requirements,  82 
safety,  117-118,  839,  900.  See  also. 

Safety 

goggles,  118 

screens  and  curtains,  11 8,  904 
solid  state,  103.  See  also.  Solid-state 
lasers 

semi-conductor,  105-107 
thin-disk  lasers,  109 
spectrum  of  light,  44 
stabilization,  85-87 
active,  87 

titanium: sapphire  laser,  82,  84—86,  88-91, 
94,  100-103,  105,  107,  109, 

111-112,  114,  123-124,  165,  346, 
358,  415,  423-424,  459,  541,  550, 
551,  645-647,  688,  706-708,  713, 
727,  750,  756,  759 
4Pi,  563-564,  567 
brain  slices,  731 
CARS,  599 

compared  to  other  fast  lasers,  82-83, 
85,  110,  112-113 
embryos,  731,  750,  756,  759,  764 
maintenance,  116 
and  OPO,  114-115 
plants,  415,  423-424,  706-708, 
713-714,  717,  781-783 
popular  models,  specs,  table,  120 
STED,  575 

transverse  modes,  82-83,  85.  110 
tweezers,  89.  See  Laser  trapping 
types,  90 

ultrafast  fiber,  113-114,  See  also. 

Ultrafast  lasers 

wavelength  expansion  by  sum-and- 
difference  mixing,  114 
optical  parametric  oscillators,  114-115 
second/third  harmonic  generation,  114 
white  light  continuum  lasers,  88,  109,  113 
continuum,  88,  109 
He:Cd,  113. 

Laser  cavity  stabilization,  active,  87. 

Laser  cutters,  686-687. 

integration,  218-219 
Laser  illumination,  conditions  for,  8. 

Laser  lines,  using  acousto-optical  tunable 
filters,  56. 

Laser  media,  maintenance,  115-116. 

Laser  printers,  593. 

Laser  rods,  maintenance,  116. 

Laser  Safety  Officer,  117. 

Laser  sources,  9,  80-125.  See  also.  Lasers. 
Laser  speckle,  84,  90,  130-132,  448. 
removing,  84.  See  also.  Scramblers 
source,  130 

Laser  trapping,  80,  89,  110,  218-219,  383, 
385,  539,  646,  680. 
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Laser  tubes,  operational  lifetime,  102,  115. 

components  likely  to  fail,  115 
Laser  tweezers.  See  Laser  trapping. 

LaserPix,  282. 

Laser  flying-spot  microscope,  7. 

Lasersharp,  confocal  microscopes,  282,  284, 
285,  288,  292,  296,  302-306. 
LaserVox,  281-282. 

Lateral  chromatic  aberration  (LCA),  14, 
155-156,  239,  242-243,  287,  640, 
657-658. 

correction  in  conventional  optics,  155 
measured,  657-658 
Lateral  coherence,  8,  84,  267. 

Lateral  resolution,  1-4,  9,  11-13,  28,  207, 
209,  222,  225,  230,  238,  270,  320, 
409,  453,  511,  513,  542,  552,  554, 
563,  568,  651,  654-656,  747.  See 
also ,  Resolution. 

4Pi,  568 

CARS,  596-597,  599 
confocal  endoscopy,  511,  513 
confocal  optics,  improvement,  9,  651, 
654-656 
of  display,  292 
light  microscopy,  1-3 
optical  coherence  tomography,  609-610 
with  pinhole  and  slit  disks,  225 
and  spherical  aberration,  409 
SPIM,  613,  674 
STED,  573-575,  578 
table,  209,  409 

Laterally-modulated  excitation  microscopy, 
see  Stuctured-illumination. 

LCA.  See  Lateral  chromatic  aberration. 

LCD.  See  Liquid  crystal  display. 

LCOS.  See  Liquid-crystal-on-silicon. 

LCS  (Leica  Microsystems  AG),  282,  312, 
910. 

Lecithin  myelin  figures,  CARS  image,  204. 
LED.  See  Light-emitting  diode. 

Leica,  confocal  manufacturer,  51-53,  56-57, 
160,  218,  797,  910. 

acousto-optical  beam-splitter,  160,  218 
objective  lens  transmission,  160 
RS  Scanner,  52-53 

spectral  confocal,  TCS  SP2,  51,  56-57, 
910 

tube  length  conventions,  157,  239 
Leica  Microsystems  AG,  282,  910. 

Leica  TCS  4Pi,  119-120,  565-568. 

4Pi  microscopy  type  C,  565-568 
imaging  of  living  cells,  568 
lateral  scanning,  567,  910 
mitochondrial  network  image,  568 
optical  transfer  function  (OTF),  567 
sketch,  566,  910 

thermal  fluctuations  minimized,  567 
Lempel-Ziv-Weich  (LZW),  580-582,  584. 
Lens  aberrations,  13-15.  See  also. 
Aberrations. 

Lens  focal  length,  change,  with  wavelength, 
152. 


Leonardo  da  Vinci,  early  optical  studies, 
788-790. 

Leukocytes,  347,  387,  520,  815,  854. 
automatic  analysis,  815 
multi-photon,  phase-based  FLIM,  521 

Lifetime.  See  Fluorescence  lifetime  imaging 
microscopy. 

Ligand-binding  modules,  256,  348,  741, 

846. 

Light  detection,  general,  28-33,  251-264. 
See  also.  Detectors;  specific 
detectors:  CCDs,  PMTs,  etc. 
assessment  of  devices,  260-262 
charge-coupled  device  (CCD),  254 
comparison,  table,  233,  255-256,  647 
conversion  techniques,  259-260 
direct  effects,  252 
future  developments,  262-264 
history,  262-264 
image  dissector,  254-255 
microchannel  plate,  232-233,  255,  262 
gated,  519,  523-524,  527,  532 
MCP-CCD,  262 
noise  internal  to,  256-259 
internal  detection,  256 
noise  currents  table,  256 
photoemissive  devices,  256-257 
photon  flux,  257-258 
pixel  value  representation,  258-259 
photoconductivity,  252,  253 
photoemissive,  254 
photon  interactions,  252-256 
work  functions,  table,  252-253 
photovoltaic  effect,  252-253 
point  detectors,  260-261.  See  also,  PMT 
quantal  nature  of  light,  25 1-252 
thermal  effects,  252 

vacuum  avalanche  photodiode,  254,  255 

Light  dose,  related  to  pixel/raster  size,  64. 

Light,  effects,  on  plant  cells,  770.  See  also. 
Bleaching,  Phototoxicity. 

Light-emitting  diode  (LED),  34,  54, 
132-133,  135-139,  143,  237. 
aligning,  135 

control  by  current-stabilized  supply, 
138-139 
definition,  105 

to  detect  galvanometer  rotor  position,  54 
excitation  wavelength  for  fluorophores, 
136 

expected  cost  reduction,  237 
fluorescence  image,  142 
galvanometer  position  feedback,  53 
lifespan,  137 

to  measure  photon  efficiency,  34 
microscope  illumination,  131-139,  141, 
143 

organic,  projected  development,  143 

radiance,  138 

spectra,  133 

stability,  136 

temperature  effects,  137 

wavelength  vs.  current  change,  137 


Light  flux,  light-emitting  diode  temperature, 
133. 

Light  intensity,  71,  163. 

Light  microscopy  history,  1-4. 

Light  paths.  See  also.  Commercial  confocal 
light  microscopes. 

separating  excitation/emission,  44-45 

Light  piping  by  specimen  vs.  depth,  182. 

Light-sheet  illumination,  672-673. 

Light  sheet  microscopy,  613. 
chapter,  672-679 
optical  setup  for,  613 
white-light  continuum  lasers,  113 

Light  sources,  widefield,  132-139,  143.  See 
also.  Chapters  5  and  6,  Arc  lamps, 
LEDs,  Lasers;  Nonlaser  light 
sources;  Filaments;  Halogen, 
commercial  sources,  143 
solar,  126-127,  131,  135 
stand-alone,  143 

table,  comparative  performance,  140 
types,  132-139 

Light  transmission.  11,  139,  160-161, 
223-229. 

cummulative  loss  along  optical  path,  139 
of  Nipkow  disk  system,  11,  223-229 
specifications  for  objectives,  table, 
160-161 

Lighting  models,  3D  image  display, 

306-312. 

absorption,  309-312 
advanced  reflection  models,  309 
artificial  lighting,  309-312 
Gourard  shading,  308 
gradient  reflection  models  for  voxel 
objects,  309 

Phong  shading,  308-309 
Phong/Blinn  models,  308 
simulated  fluorescence  process,  310 
surface  shading,  310 
transparency,  280,  284,  287,  300,  304, 

309,  311-312 

Lilium  longiflorum,  image,  783. 

Limitations,  confocal  microscopy,  chapter, 
20-42. 

fundamental,  20-42 
table,  41,  647 
typical  problem,  21,  24 

Linear  galvanometers,  54. 

Linear  longitudinal  chromatic  dispersion 
(LLCD),  stereoscopic  confocal 
image,  154. 

Linear  unmixing.  See  Spectral  unmixing. 

Line-scanning  confocal  microscope,  50,  51, 
231-232,  237,  784,  908,  916. 

Linearity,  72,  490. 

deconvolution  for  image  enhancement, 

490 

display  advantages  and  disadvantages,  72 

Links  (Internet  addresses). 

2  photon  excitation  spectra,  546,  727, 

729,  782 
brain  slices,  727 
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Links  (Internet  addresses)  ( cont .) 

CCDs,  76,  234,  927,  931 
components,  58 
confocal  Listserve,  390,  901 
deconvolution,  495 
dyes,  221,  343-344,  782 
fluorescent  beads,  653 
FRET  technique,  185,  803 
high-content  screening  systems,  811 
image  management,  865 
lasers,  104,  115,  120,  123-125 
live-cell  chambers,  388-389,  870 
movies  related  to  book,  235,  392 
muscles,  237 

non-laser  light  sources,  138,  143 
plants,  769 
safety,  900 

software,  282,  376,  594,  734,  762,  764, 
776,  777,  820,  824,  827,  831-833, 
844,  845,  864-862,  865-867,  869 
SPIM,  672 

Lipid  dyes,  236,  355,  359-360,  389,  556, 
755,  760-761. 

Lipid  receptors,  790. 

Liquid  crystal-on-silicon  (LCOS),  266. 

Liquid  crystal  display  (LCD),  39,  67,  73, 
291,  293,  589-590. 
digital  projectors,  590 
filters,  928 
non-linearities,  73 
shutters,  299,  929 
supertwisted  nematic  (STN),  589 
thin-film  transistor  (TFT),  589 

Liquid  crystal  technology/dynamic 

polarization  microscopy,  188.  See 
also ,  Pol-scope. 

Lissajous  pattern,  circular  scanning.  554. 
“tornado”  mode,  SIM  scanner,  52 

List  servers,  125. 

Lithium  triborate  (LBO),  as  non-linear 
crystal  for  multiplying  infrared 
output,  109,  115. 

Living  cells,  80,  90,  114,  136,  145-161, 
167,  219,  221-222,  381-399, 
429-439,  480,  564-566,  568, 
746-766,  770,  772-773,  788-806, 
811,  813.  See  also.  Brain  slices. 
Plants  cell  imaging,  and  by 
cell/organism  name. 

2-photon,  penetration,  749-751 
2D  plus  time,  753-754,  762-764 
3D  projection,  763 
4D  data,  746-747,  764 
4Pi  microscopy,  564-565,  568 
acquisition  speed,  222,  753-754 
algorithms,  763-764 
assays,  811 

beauty  and  functionality,  790 
bleaching  of,  797.  See  Bleaching; 
Photodamage 

cell-chamber,  11,  22,  191,  219,  370-371, 
386-387,  394,  429-430,  564, 
610-611 


for  4Pi  confocal,  564 
for  biofilms,  870-873,  875,  877,  880, 
885 

for  brain  slices,  394,  723,  727,  729 
for  epithelial  cells,  370-371,  377,  386 
finder  chamber,  683 
flow  chamber,  870-873,  875,  877,  880, 
885 

for  high-content  screening,  810 
for  optical  projection  tomography, 
610-611 
perfusion,  394 

for  plant  cells,  191,  429-430 
simple,  22,  394 
for  SPIM,  613,  625,  673 
table  of  required  functions,  380 
table  of  suppliers,  388-389 
test  chamber/dye,  654,  661 
cell-cycle  effects,  790 
chromatin,  385,  390-392,  684,  693-695, 
812 

chromatin  dynamics,  390-392 
CNS  tissue  slice  preparation,  393 
confocal  microscopy,  381-399,  746,  813 
difficulties,  381 
future  directions,  398-399 
considerations,  386-390 
antioxidants,  390 
experimental  variables,  table,  386 
fluorescent  probes,  387-389 
maintenance  of  cells/tissues,  387 
minimizing  photodynamic  damage, 

136,  389 

photon  efficiency,  141-161,  389-390 
in  vitro  preparations,  see  In  Vitro 
in  vivo  preparations,  see  In  Vivo 
contrast,  747 

dyes,  748.  See  also.  Dyes;  Fluorophors 
etc. 

for  rapid  assessment,  table,  360 
embryos,  imaging,  746-766.  See  also. 
Living  embryo  imaging 
external  membranes,  SHC  image,  90 
fluorescent  staining,  393 
microglia,  393 
nuclei,  living/dead  cell,  393 
fluorophore  effects,  748 
FRET  imaging,  chapter,  788-806 
future,  221-222 
handling  data,  395-396 
imaging  techniques,  382-386,  394-395 
low-dose  imaging,  391-392 
microglial  cell  behavior  example, 
392-398 

no  damage  from  SHG  imaging.  114 
online  confocal  community,  390 
photon  efficiency,  141-161,  389-390 
phototoxicity,  390-391 
assays  for,  813 

plant,  429-439.  See  also.  Plant  cell 
imaging  reflectance  imaging,  167 
second  harmonic  generation.  See  also, 
SHG 


of  external  membranes,  90 
no  damage,  114 
test  specimen  for,  390 
widefield,  646-647,  751-753 
working  distance,  5,  9,  129,  145,  154, 
157,  198,  249,  511,  568,  598,  634, 
673,  678,  727-728,  747,  774,  779, 
781,  872 
table,  158 

Living  embryo  imaging,  749-751,  762-764. 
aberrations  caused  by,  747 
apparatus,  748 
C.  elegans,  746,  748 
deconvolution  helps  confocal,  751-753 
developmental  changes,  746 
Drosophila,  273,  675-676,  747-748, 
751-752,  754,  756,  759,  804,  810 
dyes,  748 

introduction  of,  755 
embryo  size  vs.  speed  acquisition, 
753-754 

explants,  748-749 
future  developments,  766 
fluorescent  probe 
four  dimensional,  746-747,  749 
cellular  viability,  747-748 
challenges,  762 

dataset  display  strategies,  761-764 
photodamage  during,  746-748 
high  speed  acquisition 

disk-scanning  confocal  microscopy, 

754 

hardware,  754-755 
light  scattering,  747 

optimal  acquisition,  parameters,  753-754 
refractile  specimens,  747 
superficial  optical  sections,  748 
thick  specimens 

effective  strategies,  748-753,  755-761 
inherent  trade-offs,  747-748 
selective  plane  illumination  (SPIM), 
751 

"Test  drives,”  for  living  embryo  imaging, 
752. 

widefield/deconvolution,  751-752 
LLCD.  See  Longitudinal  chromatic 
dispersion. 

LMA-PCF.  See  Large  mode  area  photonic 
crystal  fiber. 

Loading  methods,  fluorescent  probe,  347, 
358-360,  430,  732-734,  738,  739. 
acetoxymethyl  esters,  359,  360.  See  also, 
Acetoxymethyl  esters 
ATP-gated  cation  channels,  359 
ballistic  microprojectile  delivery,  360, 
726,  803 

direct  permeability,  358-359 
electroporation,  359-360,  795,  803 
ion  indicators,  738-739,  742 
low  level,  430 

membrane  permeant  esters,  359-360 
microinjection,  360,  361,  388,  739,  748, 
755,  795,  803-804 
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neurons,  722,  726,  730,  732-734 
osmotic  permeabilization,  359 
peptide-mediated  uptake,  359 
plant  cells,  769 

stabilizing  chemicals,  341-342,  362 
transient  permeabilization,  359 
whole-cell  patch  pipet,  360 

Local  projections,  display,  305-306,  307. 

Location  proteomics,  818. 

Longitudinal  chromatic  aberration,  152-155. 

Longitudinal  coherence  length,  7,  8,  84, 

130,  131. 

Longitudinal  linear  chromatic  dispersion 
(LLCD)  objectives  for  3D  color- 
coded  BSL  confocal,  154. 

Long-pass  filters,  43-44. 

Low-voltage  scanning  electron  microscope 
(LVSEM),  846-847,  849-850,  852. 

LSM.  See  Laser-scanning  confocal 

microscopes;  Laser-scanning  flying- 
spot  microscope.  6-7 

Lucoszs  formulation,  273. 

Luminescent  nanocrystals,  343,  345. 

Luminous  intensity  vs,  color,  dye  molecule, 
138. 

LVSEM,  846-847,  849-850,  852. 

LysoTracker  Red  DND-99,  359-360, 
709-710. 

rapid  assessment  table,  360 
spectra,  710 

LZW  compression.  See  Lempel-Ziv-Welch. 

M 

Mach-Zehnder  interferometry,  245. 

Machine  learning.  See  Automated 

interpretation  of  subcellular  patterns. 

Macrography,  3D  light  scanning,  672. 

Magnesium  fluoride  (MgF2). 
for  anti-reflection  coating,  158 

Magnetic  disks,  586. 

Magnetic  resonance  imaging  (MRI),  618. 

Magnetic  resonance  microscopy  (MRM). 
618-624. 

amplitude  modulation  for  RF  carrier,  620 
applications,  623-624 
botanical  imaging,  624 
developmental  biology,  624 
histology,  623 
phenotyping,  623 
basic  principles,  618-619 
Fourier  transform/image  formation,  620 
future  development,  624 
hardware  configuration,  621-622 
image  contrast,  622-623 
image  formation,  619-621 
Larmor  frequency,  620 
Schematic  diagram,  618-619 
strengths/limitations,  622 

Magnification,  24,  35-41,  62,  131,  215,  443. 
See  also,  Nyquist  sampling;  Over- 
sampling;  Undersampling, 
calibrating,  653,  658 
and  CCD  pixel-size,  62,  70 


confocal,  52-53,  62-64 
effect  on  pixel  size,  24,  928 
factor,  24,  28 

and  lateral  chromatic  aberration,  278 
for  line-scanner,  232 

over-sampling,  68-70,  493,  509,  635,  729 
high-content  screening,  816 
and  pinhole  size,  28 
under-sampling,  68 

zoom  magnification,  11,  24,  37,  63-34, 
66,  70,  79,  317,  389,  493,  627, 
634-636,  731 
Maintenance. 

cell  viability,  387 
dye  lasers,  114 
lasers,  115-117,  124 
remote  logging  of,  864 
troubleshooting  reference,  124 
Maize  (Zea  mays),  167-168,  172.  179,  202, 
417^124,  428,  438,  710-711, 
713-714. 

2-photon,  time-lapse  microspectroscopy, 
423 

abnormal  vasculature,  437 
anther,  420,  433 
attenuation  spectrum,  leaf,  418 
cross-sections,  stem,  172,  707 
emission  spectrum,  710,  711,  713 
fluorescence  spectra,  422-424 
leaf, 

attenuation  spectrum,  418 
optical  section,  172,  179 
reflectance,  167 
surface,  436 

meristem,  420,  430-432,  707 
multi-photon  excited  signals,  422-424 
polarization  microscopy,  707,  7 1 1 
pollen  grain,  202,  433-434 
protoplast,  424 
root,  432 

second  harmonic  imaging,  707,  711 
silica  cells,  428,  437,  707 
spectrum,  422,  423,  710 
starch,  420,  435-436,  707,  711 
stem 

attenuation  spectra,  417,  418,  713 
optical  sections,  419,  714 
storage  structures,  420,  435-436,  707, 

711 

Manufacturers.  See  also.  Commercial 
confocal  light  microscopes; 
Appendix  2. 

listing  with  web  addresses,  table, 

104-105. 

Mapping  conventions,  in  image  processing, 
294-296,  300-304. 
data  values,  300-304 

choosing  data  objects,  300-301 
object  segmentation,  301-302 
projection  rules,  302-304 
scan  conversion,  301-302 
table,  300 
visualization,  300 


multi-dimensional  image  display, 

294-296 
G  function,  294 
image/space  view,  296 
orthoscopic  view,  294 
reducing  geometric  dimensions,  294 
rotations,  294-296 
visualization  process,  294 

MAR.  See  Mark/area  ratio. 

Marching  cubes  algorithm,  301-302,  304, 
776. 

Marconi,  CAM-65  electron  multiplier  CCD 
camera,  76.  See  also,  EM-CCD. 

Mark/area  ratio  (MAR),  279. 

Marsilea  quadrifolia,  416,  419. 
attenuation  spectra,  416 
optical  section,  419 

Mass  balancing,  to  reduce  scanner  vibration, 
54. 

Mass  storage,  580-588,  593-594. 
data  compression  for,  288-289,  292-293, 
295,  319,  499,  580-585.  See  also. 
Data  compression 
algorithms,  319,  580 
archiving  systems,  580 
color  images,  581 
file  formats,  580-588 
removable  storage  media,  585-588.  See 
also.  Removable  storage  media 
random-access  devices,  586-588 
sequential  devices,  585-586 
solid  state  devices,  588 
time  required,  table,  581 

Materials,  silicon,  fused  quartz,  beryllium, 
52. 

Mathematical  formulas,  for  confocal 
microscope  performance,  table, 

209. 

Maximum  intensity  projection,  180, 

284-285,  292,  294,  298,  302-304, 
307,  313-314,  319,  325-326, 
330-331,  585,  755,  763-764,  770, 
774,  881,  884. 
local,  305 

Maximum  likelihood  estimation  (MLE), 
472-475,  495,  497^198,  669. 
blind  deconvolution,  472-475,  498,  784 
effect  on  colocalization,  table,  669 

M-CARS.  See  Multiplex  CARS 
microspectroscopy. 

MCR  See  MicroChannel  plate,  232-233, 

255,  262. 

MCP-CCD,  262 

Gated  intensified,  519,  523-524,  527, 
532 

MCP-PMT.  See  MicroChannel  plate 
photomultiplier. 

MDCK  cell,  372-374. 
actin  cytoskeleton,  374 
Golgi  apparatus,  image,  374 
morphologic  changes,  374 
stereo  image,  373,  374 
vertical  sections,  image,  372 
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Measurements,  20,  33-36,  76,  139-141, 

159. 

achromat  performance,  194 
buffering  of,  ion  measurement,  738,  740 
field  flatness,  26-28 
geometric  distortion,  653-654 
laser  pulse  length,  109,  112,  115,  507, 
537,  538,  902-903 
light  throughput,  139-141 
limits  on  confocal  intensity,  accuracy,  20 
photon  efficiency,  33-36 
pinhole,  effective  size,  34 
intensity  spread  function  histogram, 

74-78 

resolution,  241-245,  657,  658 
shrinkage,  specimen  preparation, 

371-373 

spectral  transmission  of  objective,  159 
spherical  aberration,  145,  407 
surface  height,  using  LLCD  BSL 
confocal,  224 
z-resolution,  194 
Mechanical  scanners,  51-54. 

Melles  Griot  catalog,  real  lens,  performance, 

210. 

Membrane  permeant  esters,  361,  358-359, 
361,  726,  738-739,  744. 

Membrane  potentials,  179,  188,  204-205, 
346,  353,  383,  517,  743,  811-813. 
Memory  stick,  588. 

Mercury  arc  lamp,  37,  44,  132,  135-138. 
fluorophores  matching  excitation, 
135-136,  139 

iso-intensity  plots  of  discharges,  132 
and  pinhole  size,  37 
radiance,  improvements,  137-138 
wavelengths,  44 

Mercury-halide  arc  source,  136,  138, 
143-144. 
spectrum,  144 

Mercury-iodine  (Hg-I)  arc  lamp,  radiance, 
138. 

Mercury-xenon  arc  lamps,  136-138. 

spectral  lines,  136 
Meristem,  168,  420,  430,  432,  770, 

776-778,  782. 
maize,  168,  432 

Merit  functions,  confocal  scanners,  217. 
object-dependent,  defined,  217 
object-independent,  defined,  217 
Mesophyll  cells,  169,  193,  195.  417-418, 
423,  428,  430,  711-712,  714,  779. 

A.  thaliana,  193,  196 
photodamage,  203 
protoplasts,  196,  203,  424,  425^126, 
430,  439 

harmonic  images,  711-712,  714 

image,  424 

spectra 

attenuation,  416,  418 

change  with  1-  vs.  2-photon,  421,  423 

emission,  423 

Metal-halide  light  source,  136,  143-144, 
907,  908. 


Metal  vapor  lasers,  112. 

Metamorph,  281-282,  290,  311,  817. 
MicroChannel  plate  (MCP)  image  intensifier, 
233,  255,  519,  532. 
multiplicative  noise,  233 
photocathodes,  262 
PMT,  255,  523,  532 

MicroChannel  plate  PMT  (MCP-PMT),  255. 
Micro-computerized  tomography  (Micro- 
CT),  614-618. 
contrast/dose,  614-615 
dose  vi.  resolution,  graph,  616 
layout,  614 

mouse  images,  615-617 
tumor-bearing,  617 
operating  principle,  614 
Micro-CT.  See  Micro-computerized 
tomography. 

Microdissection. 

with  multi-photon  IR  light,  686-687 
with  nitrogen  lasers,  112 
Microelectrodes,  for  introducing  indicator, 
738. 

Microglial  cell  behavior,  392-398. 
Microinjection,  360-361,  388,  739,  748, 

755,  795,  803-804. 
of  chromophores,  803-804 
Microlens  array,  12,  134,  135,  216,  225, 

231,  235. 

for  4Pi  confocal,  563-565 
for  CCD,  237 

for  disk  scanners,  6,  12,  216,  224,  226, 
231,  458 

for  light-emitting  diode  source.  134-135 
for  multi-focal,  multi-photon,  (MMM), 
537,  551-555,  558 
principle,  135 

in  Yokogawa  disk-scanning  confocal,  12, 
224-226,  231,  235 

Microscopes,  217,  226.  See  also ,  particular 
types. 

attachment  of  confocal  scanner,  217 
specification  comparisons,  table,  226 
Microscopy  laboratory  URLs,  125.  See  also , 
Links. 

Microspectroscopy,  421-425,  426,  516. 
CARS,  601-602 

fluorescence  properties  of  plants,  421-425 
lifetime,  516 
of  maize,  424 
multi-photon  setup,  424 
Microspores,  birefringence  images,  189, 
431-432. 

Microsporogenesis,  43 1-432. 

Microstructure  fibers,  504. 

Microsurgery,  112,  219,  686-687,  764-765. 
Microtubule,  11,  68,  80,  188,  222,  292,  432, 
582,  703,  714,  752-753.  759,  773, 
790,  852.  See  also,  Cytoskeleton. 
birefringence,  714-715 
Brownian  motion  of,  1 1 
electron  microscopy,  848,  850 
fixation,  369,  372-375 
fluorescence  correlation  spectroscopy,  383 


GFP.  12.  See  also.  Green  fluorescent 
protein 

in  mitosis,  759.  See  also.  Mitotic 
apparatus 

polarization  microscopy,  15,  173,  188, 
420-421 

photodamage  of,  341,  850-851 
stabilizing  buffers,  852 
STED,  576-577 
stereo  image,  752 
TIRF,  180,  183 

second  harmonic  generation,  for  tracking, 
90 

Microwave  fixation,  377-378. 

Microwire  polarizer  (Moxtec  Inc.),  85. 

Mie  scattering,  162-163,  167,  417-418. 
clearing  with  index-matched  liquid,  167 
comparison  with  Rayleigh  scattering, 

163 

light  attenuation  in  plant  tissue,  417 
by  refractive  structures,  162-163 
MIL  See  Multi-photon  intrapulse 
interference,  88. 

Mineral  deposits,  plant,  163-420,  436-437, 
703. 

Miniaturized  fiber-optic  confocal 
microscope,  508-512. 
bundle  imagers  for  in  vivo  studies,  509 
clinical  endoscope,  514 
objective  lens  system,  509 
optical  efficiency,  509 
optical  schema,  508 
resolution,  509 
rigid  endoscope,  511 
single  fiber  designs,  510 
vibrating  lens  and  fiber,  510-511 
in  vivo  imaging  in  animals,  512 
Minolta,  CS-100  radiospectrometer,  139. 
Minsky,  Marvin,  2,  4-6,  11,  141,  216,  890. 
Mirror  coupling,  pulse  width  and  pulse 
shape,  88. 

Mirrors,  26,  48,  54,  63,  209-210,  214. 
galvanometer,  54.  See  also. 
Galvanometers 

internal,  testing  reflectance  losses,  26 
laser-line,  48 
performance,  54,  63. 
scan  angle  and  magnification,  63 
size  calculation  for  LSCM,  209 
x-y  scanning  mirror  orientations,  214 
Mismatch,  893. 

probe  shape/pixel,  39,  466 
caused  by  chromatic  aberration,  243 
refractive  index,  377,  404-412,  411,  654, 
658,  747,  863,  893 
4Pi,  568 

causing  signal  loss,  148-150,  408-409, 
654 

chapter,  404-412 
corrections,  411-412 
embryos,  747 
film  vi.  CCD,  590 

harmonic  signal  generation,  704-705 
less,  at  long  wavelength.  416 
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measurement,  148-150.  655-656 
of  movie  frame  rate,  839 
MRM,  contrast  agent  vi.  imaging  time 
mylar  flakes,  198 

resolution  loss  measured,  192-194 
vector  mismatch  in  CARS,  596-597, 
600 

z-distortion,  287 

Mitotic  apparatus,  15,  173,  373-374,  377, 
386,  421,  431,  693,  749,  752,  799. 
See  also ,  Microtubules, 
damage,  693 
fixation,  373-374,  377 
FRET,  765 

marker,  cyclin-B,  790 
Pol-scope,  13,  188,  432,  468,  479-480. 
deconvolved,  479 
images,  15,  188,  479,  717 
SHG  imaging,  702,  718-719 
in  vitro  fertilization,  188 
Mitotracker  stain,  142,  170,  353,  358,  360, 
430-431,  692,  750. 

living  cells  rapid  assessment,  table,  360 
Mixing,  sum  or  difference,  to  generate  laser 
wavelengths,  114. 

MLE.  See  Maximum  likelihood  estimation. 
MMM.  See  Multi-focal,  multi-photon 
microscopy. 

MMM-4Pi  microscopy,  556. 

MO  (magneto-optical)  disks,  586. 
Model-based  object  merging,  323-325. 
Mode-locked  lasers,  87,  101,  111-114,  118, 
124,  358,  520,  646,  728-729,  749, 
901-904. 

active,  pulsed  laser  class.  111 
adjustment  of,  901-904 
for  CARS,  599-600 
colliding  pulse,  112 
fiber,  ytterbium  and  neodymium, 

113-114 

fiber/diode,  ultrafast,  113-114 
FLIM,  520 

interference  with,  by  specimen,  171 
Kerr  lens.  111 
modulator,  fiber  lasers.  111 
multi-photon,  535-536,  540-541, 
550-551,  563-564,  567,  646, 
728-729,  749 
passive,  111,  113-114 
saturable  Bragg  reflector.  111 
SHG,  THG,  706-707 
Mode-locked  oscillators.  See  also.  Mode- 
locked  lasers. 

nanojoule  pulse  energies.  111 
Moire  effects,  270-271,  755. 

ambient  fluorescent  room  lighting,  201 
banding  patterns,  755 
disk-scanners  and  CCDs,  231,  754-755 
structured-illumination  methods,  268-271, 
273 

Molecular  imaging,  in  vivo,  387,  618,  624, 
790,  803-806. 

FRET,  790,  803-806 
micro-CT,  618 


MRM,  624 

Monitors,  computer  display,  588-589. 
Monkey  cells,  693,  803. 

Monomeric  red  fluorescent  protein  (mRFP) 
constructs,  756,  760,  798. 
with  CFP  or  GFP  molecules,  as  FRET 
pair,  798 

Montage  synthesis  method,  281,  312,  318, 
328-331,  748,  753,  851-852,  855, 
858-859. 
defined,  329-330 
examples,  330-332,  780-781 
scanning  electron  micrographs,  85 1-852, 
855 

TEM  methods,  858-859 
Moon,  early  phase  measurements,  788,  789. 
Morphological  filters,  285,  300-301, 

316-317,  320-322,  730-734,  817, 
826. 

high-content  screening,  812,  819,  826 
Morphometry,  145,  316,  319,  331,  726,  728. 
group  properties,  331 
intensity/spectral  measures,  331 
interest  points,  33 1 
invariants,  331 
location/pose,  331 
shape  measures,  331 
size  measures,  331 
texture  measures,  331 
topological  measures,  331 
Mosaicing.  See  Montage  synthesis  method. 
Mounting  medium,  166,  198,  342,  370-371, 
373-377,  404-413,  418,  454,  457, 
473,  493-794,  499,  564,  631,  642, 
652,  655,  696,  730,  774  See  also. 
Clearing  agents, 
brain  slices,  730 
chapter,  404-413 

clearing  solutions,  166,  417-418,  420, 
439,  610,  624,  774-775 
effect  of  glass  bead,  199 
plant  specimens,  418,  431,  774 
refractive  index,  tables,  377 
selection,  198,  631 

Mouse,  192,  376,  393,  608,  612,  615+,  723, 
726. 

confocal  colonoscopy,  509,  512 
embryo 

optical  projection  tomography  image, 
612 

SIM/EFIC  image,  608 
GFP  transgenic,  726 
hippocampus,  393 
micro-CT  image,  614-615,  617 
femur,  616 
tumor,  617 

spectral  unmixing  image,  192,  382, 
664-667 

examples,  665-666 
visual  cortex  brain  slices  image,  723 
Movement  contrast,  190. 

Movie  compression,  836-840. 

Moving-coil  actuators,  galvanometer,  52. 
Moxtec  Inc.,  Micro  wire  polarizer,  85. 


MPA.  See  Multi-photon  absorption. 

MPE.  See  Multi-photon  excitation. 

MPEG  display  formats,  836-841. 

MPEM.  See  Multi-photon  microscopes. 

MPLSM.  See  Multi-photon  laser-scanning 
microscopy. 

MPM.  See  Multi-photon  microscopy. 

MQW.  See  Multiple  quantum  wells. 

mRFP.  See  Monomeric  red  fluorescent 
protein. 

MRI.  See  Magnetic  resonance  imaging. 

MRM.  See  Magnetic  resonance  microscopy. 

Multi-channel  experiments,  813. 
filters  and  dispersive  elements,  5 1 
time-lapse  fluorescence  imaging,  382, 

384 

toxicity,  755 

Multi-dimensional  microscopy,  display, 
280-314.  See  also.  Automated  3D 
image  analysis  methods. 

2D  pixel  display  space,  291 
efficient  use,  292 
animations,  292-293 
artificial  lighting,  306-308 
CLSM  images,  286-290 
anisotropic  sampling,  287 
calibrating  image  data,  286 
data  type/computational  precision, 
288-289 

dimensions  available,  286-287 
file  formats  for 

calibration/interpretation,  288-289 
image  data,  286 
image  size  available,  287 
image  space  calibration,  287-288 
image/view  dimension  parameters, 
table,  288 

processor  performance,  289-290 
storing  image  data,  286 
voxel  rendering  speed,  290 
color  display  space,  291-292 
commercial  systems,  tables 
available  systems,  282-283 
desirable  features,  288-289 
display  options,  293 
geometric  transformations,  295 
projection  options,  300 
realistic  visualization  techniques, 

307 

criteria  for  choosing  visualization,  281 
data  values,  definition,  222,  280 
dimensions,  280,  323 
degrees  of  freedom,  optical  image,  8-9 
depth-weighting,  304,  306 
exponential,  304 
linear,  304 
recursive,  304 

display  view,  definition,  280 
hidden-object  removal,  304-305 
local  projections,  305-307 
z-buffering,  304-305 

highlighting  previously  defined  structures, 
284 

image,  definition,  280 
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Multi-dimensional  microscopy,  display 
(cont.) 

image/view  display  options,  table,  293 
geometric  transformations,  table,  295 
intensity  calibration,  304 
iso-intensity  surface,  304 
laser-scanning  microscopy,  280 
lighting  models,  306-312 
absorption,  309-312 
advanced  reflection  models,  309 
artificial  lighting,  309-312 
Gourard  shading,  308-309 
gradient  reflection  models/voxel 
objects,  309 

Phong  shading,  308-309 
Phong/Blinn  models,  308 
simulated  fluorescence  process,  310 
surface  shading,  310 
transparency,  309-312 
living  cells  of  rodent  brain,  392-398 
mapping  data  values,  300-304 
choosing  data  objects,  300-301 
object  segmentation,  302 
projection  rules,  302-304 
scan  conversion,  301-302 
segmenting  data  objects,  301 
visualization  model,  300 
mapping  into  display  space,  294-296 
G  function,  294 
image/space  view,  296 
orthoscopic  view,  294 
reducing  geometric  dimensions,  294 
rotations,  294-296 
visualization  process,  294 
measurement  capabilities  See  also. 
Chapter  15 

reconstructed  views,  312-313 
results,  284-285 

objective  vs.  subjective  visualization,  281 
prefiltering,  281 
principle  uses,  281-285 
projection/compositing  rules,  302-304 
alpha  blend,  302,  304 
average  intensity,  302 
first  or  front  intensity,  302 
Kalman  average,  304 
maximum  intensity,  302 
pseudo  color,  173-175,  190,  291 
purpose,  281-285,  293-295 
realism  added  to  view,  306-308 
techniques  for,  table,  307 
reconstructed  view  generation,  290-312 
5D  image  display  space,  291-294.  See 
also,  5D  image  display  space 
choosing  image  view,  291-294 
subregion  loading,  290-291 
reconstruction,  definition,  280 
reflection  models,  306-308 
rendering,  definition  for,  280 
software  packages,  table,  282-283 
stereoscopic  display,  293,  296-299 
color  space  partitioning,  297 
interlaced  fields  of  frame,  297 


pixel-shift/rotation  stereo,  297 
stereo  images  example,  298 
synchronizing  display,  297 
true  color,  291 

unknown  structure  identification,  281-284 
viewing  data  from,  283 
visualization  parameters,  table,  285 
z-coordinate  rules,  304 
z-information  retained  by,  296-300 
non-orthoscopic  views,  299 
stereoscopic  views,  296-299 
temporal  coding,  299-300 
z-depth,  299-300 

Multi-fluorescence,  systems  for  utilizing, 

217+. 

Multi-focal,  multi-photon  microscopy 

(MMM),  221,  276,  550-559,  797. 
4Pi-MMM,  563-564 
basics,  565 
scheme,  563 

alternative  realizations,  554-555 
background,  550 

beam  subdivision  approaches,  table,  558 
current  developments,  558-559 
experimental  realization,  551-555 
FRET,  797 

imaging  applications,  556 
boar  sperm  cells,  557 
Convallaria,  556 
FRET,  556 

hippocampal  brain  slices,  557 
pollen  grains,  556 
Prionium,  556 

interfocal  crosstalk,  553-554,  556 
time-multiplexing,  553-555 
limitations,  556-558 
localization,  538 

Lissajous  pattern  of  scanning  foci,  554 
“tornado”  mode,  SIM  scanner,  52 
Nipkow-type  microlens  array,  551-552 
optimum  degree  of  parallelization, 
550-551 

resolution,  552-553 
schematic  diagram,  552 
time  multiplexing,  553-554 
variants,  555-556 
FLIM,  555-556 
MMM-4PL  556 
SHG,  556 

space  multiplexing,  555 

Multi-length  fiber  scrambler,  8.  See  also. 
Scramblers,  light. 

Multi-photon  absorption  (MPA),  535. 

Multi-photon  excitation  (MPE),  356-358, 
535-545,  894.  See  also.  Multi-focal 
multi-photon  microscopy, 
absorption,  705-707 
advantages/disadvantages,  644-647, 
749-751 

autofluorescence,  plants,  424,  427 
background  from  SHG/THG,  361, 
708-709,  728 

backscattered  light  imaging,  429 


bleaching,  218,  338,  539-540,  680-689, 
692-693,  905.  See  also.  Bleaching; 
Chapter  38 

caged  compounds,  187,  383,  543-544, 
692,  729,  912 

cell  viability  during  imaging,  544-545 
chromophores  for,  543-544 
detection,  538 
duty  cycle,  644 
excitation  localization,  538 
excitation  spectra,  125 
FLIM,  576 

fluorophores  for,  543-544 
FRET,  797 
heating,  539-540 
history,  535 

image  formation,  535-540 
instrumentation,  540-543,  900-905.  See 
also,  lasers  for.  See  also.  Ultrafast 
lasers 

Alexandrite,  109 

Cr:Forsterite,  109,  114,  415,  541, 
706-709,  712-714 
Nd:glass,  706-708 
Nd:YAG,  88-89,  107-109,  514,  680, 
798 

Nd:YLF,  89,  112-114,  750,  760-761 
Nd:YV04,  89,  95,  107-109,  113-114, 
541 

Ti:Sapph.  See  Laser,  titanium-sapphire 
laser 

multi-focal,  multi-photon  microscopy 
alignment,  900-901 
beam  delivery  requirements,  541 
control  of  laser  power,  543 
CPM  laser,  540 

descanned  detection,  166,  208,  212, 
428,  537,  540-542 
excitation  wavelengths,  541 
focal  plane  array  detection,  542 
hybrid  mode-locked  dye  laser,  540-541 
lasers/excitation  wavelength  choice, 

540- 542 

non-descanned  detection,  185,  201, 

218,  381,  447,  456,  507.  542,  552, 
559,  643,  646,  727,  750,  779,  904, 
909,  910 

non-mechanical  scanning,  543 
optical  aberrations,  542 
power  requirements,  541,  903,  904 
pulse  spreading  due  to  GDD,  547,  538, 
543 

resonant  scanning,  543 
whole-area  and  external  detection, 

541- 542 

optical  pulse  length,  537-538 
group  delay  dispersion,  537-538,  543 
group  velocity  dispersion,  88,  111,  210, 
537,  606,  903 

measurement,  115,  901-903 
penetration,  749-750 
photodamage,  539-540,  680-688, 
692-693 
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physical  principles,  535-540 
refractive  index  mismatch,  404-413 
resolution,  539 

SHG  and  THG  background,  361, 

708-709,  728 

two-photon  absorption  cross-sections,  125 
(URL)  543-544 
wavelengths,  538-539 
Multi-photon  intrapulse  interference  (Mil), 
88. 

Multi-photon  microscopy  (MPM).  10-11, 

56,  172-177,  210,  535-545,  681, 

682,  685-688,  746-766,  894, 

900-905. 

advantages/disadvantages,  644-647, 
749-751 

alignment,  901-902 
autofluorescence,  425-427,  545 
SHG,  THG,  361,  708-709,  728 
calcium  imaging,  545 
cell  damage  during,  544—545,  682,  685 
1-photon  vs.  2-photon  excitation,  681 
absorption  spectra  of  cellular  absorbers, 
681 

intracellular  chromosome  dissection, 
688 

mitochondria,  686 
nanosurgery,  human  chromosomes, 
686-687 

by  optical  breakdown,  198,  680,  682, 
685,  687,  703,  705 
photochemical,  682-685 
photothermal,  539,  545,  681,  685, 

904 

reproduction  affected  by  ultrashort  NIR 
pulses,  686 

ultrastructural  modifications,  685-686 
cell  viability,  544-545 
compared  with  other  3D  methods, 
644-647,  748-751 
deconvolution,  495-498 
developmental  biology,  545,  746-754, 
757,  759-760,  764 
dispersion  as  problem,  56.  See  also , 

GVD;  GDD 

fluorescence,  contrast,  172-177 
for  living  embryo  imaging,  chapter, 
746-766 

need  for  efficient  illumination  light  path, 
210 

optical  layout,  540 

photobleaching,  545,  680-688,  692-693 
practical  operation,  900-905 
protein  damage/interactions,  765 
resolution,  552 

setup/operation,  540,  900-905 
schematic  diagram,  540,  901-902 
in  vivo  (intact  animal)  imaging,  545 
ultrafast  lasers,  88,  90,  109.  See  also. 
Ultrafast  lasers 
Alexandrite,  109 

Cr:Forsterite,  109,  114,  415,  541, 
706-709,  712-714 


Nd:glass,  706-708 
Nd:YAG,  88-89,  107-109,  514,  680, 
798 

Nd:YLF,  89,  112-114,  750,  760-761 
Nd:YV04,  89,  95,  107-109,  113-114, 
541 

Ti:Sapph.  See  Laser,  titanium-sapphire 
laser 

uncaging,  545 

Multi-photon-based  photo-ablation,  764. 

Multi-slit  design,  for  disk-scanning 
confocal,  229. 

Multi-view  deconvolution,  330,  675-677. 

Multiple  quantum  wells  (MQW),  diode 
injection  lasers,  106. 

Multiplex  CARS  microspectroscopy  (M- 
CARS),  601,  602. 

Multiplicative  noise,  28-33,  51,  77-78,  224, 
234,  256-258,  262,  275,  443,  460, 
633,  661,  667. 

of  EM-CCD,  30-31,  77-78,  264,  256, 

262 

losses  in  effective  QE  from,  33,  234, 

443 

from  PMT,  29,  51,  77-78,  233,  256-258, 
460,  633,  661,  667 
and  quantum  efficiency,  33,  234,  443 
photon  counting,  32-33,  78 
pulse  pile-up,  32-33,  35,  78,  521,  523, 
526-527 
table,  256 

why  it  is  usually  unnoticed  in  LSCM, 

633,  661 

Muscle,  737,  739-742. 
fatigue,  739-740 


N 

NA.  See  Numerical  aperture. 

Nanobioscopy,  protein/protein  interactions, 
795-798. 

acceptor  bleach,  797-798 
donor  fluorescence,  796-797 
FRET  measurement,  795 
sensitized  acceptor  emission,  795-796 
Nanoscale  resolution  with  focused  light, 
571-578.  See  also.  Stimulated 
emission  depletion  (STED) 
microscopy. 

breaking  the  diffraction  barrier,  571-573 
different  approaches,  573-574 
ground  state  depletion  (GSD),  573 
STED,  573-574 
outlook,  577 

RESOLFT  concept,  571-573 
resolution,  new  limiting  equation,  57 1 
measured,  578 

stimulated  emission  depletion  (STED), 
573-578 

axial  resolution  increase,  576 
compared  to  confocal  microscopy,  576 
dyes,  suitable,  table,  575 
OTF  comparison,  578 
PSF  comparison,  578 


Nanosurgery,  219. 

with  multi-photon  systems,  90 
NCI60  CMA,  standard  encapsulation,  816. 
NCPM.  See  Non-critical  phase  matching. 
ND.  See  Neutral-density  filters. 

Near  infrared  (NIR)  lasers,  10,  90,  106.  See 
also.  Lasers:  titanium-sapphire;  Nd:; 
Cr:Forsterite. 

Near  infrared  (NIR),  10,  90,  106. 
diode  injection  lasers,  106 
for  laser  tweezers,  90 
objective  lenses  designed,  174 
Nearest-neighbor  deconvolution  algorithm, 
476. 

image  enhancement,  495-496 
Negative  contrast,  for  fluorescence 
microscopy,  173-174. 

Negative  feedback,  to  correct  mirror  motion, 
53. 

Neodymium  glass  laser,  706-708. 
Neodymium-yttrium  aluminum  garnet 

(Nd:YAG)  lasers,  88-89,  91,  95,  97, 
103,  107-109,  111,  113-115,  117, 
245,  514,  680,  798. 
infrared  range,  108 

pumping  non-linear  crystal/green  light, 
114-115 

Neodymium-yttrium  lithium  fluoride 

(Nd:YLF)  laser,  89,  98,  100,  103, 
109,  112-114,  750,  760-761. 
Neodymium-yttrium  orthovanadate 

(Nd:YV04)  laser,  89,  95,  100,  103, 
107-109,  111,  113-114,  541. 
kits  utilizing,  1 1 3 
Nerve  cells,  images. 

Alexa  stained,  330 
backscattered  light  images,  167 
eye,  optic  nerve,  481 
Golghi-stained,  298 
Lucifer-yellow,  314 
microglia,  396-398 
rat-brain  neurons,  398 
transmitted  light,  475 
Neutral-density  filters  (ND),  43,  76,  126. 
in  fixed-pattern  noise  measurements, 

76 

to  reduce  source  brightness,  43,  126 
NFP.  See  Nominal  focal  position. 

Nikon,  confocal  manufacturer,  13,  15, 
119-120,  161,  199,  201,  507, 
638-640,  657,  750,  910. 

Cl  confocal  microscope,  119-120,  507 
Cl  si  spectral  confocal  microscope,  908, 
910 

CF  objectives,  154-156,  217,  669,  779 
confocal  x-z,  BSL  image,  22 
Plan  Apo  objective,  13,  15,  638 
resolution,  measured,  16,  638-640, 

657 

water-immersion  lenses,  15 
high-content  screening,  810 
tube  length  conventions,  157,  239 
Nile  Red,  dye,  435,  528,  575,  774,  782 
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Nipkow  disk  scanning,  2,  5-6,  11,  12,  41, 
215,  223,  231,  276,  551,  754, 
783-784,  810,  894.  See  also, 
Yokogawa;  Disk-scanning  confocal 
microscopy. 

commercial  systems,  907,  913,  915 
compared  to  single-beam  scanning,  458 
for  high-content  screening,  810 
micro-lens  system,  6,  12,  216,  224—226, 
231,  234,  237,  551-552 
multi-photon,  537,  551-558,  563-565. 
See  also.  Multi-focal,  multi-photon 
microscopy  rotation,  754 
for  single-sided  confocal,  6,  141,  223, 

229 

source  brightness,  141 
speed  of  image  acquisition,  11,  220, 
222-226,  227,  231 
for  tandem-scanning,  141,  215 
visualization,  of  cells,  458,  667,  754, 

784 

Nipkow,  Paul,  5-6,  109 
NIR.  See  Near  infrared. 

Nitrogen  lasers,  112. 

nanosurgery  using,  219 
NLO.  See  Non-linear  optical  effects. 

NMR.  See  Nuclear  magnetic  resonance. 
Noise,  21,  28,  74-77,  83,  87,  114,  190,  232, 
256-259,  442-444,  495.  See  also, 
Signal-to-noise  ratio;  Poisson  noise; 
Quantum  noise, 
background,  443-444 
of  CCD  detectors,  30-31,  77-78, 
232-233,  256,  262 
equations,  256 
table,  256 

vs.  photomultiplier  tube  detectors,  74, 
77 

CIC,  clock-induced  charge,  EM-CCDs, 
234,  926 

in  counting  quantum-mechanical  events, 

21 

deconvolution  reduced  noise,  39-40,  114, 
392,  495,  498,  667,  783,  835-836 
detector,  28 

fixed-pattern,  74,  76,  278,  924,  927,  931 
in  fluorescence  microscopy,  defining, 
74-75 

in  lasers,  sources,  85-86 
reducing,  87 
limits  grey  levels,  443 
measurement,  74-75 
multiplicative.  28-33,  51,  77-78,  224, 
234,  256-258,  262,  275,  443,  460, 
633,  661,  667 

in  photon  detectors,  256-259 
noise  currents  table,  256 
photo  flux,  257-258 
photoemissive  devices,  256-257 
pixel  value  represented,  258-259 
Poisson.  See  Poisson  noise 
polarization,  in  laser  systems,  83 
read,  and  readout  speed,  77 


shot,  442-443.  See  also,  Poisson  noise 
single-pixel,  65,  67,  190,  635,  832, 
835-836 

deconvolving,  to  reduce,  39-40,  392, 
498,  667,  784,  835-836 
reducing,  39,  40,  190,  41,  65,  392,  498 
sources  of,  442-444 
wavelet  transform  to  reduce,  733-734, 
819-820 

Nomarski  DIC  contrast,  2,  368,  746,  892. 
See  also.  Differential  interference 
contrast. 

Nominal  focal  position  (NFP),  405,  408, 
409. 

calculations  for  glycerol,  409 
calculations  for  water,  409 
z-responses,  diagram,  408 

Non-confocal  microscopy  vs.  confocal,  746. 
high  content  screening,  table,  811 

Non-critical  phase  matching  (NCPM), 
114-115. 

Non-descanned  detection,  for  MPM,  185, 
201,  218,  381,  447,  456,  507,  542, 
552,  559,  643,  646,  727,  750,  779, 
904,  909,  910. 
for  CARS,  559 

No-neighbor  algorithm,  476^-77,  496. 

Non-laser  light  sources,  chapter,  126-144. 
arc  sources,  130,  132,  140 
commercial  systems,  table,  143 
comparative  performance,  table,  140 
control,  138 

for  disk-scanning  confocal,  141 
filament  sources,  135-136 
LEDs,  132-133,  135,  138-139,  143 
light  scramblers,  131-132 
measured  performance,  139-141 
results,  142 

solar,  126-127,  131,  135 
stability,  136-137 

Nonlinear  constrained  iterative 

deconvolution,  68,  458,  475-476, 
496-497,  499,  520,  568. 

Nonlinear  conversion,  tunable  laser,  114. 

Nonlinear  crystals,  frequency  multiplying, 
109. 

Nonlinear  optical  (NLO)  effects,  in 

microscopy,  90.  114,  163,  165,  177, 
179,  188,  190,  195,  416-417, 
426-427,  430,  442,  504,  535,  507, 
703-720,  728,  741,  751.  See  also. 
Multiphoton/microscopy;  Harmonic 
signals;  SHG,  THG. 

absorption,  188,  415-418,  426-427,  430, 
705 

bleaching,  536,  550,  558,  645,  680-685, 
693,  697,  707,  729.  See  also. 
Bleaching;  Photodamage 
CARS,  595-598,  600 
DIC,  473-474.  See  also.  Differential 
interference  contrast 
fluorescence,  172,  179 
focus  shift  with  spherical  aberration,  409 


harmonic  generation,  704-705 
emission,  710-711 
energy  state  diagram,  705 
multi-photon  absorption/fluorescence, 
705 

second  harmonic  generation  (SHG), 
704-705 
setup,  708-709 

third  harmonic  generation  (THG),  705 
light  sources/detectors,  706-708 

light  attenuation  spectra  in  plants,  706 
photodetector  characteristics,  707 
pulsed-laser,  table,  706.  See  also. 
Ultrafast  lasers 
in  optical  fiber,  504-508 
optically  active  animal  structures, 
714-717 

man-made  collagen  matrix,  717 
signal-producing  structures,  table,  715 
spindle  apparatus,  717-718 
zebrafish  embryo,  716,  718 
optically  active  plant  structures,  710-714 
Canna,  710 

Commelina  communis,  712 

emission  spectrum  of  maize,  710,  711 

maize  stem,  711,  714 

potato,  712 

rice  leaf,  712,  715 

polarization  dependence  of  SHG,  717, 

719 

setup  for,  708-710 
spectra,  415,  417,  435 
Euphorbia  pulcherrima,  710 
maize  leaf,  710 
Pyrus  serotina,  711 
STED  microscopy,  571-579.  See  also, 
STED  microscopy, 
structured  illumination,  270,  276 
Non-radiative  dipole-dipole  interactions, 
790. 

Non-specific  staining,  27,  44,  74,  303,  345, 
357-358,  442,  467,  472,  617,  660, 
667-668,  760,  820,  878,  882.  See 
also.  Background. 

Non-tunable  solid-state  laser,  103. 

Normal,  free-running,  pulsed  laser,  111. 
Northern  Eclipse,  software,  282. 

Notch  filter,  to  transmit  laser  line,  49. 
Novalux  Inc.,  Protera  488  laser  system,  107. 
NSDC.  See  Nipkow  spinning-disk  confocal. 
Nuclear  import  analysis,  802. 

Nuclear  magnetic  resonance  (NMR),  618. 
Numerical  aperture  (NA),  1,  4,  24,  28,  61, 
126,  141,  145,  148,  168,  180,  195, 
198,  239-250. 

affects  surface  reflection  contrast,  180 
defined,  1 

determining  axial  resolution,  4,  241-242, 
657 

determining  lateral  resolution,  1, 

241-242,  656 

diffraction  orders  accepted  by,  61 
effect  on  self-shadowing,  168,  198 
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in  fiber-based  mini-confocal  endoscopes, 
509 

image  brightness,  126 
matching  to  CCD  pixel  size,  62,  928 
objective  lenses  with  high,  145,  239-250 
empty  aperture,  248 
with  oil-immersion  vs.  water  objective, 

148 

pinhole  size  as  function,  28 
and  refractive  index  mismatch,  147-148. 
See  also ,  Spherical  aberration  in 
tandem  scanning  confocal 
microscopy,  141 
vertical  shadowing,  195 
and  zoom  setting  optimal,  24 

Nyquist  criterion,  and  digitization,  38-39, 
64-68. 

Nyquist  digitizing,  65,  67. 

Nyquist  filtering,  70-79,  281. 

Nyquist  frequency,  64,  301.  See  also , 
Shannon  sampling  frequency. 

Nyquist,  Harry,  64. 

Nyquist  noise,  256. 

Nyquist  reconstruction,  limit  output 

bandwidth,  59,  66-67,  69,  70,  173, 
235-236,  280-315,  458,  468-469, 
474-475,  496-497,  563,  585,  603, 
607,  610,  615,  635,  672,  675, 
677-678,  690,  772,  730-731,  762, 

77,  774-776,  778,  784,  883. 

Nyquist  sampling,  24,  37,  39,  40,  53,  60, 
64-70,  73,  75-76,  78-79,  142,  146, 
152,  205,  222,  258,  271,  273,  289, 
386,  391,  448,  635-636. 
blind  spots,  38 

for  CCD  camera,  70,  233,  273,  928 
and  deconvolution,  59,  65,  67-68, 
222-223,  635 
diagram,  60 

optimal,  results  of  deviating  from,  24 
practical  confocal  microscopy,  448, 
635-636 

reconstruction,  see  Nyquist 
reconstruction. 

relationship  with  Rayleigh-criterion  and 
PSF,  39,  60,  64,  66 
signal-to-noise  ratio,  67,  448 
subpixel,  resampling,  478-479 

O 

Object  scanners,  image  quality,  216. 

Objective  lenses,  13,  15,  25-26,  34,  49,  145, 
152,  156,  239-250,  652-660.  See 
also ,  Aberrations, 
apodization,  250 

axial  chromatic  registration,  287,  658 
axial  resolution  measurement,  656-657 
Vi.  pinhole  size,  656 
chromatic  aberrations  14,  145,  152-156, 
160,  177-178,  209,  242-243,  641, 
659 

apparatus  in  measuring,  243-244,  654, 
659 


axial  shift,  243-245,  657-658 
chromatic  registration,  657-658 
cleaning,  642 

confocal  performance,  145-161,  652-660 
contrast  transfer  function  (CTF),  16,  35, 
37-39,  59-62,  656,  747 
coverslip  thickness,  table,  654 
dipping  lenses,  161,  209,  411,  429,  568, 
613,  727,  737,  870,  872 
dry,  high-NA,  aberrations,  15 
field  illumination,  34-35,  127-128,  139, 
461,  627,  658 

flatness  of  field,  145,  151-152,  154,  418, 
457,  639,  659 
Focal  Check™  beads,  657 
high-NA  planapochromat,  13,  145, 
239-250 

infinity  correction,  155-157,  166,  239, 
405 

advantages,  49 

lateral  chromatic  registration,  657-658 
lateral  resolution.  See  CTF 
light,  vector  nature,  267 
mounting  media.  See  Mounting  media 
photon  efficiency  losses,  25-26 
plan  objectives,  table,  152 
point  spread  function  of  high  NA, 
239-250 

measuring,  240-242,  455,  462,  471, 
656 

polarization  effects,  249-250 
pupil  function,  measured,  245-248 
3D  point  spread  function  restored, 
247-248 

empty  aperture,  248 
Mach-Zehnder  interferometry,  245 
phase-shifting  interferometry,  245 
Zernike  polynomial  fit,  245-247 
table,  247 

resolution  test  slide,  169,  656 
spherical  aberration.  See  Spherical 
aberration 
correction,  654—655 

sub-resolution  beads,  181-182,  196,  454, 
477,  493,  499,  527,  652-656,  784, 
900,  904,  930 
images,  656 
table  of  suppliers,  653 
temperature  variations,  248-249 
transmission,  optical,  154,  158,  159-161. 
See  Transmission,  objective 
table  of  objective  lenses,  159-161 
water-immersion,  145,  149-150 
dipping  objectives,  161,  209,  411,  429, 
568,  613,  727,  737,  870,  872 
use  and  limitations,  15 
working  distance,  5,  9.  129,  145,  154, 
157,  198,  249,  511,  568,  598,  643, 
673,  678,  727-728,  747,  774,  779, 
781,  872 

x-y  and  z  resolution  using  beads,  656 
OCT.  See  Optical  coherence  tomography. 
OLED.  See  Organic  light-emitting  diodes. 


Olympus,  confocal  manufacturer,  52-53,  54, 
119-120,  161,  184,  187,  204,  229, 
230,  234-236,  419,  421,  427,  557, 
708-709,  727-730,  797,  908, 

912. 

Fluoview-1000,  119-120,  184,  187,  204, 
908,  912 

DSU  disk-scanning  confocal  microscope, 
229-230,  234-235,  908,  913 
FRAP  system,  210 
FRET,  797 

high  content  screening,  811 
objectives,  557,  727-730 
stick,  in  vivo  objectives,  806 
TIRF  objectives,  183 
transmission,  table,  159,  161 
SIM  scanner,  52-54 
tube-length  conventions,  157,  239 

On-axis  reflections,  artifact,  171. 

Onion  epithelium  (Allium  cepa),  390. 

Online  confocal  community,  Listserv, 

390. 

OPA.  See  Optical  parametric  amplifiers. 

OpenLab,  282. 

Operational  lifetime,  of  laser  tubes,  102. 

OPFOS,  Orthogonal-plane  fluorescence 
sectioning,  672-673. 

OPO.  See  Optical  parametric  oscillators. 

OPT.  See  Optical  projection  tomography. 

Optical  aberrations,  109,  542.  See  also , 
Aberrations. 

thin-disk  laser  optics,  109 

Optical  layout  of  confocal  microscopes, 
212-213.  See  also ,  Optical  paths 
by  class,  213 
evaluation,  212-213 
class  1  systems,  212 
class  2  systems,  212-213 
class  3  systems,  213 

Optical  bandwidth/electronic  bandwidth,  32. 
See  also ,  Bandwidth. 

Optical  breakdown,  198,  680,  682,  685,  687, 
703,  705. 

Optical  coatings,  maintenance,  116. 

Optical  coherence  tomography  (OCT), 
609-610. 

of  human  retina,  609 

schematic,  610 

Xenopus  laevis  embryo,  610 

Optical  components,  chapter,  43-59. 

Optical  density  (OD),  71,  81,  416. 
filters,  43,  49-50 

Optical  disks,  586. 

Optical  efficiency,  improvements,  143-144, 
216.  See  also.  Photon  efficiency, 
of  disk  scanners,  216 
of  light-emitting  diodes,  143-144 

Optical  elements,  43-58,  128,  211. 
confocal  light  beam  affected  by,  211 
of  Kohler  illumination  components,  128 
light  beam  characteristics  affected  by, 
211 

chapter,  43-58 
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Optical  excitation,  diagram,  82. 

Optical  fiber.  See  Fiber  optics. 

Optical  fiber,  for  scanning  by  moving  fiber 
tip,  213-214. 

Optical  heterogeneity,  specimen,  22-23. 

reflection,  refraction,  scattering,  192-197 
Optical  images,  electronic  transmission,  5-6. 
Optical  materials,  158,  501. 

thermal  properties,  158,  248-249 
Optical  parametric  amplifiers  (OPA), 

100-101,  112,  114-115,  118,  124. 
components.  115 
table,  101 

Optical  parametric  oscillators  (OPO), 

100-102,  111-112,  114-115,  118, 
541,  600. 

for  CARS  microscopy,  600 

cavity  dumped,  to  increase  white  light, 

113 

tunable,  114-115 
table,  101 
Optical  path.  of. 

4Pi,  confocal,  563 
commercial,  566 
acousto-optical  device,  55 
compound  light  microscope,  156-157 
CARS,  599,  601,  907 
CARV-2  disk  scanner,  230 
confocal,  10,  208-209,  212,  632,  681 
beam-splitter,  213 
disk-scanner,  12,  216 
folded,  166 
scanning  systems,  214 
fiber-optic  confocal,  508 
interferometers,  243,  245 
Kino  single-sided  disk  scanner,  229 
La  Vision-Biotec,  Trimscope,  907 
Leica,  TCS  AOBS,  910 
magnetic  resonance  imaging,  621 
Minsky  confocal,  5,  25 
for  measuring  photon  efficiency,  34 
multi-photon,  540,  681,  708-709 
multi-focal,  552,  555 
spectrometer,  424 
Nikon  Cl  si,  911 

Olympus  DSU  disk-scanner,  230 
Olympus  Fluoview-1000,  912 
optical  coherence  tomography,  610 
optical  projection  tomography  (OPT), 

611 

Petran  tandem  scanner,  228 
selective  plane  illumination  (SPIM),  613, 
673 

or  simultaneous  BSL  and  fluorescence, 

128 

surface  3D  imaging,  SIM/EFIC,  608 
surface  spherical  aberration,  405-406 
STED,  573 

structured-illumination,  266 
Visitech  VT-Infinity  and  VT-eye,  914 
Yokogwawa  dual-disk-scanner,  231, 

915 

Zeiss  LSM-510,  META,  916-917 
Zeiss  LSM-5-Live,  50,  232,  916 


Optical  performance,  practical  tests, 
652-660. 

axial  chromatic  registration,  658 
axial  resolution  using  mirror,  656-657 
chromatic  aberration,  659 
chromatic  registration,  657-658 
contrast  transfer  function  (CTF),  656 
coverslip  thickness  vs.  RI,  table,  654 
field  illumination,  658 
flatness  of  field,  659 
Focal  Check™  beads,  657,  658 
lateral  resolution,  655 
resolution  test  slides,  655-656 
specimen  self-lensing  artifacts,  659 
spherical  aberration  correction,  654-655 

Optical  power,  specimen  plane,  table,  140, 
644. 

Optical  probes,  737.  See  also.  Dyes; 

Fluorescent  indicators;  Fluorophors; 
Fluorescent  labels. 

Optical  projection  tomography  (OPT), 
610-613. 

lamprey  larva,  612 
mouse  embryo,  612 
refractive  index,  613 
setup,  611 

Optical  pulse  length,  537-538.  See  also. 
Pulse  broadening, 
group  delay  dispersion,  537-538 
group  velocity  dispersion,  537 
measurement,  115,  901-903 

Optical  resonator  in  laser,  81-82,  116. 
laser,  81-82 
maintenance,  116 

Optical  sectioning,  9-10,  13,  180,  182,  222, 
223,  236,  268-270,  469,  748, 
763-764,  772,  774,  775,  784.  See 
also.  Deconvolution,  Confocal, 
etc. 

algorithms  for  widefield,  763-764 
of  A.  Thaliana  root,  772,  775 
with  confocal  laser-scanning  microscope, 
9-10 

example,  182,  463,  471,  492,  656 
dynamic  imaging,  784 
improvement,  with  deconvolution,  752 
latex  bead,  3D  image,  196 
limiting  excitation,  223 
near  surface  of  living  embryo,  748 
near  to  refractive  index  interface,  180 
selective  plane  illumination,  748 
structured  illumination,  268-270 
with  widefield  phase-dependent  imaging, 
13 

Optical  system,  losses,  25-32,  217. 

Optical  transfer  function  (OTF),  164-165, 
490-491,  562,  563,  567,  569-570, 
578.  See  also.  Point-spread  function; 
Contrast  transfer  function. 

4Pi  microscopy,  562,  563,  567 
contrast,  164—165 

deconvolution  for  image  enhancement, 
490-491 
I5M,  569-570 


point  spread  function,  490-491.  See  also. 
Point-spread  function 
STED  comparison,  578 

Optical  tweezers,  89-90,  110,  218,  383,  385. 
setups  for  integrating,  218 
table,  385 

trapping  wavelength,  89-90 

Optics,  general,  12,  125,  156-157. 
finite  i's.  infinity,  156-157 

Optiscan  confocal  endoscope,  213-214. 

Organic  dyes,  109,  203,  342-343,  353-356. 
See  also.  Dyes;  Fluorophores; 
Fluorescent  labels;  Fluorescent 
probes. 

AlexaFluor,  353-355 
BOPIDY,  353-355 
classes,  table,  355 
coumarin,  353,  355 
cyanine,  353-355 
fluorescein,  353-355 
rhodamine,  109,  203,  353,  355 

Organic  light-emitting  diodes  (OLED),  143. 

Orthogonal-plane  fluorescence  sectioning 
(OPFOS),  672-673. 

Oryza  sativa.  See  Rice. 

Oscillating-fiber  scrambler,  8. 

Osmotic  permeabilization,  359. 

OTF.  See  Optical  transfer  function. 

Out-of-focus  light. 

deconvolution  vs.  confocal  microscopy, 
461. 

information,  26,  32,  487,  644-646. 

Output  amplifier,  reconstructing  analog 
signal,  64. 

Output  modulation,  of  semiconductor  lasers, 
108. 

Overheating,  of  filters,  43.  See  also. 

Thermal  variables. 

Overlap  alignment  protocol,  montaging, 

732. 

Over-sampling,  60,  70,  728. 

vs.  duplicate-and-smooth  process,  70 
reasons  for,  68 

subpixel,  resampling,  478^179 

Oxygen  sensor,  45,  347. 

P 

Pack-and-go  mode.  Power  Point,  842,  844. 

Paeonia  suffruticosa,  421. 

Panda  pattern,  polarization-preserving  fiber, 

88. 

PAS.  See  Periodic-acid  Schiff. 

Passively  mode-locked  lasers.  111. 

Patch  clamp,  for  loading  dye,  360,  726-727 , 
734,  738-740. 

Patch  pipette,  738. 

Pattern  analysis.  See  Automated 

interpretation  of  subcellular  patterns. 

Patterned-illumination  microscopy,  see 

Structured  illumination  microscopy 

PC.  See  Personal  computer. 

PC  A.  See  Principal  component  analysis. 

P-CARS,  Polarization-sensitive  detection 
CARS. 
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PCF.  See  Photonic  crystal  fiber. 

PE.  See  Photoelectrons. 

Pear  ( Pyrus  serotina),  spectrum,  image, 
711. 

Pearson’s  correlation  coefficient,  668. 
Pellicle  beam-splitter,  216,  228-229,  231, 
346. 

Peltier  cooling. 

CCDs,  234,  447 
cell  chamber,  387-389 
lasers,  85,  106-108,  111,  117 
Penetration  depth,  177,  343,  643,  672,  731, 
765. 

of  dyes,  360,  387,  731,  739,  882,  874 
of  fixative,  369-370,  376,  857 
FRET  sensors,  798-799 
long  laser  wavelengths,  109,  416,  418, 
427-428 

multi-photon,  381,  418,  433,  435,  439, 
543,  545,  558,  646,  684,  708,  714, 
728,  749,  904 
in  plant  imaging,  779 
in  scanning  electron  microscopy,  847 
in  SPIM,  613,  675-678 
TIRF,  177-178 

Peony  flower,  autofluorescent  petals, 
173-174,  176,  421,  423. 
Peptide-mediated  uptake,  359. 

Perfusion. 

chambers,  381,  386-389,  394,  726,  729, 
769,  870-873 
fixation,  376 

Periodic  grating.  See  Grating. 

Periodic-acid  Schiff  (PAS)  reagent,  262, 
369,  770,  774-775,  778. 
maize  pollen  grain,  202 
Periodically  poled  (PP)  waveguides, 
114-115. 

Perrin-Jablonski  diagram,  516,  517,  697, 
792. 

photobleaching,  697 
Personal  computer  (PC),  performance 
needed  for  image  processing, 
289-290. 

Perspectograph,  early  studies,  789. 

Petran  disk,  2,  6,  11,  135,  141,  215, 

223-224,  228,  251,  265,  381,  387, 
447,  458,  554. 

Petran,  Mojmir,  2,  6,  11,  215,  223,  228. 
pH  imaging,  188-189,  221,  346,  348,  359, 
386,  421,  517,  529-530,  664, 
739-740,  743,  744. 
calibration,  421,  530,  745 
display,  287 

intensity  image,  529,  530,  739,  740, 

744 

lifetime  image,  530 
pH  indicators,  346,  739-742. 
pH  shift/formaldehyde  fixation,  370-371, 
373. 

Phalloidin,  as  correlative  marker,  235-236, 
344,  376,  378,  694,  696,  756,  804, 

854-856. 

Pharmacological  screening,  813-814. 


Phase  and  intensity  determination  from 
correlation  and  spectrum  only 
(PICASO),  115. 

Phase  contrast,  9,  171,  368,  372,  453,  506, 
643,  649,  731,  851,  854,  890,  892. 
coherent  light  for,  130 
depth  of  field,  13 
and  holography,  7 
scanning,  9,  13,  386 
Phase  fluorometry,  518-519,  526. 

comparison  of  FLIM  methods,  table,  526 
excitation/emission  signals,  519 
fluorescence  lifetime  imaging,  518-519 
Phase  randomization,  to  scramble  light,  8, 

13,  84,  131-132,  143,  507. 
Phase-dependent  imaging,  depth  of  field,  13. 
Phase-shifting  interferometry,  245. 
Phenotyping,  623-624. 

Phong  shading,  308-309. 

Phong/Blinn  models,  308. 

Phosphoinositide  signaling,  799. 

Photo  efficiency.  See  Photon  efficiency. 
Photoactivatable  dyes.  See  Photoactivation. 
Photoactivation,  187,  224,  383,  385,  541, 
543-545,  693,  759. 
example,  759 
genetically  encoded 
Kaede,  187,  383,  385 
Kindling,  574,  760 

PA-GFP,  187,  383,  385,  752,  759-760 
table,  385 

Photobleaching,  174,  218,  224,  275, 

341-342,  362-363,  545,  690-700, 
729,  747-748,  759.  See  also. 
Bleaching,  and  Chapter  39. 
autofluorescence,  698 
defined,  218,  691 
dynamics,  as  a  source  of  contrast, 

202-203 

effect  on  contrast,  174 

fluorescence  intensity  loss,  691,  694,  696, 

698+ 

fluorescent  image  of  single  protein,  699 
fluorescent  probes,  362-363 
fluorescent  recovery  vs.  irradiation  time, 
699 

fluorophores  signal  optimization,  341-342 
choice  of  fluorophore,  342 
fluorophore  concentration,  342 
light  collection  efficiency,  217,  341 
protective  agents,  36,  341-342,  363, 
368,  375,  499,  694 
spatial  resolution,  341 
in  four-dimensional  imaging,  747-748 
green  fluorescent  protein  (GFP),  690,  692, 
698 

intentional  See  Fluorescence  recovery 
after 

photobleaching  (FRAP) 
kinetics,  695 

mechanisms,  340,  691-693 
FRET,  691 

multi-exponential  fluorescent  bleaching, 
697 


multi-photon  microscopy  (MPM),  545 
Perrin-Jablonski  diagram  of  bleaching, 

697 

photocycling,  fluorescent  proteins,  698 
propidium  bound  to  DNA,  plot,  695 
reactive  oxygen  species,  341-342, 

362-363,  390,  544,  682-684,  691, 
693-694,  852-853 
reduction  in,  693-696 
antifade  agents,  36,  341,  368,  375,  499, 
694 

disk-scanning  microscopy,  224 
quantum  dots,  694 
results,  in  living  embryos,  759 
of  single  molecules,  696-698 
structured-illumination  methods,  275 
two-photon  excitation  microscopy 
(TPEM),  690,  697 

Photocathode,  PMT,  28-29,  232-233. 
quantum  efficiency,  232-233 
to  reduce  transmission  losses,  28-29 

Photoconductivity,  in  photodetectors,  252, 
253. 

Photocycling,  fluorescent  protein  molecules, 
698. 

Photodamage.  See  Phototoxicity. 

Photodetector.  See  Detectors;  Light 
detectors; 

CCD;  EM-CCD;  PMT  etc. 

Photodiode,  134-135,  253-255,  610, 
707-708. 

feedback,  to  stabilize  laser,  87,  682 
feedback,  to  stabilize  arc/filament, 
134-135,  137 
in  hybrid  PMT,  29,  30 
infrared  sensitive  for  IR  lasers,  707 
photometer  sensor,  air  space,  26 
quadrant,  for  alignment,  87,  134 
of  self-aligning  source,  134-135 
for  testing  display  software  response,  830 
vacuum  avalanche,  254,  255 

Photoelectric  effect,  and  LED  operation, 

137. 

Photoelectrons  (PE),  29,  30,  62-63,  77, 
232-234,  254-255,  257,  259-264, 
339,  633,  863. 
amplification  of,  62-63 
in  the  CCD,  232-234,  495,  918,  931 
production  in  PMT,  30 

single-PE  pulse-height  spectrum,  29,  77 
secondary  electrons,  as  source  of  PMT 
multiplicative  noise,  77 

Photoemissive  devices,  256-257. 

Photoemissive  effect,  254. 

Photographic  recording  systems,  6-7, 

11-12,  20,  22,  30,  71-72,  132,  139, 
141,  162,  207,  217,  263,  280,  488, 
581+,  588,  590-591,  593-594,  607, 
613,  628-629,  633,  640,  643,  712, 
829,  862,  865-867. 

“toe”  response,  quadratic,  71 

Photometer  paddle,  to  measure  light  beam, 
26,  35,  139-140,  159,  391,  650-651, 
665. 
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Photometric  response,  and  HD  curves,  71. 

Photomicrography  (Loveland),  139. 

Photomultiplier  tube  (PMT),  9,  28-31, 

35-36,  51,  62-63,  74-75,  222,  232, 
251,  254,  255,  258-261,  443,  527, 
661-662. 
after  pulsing,  257 
Bio-Rad,  260-261 

as  confocal  detectors,  pros/cons,  222 
for  epi-fluorescence  confocal  microscope, 
9 

functioning,  62-63 

GaAs  photocathode,  28-29,  232,  252, 
255,  263,  527,  931 

gain  from  collisions  at  first  dynode,  31 
grey  levels,  443 

hybrid,  single-pixel  signal  levels,  31, 
254-255 

linearity,  661-662 

microchannel  plate,  232-233,  255,  262 
mini-PMT  arrays,  51,  667 
multiplicative  noise,  28-30,  77,  633, 

677,  926.  See  also.  Multiplicative 
noise 

in  multi-channel  detection  systems,  51 
noise  and  gain,  74-75 
number  of  photons  striking  per  unit  time, 
35-36 

optical  enhancer  to  increase  QE,  28-30 
photon  counting,  21,  29-30,  32-35, 
258-259,  260-263,  542 
quantum  efficiency,  527 
Vi.  cooled  CCD,  26-28 
signal  variation  with  time,  232 
transit  time  spreads,  527 

Photon(s),  20-21,  30,  33-36,  63-64,  132. 
counting  precision,  20-21 
uncertainty,  63-64 

interactions  with  photomultiplier  tube,  30 
lost,  33-36 

Photon  counting,  21,  29-30,  32-35, 
258-259,  260-263,  542. 
circuits,  33-34,  258,  521 
digital  representation  of  optical  data, 
32-33 

effects,  34-35 
examples,  35,  263 
hybrid  PMT,  29-30 

pile-up  losses,  32-33,  35,  78,  521,  523, 

526-527 

with  PMT,  29-30,  32-35,  258-259,  260, 
263 

Photon  detector  types.  See  Detectors  and 
entries  by  each  detector  type. 

CCD,  254 
direct  effects,  252 
image  dissector,  254-255 
microchannel  plate,  232-233,  255,  262 
MCP-CCD,  262 
gated,  519,  523-524,  527,  532 
photoconductivity  effects,  252,  253 
photoemissive,  254 
photovoltaic,  252-253 


thermal  effects,  252 

vacuum  avalanche  photodiode,  254,  255 
work  functions,  table,  252-253 

Photon  efficiency,  24-36,  215,  217,  341, 
631. 

defined,  24 

as  a  limitation  of  confocal  systems,  24, 
223 

measuring,  26,  33-36,  217 
practical  confocal  microscopy,  631 
of  scanners,  215 
table  listing  photon  losses,  217 

Photon  flux,  statistics,  256-258. 

Photon  interactions,  252-256. 

Photon  (shot)  noise,  660-661.  See  also, 
Poisson  noise. 

Photonic  crystal  fiber  (PCF),  laser  delivery, 
1,  88,  109-110,  113,  504,  541. 
for  white  light  source,  113 

Phototoxicity,  112,  363-364,  390-391,  651, 
729,  746,  770. 
chapter,  680-689 
in  brain  slices,  729 

damage  is  higher  to  either  side  of  raster, 
54 

factors  influencing,  table,  363 
fluorescent  probes,  363-364 
live  cells,  390-391 
reduction,  391 

from  uneven  scan  speed,  65 1 

Photo-uncaging,  187,  210,  383,  385,  541, 
544-545,  692,  729,  760,  912.  See 
also.  Photoactivation. 

Photovoltaic  effect,  252-253. 

Phycobiliproteins,  338,  341,  343,  355-357, 
693. 

Physical  limitations,  20,  24,  63-64. 
on  accuracy  and  completeness  of  data, 

20 

Poisson  noise,  63-64.  See  also.  Poisson 
noise 

Physiological  fluorescence  imaging,  383, 
385. 

PICASO.  See  Phase  and  intensity 

determination  from  correlation  and 
spectrum  only. 

Piezoelectric  effect,  defined,  57. 

Piezoelectric  focus  controls,  166,  215,  219, 
222,  231,  241,  245,  268,  468,  754, 
909. 

Piezoelectric  scanning  systems,  57,  215, 
238,  510,  555,  610. 

Piezoelectric  devices. 

AOD  driver,  54-55,  57 
acousto-optical  components,  54—55,  57 
to  align  objective,  166 
dithering  to  increase  CCD  resolution, 

70 

effect  described,  57 
to  focus  objective,  166,  215,  219,  222, 
231,  241,  245,  268,  468,  754,  909 
laser  alignment,  87 
light  scrambler,  84 


to  move  optical  fiber,  84 
to  move  scanning  mirror,  57,  215,  238, 
510,  555,  610 
to  move  stage,  215,  567 
phase-shifter 

in  4Pi  confocal,  609 
in  structured  illumination,  268 
optical  coherence  tomography, 

609-610 

stretching  optical  fiber,  609 

Pile-up,  of  pulses. 

in  avalanche  photodiode,  253 
in  photomultiplier  tube  output,  32-35 
measuring  risk  of,  34—35 

p-i-n  diode,  253. 

Pinhole,  26-28,  33-35,  149,  150,  154,  201, 
210,  213,  215,  224-228,  395, 
631-632. 

advantages  and  disadvantages,  26-28 
calibrating  diameter,  33-34 
confocal,  proper  use,  28 
disk-scanning,  224-228 
mini-image  detection,  32 
optical  fiber  as,  506-507 
optimal  size,  226-227,  Chapter  22 
Fraunhofer  formula,  225 
position  in  confocal  microscope,  210 
practical,  in  confocal,  631-632 
radius,  effective,  35 
ray  paths,  different  sizes,  226-227 
single-mode  polarization  preserving  fiber, 
213 

small  pinholes,  effect,  225 
of  tandem  scanners,  215 
vibration  shifts  relative  positions,  201 

Pinhole  disks,  critical  parameters,  224-228. 

Pinhole  energy,  with  spherical  aberration, 
149,  150,  154,  631-632. 
penetration  into  water,  149,  150 
defocus  and  NA,  150 
defocus  and  wavelength  planapochromat, 
154 

Pixel  clock,  digitization,  62,  64-65,  201, 
234-235,  258,  903,  923,  929. 

CCD,  table,  929 

Pixels,  38-39,  60,  62-63,  65,  258-259. 
defining,  60 
digitization,  62-63 
optimal,  63-64,  66 
representing  intensity,  258-259 
and  resolution,  38-39 
and  Abbe  criterion  resolution,  38-39,  65 

PKzip,  580. 

Plan  objectives,  Zeiss,  field  diameter,  table, 
152. 

Planapochromat,  152,  155.  See  also. 
Objectives. 

flatness  of  field  and  astigmatism,  152 
lateral  chromatic  aberration,  155 

Plancks  law,  energy  of  photon,  35,  137,  252, 
424. 

Planar  illumination,  SPIM,  optical 
sectioning,  751. 
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Plane  of  focus,  distortion,  16,  23. 
by  beam  deviations,  16 
by  refractile  cellular  structures,  23 
Plant  cell  imaging,  769-785. 
autofluorescence,  770-772 
birefringent  structures,  162-164. 

420-421.  See  also.  Birefringence 
chamber  slides  for  plants,  429 
clearing  intact  plant  material,  166, 
417-418.  420,  439,  610,  624, 
774-775 

computer  visualization  methods,  778 
deconvolution,  784-785 
direct  imaging,  772-773 
dynamic  imaging,  783-784 
effect  of  fixation,  195,  428 
Equisetum,  774 

fluorescence  properties,  421-428 
emission  spectra,  421-423 
microspectroscopy,  421-426 
fluorescence  resonance  energy  transfer, 
425.  See  also,  FRET 
harmonic  generation  See  Harmonic 
signals  fungi,  438-439 
genetically  encoded  probes,  769,  773, 

783 

green  fluorescent  protein  fusions,  773, 

783 

of  green  tissues,  770 
hairs,  434^-35 
history,  769 

light  attenuation  in  plant  tissue,  4 1 4 — 4 1 8 
A.  thaliana,  example,  416 
absorption  spectrum,  415 
effect  of  fixation,  428 
maize  stem  spectra,  417,  418 
M.  quadrifolia  spectra,  416 
M.  quadrifolia  optical  sections,  419 
Mie  scattering,  162-163,  167,  417 — 418 
nonlinear  absorption,  416-417 
Rayleigh  scattering,  162-163,  167,  417, 
703 

light  effects  on,  770 
light-specimen  interaction,  425-428 
living  plant  cell  specimens,  429-439 
calcofluor  staining  procedure,  424, 

438 

callus,  429 

cell  walls,  168-169,  188-189,  303, 

306,  416-417,  420-421,  428-431, 
435-136,  438,  439,  710-711, 
713-715,  769-776,  779-781 
chamber  slides,  use,  429 
cuticle,  434-437,  715,  717,  779 
fungi,  438-439,  624,  782,  870 
hairs,  431,  434-436,  772 
meristem,  168,  420,  430,  770,  776-778, 
783 

microsporogenesis,  431-432 
mineral  deposits,  163,  420,  436-438, 
703 

pollen  germination,  420,  433-434,  781, 
783 


pollen  grains,  202,  305,  313,  420, 

431- 433,  553,  558,  781,  783 
protoplasts,  195-196,  203,  416,  421, 

423-427,  429-431, 438-439,  693 
root,  172,  174,  303,  307,  421,  429, 
430+,  438,  464-465,  556,  772-773, 
775,  777,  779-783 
culture  chamber,  429 
starch  granules,  202,  420-421,  428, 

432- 433,  435,  703,  710-712,  715, 
719 

stem,  168,  172,  180,  417-419,  421, 
424,  429,  556,  707,  710-711, 
713-714 

storage  structures,  435-436 
suspension-cultured  cells,  189, 
429^130 

tapetum,  433-434,  779 
waxes,  420,  428,  434-435,  714-715 
new  spectral  tools,  770 
obtaining  spectral  data,  methods,  772 
penetration  values,  779 
photodamage,  770 
point  spread  function,  722,  784 
refractive  index  heterogeneity,  192, 

418 — 420 

single-photon  confocal  excitation, 
772-778 

specific  methods,  769 
spectral  unmixing,  770 
examples,  665-666 
staining,  774 

technological  developments,  769 

textbooks,  769 

three  dimensional,  77 1 

clearing  agents,  166,  417-418,  420, 
439,  610,  624,  774-775 
deconvolution  protocols,  784 
reconstruction,  775-776 
segmentation,  776-778 
two-photon  excitation,  415-419,  421, 
423 

advantages,  778-779 
best  conditions,  781 
compared  with  one  photon,  421 
cell  viability,  779-782 
deconvolution  protocols,  784 
dyes,  782 

green  fluorescent  protein,  782-783 

light-specimen  interaction,  425-427 

microspectrometer,  424 

pitfalls,  782 

thick  specimens,  779 

in  vivo,  781 

Plasma  membrane,  microscopy.  See  Total 
internal  reflection  microscopy 
(TIRF). 

Plasma  light  sources,  spectra,  44. 

Plasmid  DNA,  nick-damage,  684,  724, 

802-804.  See  also.  Microinjection; 
Electroporation;  Biolistic 
transfection. 

Plasmodesmata,  777 . 


Plumbago  auriculata,  fluorescence  spectra, 
422. 

PMT.  See  Photomultiplier  tube. 

p-n  diode,  253.  See  also.  Photodiode. 

PNG  (Portable  network  graphic),  581,  584. 

Pockels  cell,  variable  beam  attenuator,  25, 
54,  57,  87,  116,  543,  701,  903-904. 

Pockels  effect,  in  crystals,  57. 

Point-spread  function  (PSF),  4,  10,  23,  27, 
39,  68-70,  145-146,  189-190,  208, 
223,  239-250,  271,  275,  330,  378, 
405,  407,  409,  446,  448,  453-457, 
485-486,  489^194,  536,  562-564, 
570,  574,  578,  635,  656,  674,  750, 
784,  830,  895. 

3D.  68-70,  247-248 
4Pi  microscopy,  562-563 

additional  information  from,  570 
space  invariance  of  PSF  for,  564 
apodization,  240,  243,  249,  250.  272,  567, 
889 

blind  deconvolution,  468,  485 
in  botanical  specimens,  772,  784 
in  brain  slices,  729 
calculations,  Rl-mismatch,  407 
for  glycerol,  table,  409 
for  water,  table,  409 
CARS,  596 

comparing  widefield  with  confocal,  27, 
453-457,  493,  644-647 
confocal,  10,  12,  208+,  212,  216,  405, 

632,  681 

vs.  deconvolution,  27,  453-457,  493, 
644-647 

deconvolution,  189-190,  223,  489, 
490-494,  784.  See  also. 
Deconvolution 
quantifying  PSF,  492-494 
deformation  caused  by  RI  anomalies, 
22-23 

Fourier  transform,  489,  490 
lateral  resolution.  See  Lateral  resolution 
measuring,  240-242,  455,  462,  471, 
656 

amplitude/phase,  242 
fiber-optic  interferometer,  240-241 
images,  246-248 

high-NA  objectives,  239-250,  492,  656 
pupil  function,  240 
for  3D  deconvolution,  145-146 
non-linear,  552,  750 
and  Nyquist,  635,  636,  751,  752 
optical  transfer  function,  related  to, 
490-491 

polarization  effects,  249-250 
pupil  function,  245-248.  See  also.  Pupil 
function 

Rayleigh-criterion  and  Nyquist  sampling, 
39 

refractive  index  mismatch,  405,  407 
spherically  aberrated,  148-150,  407,  492 
shape  in  telecentric  systems,  208 
SPIM,  674 
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Point-spread  function  (PSF)  ( cont .) 

STED,  diagram,  574,  578 
structured  illumination  see  Structured 
illumination  microscopy 
temperature  effects,  25,  85,  248-249,  630 
terminology,  405 
Wiener  filtering,  494 
Points,  defined,  59. 

Point-source,  for  measuring  photon 
efficiency,  33. 

Poisson  noise,  20-21,  29,  37,  63-64,  67,  69, 
74-75,  81,  164-165,  211,  232,  234, 
442,  456,  460-463,  468,  487,  495, 
497,  633-636,  647,  651,  655,  660, 
693,  784,  835,  923-924,  926.  See 
also ,  Quantum  noise.  Shot  noise, 
bleaching,  693 
of  CCD 

charge  transfer,  920 
dark  charges,  921-922 
CT  imaging,  615 
and  display  linearity,  72-73,  588 
digitization,  as  part  of  signal,  65,  69, 
633-636 

of  EM-CCD,  233-235,  262,  927-928 
and  FLIM,  524-525 
and  gray  levels,  74 

importance  of  deconvolution,  38-41,  60, 
69,  189-190,  222-223,  320,  399, 
471-472,  481,  495,  751-753,  835 
intensity  spread  function,  75-78 
photomultiplier  tube,  74-75 
affects  effective  QE,  3 1 
multiplicative  noise,  29,  647,  660 
in  photon  detection,  63-64 
and  pixel  size,  64,  68,  633-636,  928 
practical  effects,  67 

single-pixel  noise,  65,  67,  190,  635,  832, 
835-836 

spectral  unmixing,  667,  770 
examples,  665-666 
structured  illumination,  278 
uncertainty  in  contrast,  74,  164—165 
and  visibility,  37,  667 
Polarization,  13,  49,  57,  83,  88,  89, 

211-212. 

attenuator,  43,  543,  907 
beam-splitter,  13,  50-51,  57,  85,  87,  100, 
171,  217,  513,  631,  904 
to  avoid  spectral  distortion,  49 
circular  or  phase  randomized,  211-212, 
229 

effect  on  AODs,  55 
effect,  of  dichroic  beam-splitters,  34, 
49-50 

Kerr  cell.  111,  113,  516 

of  laser  light,  8,  83,  88-89,  113,  478, 

558 

optical  components,  57,  155,  211 
optical  fibers,  213 

Pockels  cell,  25,  54,  57,  87,  116,  543, 

701,  903+ 

rectified  DIC  optics,  846 


to  reduce  reflections,  6,  25,  141,  158,  171, 
516,  229.  See  also.  Antiflex  system 
scramblers,  8,  84,  132,  143 

Polarization  effects,  211,  249-250,  503. 
birefringence,  188,  420-421,  431,  434, 
436,  438,  480,  503.  See  also. 
Birefringence 
blind  deconvolution,  479 
and  CARS  microscopy,  595,  600-604 
high-NA  objective  lenses,  249-250, 

267 

interaction  with  nucleus,  23 
optical  fibers,  503,  507 
stereo  image  displays,  299,  589 

Polarization  microscopy,  43,  50-51,  154, 
156,  162,  188,  288,  348,  438, 
479^180,  513,  555,  711,  714-715, 
717,  719,  891,  894. 
centrifuge  microscope,  8 
of  collagen  fibers,  164,  188,  717 
DIC,  10,  14,  127,  146,  468,  473 
and  FRET,  793 

and  harmonic  generation,  179,  428, 
704-706,  717,  719 
MFMP,  555 

mitotic  apparatus,  15,  717 
p-  and  s-,  and  incidence  angle,  50-51 
Pol-scope,  13,  188,  432,  468.  480 
PSF,  406-407 

to  regulate  light  intensity,  43 
STED,  578 

Polarization  noise,  in  lasers,  83. 

Polarization-preserving  fiber,  49,  87,  503, 
505,  507. 
as  a  pinhole,  213 

Polarization-sensitive  detection  CARS  (P- 
CARS),  600,  601,  604. 
adipocyte  cells,  604 

Polarized  light,  7,  14,  83-85,  146,  158,  162, 
171,  229,  406-407,  420,  479,  894. 
deconvolution,  479 
image  formation,  406-407 
PSF,  479 

Polarizer,  83,  128,  188,  249,  268,  275,  420, 
479,  711,  903-904. 
for  antiflex,  6,  84,  141.  158,  229 
for  attenuation,  43,  85,  87-88,  543, 
903-904 
for  CARS,  601 
Glan-Taylor,  85,  87,  100,  171 
Glan-Thompson,  85,  904 
LCD,  589,  715 
micro-wire,  85 
structured  illumination,  264 
tunable,  715 

Pollen  germination,  433-434. 

Pollen  grains,  202,  305,  431-433,  438,  553, 
556,  558,  678,  781,  783. 
germination,  433-435,  783-784 
multi-focal  multi-photon  imaging,  556 
Pol-scope,  13,  188,  432,  468,  479-480 
test  specimen,  195,  269,  313,  553,  556, 
678 


Pol-scope,  13,  188,  432,  468,  479^180. 
deconvolved,  479 
images,  15,  188,  479,  717 
Portable  network  graphic.  See  PNG. 
Position,  accuracy  in  CLSM,  40. 

Position  sensors,  galvanometer,  53-54. 
Posterizing,  591. 

Potassium  titanium  oxide  phosphate  (KTP) 
crystal 

for  non-linear  optical  frequency  conversion, 
107. 

Potato  ( Solarium  tuberosum)  SHG  signal, 
712. 

Power  requirements,  for  lasers,  65,  80-81. 
Power  spectrum.  See  Contrast  transfer 
function. 

Power  supply,  laser  as  noise  source,  86. 

PP,  Periodically  poled  waveguide, 

114-115. 

Practical  confocal,  2-photon  microscopy, 
tutorial.  See  also,  each  topic  as  a 
major  entry. 

2-photon 

excitation  duty  cycle,  644 
peak  power  level,  644 
photodamage  vi.  penetration,  645 
power  vi.  penetration,  646 
3D  microscopy  methods  compared,  table, 
647+ 

best  3D  method  for,  644-647 
biological  reliability,  63 1 
bleaching  pattern,  627-628 
quantum  efficiency,  628 
chapter,  627-649 

confocal  images  with  few  photons,  634 
deconvolution,  factors,  646 
filling  back-focal  plane,  210,  509,  629, 
633 

focus,  compensating  drift,  395,  732 
getting  a  good  confocal  image,  629-631 
alignment  of  optics,  629-630 
back-focal  plane  (BFP),  210,  509,  629, 
633 

focus,  629 

low  signal,  631 

mirror  test  specimen,  630 

no  signal,  631,  660 

simultaneous  BSL/fluorescence, 

631 

getting  started,  627 

Kohler  illumination  for  transmission,  34, 
127-128,  131,  229,  627,  648-649 
multi-photon  vs.  single-photon,  646 
new  controls,  631-636 
biological  reliability,  63 1 
pinhole  size,  631-632 
pixel  size,  62,  634-635,  784,  928 
Nyquist  reconstruction/deconvolution, 
635-636 

over-sampling,  635 

photon  efficiency,  24—26,  215,  217,  341, 
631 

pinhole  summary  for,  26-28,  633 
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pixel  size,  62,  634-635,  784,  928 
measuring,  635 
summary  for,  636 

poor  performance,  reasons,  640-643 
air  bubbles,  643 
curvature  of  field,  641 
dirty  objective,  642-643 
imaging  depth,  643 
under  filling  objective  pupil,  642 
optical  problems,  640-641 
sampling  problems,  640 
singlet-state  saturation,  643 
under-sampling,  635 
schematic  diagram,  632 
statistical  considerations,  633-634 
stray  light,  201,  632,  904 
test  specimen,  636-640 
description,  636-637 
diatom,  638-640 
figures,  637-640 
reasons  for,  636 

widefield  vs.  beam  scanning,  647 

Prairie  Technologies,  LiveScan  Swept  Field 
design,  237. 

Pre-amplifier,  in  digitizing  analog  signal, 

64. 

Precompensation,  in  fiber  optic  cables,  88. 

Presentation  software,  829-845. 
helpful  URLS,  844-845 
movies,  837-844 
artifacts,  839-840 
coding  limitations,  838 
compression  of  large  movies,  graph, 
843 

compression  of  PAL  TV  movies,  table, 
842 

digital  rights  management,  844 
entropy,  841 

frame  count  matching  display  cycle, 
838-839 

MPEG  display  formats,  840-841 
overlaying,  844 
Pack-and-go  mode,  842,  844 
performance  benchmarks,  841-842 
region  code,  844 
remote  use,  842-844 
rules  837-838,  844 
up-sampling,  838-839 
very  high  resolutions,  841 
precautions,  829-830 
testing,  830-836 
aliasing  gallery,  834 
aligning  images,  835 
brightness,  832 

changing  display  size,  832-835 
codecs,  831 
compression,  835-836 
compression  artifacts,  837 
cropping,  835 

digital  rights  management  (DRM),  830 
down-sampling  in  PowerPoint,  834 
fast  graphics  cards,  831,  832 
gamma,  832-833 


measuring  display  speed/sensitivity, 
830 

random  color  dot  image,  836 
reference  images,  830-831 
removing  distortion,  835 
resolution,  832-835 
rotating,  835 
scaling,  835 
screen  capture,  830 
static  image  performance,  831 
step  image,  833 
under-sampled  image,  835 
up-sampling,  example,  834 
viewer,  830 

Preventive  maintenance,  lasers,  115-116. 

Principal  component  analysis  (PCA), 
731-732. 

Printers,  591-593. 
aliasing,  592 
color  images,  592 
grey  levels,  592 
ink  jet,  593 
laser,  593 
posterizing,  591 
scaling  techniques  for,  592 

Prionium,  MMM  image,  556. 

Probe,  mismatch  with  pixel  shape,  39. 

Processor  performance,  3D-image  display, 

289+. 

Projection/compositing  rules,  3D-image 
display,  302-304,  763-764. 
alpha  blend,  302,  304 
average  intensity,  302 
first  or  front  intensity,  302 
Kalman  average,  304 
maximum  intensity,  302 

Propidium  iodide,  344,  355,  360,  426, 

651,  693-695,  773,  778-779,  782, 
812. 

dead  cell  indicator,  426,  651,  875,  877 

Proteins,  195,  756,  760,  794-795,  804.  See 
also,  Green  fluorescent  protein,  etc. 
chimeric  fusion,  794 
fluorescent,  FRET,  794-795 
Kaede,  187,  383,  385 
Kindling,  760 
microinjection,  804 
PA-GFP,  187,  383,  385,  752,  759-760 
tagged,  756,  758 
translational  fusions,  756 
UV  absorption,  195 

Proteomics,  237,  790,  804,  809,  818,  867. 
location,  825 

Protoplasts,  195,  416,  429,  430,  431. 

A.  thaliana,  195-196,  203,  416,  421, 
423-427,  429-431, 438-439,  693 

Proximal  tubule,  labeled,  744. 

Pseudo  color  display,  173-175,  190,  291. 

PSF.  See  Point  spread  function. 

Pulse  broadening,  88,  111,  210,  537-538, 
543,  606,  609,  728,  903. 

Pulse  length  measurement,  115,  901-903. 

Pulse  spreading.  See  Pulse  broadening. 


Pulsed  lasers,  81,  96-100,  110-114,  120, 

137.  See  also.  Lasers;  Ultrafast 
lasers. 

broadband  tunable,  table,  120 
diode,  table,  96-97 
DPSS,  table,  98 
dye,  table,  96 
excimer,  table,  96 
for  FLIM,  537 
kits,  table,  98,  100 
nitrogen,  table,  96 

scanning  only  region  of  interest,  237 
for  2-photon  excitation,  8 1 
ultrafast,  table,  99-100 
vapor,  table,  97 

Pulse-counting  mode,  21,  29-30,  32-35, 
258-259,  260,  263. 

Pump  sources,  for  dye  lasers,  103. 

Pumping  media,  maintenance,  116. 

Pumping  power  vs.  frequency  cubed,  65,  82. 

Pupil  function,  211,  245-248. 

3D  point  spread  function  restored, 
247-248 
4Pi,  566-567 
AOD,  56 

empty  aperture,  248 
of  human  eye,  72,  128 
intermediate  optics,  211,  222,  225,  250 
Kohler  illumination,  34,  127-128,  131, 
229,  251,  627,  648-649 
Mach-Zehnder  interferometry,  245 
measurement,  246-248 
images,  246-248 
objective,  24,  155,  158-159,  211, 

239-240,  242,  492,  551-552,  554, 
566-567,  650 

orthonormal  Zernike  polynomial  for, 
table,  247 

phase-shifting  interferometry  measuring, 
245 

polarizing  effects,  249 
pupil  plane,  50  See  also.  Back-focal  plane 
transfer  lens,  728 
view  of  pupil  image,  629 
Zernike  polynomial  fit,  245-247 

Purkinje  cells,  Golgi-stained,  167-168. 

Pyrus  serotina.  See  Pear. 

Q 

QE.  See  Quantum  efficiency. 

Q-switched  pulsed  laser  systems.  111, 
114-115. 

Quantitative  analysis,  flying-spot 
microscope,  6-7. 

Quantization,  limitations  imposed  by,  37-39. 
See  also.  Chapter  4. 

Quantum  dots,  221,  343,  357-358,  360-361, 
656,  694,  696,  757,  801,  814,  846, 
853.  See  also.  Semiconductor 
nanocrystals, 
assays  for,  814 

in  electron  microscope,  852-854 
FRET,  801 
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Quantum  dots  ( cont .  ) 
labeling,  853 
toxicity,  357,  694 

Quantum  efficiency  (QE),  25-30,  74-78, 
222,  232-234,  238,  251,  254-255, 
349,  355,  375,  383,  390,  442^43, 
459,  516,  527,  575,  628,  646,  556, 
703,  751,  793,  920-922. 
of  back-illuminated  CCD,  77-78 
charge-coupled  device  (CCD),  26-28, 
74-76,  142,  215,  232,  234,  257-258, 
261,  644,  707,  751,  754,  810, 
920-921 

comparative  among  CCD  cameras,  76 
effect  on  Poisson  noise,  74-75 
effective  QE,  of  photon  detectors,  28,  29 
of  electron-multiplier  CCD,  4,  30,  59, 
234,  920 

FLIM,  516-517,  520,  523,  526-527,  529, 
530 

FRET,  792 
of  human  eye,  25 1 
and  intensity  spread  function,  74—75 
and  multiplicative  noise,  77 
optical  enhancer,  to  increase  QE,  28-30 
optimal  3D  microscopy,  644 
photomultiplier  tube  (PMT),  26-28,  51, 
77,  222,  257,  262,  527,  707 
graph,  29 
table,  707 

signal-to-noise  ratio,  263,  442-443 
variation  with  wavelength,  29 
vs.  wavelength,  922 

Quantum  noise,  21  63-64,  69-70,  468,  472. 
See  also,  Poisson  noise, 
and  approximation,  for  reconstruction, 
69-70 

Quantum  wells,  as  absorbers.  111. 

Quantum  yield,  of  fluorescent  dyes,  172, 
180,  338-345,  353,  360,  363,  383, 
421,  543-544,  574,  661,  683,  690, 
710,  737,  792,  794-795. 

Quartz-halogen  lamp,  control,  138-139.  See 
also.  Halogen  lamps. 

R 

Rabbit,  237,  744. 
antibodies,  855,  877-878 
kidney  proximal  tubule,  pH,  744 

Radiance,  of  non-laser  light  sources,  126, 
132,  137-139,  141. 
measuring  with  radiospectrometer, 

139  ^ 

Table,  1140 

Radiospectrometer,  radiance  vs.  wavelength, 
139. 

Raman  background,  in  glass  fibers,  88,  90, 
162,  506-507. 

lower  in  large-mode-area,  fiber,  110 

Raman  scattering,  162-163,  167,  339-340, 
348,  506-507,  545,  697. 
and  bleaching,  697 
defined,  162 


Raman  spectroscopy,  48-49,  90,  167,  254, 
339-340,  507,  545,  697.  See  also, 
CARS. 

CARS,  204,  550,  577,  595-605 
chemical  imaging,  90 
hard-coating  on  interference  filters  used, 
48-49 

image  contrast,  167 
Ramp-up,  for  light  sources,  136,  137. 
and  long-term  stability,  137 
and  short-time  stability,  136 
Rare  earths,  for  doping  fiber  lasers,  110. 
Raster,  62-64. 

convolution,  485-486 
dimensions,  in  specimen,  63 
retrace,  25,  33,  53-54,  219,  338,  389, 
543,  628,  651.  908.  See  also. 
Retrace,  raster  scanning  shape, 

63 

size,  V5.  pixel  size  and  light  dose,  64 
temporal  limitations.  141 
Raster  scanning.  5-6,  25.  141-142,  223, 
540,  596. 

alignment,  629-630,  651 
assymmetrical  sampling,  38-40 
bleach  pattern,  3D,  538,  628,  693 
chromatic  aberration  limitations,  156, 
640-641 

damage  is  higher  to  either  side  of  raster, 
54 

display,  830-831,  835 
distortion,  40 

and  electronic  bandwidth,  70,  238 
for  fast  confocal  imaging,  223 
fiber-scanning,  214,  508 
galvanometer  limits,  52-54,  223,  65 1 
limitations  imposed  by  AODs,  56 
MPEG  formats.  840 
Nyquist  sampling,  38,  41,  59-60,  62, 
634-635 

off-axis  aberrations,  151,  640-641, 
659-662 

pattern  on  Nipkow  disk,  5-6,  223-225. 

See  also,  Nipkow  disk  scanning 
retrace  gating.  25,  54,  56,  219,  389,  543, 
628,  651,  908 
scan  angles,  209,  214 
stability,  708 

sampling  in  time  and  space,  141-142 
timing,  33,  53,  753 

zoom,  raster  size  and  magnification,  1 1 , 
24,  37,  63-64,  66,  70,  79,  317,  389, 
493,  627,  634-636,  655-658,  683, 
731 

Rat,  cells  and  tissues,  205,  320,  323,  330, 
398,  739,  813. 
brain  slices,  393,  398,  686 
CA1  region,  323 
cardiac  muscle,  498,  529,  556 
cerebellar  granule  neurons,  813 
EDL  muscle,  calcium,  740 
fixation,  370,  372,  393 
hippocampus,  268,  317,  341 


interossi  muscles,  SNARF-1,  pH  image, 
739 

intervertebral  disk.  310-311 
kidney,  511,  803 
leukemia  cells,  347,  520-521 
FLIM  image,  521 
neuron,  membrane  potential,  205 
tooth,  667 

Rate,  imaging,  limited  by  signal  level,  73. 

Ratiometric  imaging,  189,  346-347.  See 
also.  Calcium  imaging,  pH,  etc. 
bleach  ratio,  697-698 
calcium.  736-737 ,  850.  See  also.  Calcium 
imaging 

CARS,  600,  602,  604.  See  also,  CARS 
concentration  calibration,  742-745 
to  detect  colloidal  gold  labels,  167 
to  determine  ionic  concentration,  36 
FLIM,  516-532.  See  also,  FLIM 
FRET  174,  184,  790,  794-795,  797-798. 
See  also,  FRET 

glutaraldehyde  autofluorescence  assay, 

369 

HCS,  high-content  screening,  813, 
823-824. 

indicator  choice,  738 
interpretation,  740-741 
live/dead  assay,  875 
pH,  739-744.  See  also,  pH  imaging 
structured  illumination.  See  Structured 
illumination  microscopy 
water-immersion  objectives,  737 

Rayleigh  criterion  (Abbe  criterion).  1-3,  9, 
37-39,  60-61,  66,  129,  146,  486, 
703,  822,  928. 

breaking  the  Abbe/Rayleigh  barrier, 
571-573 

Nyquist  sampling,  39,  60.  66 
of  two  point  images,  1-3,  146 

Rayleigh  scattering,  162-163,  166,  167, 

339,  342,  417,  703,  747. 
compared  to  Mie  scattering,  163 
in  embryos,  747 
by  colloidal  gold  labels,  167 
light  attenuation  in  plant  tissue,  417 
wavelength  dependency,  162-163 

Rayleigh  unit,  147. 

Reactive  oxygen  species  (ROS),  341-343, 
362-363,  390,  544,  682-684,  691, 
693-694,  852-853.  See  also. 
Bleaching;  Phototoxicity, 
as  basis  of  correlative  TEM  staining, 
852-853 

Readout  noise,  74-75,  77,  232.  See  also. 
Noise. 

and  readout  speed,  77 

Real  image,  disk-scanners,  224. 

Real-time  2D  imaging,  12-13,  167-168, 

215,  222-224,  232,  235,  307,  496, 
542. 

Real-time  3D  imaging,  154. 

Receptors. 

cholera  toxin,  790-791,  796-797,  802 


Index  975 


deconvolution,  495 
EGF,  533 
ERD2,  791,  796 
fibrinogen,  846-847,  850 
high-content  screening,  809,  812-814 
KDEL,  790,  797 
ligands,  354 
lipid,  790,  791 
proteins,  357 
Streptococcus,  879 
transferin,  819 
uncaging,  545 
Reconstruction,  3D. 
definition,  280. 

Nyquist  and  filtering/deconvolution,  59, 
66-67,  69,  70,  173,  235-236, 
280-315.  458,  468-469,  474-475, 
496-497,  563,  585,  603,  607,  610, 
615,  635,  672,  675,  677-678,  690, 
772,  730,  731,  762,  77,  774-776, 
778,  784,  883 
Recording  times,  141-142. 

in  widefield  microscopy,  141-142 
using  LED  source  systems,  141-142 
Recovery  curve,  after  bleaching,  187. 

Red  fluorescent  protein  (RFP),  221-222. 
Reference  list, 
historic,  889-899 
lasers,  123-125 

Reflected-light  images,  180,  181.  See  also, 
Backscattered  light, 
confocal,  of  integrated  circuit,  180 
of  glass  bead,  in  water,  181 
Reflecting  objectives,  constraints,  156. 
Reflection  contrast  technique,  Antiflex,  159. 
Reflection  mode,  low  coherence  light,  130. 
Reflectivity,  optical  surfaces,  159,  163, 
167-171. 

anti-reflection  coatings,  158 
on-axis,  artifact,  168-171 
refractive  index,  159,  163,  167 
Refracting  regions  affect  imaging  beam, 
15-16. 

Refractive  index,  (RI),  14-15,  23,  45,  148, 
152,  163,  198,  377,  404-413, 
418-420,  613,  654.  See  also. 
Spherical  aberration;  Dispersion, 
anomalies  in,  effect  on  PSF,  23,  418-420 
of  biological  structures,  163,  377 
table,  277 

of  botanical  specimens,  418-420 
coverslip  thickness,  importance,  table, 
654 

of  immersion  medium,  277,  411 
effect  on  PDF,  23,  418-420 
effect  on  sharpness,  14—15 
effect  of  wavelength  and  temperature 
on,  148,  248-249,  411 
and  intensity,  and  spectral  broadening, 

111 

of  layers  in  interference  filters,  45 
of  mounting  media,  table,  198,  342, 
370-371,  373-377 


of  optical  glass  vs.  wavelength,  152 
optical  projection  tomography  (OPT),  613 
self-shadowing,  198 
temperature,  148,  248-249,  411 
of  tissue/organs,  table,  377 
Refractive  index  mismatch,  effects, 

404-413.  See  also.  Spherical 
aberration. 

table  for  glycerol,  409 
table  for  water,  409 
calculation,  404-407 
dependence  of  focal  shift,  410 
diagram,  404 
dry  objectives,  4 1 0 — 4 1 1 
experiments,  409^110 

water/glycerol  results,  table,  410 
field  strength  calculation,  405 
other  considerations,  410-413 
spherical  aberration  correctors,  15,  151, 
147,  192,  411-412 
terminology,  405 

actual  focal  position  (AFP),  405 
focal  shift,  405 

nominal  focal  position  (NFP),  405 
theory,  404-407 

Region  code,  for  MPEG-encoded  movies, 
844. 

Region-of-interest  (ROI),  835. 
brain  slice,  726,  733 
diagonal,  658 
display  presentation,  835 
embryos,  747,  759 
FRAP,  51.  187 
FRET,  797,  801 

in  image  processing,  289,  300,  323,  330, 
676 

labeling,  353 

must  be  smaller  at  high  resolution,  577 
nanosurgery,  219,  686 
photobleaching,  690 
preprocessing,  676 
rapid  acquisition,  236-237 
structured  illumination,  example,  272 
viability  studies,  683 
Registration  synthesis  method,  328-331. 
defined,  328 

landmark-based,  328-329 
multi-view  deconvolution,  330 
Relationships,  in  fluorescence  microscopy, 
80. 

energy  per  photon,  80 
flux  per  pixel,  80 
photons/s  vi.  wavelength,  80 
Relative  motion,  objective  vs.  specimen, 
39-40. 

Relaxation,  in  laser  energetics,  82. 

Relay  optics  (telan  lenses),  145,  157,  214, 
455. 

Reliability. 

of  3D  image,  461,  517 

biological,  vs.  damage,  24,  68,  631,  633 

lasers,  80,  102,  115 

living  cell  work,  387 


mirror  position,  40 
photometric,  312 
spectral  detectors,  662 

Removable  storage  media,  585-588. 
random-access  devices,  586-588 
compact  disks  (CD),  586-587 
digital  video  disks  (DVD),  587-588 
floppy  disks,  586 
magnetic  disks,  586 
MO  (magneto-optical)  disks,  586 
optical  disks,  586 

WORM  (write  once,  read  many)  disks, 
586 

Rendering,  of  3D  views,  280,  285,  290,  301, 
307,  309,  311,  377,  749,  762,  764. 
definition,  280 
voxel  speed,  290 

RESOLFT  microscopy,  571-574,  577.  See 
also,  STED. 

breaking  diffraction  barrier,  571-573 
concept,  571-573 
different  approaches,  573-574 
ground  state  depletion  (GSD),  573 
STED,  573-574 
outlook  for,  577 

resolution,  new  limiting  equation,  571 
triplet-state  saturation,  573 

Resolution,  1,  4,  13,  16,  24,  36-41,  59,  61, 
65-67,  210.  See  also,  PSF;  FWHM. 
adequate  levels,  36 — 41 
axial,  13 

axial-to-lateral  ratio  vs.  NA,  4 

back-focal  plane  diameter,  table,  210 

confocal  t's.  non-confocal,  16 

and  contrast  transfer  function,  37,  59,  61 

estimating,  65-67 

measured,  widefield,  16 

minimum  resolvable  lateral  spacing,  1,  16 

spatial  and  temporal,  24 

sufficient,  36-37 

Resolution,  structured  illumination. 
Fourier-space,  270-271 
linear  image  reconstruction,  27 1 
Lucosz’s  formulation,  273 
methods,  270-276 
Moire  effects,  270-27 1 
photobleaching,  275 
reconstruction  results,  272 
standing-wavefield  microscope,  275 
thick  samples,  274,  275,  278-279 

Resolution  scaling,  STED  comparison,  578. 

Resolution  test  slides,  16,  656. 

Resonant  cavity,  laser,  81-82,  111,  115. 

Resonant  scanners,  52-54,  56-57,  223,  447, 
543. 

acceleration  distorts  mirror  shape,  53 
blanking,  25,  218,  338,  389,  543,  628, 
651,  908 

compared  to  acousto-optical  deflector,  56 
duty  cycle,  52 
galvanometer,  52 

multi-photon  excitation,  543 
raster-scanning,  33,  53-54,  56 
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Resonant  scanners  (cont.) 

retrace,  54,  56.  See  also ,  Retrace,  below, 
scan  speed,  54 

Retrace,  raster  scanning  shape,  25,  54,  56, 
219,  389,  543,  628,  651,  908. 
acousto-optical  deflector,  56 
blanking,  25,  219,  338,  389,  543,  628, 
651,  908 

raster-scanning,  33,  53-54,  56 
Review  articles,  listing,  889. 

RFR  See  Red  fluorescent  protein. 
Rhodamine.  dyes,  81,  109,  116.  136,  140, 
203,  264,  292,  339,  342-345,  353, 
355,  362-363,  375-378,  409,  538, 
553,  592,  693,  697-698,  762, 
783-784,  794,  851,  854-856. 
arsenical  derivatives,  348 
bleaching,  697,  698 
calibration  plot,  661,  851 
excitation  of,  181,  109 
fluorescence  correlation  spectroscopy, 
693 

FRET,  347 

photobleaching  quantum  yield,  363 
planar  test  specimen,  538 
power  for  1-,  2-photon  excitation,  81,3 
41 

Rhodamine- 123,  374,  389 
resolution  measurement,  409 
stability  and  cost,  116 
Rice  ( Oryza  saliva),  168,  171,  414,  415, 
712,  715. 

absorption  spectrum,  415,  706 
backscattered  light  image,  168,  171 
emissions  spectra,  autofluorescence,  713 
leaf  fluorescence  images,  714-715 
light  attenuation  in  plant  tissue,  414 
silica  deposits,  714-715,  717 
Richardson-Lucy,  deconvolution,  497,  568. 
Richardson  Test  Slide  Gen  III,  652,  656. 
RLE.  See  Run-length  encoding. 

RNA,  microinjection  of,  803,  804. 

RNA  labels,  344,  369,  465,  531-532,  612, 
691,  758,  874-875. 

ROI.  See  Region  of  interest. 

Room  light,  as  stray  signal,  201,  632,  904. 
Roots,  plants,  172,  174,  303,  307,  421, 
430-432,  438,  464-465,  556, 
772-773,  775,  777,  779-783. 
maize,  image,  432 
mounting,  429,  43 1 
ROS.  See  Reactive  oxygen  species. 

Rose  Criterion,  37-38,  68,  164,  633. 
relationship  with  signal-to-noise  ratio, 
164 

for  visibility,  37 

Rotating,  specimen,  188,  568,  835. 
micro-CT,  615 

optical  projection  tomography,  610-611 
SPIM,  672-673,  676,  751 
Rotor,  galvanometer,  detecting  position, 
53-54. 

Run-length  encoding  (RLE),  580. 


S 

Safety,  83,  85,  90,  115,  117-118,  124, 
132-139,  900,  903,  904. 
arc  sources,  132-139 
beam-stop  design  and  use,  118.  903-904 
classification  of  laser  systems  by  hazard, 
117 

cleaning  objectives,  642 
display  geometry,  297 
equipment  needed,  900 
eye  protection 

against  Brewster  surface  reflections,  83 
goggles,  118 

with  external-beam  prism  method,  90 
fiber  optics  for  transporting  laser  light, 

88 

hazardous  materials 

fluorescent  laser  dyes,  85,  103,  116 
used  beryllium  oxide  tubes,  115 
high  pressure  Xe  lamps,  136 

monitor  power  to  avoid  explosions, 
138-139 

in  disk-scanning  confocal  microscope, 

231 

laser,  117-118,  839,  900,  903-904 
installation  requirements,  85 
monitor  power  to  avoid  explosions, 
138-139 

references,  list,  123 
safety  curtains,  117,  904 
training,  118 

SAM,  saturable  absorber  mirror.  111. 
Sampling.  See  Digitization,  20,  63-64. 
non-periodic  data,  38 
optimal,  63 

Saponin,  formaldehyde  fixation,  359,  375, 
856. 

Saturable  absorber  mirror,  pulsed  lasers, 

111. 

Saturable  Bragg  reflector  (SBR),  111. 
Saturable  output  coupler  (SOC),  107,  111. 
Saturation,  singlet-state  fluorescence,  21-22, 
41,  142,  265,  276,  339,  442,  448, 
450,  643,  647,  899. 
performance  limitations,  81,  450,  928 
SBR,  saturable  Bragg  reflector.  111. 

SBT.  See  Spectral  bleedthrough. 

Scaling  techniques,  592,  835. 

Scan  angle,  and  position  in  image  plane, 
209-210. 

Scan  instability,  detecting,  40-41. 

Scan  raster,  testing,  65 1-654. 
malfunctioning  system,  653 
phototoxicity  from  uneven  scan  speed, 

651 

sources  of  fluorescent  beads,  table,  653 
well-calibrated  system,  652-653 
x  and  y  galvanometers,  65 1-652 
z-positioning  calibration,  652,  654 
stability,  652 

Scanned-slit  microscopes,  table,  224. 
Scanner  arrangements,  evaluation, 

213-215. 


Scanners,  51-55,  57,  214-216. 

acousto-optical  deflectors,  55.  See  also, 
AODs 

mirror  arrangements,  214 
evaluating,  215-216 
mechanical,  51-54.  See  also. 
Galvanometers 

piezo-electric,  57,  215,  238,  510,  555, 

610 

single  mirror/double  tilt,  215 
sinusoidal,  “tornado”  mode,  SIM  scanner, 
52 

Scanning  electron  micrographs,  428,  434, 
437,  846-848,  850-852. 

Scanning  laser  ophthalmoscope  (SLO),  480. 

Scanning  fiber-optical  microscopy.  See 
Fiberoptic  confocal  microscope. 

Scatter  labeling  for  tracing  lineage,  461, 

462. 

Scanning  systems  for  confocal  light 
microscopes.  See  also. 
Galvanometers;  Disk-scanning 
confocal  microscopy;  Acousto- 
optical  deflectors;  Linescanning 
confocal  microscopes;  Raster. 
Lissajous  pattern,  circular  scanning.  554 
“tornado”  mode,  SIM  scanner,  52 

Scattering,  162-163,  167-171,  550. 

coherent  anti-Stokes  Raman  (CARS),  550 
elastic,  Rayleigh,  162-163,  166-167,  339, 
342,  417,  703-747 
Raman,  162,  167,  339-340,  348, 

506-507,  545  and  reflection  contrast, 
167-171 

Scattering  object,  viewed  by  TIRM  177.  See 
also,  Backscattered  light. 

Schiff  reagents,  262,  369,  770,  774-775, 

778. 

Schottky  diode,  photodetector,  253. 

Scientific  thought,  four  aspects,  789-790. 

Scion  Image,  281-282,  395,  730. 

Scramblers,  light,  8,  13,  84,  131-132,  143, 
507. 

Screen  capture,  830. 

Screens,  to  enclose  laser  beams,  118. 

SD.  See  Static  discharges. 

SDA.  See  Stepwise  discriminant  analysis. 

Sea  urchin,  S.  purpuratus,  173,  198,  200. 

Second  harmonic  generation  (SHG),  90, 
114-115,  166-167,  179,  188,  550, 
552,  556,  703-719,  729-730.  See 
also.  Harmonic  signals, 
as  autofluorescence,  361 
cell  chambers,  166,  429,  552 
detectors,  706-708,  728 
disk-scanning,  552,  556 
double-pass  detection,  166-167 
table,  706-708 

crystals  for  SHG,  103,  107,  114-115,  188, 
703 

energy  relations,  705 
in  lasers,  103,  107,  114-115 
layout,  166,  191,  552,  708-709,  712 
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light  attenuation  spectra,  706 
light  sources,  706-708 
brain  slices,  729-730 
non-linear  optical  microscopy,  704-705 
optically  active  animal  structures  in, 
714-717 

brain  slice,  729-730 
collagen  structure,  703,  717 
sarcomeres,  716 
spindle  in  mouse  zygote,  717 
spindle  in  zebrahsh  embryo,  718 
structures  producing  SHG,  table,  715 
table  of  structures,  715 
zebrahsh  embryo,  716,  718 
optically  active  plant  structures,  428, 
710-714 

Canna ,  nonlinear  absorption,  710 
cell  wall,  428,  711,  714 
Commelina  communis,  712 
emission  spectrum  of  maize,  710,  711 
Euphorbia  pulcherrima,  spectrum,  710 
mineral  deposits,  436 
Pyrus  serotina,  spectrum,  711 
rice  leaf,  712,  715 
starch  granules,  433 
maize,  710-711,  713-714 
emission  spectrum,  710-711 
leaf  spectrum,  710 
pol-microscopy,  711 
stem,  optical  section,  714 
stem,  spectrum,  710,  713 
chloroplasts,  tumbling,  713 
membranes  of  living  cells,  90 
mineral,  deposits,  436 
photodetector  suitability,  table,  706-707 
polarization  dependence,  71,  717-720 
potato,  as  SHG  detector,  712 
pulsed  laser  suitablity,  table,  706 
signal  generation,  179,  552,  597,  704-705 
spectra,  706 

spectral  discrimination,  421 
starch  granules,  433 
Segmentation,  FLIM  image,  527-528. 
Segmentation  methods,  281,  283-285,  290, 
300-302,  304-306,  309,  311-312, 
316-319,  321-330,  333-334, 
527-528,  776-778,  812. 

3D,  776,  822,  828 
automated,  818,  821-822,  828 
background,  321 

blob  segmentation  example,  322-324 
gradient-weighted  distance  transform, 
323 

model-based  object  merging,  323-325 
watershed  algorithm.  323-324 
bottom-up,  321 

combined  blob/tube  segmentation, 
328-330 
foreground,  321 

hybrid  bottom-up/top-down,  322 
integrated,  322 
intensity  threshold-based,  321 
object,  321 


for  plant  cells.  774-777 
balloon  model,  776 
watershed  algorithm,  322-325,  777, 

822 

region-based,  321-322 
top-down,  322 

tube-like  object  segmentation,  324-328 
mean/median  template  response,  328 
skeletonization  methods,  324-325 
vectorization  methods,  324—327 
validation/correction,  333-334 
manual  editing,  333-334 
Selective  plane  illumination  microscopy 
(SPIM),  613,  614,  672-679,  751. 

3D  scanning  light  macrography,  672 
anisotropic  resolution,  678 
applications,  675 
axial  resolution,  674-675 
Vi.  CLSM,  678 

Drosophila  embryogenesis,  675-676, 
747-748,  751-752,  754,  756.  759, 
804,  810 
and  FLIM,  527 
images  processing,  675-678 
image  fusion,  676-677 
pre-processing,  676 
registration,  676 
lateral  resolution,  674 
light-sheet  illumination,  672-674 
light  sheet  thickness,  674-675 
Medaka,  614-615 
heart  image,  614 
embryo  image,  675 
multi-view  reconstruction,  675-678 
point  spread  functions  (PSF),  674 
schematic  setup,  613,  673 
thin,  laser  light-sheet  microscope.TLLSM, 
672 

Self-aligning  arc  source,  135. 
Self-shadowing,  165.  174,  194,  195. 
in  confocal  optical  sections,  174 
spherical  structure,  195 
in  epi-fluorescent  mode,  165,  194 
SEM.  See  Scanning  electron  microscope, 
Semi-apochromat,  pros  and  cons,  158. 
Semiconductor  lasers,  86,  105-108. 
noise  sources,  86 

Semiconductor  nanocrystals  (quantum  dots), 
221,  343,  357-358.  360-361,  656, 
694,  696,  757,  759,  801,  814,  846, 
853. 

as  probes,  221,  757,  759 
Semiconductor  saturable  absorber  mirror 
(SESAM),  107,  111. 
for  self-starting  intense  optical  pulse 
trains,  111 

Sensitivity,  video  photodetectors,  6-7. 
Sensitized  emissions,  of  acceptor,  795-796. 

See  also,  FRET. 

Sequential  devices,  585-586. 

Serial  sampling,  single-beam  confocal,  20. 
SESAM,  Semiconductor  saturable  absorber 
mirror,  107,  111. 


SFP.  See  Simulated  fluorescence  process. 

Shannon,  Claude,  64-65. 

Shannon  sampling  frequency,  defined,  64, 
443. 

SHG.  See  Second  harmonic  generation. 

Shift  invariance,  deconvolution,  457,  490, 
564. 

Short-pass  filters,  43-44. 

Shot  noise,  232,  256-257,  286,  442, 

460-461,  495,  558,  660-661,  831. 
See  also,  Poisson  noise.  Quantum 
noise. 

Signal,  27,  62.  See  also  Speed  relationship 
to  magnification,  62 

Signal  attenuation-correction,  320-321. 

Signal  detection,  basics,  660-663,  918-931. 
See  also.  Detectors, 
coefficient  of  variation.  660 
instrument  dark  noise,  660 
photon  (shot)  noise,  660-661 
PMT  linearity,  661-662 
signal-to-noise  ratio,  660 
spectral  accuracy,  662 
spectral  resolution,  662-663 
wavelength  response,  663 

Signal  levels,  16-photon  peak  signal,  73-74. 

Signal-to-background  ratio,  of  titanium- 
sapphire  laser,  112. 

Signal-to-noise  (S/N)  ratio,  37,  53-54,  67, 
81,  164,  251,  257,  265,  330,  340. 
386,  391,  442-451,  470,  481,  495, 
498-499,  528,  542,  562,  567,  582, 
599,  621-622,  660,  690,  696,  699, 
707,  736-737,  740,  753,  769,  772, 
778-780,  810,  813. 

3D  imaging,  448-451 
4Pi  microscopy,  562-567 
bleaching,  391.  442,  690,  696 
in  calcium  imaging,  737 
chapter,  442-451 

comparative  performance,  256,  448-451 
bleaching-limited  performance, 

448-450 

configurations  of  microscope,  448,  449 
disk-scanning  microscope,  449 
line  illumination  microscope,  449 
saturation-limited  performance,  450 
scanning  speed  effects,  53,  450 — 45 1 
structured  illumination,  265-266,  270, 
275-276,  279-280 
S/N  ratios  for,  table,  450 
widefield  (WF)  microscope  for,  450 
confocal  microscope,  444^147,  660 
calculations.  444 
detectability,  446-447 
methods  compared,  450 
noise  model  Nl,  444^145 
noise  model  N2,  446^147 
deconvolution,  470,  481,  495,  498-499 
designs,  confocal,  212-216,  447-448,  450 
disk-scanners,  221 

dynamic  range,  2-photon,  644,  778-780 
high-content  screening,  810 
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Signal-to-noise  (S/N),  ratio  ( cont .) 
improvements,  736 
micro-CT,  615 

magnetic  resonance  microscopy,  621-622 
multi-photon  fluorescence  microscope, 
112,  427,  447,  542,  779 
Nyquist  sampling,  67,  448 
optimal  excitation  power,  81,  340 
Rose  criterion,  visibility,  37-38,  68,  164, 
633 

saturation,  442 
t's.  scan  rate,  53 
signal  level,  67,  75,  528 
sources  of  noise 

background  noise,  443-444 
grey  levels,  443 
quantum  efficiency,  442^143 
shot  noise,  442-443 
sources  of  noise,  442-444 
STED,  574 
and  visibility,  37 

Silica  glass,  transmission  losses,  502. 

Silicon  diodes,  near  infrared  emission,  132. 
Silicon-intensified  target  (SIT)  camera,  730. 
brain  slices,  730 

SIM.  See  Surface  imaging  microscopy. 
Simplicity,  as  design  goal,  43,  66,  220,  229, 
387,  508,  647. 

Simulated  fluorescence  process  (SFP),  310. 
Single-cell  automatic  imaging,  809,  812. 
Single-cell  calcium  imaging,  812. 
Single-longitudinal-mode  fiber  laser,  110. 
Single-mirror/double  tilt  scanner,  215. 
Single-molecule,  80. 
biochemistry,  221-222,  575,  690,  693, 
696 

bleaching,  690,  693,  696,  697-698,  699 
Single-photoelectron  pulse  heights,  30. 
Single-photon,  energy,  equation,  35. 

Single -photon  counting  avalanche 
photodiodes  (SPAD),  527. 
Single-photon  excitation,  plant  imaging, 
772-778. 

Single-photon  pulses.  See  Photon  counting. 
Single-scan  images  measure  scan  stability, 
40 — 41. 

Single-sided  disk  scanning,  confocal 

microscopy,  132,  141-142,  168,  171, 
175,  215-216,  229,  231,  907,  913. 
See  also.  Disk-scanning  confocal 
microscopy. 

advantages  and  disadvantages,  215-216 
basic  description,  141 
commercial,  907,  913 
light  source,  132,  141-142 
Singlet  state  saturation,  21-22,  41,  81,  142, 
265,  276,  338-339,  442,  448,  450, 
643,  647,  899,  928. 

Sinusoidal  bidirectional  scanning,  25, 

52-54.  See  also.  Resonant  scanners, 
duty  cycle,  53,  260 
Sinusoidal  image,  831,  838. 
fiber-optic  confocal,  510 


Sinusoidal  modulation,  in  FLIM,  524—526. 

SIT.  See  Silicon-intensified  target  camera 
imaging. 

SFF.  See  Subcellular  location  features. 

Slice  AM-dye-painting  protocol,  726-727. 

Slice  chamber  protocol,  727. 

Slit  scanning  confocal,  12,  25,  37,  50,  51, 
56,  221-226,  231+,  235,  238,  519, 
522,  664,  741,  914,  916. 

Achrogate,  50,  212,  231-232,  916 
with  AOD  scanning,  56,  914 
commercial,  913-914,  916 
critical  parameters,  224-228 
optical  sectioning,  228,  444-449 
optimal  slit  size,  225 
point  excitation,  slit  detection,  914 

SFM.  See  Spatial  light  modulator. 

SET.  See  Subcellular  location  tree. 

Smart  media,  digital  storage,  588. 

SMD.  See  Surface  mount  device. 

SNARF-1,  345,  346,  531,  739,  744-745. 
ratiometric  pH  label,  744-745 
stained  rat  interossi  muscles,  739 
table  of  variants,  531 

Snell’s  law  of  refraction,  167,  654. 

SOC.  See  Saturable  output  coupler. 

Software  packages,  visualization,  table, 
282-283. 

SoftWorx,  3D  display  software,  282. 

Solanum  tuberosum,  potato,  712. 

Solid  state  memory  devices,  588. 
compact  flash  cards,  588 
memory  stick,  588 
smart  media,  588 

Solid-state  photodetector,  30-31,  918-931. 
See  also,  CCD;  EM-CCD. 

Solid-state  lasers,  86,  103-118,  236-237. 
cooling,  108 
noise  sources,  86 
thin-disk  lasers,  109 
tunability,  109 
use,  236-237 

Source  brightness,  measure,  radiance  units, 
126. 

Source  optics,  reflecting  and  collecting  light, 
134. 

Space  invariance,  telecentric  systems, 
207-208. 

Space  multiplexing,  in  MMM,  555. 

Spacer,  material  in  interference  filters,  46. 

SPAD,  single-photon  counting  APD,  527. 

Spatial  coherence,  84. 

Spatial  filter,  89,  107,  391,  542,  708,  729. 
optical  devices  for,  89,  222-223,  729 
digital,  391-392.  See  also,  Gaussian 
filtering 

Spatial  frequency,  37,  60,  65,  66.  See  also, 
CTF. 

and  contrast  transfer  function,  37 
and  geometry,  66 

response  of  microscope,  and  pixel  size, 

65 

zero,  as  measure  of  brightness,  60 


Spatial  laser  beam,  characteristics,  89. 

Spatial  light  modulator  (SFM),  266. 

Spatial  orientation  factor,  for  FRET, 
792-793. 

Spatial  resolution,  in  confocal  microscopy, 
24.  See  also.  Resolution,  PSF,  CTF. 

Special  setups,  for  CFSM,  218-219. 

Specifications,  general,  for  scanner,  54. 

Specimen,  general  considerations,  192-197, 
228,  361-362,  779.  See  also,  Fiving 
cells,  Fiving  embryo  imaging, 
fluorescent  probes  interactions,  361-362 
cytotoxicity,  362 
localization,  361-362 
metabolism,  361-362 
perturbation,  362 
optical  heterogeneity,  22,  23 
plants.  See  Plant  cell  imaging;  Botanical 
specimens 

Specimen  chambers.  See  Fiving  cell 
chambers. 

Specimen  heating,  in  2-photon,  539. 

Specimen  holder,  for  scanning  specimen,  9. 

Specimen  preparation,  for  automatic  3D 
image  analysis,  319-321. 
image  analysis,  319-321 
imaging  artifacts,  320 

Specimen  preservation,  general,  368-378. 
antibody  screening  on  glutaraldehyde- 
fixed  specimens,  377 
evaluation,  371-374 

cell  height  to  measure  shrinkage, 
371-373 

defined  structures,  distortion,  373-374 
MDCK  cell,  stereo  image,  373 
MDCK  cell,  vertical  sections,  372 
fixation/staining,  370-371 
fixative  characteristics,  368-370 
chemical  fixatives,  369 
cross-linking  fixatives,  369 
freeze  substitution,  369 
microwave  fixation,  369 
protein  coagulation,  369 
formaldehyde,  369-370,  373 
general  notes,  374-378 
glutaraldehyde,  369,  370 
immunofluorescence  staining,  371 
improper  mounting,  376 
labeling  thick  sections,  376-377 
microwave  fixation,  377-378 
mounting  methods,  370-374 
critical  evaluation,  37 1-374 
mounting  media,  table,  377 
pH  shift/formaldehyde  fixation,  370-371, 
373 

refractive  index  mismatch,  377 
mounting  media,  table,  377 
refractive  index  of  tissue/organs,  table, 
377 

tissue  preparation,  376 
triple  labeling,  375-376 

Specimen-scanning  confocal  microscope, 

9. 
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Speckle,  from  high-coherence  sources,  8, 

84,  90,  130-132,  448. 

Speckle  microscopy,  13,  383,  385,  889. 

Spectra,  emission, 
arcs,  130 
black  body,  136 
LEDs,  133 
solar,  127 

tungsten  source,  153 

Spectral  accuracy,  662. 

Spectral  bleedthrough  (SBT),  185,  203-204, 
664. 

in  intensity-based  FRET,  185 

Spectral  confocal  image  A.  thaliana 
seedling,  175. 

Spectral  detector,  203-204,  662-663, 
666-667 . 
testing,  662 

Spectral  discrimination,  filter  bandwidths, 
44. 

Spectral  imaging,  175,  382,  384. 
table,  384 

Spectral  leakage,  inter-channel  signal 
imbalance,  185,  203-204. 

Spectral  phase  interferometry,  for  direct 
electric  field  reconstruction 
(SPIDER),  115. 

for  pulse  length  measurement,  115, 
901-903 

Spectral  properties,  of  filters  V5.  angle,  49. 

Spectral  resolution,  of  detection  system, 
203-204,  662-663,  666-667. 

Spectral  response, 
of  CCD  chips,  29,  234,  922 
of  eye,  153 

PMT  photocathodes,  29 

Spectral  transmission,  objectives,  plots, 
159-161. 

Spectral  unmixing,  190-192,  319,  361,  382, 
384,  386,  423^125,  431,  664-667, 
770,  905. 

detectors  for,  51,  667 
examples,  665-666 
limitations,  51,  382,  667 
overlapping  fluorophore  emission,  190, 
319,  423-425,  664-667 
removing  autofluorescence  using,  667 

Spectrofluorimetry,  for  FRET,  793,  795. 

Spectroscopic  ruler,  765. 

Speed,  in  confocal  imaging,  7,  11-12,  36, 

41,  53,  142,  222-224,  235-236,  434, 
447,  450,  458,  460,  482,  526,  536, 
563-564,  748,  753-755,  784.  See 
also ,  Temporal  resolution. 
4Pi-MMM,  563-564 
AOD,  55-56 
calcium  imaging,  741 
CARS,  599-600,  604 
charge-coupled  device  cameras,  77-78, 
142,  229,  231-235,  259,  647,  651, 
754-755,  885 

data  compression,  581-582,  586-588 
detector,  in  FLIM,  523,  558 


disk-scanning  confocal,  141,  216,  224, 
754 

for  display,  processing,  803,  839,  841- 
842,  862 

factors  affecting,  235-236,  482,  496, 
753-754 
of  fixation,  370 
FRET,  795,  805 
galvanometer,  52-54,  211,  214 
high-content  screening,  809-810,  813 
MMM,  551-555,  563-564 
need  for,  in  living  cell  imaging,  222, 
753-754 

rendering,  3D  display,  831 
SPIM,  613,  678 

Spermatocyte,  crane  fly,  15. 

Spherical  aberration,  15,  34,  147-149,  151, 
160,  192-197,  208,  241,  244,  247, 
330,  395,  404-413,  454-455,  463, 
466,  471,  480-481,  542,  629,  640, 
654-655,  657-658,  728,  772,  774, 
893,  903-904.  See  also.  Aberrations, 
spherical. 

blind  deconvolution,  47 1 ,  480-48 1 
chapter,  404-413 

confocal  microscopy  performance,  654 
correction  of  RI  mismatch,  192,  287,  411, 
542 

correction  of,  figure,  145,  411-412, 
654-655 

corrector,  92,  395,  398,  411,  477.  640, 

654 

deconvolution,  463,  466,  468^169,  471, 

480,  498^199,  658,  728,  784 
effect  of  specimen,  192-197,  418,  454, 

747 

index  mismatch.  See  Index  mismatch 
measurement,  145,  407,  455,  471, 

481,  492,  657 

signal  loss,  330,  389,  395,  413,  457,  661 

SPIDER,  Spectral  phase  interferometry  for 
direct  electric  field  reconstruction, 
115,  901-903. 

Spill-over,  between  detection  channels.  See 
Bleedthrough. 

SPIM.  See  Selective  plane  illumination 
microscopy. 

Spinning  disk,  3,  5-6,  11,  40,  141,  176,  216, 
223-224,  231-232,  235-236, 
260-265,  459^160,  464,  468, 
481-483,  783-784,  810-811.  See 
also.  Diskscanning  confocal 
microscopy. 

commercial,  907,  913,  915 
FLIM,  519-520,  522 
high-content  screening,  810-811,  820 
MMM,  554,  558 
performance,  449-450 
systems  for,  cytomic  imaging,  810 
vs,  TPE  imaging,  in  plant  cells,  783 
Yokogawa  CSU-10/22,  231.  915 

Spinning-disk  light  scrambler,  ground  glass, 

8. 


Spinning  filter  disk,  digital  projector,  590. 

Spirogyra,  and  depth  of  optical  sections, 

195. 

Spot  scanning,  to  avoid  coherence  effects, 

84. 

Spot  size,  full-width  at  half-maximum.  See 
Pointspread  function.  Full-width 
half-maximum. 

Square  pixels,  advantage  of  using,  62. 

Stability,  86,  102,  103,  136-139,  826. 
algorithmic,  473 
arc  sources,  136-137,  477 
argon-ion  laser  vs.  krypton  laser,  102 
disk  scanners,  215 
of  DVDs,  587 

dye.  See  Dyes;  Bleaching  from  fiber-optic 
coupler,  505-506 
galvanometer,  54 
halogen  sources.  136-139,  346 
interferometer,  240-241,  267 
laser,  81,  85-89,  704 
diode,  106,  108-109 
fiber  output,  505 
helium-cadmium,  (low),  103 
intensity,  85-87,  113,  116,  136,  477, 

903 

measurement,  650-651 
pointing,  87,  903 
results,  86,  103 
structure,  82-85,  103 
thermal.  111 

wavelength.  106-108,  115,  118 
mechanical,  39,  82,  85,  201,  267,  512, 

652 

objectives,  146 

photostability,  363,  369,  690-702,  802. 

See  also.  Dyes;  Bleaching 
scan,  40,  638-639,  651 
shutter,  CCD  camera,  929 
thermal.  111,  219,  387,  389,  394,  539.  See 
also.  Thermal  variables 

Stage-scanning  confocal  microscope,  11. 
piezoelectric  scanners,  57,  708 

Staining,  plants,  438,  774.  See  also.  Dyes; 
Livingcells;  Botanical  specimens; 
Plant  cell  imaging;  Fluorophors. 
calcofluor  procedure,  438 
of  plant  tissues,  774 

Standards,  ISO  (DIN)  microscope  design, 

156+. 

Standing-wavefield  microscope,  275. 

Starch  granules,  plant,  202,  420^121,  428, 
432-433,  435,  703,  710-712,  715, 
719. 

Static  discharges,  destroy  semiconductors, 
109. 

Statistical  noise,  in  counting  quantum- 
mechanical.  See  Poisson  noise. 

STED.  See  Stimulated  emission  depletion. 

Stem-cells,  623,  678,  762,  790,  813. 

Stem,  plant,  168,  172,  180,  417-419,  421, 
424,  430,  556,  707,  710-711, 
713-714. 
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Stentor  coeruleus,  backscattered  light  image, 
168. 

Step  index  optical  fibers.  501-503. 

Stepwise  discriminant  analysis  (SDA),  818, 
820. 

Stereo  Investigator,  software,  282. 
Stereology,  316,  319. 

Stereoscopic  image,  about,  6-7,  9,  11,  154, 
224,  298-299,  317,  396. 
biofilms,  880 
cheek-cell  specimen,  23 
diatom,  640 

Drysophila,  microtubules,  752 
embryo,  200 

fat  crystal,  polarization,  479 
neurons,  298,  314 
Alexa  stained,  330 
backscattered  light  images,  167 
eye,  optic  nerve,  481 
Golghi-stained,  298 
Lucifer-yellow,  314 
microglia,  396-398 
rat-brain  neurons,  398 
transmitted  light,  475 
lung,  292 

MDCK  cells,  373-374,  378 
Milium  chromosomes,  Fuelgen-stained, 
298 

Paramecium,  chromosomes,  298 
pea  root,  RNA  transcript,  465 
platelet,  high-voltage,  EM,  848-849 
sea  urchin,  S.  Purpuratus,  173,  198,  200 
skin,  298 
Spirogyra,  195 

tandem-scanning  confocal  microscope, 

6 

Stereoscopic  views,  image  processing  and 
display,  290,  292,  293,  295-299, 

451,  764. 

color  space  partitioning,  297 
display,  293,  299 
interlaced  fields  of  frame,  297 
movie  projection,  838 
pixel-shift/rotation  stereo,  297 
stereo  images  example,  298 
synchronizing  display,  297 
Stick  objective,  for  in  vivo  confocal,  806. 
Stimulated  emission  depletion  (STED) 
microscopy,  3,  539,  561,  568, 
571-578. 

axial  resolution  increase,  576 
breaking  the  diffraction  barrier,  571-573 
challenges,  577 

compared  to  confocal,  575-576 
diagram,  573 
different  approaches,  573 
dyes  used  successfully,  table,  575 
OTF  compared  to  confocal,  578 
outlook,  577 

PSF  compared  to  confocal,  578 
RESOLFT,  the  general  case,  572-573 
results,  576,  578 
triplet-state,  573 


Stimulated  emission  of  radiation,  defined, 
82-83,  124. 

Raman  scattering,  167 
semiconductor,  106 

and  stimulated-emission  depletion,  573, 
577 

STN,  supertwisted  nematic,  589. 
Stochiometry,  ion  kinetics,  741. 

Stokes  held  intensity,  595,  597. 

Stokes  laser,  in  CARS  microscopy,  595, 
597-604. 

Stokes  shift,  44-45,  268,  338,  341,  343, 
443^147,  539,  542,  690,  759, 
792-793. 
anti-Stokes, 

CARS,  550,  595-604 
defined,  44-45 

in  fluorescence  resonance  energy  transfer, 

792+ 

large,  in  2-photon,  539,  646 
of  quantum  dots,  694,  759 
size  of  fluorophores,  45 
Storage,  digital.  See  Data  storage. 

Storage  structures,  plant,  435-436. 

maize,  image,  436 
Stray  light,  58,  632,  904. 
laser  light,  632 
non-descanned  detection,  904 
practical  confocal  microscopy,  632 
room  light,  201,  632 
Streak  camera,  FLIM  detector,  520. 

Strehl  ratio,  measure  of  image  sharpness, 
247. 

S.  purpuratus  (Sea  urchin),  173,  198,  200. 
embryo,  173,  198,  200 
first  mitotic  division,  173 
image  degradation,  from  top  and 
bottom,  198 

stereo-pairs  of  embryo,  200 
Structural  contrast,  188.  See  also.  Harmonic 
signals. 

Structure,  optical,  59,  68.  132-135. 
of  light-emitting  diodes  (LED),  133 
of  microscope  sources,  132-135 
recognizing  features  in  noisy  images,  68 
chapter,  265-279 

Structured  illumination  microscopy, 
265-279. 

advantages/disadvantages,  265 
computing  optical  sections,  268-270 
Vi.  confocal  microscopy,  265 
degree  of  spatial  excitation  modulation, 
268-270 

absolute  magnitude  computation, 
268-269 

homodyne  detection  scheme,  268-269 
max/min  intensity  difference,  268 
scaled  subtraction,  269-270 
square-law  detection,  268-269 
synthetic  pinholes,  268,  269 
experimental  considerations,  265-268 
illumination  masks  for,  266 
light  source  for,  267 


pattern  generation,  266-268 
schematic  setup,  266 
nonlinear,  276 

resolution  improvement,  270-276 
Fourier-space,  270-271 
linear  image  reconstruction,  27 1 
Lucosz’s  formulation,  273 
Moire  effects,  270-271 
photobleaching,  275 
reconstruction  steps/results,  272 
standing-wavefleld  microscope,  275 
test  results,  274 

thick  samples,  274,  275,  278-279 

Subcellular  location  features  (SLF)  in 

automatic  image  analysis,  819-820, 
822-824,  828. 

2D,  819-820 

2D  SLF  feature  descriptions  table,  819 
3D  SLF,  822-823 
test  results,  table,  824 

Subcellular  location  tree  (SLT),  825. 

Subpixel  deconvolution,  478-479. 

Subresolution  beads,  655-656.  See  also. 
Beads. 

Sun,  microscope  light  source,  126-127,  131, 

135. 

spectrum,  127 

Superficial  optical  sections,  living  embryo, 
748. 

Supertwisted  nematic  (STN),  589. 

Surface  imaging  microscopy  (SIM), 
607-608. 

mouse  embryo,  608 
setup,  608 

Surface  mount  device  (SMD),  for  LED, 

133. 

Surface  orientation,  affects  reflected  light, 
181. 

Surface  structures,  distortion,  197. 

Surface  topography,  maximum  intensity, 

180. 

Surfaces,  of  interference  Alters,  47. 

Suspension-cultured  cells,  189,  429-430. 
bacteria,  876,  878 
image,  430 
frozen,  854 

Swept-fleld  confocal  microscope,  238. 

Synchrotron,  wide-spectmm  light  source, 

135+. 

Synthetic  pinholes,  in  structured- 
illumination 

microscopy,  268,  269. 
images,  269 

SYTO,  396,  874-876,  879-885. 

T 

Tagged  image  Ale  format.  See  TIFF. 

Tandem-scanning  confocal  microscope 

(TSM),  2-6,  11,  13-15,  39-40,  141, 
167,  215-216,  223-224,  228-229, 
447. 

comparison  with  other  confocals, 

13-15 
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description,  6,  141,  215-216,  228-229 
development,  5-6 
evaluation,  215,  216 
observing  ciliate  protozoa,  141 
rate  of  data  acquisition,  11 
real-time  imaging  of  tooth.  167 
sources  of  vibration,  39^-0 
viewing  color/depth-coded,  real-time, 
stereo 

images,  154,  304 

Tapetum,  plant,  433,  434,  779. 

TEC,  Thermo-electrically  cooled,  see  Peltier 
cooling. 

Telan  systems,  129,  157. 

Telecentric  plane,  208-209,  211. 
conjugate,  208-209 
effect  of  angular  deflection  in,  211 

Telecentricity,  207,  214. 

of  closely-spaced  scan  mirrors,  214 
defined,  207 

Tellurium  oxide  (Te02),  for  use  in  AODs, 
55 

TEM.  See  Transmission  electron 
microscope. 

TEM.  See  Transverse  electromagnetic 
modes. 

Temperature,  29,  56,  133,  135-136,  856, 
885.  See  also.  Thermal  variables. 

Temperature  tuning,  of  diode  lasers,  108. 

Temperature  effects  on  high  NA  objectives, 

248+. 

Temporal  aliasing,  39,  41,  391,  836-837, 
839. 

Temporal  coding,  299-300. 

Temporal  coherence,  7-8,  82-85,  131. 
defined,  84 

Temporal  dispersion,  502.  See  also.  Pulse 
broadening. 

Temporal  displays,  292-293,  297,  836. 

Temporal  experiments,  biofilms,  885-886. 

Temporal  pulse  behavior,  pulsed  laser.  111. 
See  also.  Pulse  length  measurement; 
Pulse  broadening. 

Temporal  resolution,  12,  24,  36-38,  41, 
221-222,  322,  334,  386,  391,  399, 
458,  558,  577,  618,  620,  622,  651, 
667,  730,  737,  746,  772,  784,  801, 
809.  See  also.  Fluorescence  lifetime 
imaging  (FLIM). 
of  photodetectors,  263 

Temporal  signals,  162,  286,  331,  383. 

“Test  drives,”  for  living  embryo  imaging, 
752. 

TFT.  See  Thin-film  transistor. 

Tetracysteine,  labels,  221,  348,  359,  853. 

Thalamocortical  slice  protocol,  724. 

Thermal  lensing,  pulsed  lasers,  109,  113, 
543. 

Thermal  variables,  219,  856. 
active  medium,  lasers,  8 1 
of  AODs,  56-57 

arcs,  peak  emission  wavelengths,  129 
automated  confocal  imaging,  810 


cell  chambers,  117,  386-389,  394,  727, 
790,  810,  814,  885-886.  See  also. 
Cell  chambers  cooling,  108,  133 
cryo  preparation  for  EM,  856-857 
on  detectors,  29,  252,  256-257,  495 
drift,  16,  115,  219,  386,  567,  489,  652 
compensating,  396,  732 
on  dye  labeling,  359,  361,  738-739 
effects  of  anti-bleaching  agents,  694 
effect  on  bleach  rate,  696-689 
effect  fiber  pinhole  size,  506 
fiber-optic,  pol-preserving  fiber,  503 
filament  spectra,  135-136 
fixation,  369-372,  375,  377 
incandescent  lamp  emission,  135-136 
laser  cavity,  34,  82,  85-88.  107,  109,  111, 
541 

of  LED,  133,  136-138 
brightness,  133 

lensing,  in  pulsed  lasers,  109,  113,  543 
and  light-source  output,  136,  138,  650 
noise  signal,  254,  257,  232-234, 

261-262,  495,  660,  734,  921,  924, 
925 

on  objective  lenses,  248-249 
in  photography,  71 
properties  of  ice,  856 
properties  of  optical  materials,  158, 
248-249 

and  photomultiplier  tube,  (PMT),  29 
on  refractive  index,  15,  56,  145,  411 
immersion  oil,  148-149,  248-249, 

411 

retinal  exposure,  117-118 
sensors,  255-256,  727 
solid-state  laser,  86,  108 
specimen  damage,  84-85,  139,  685 
specimen  heating,  539,  545,  681,  685, 

904 

temperature  tuning,  laser,  108,  115 
thermomechanical  effects,  685 
time  constant,  38 

Thermo-electrically-cooled,  see  Peltier- 
cooled. 

diode  lasers,  85,  107-108,  111,  117 

THG.  See  Third  harmonic  generation. 

Thick  samples,  274,  275,  278-279.  See  also. 
Living  embryo  imaging;  Brain  slices; 
Biofilms, 
background,  278 
structured  illumination,  274,  275, 

278-279 

close  focus  region,  279 
distant  focus  region,  279 
in  focus  region,  278 
number  of  collected  photons,  279 

Thin  disk  lasers,  109-110. 

Thin  Laser  Light  Sheet  Microscope 

(TLLSM),  672.  See  also,  SPIM. 

Thin-film  transistor  (TFT),  589. 

Third  harmonic  generation  (THG),  90, 

166-167,  179-180,  188,  428,  435, 
550,  705-718. 


CARS,  596-597 
contrast  mechanism,  166-167 
deposits  no  energy,  361 
detectors  for,  421,  706-708 
table,  707 

double-pass  detection  method,  166-167 
intracellular  inhomogeneities  tracked, 

90 

light  attenuation  spectra,  706 
light  sources,  706-708 
to  make  more  laser  lines,  109,  114 
mechanism,  705 
microspectroscopy,  421 
MMM,  551,  559 

non-linear  optical  microscopy,  705 
optical  sectioning,  704 
optically  active  animal  structures, 

714-717 

collagen  mat,  polarization  microscopy, 
717 

mouse  zygote  spindle,  717 
structures  producing  THG,  table,  715 
zebrafish  embryo,  716,  718 
optically  active  plant  structures,  710-714 
cell  walls,  438 
Commelina  communis,  712 
Euphorbia  pulcherrima  spectrum,  710 
maize,  emission  spectrum,  710,  711, 
713 

maize,  polarization  microscopy,  711 
maize,  stem  section,  714 
phytoliths,  polarization  microscopy, 

720 

potato,  712 

Pyrus  serotina  spectrum,  711 
rice  leaf,  image,  712,  715,  719 
photon  interactions,  179 
pulsed  lasers  suitable,  table,  706 
STED,  577 

structural  contrast,  188 
Three-decibel  point  (3dB ),  for  bandwidth, 
59,  65. 

Three  dimensional  cell  pellet,  815. 

Three  dimensional  microscopy,  766,  771, 

804+. 

future  perspectives,  804-805 
living  embryos,  766 
of  plant  cells,  771 

Three  dimensional  projections,  embryo,  763. 
Three  dimensional  segmentation,  plant, 
776-778. 

Three-channel  confocal  microscopy, 
with  4  recombinant  proteins,  190 
assays  for,  814 

Three-dimensional  diffraction  image,  4,  147, 
407,  455,  463,  471,  491. 
Three-dimensional  micro-array  assays, 
815-816. 

Three-dimensional  reconstruction,  775-776, 
778,  and  Chapters  14  and  15. 
plant  imaging,  775-776 
A.  thaliana,  778 
Equisetum,  11 A 
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Three-photon  excitation  (3PE),  88,  415,  447, 
535,  550-552,  555,  558,  647,  680, 
709,  876. 

absorption  cross-section,  680 
damage,  682,  686 
fiber-optics,  507 
resolution,  447 
setup,  708-709 

TIFF  (Tagged  image  file  format),  580. 

Tiled  montage,  851,  858. 

Tiger,  ECDL  laser  system,  90. 

Time  correlated  single-photon  counting 
(TCSPC),  518,  520-523,  526. 
for  lifetime  imaging,  table,  526 
FLIM,  520-523 
FRET-FLIM,  186 
schematic  diagram,  521 

Time  multiplexing,  of  adjacent  excitation 
spots,  to  reduce  flare  in  MMM, 
553-554. 

Time-gated  detection,  FLIM,  522-524,  526, 
528+. 

diagram,  522 

FLIM  methods  compared,  table,  526 
FLIM,  image,  528-529 

Time-lapse  imaging,  136,  222,  354, 

382-384,  392-399,  652,  773, 
885-886. 

Amoeba  pseudopod,  191 
confocal  of  plant  cells,  773 
high-content  screening,  812 
illumination  stability,  136 
image  analysis,  286,  320,  333,  732-733 
mechanical  stability,  219 
microspectrometry,  maize  damage, 
424-426 

rectified-DIC,  of  platelets,  846 
SPIM,  613 
table,  384 

three-dimensional  plus  time,  222 
two-dimensional  plus  time,  222 

Time-lapse  recordings. 

Amoeba  pseudopod,  191 
Ascaris  sperm,  846 
biofilms,  885 

brain  slices,  725,  727,  729,  732-733 
embryos,  676,  749,  752,  759,  761 
meristem  growth,  430 
plant  roots,  781,  784 
rectified-DIC,  of  platelets,  846 
two-photon  microscopy,  10 

TIRF.  See  Total  internal  reflection 
fluorescence. 

TIRM,  177-179,  All. 

Tissue  specimens,  introducing  the  probe,  360. 

Titanium:  sapphire  laser  (Ti:Sa),  82,  84-86, 
88-91,  94,  100-103,  105,  107,  109, 
111-112,  114,  123-124,  165,  346, 
358,  415,  423-424,  459,  541,  550, 
551,  645-647,  688,  706-708,  713, 
727,  750,  756,  759.  See  also.  Lasers, 
titanium :  sapphire  and  Ultrafast 
lasers. 


4Pi,  563-564.  567 
brain  slices,  731 
CARS.  599 

compare  to  other  fast  lasers,  112-113 
Cr :  Fosterite,  femtosecond  pulsed  laser, 
109,  114,  415,  541,  706-709, 
712-714 

embryos,  750,  756,  759,  731,  764 
emission  stability,  86 
four-level  vibronic  model,  82,  109 
maintenance,  116 
multi-photon  excitation,  541 
and  OPOs,  114-115 

plants,  415,  423-424,  706-708,  713-714, 
717,  781-783 

popular  models,  specs,  table,  120 
STED,  575 
ultrafast,  112-113 
URLs,  124 

TLB.  See  Transmitted  light  bright-field. 

TLLSM.  See  Thin  Laser  Light  Sheet 
Microscope. 

Tobacco,  116,  189-190,  430,  693. 
smoke,  not  around  lasers!,  116 
suspension-cells, 

birefringence,  189-190 
GFP  expressing  cells,  430 
photo-bleaching,  693 

“Toe”  photographic  response,  defined,  71. 

Tornado  mode,  SIM  scanner,  54. 

Total  fluorescence  signal,  742. 

Total  internal  reflection  fluorescence 
microscopy  (TIRF),  90,  160, 
180-184,  223,  477,  801. 
blind  deconvolution,  477 
Vi.  confocal  image,  184 
contrast,  180-184 
cytoskeleton,  image,  183 
FRET,  801 

limits  excitation  to  single  plane,  223 
objectives,  for  epi-TIRF,  161 

Total  internal  reflection  microscopy  (TIRM), 
177-179,  All. 
blind  deconvolution,  477 
evanescent  wave  generation,  178 

TPE.  See  Two-photon  excitation. 

TPEM.  See  Two-photon  excitation 
microscopy. 

Trade-offs,  36.  68,  78-79,  221,  224, 
644-648,  747-748,  825. 
beam  power,  visibility/damage,  693 
blind  deconvolution,  483,  488,  499 
compression  algorithms,  581,  840 
confocal  endoscopes,  508 
when  digitizing,  68,  78-79 
embryo  specimens,  747-748 
high-content  screening,  optimal 
clustering,  825 
living  cells,  381,  693 
micro-CT,  dose/resolution,  616 
MRM,  time/resolution,  622 
and  pinhole  size,  265,  267 
processing  speed/segmentation,  301 


speed,  S/N,  sensitivity  and  damage,  221, 
224,  232,  556,  644-648 
SPIM,  resolution  and  number  of  views, 
613 

Transcriptional  reporters,  embryo  analysis 
and,  748,  755-756. 

FluoroNanoGold,  854 
rnRNA,  316-317,  465 
plants,  773,  781 
NF-kB,  814 

Transfection  buffer,  electroporation,  table, 
802. 

Transfection,  cellular,  756-758,  790,  791. 
brain  slices,  722,  724-725,  730-731 

Transfection  reagents,  for  chromophores, 
358,  360,  362,  556,  682,  790-791, 
795,  803. 

2-OST-EGFP,  566 
COS7,  693 
EB3-GFP,  183 

for  FRET,  CFP/YFP,  795-796,  798, 
801-802 
GalT-EGFP,  566 
GFP-MusculoTRIM,  184 
ligand  binding,  348 

Transfer  function,  implications  for  image 
contrast,  164-165.  See  also,  CTF. 

Transient  permeabilization,  359,  373,  375. 

Trans-illumination,  absorption  contrast,  166. 

Transistor-transistor  logic  (TTL),  259. 

Transit  time  spreads  (TTS),  527. 

Translational  fusions,  756,  757.  See  also. 
Transfection  agents, 
subcellular  specific  protein  distribution, 
756 

Transmission,  33,  49,  159,  225,  231,  804. 
AOBS,  57 
contrast,  163-164 

disk-scanning  micro-lens  array,  223-226, 
227-229,  231,  235 
dispersion,  683 
of  filters.  See  Filters 

linear  vs.  log  plots,  44—49 
of  glass  fibers,  501-505 
illuminator,  201,  127-128 
losses  due  to  refractive  optics,  33,  217 
table,  217 

of  objectives,  154,  158,  159-161,  641 
relative,  measurement,  26,  34,  36 
table,  transmission,  158,  159-161 
of  plant  tissue,  spectra,  416,  422 
of  Polaroid  materials,  85 
SHG  signal  detection,  707-709,  729-730 
by  small  pinholes  or  slits,  225 

Transmission  electron  microscope  (TEM), 
846. 

correlated  LM-TEM  images,  852-855, 
857-859 

stereo  images  of  platelets,  848-849 

Transmission  illuminator,  ghost  images, 
201-202. 

Transmission  intensity,  specimen  thickness, 
164. 
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Transmittance,  optical  system,  measured, 
25-26. 
table,  217 

Transmitted  light  brightfield,  468,  472-473, 
477. 

blind  deconvolution,  472-473,  477 
Transparency,  lighting  models,  309-312. 
Transverse  electromagnetic  modes  (TEM) 
laser,  83. 

Trends,  in  laser  design.  118. 

Triple-dichroic,  33,  46,  48,  217-218,  678, 
783. 

light  loss  due  to,  33 
performance,  46-48 
Triplet  state,  103,  338,  339-342,  348, 

362-363,  390,  516-518,  573,  646, 
684,  691-693,  697,  698,  704,  852. 
saturation,  339,  573 
as  a  RESOLFT  mechanism,  573 
Triton  X-100,  730,  852. 

formaldehyde  fixation,  370-372, 

375-377 
True  color,  29 1 . 

TSM.  See  Tandem-scanning  confocal 
microscope. 

TTL.  See  Transistor-transistor  logic. 

Tube  length/chromatic  corrections,  table, 
157. 

Tunable  lasers,  91,  103,  107,  109,  120. 
broadband,  table,  120 
continuous  wave  dye,  table,  91 
diode,  emerging  techniques,  107 
solid-state,  106,  109 
solid-state  ultrafast,  103 
Tungsten  carbide  electrodes,  radiance, 
137-138. 

Tungsten  halogen  source,  132,  137,  153. 
Turnkey  ultrafast  laser  systems.  118. 
Tutorials,  lasers  by  level,  124. 

Tweezers,  optical,  89-90,  110,  218,  383, 
385. 

setups  for  integration,  218 
single-longitudinal-mode  fiber  laser  for, 
110 

trapping  wavelength,  89-90 
Two-channel  confocal  images,  175-177, 
177,  193.  425,  522. 

A.thalicma ,  epidermal/mesophyll  cells, 
193,  425.  431-432,  434-436 
Amoeba  pseudopod,  169 
colocalization,  667 
display,  311.  841 
FLIM,  522 

harmonic  images,  714—716 
mouse  muscles,  716 
montaging,  331 
neurons,  332 
microglia,  396-398 
eye,  optic  nerve,  481 
Golghi-stained,  298 
Lucifer-yellow,  314 
rat-brain  neurons,  398 
transmitted  light,  475 


of  peony  petal,  cytoplasmic,  175-176 
rat  intervertebral  disk,  310-311 
of  zebrahsh  embryo,  177 
Two-dimensional  imaging,  60,  222, 
397-398. 

time  lapse,  222,  397-398 
Two-photon  fluorescence  excitation  (2PE), 
156,  160,  218,  535,  536,  750, 
778-783. 
chapter.  535-549 
chromatic  correction  for,  156 
for  plant  cells 

advantages  of,  778-779 
cell  viability,  779-781 
t's.  confocal  microscopy,  779 
dyes,  782 

of  green  fluorescent  protein,  782-783 
pitfalls,  782 

of  thick  specimens,  779 
in  vivo,  781 

special  objectives  for,  160 
visible  and  ultraviolet  dyes,  218 
Two-photon  microscopy,  10-12,  195,  357, 
535-549,  690,  697,  900-905.  See 
also.  Multi-photon  excitation;  Multi¬ 
photon  microscopy 
autofluorescence,  545 
basic  principles,  535 
of  biofilms,  882-885 
bleach  planes,  in  fluorescent  plastic,  193, 
194 

caged  compounds,  544 
calcium  imaging,  545 
chromophores,  543 

2-photon  absorption,  543 
diagram,  540 
detection,  538,  541 
descanned,  542 

non-descanned  (whole  area)  detector, 
541 

stray  light,  904 
fluorescence,  shadowing,  195 
group  delay  dispersion,  5443 
laser.  540-541 

alignment,  900-904 
monitoring,  901-903 
mounting,  541 
power  level,  903-904 
safety,  117-118,  839,  900. 

903-904 

living  cell  studies,  review,  544-545 
living  animal  studies,  545 
minimize  exposure  during  orientation, 

905 

mirror  scanning,  543 
optical  aberrations,  542 
photobleaching,  690,  697 
practical  tips,  900-905 
beam  alignment,  901 
bleed-through,  904 
choice  of  pulse  length,  537,  903 
pulse  length,  109,  112,  115,  507,  537, 
538,  902-903 


specific  specimens,  see  specimens  by 
name  imaging  multiple  labels, 
904-905 

neurolucida  protocol,  731 

resolution,  539 

and  speed,  12 

Vi.  spinning  disk  imaging,  in  plant  cells, 

783 

stray  light  and  non-descanned  detection, 
904 

theory,  535,  537 

wavelengths,  538-541,  See  also. 

Botanical  specimens 

U 

UBC  3D  living-cell,  microscopy  course, 

174,  183,  184,  190.  205,  364,  430, 
435,  439,  805-806. 

Ulbricht  sphere,  for  measuring  light,  140. 

Ultrafast  imaging,  two  dimensional,  222. 

3D,  235 

Ultrafast  lasers,  88,  101,  103,  112-114. 
Cr:Fosterite.  109,  114,  415,  541,  706, 
707-709,  712-713 

diode-pumped  solid-state  (DPSS),  112 
distributed  feedback  (DFB)  diode  laser, 
113 

fiber,  113-114 
table,  101 

fiber-diode,  mode-locked,  113 
Nd:  YAG,  88-89,  91,  95,  97,  103, 
107-109,  111,  113-115,  117,  218, 
245,  514,  680,  798 

Nd :  YLF,  89,  98.  100,  103,  109,  112-114, 
750,  760-761 

Nd:  YV04,  89,  95,  100,  103,  107-109, 
111,  113-114,  541 
solid-state,  tunable,  103 
spectrum,  44 

titanium :  sapphire,  112-113.  See  also. 
Laser,  titanium :  sapphire;  Titanium- 
sapphire  laser 

Ultrafast  pulses,  delivery  by  fiber  optics,  88, 

507. 

dispersion  losses,  502 

Ultraviolet  (UV),  argon-ion  laser  lines,  85, 
87,  90,  102,  339,  346. 
other  UV  lasers,  111-117 
use  for  micro-surgery,  218-219 

Ultraviolet  (UV)  confocal  microscopy,  109, 
174,  195,  571. 
absorption,  707,  713 
autofluorescence,  431-432.  434,  544 
CCD  response,  29,  255,  459,  921-922 
correct  imaging  with  planapochromats, 

14,  154 

damage,  212,  290,  439,  544,  680,  686, 
903 

disk-scanners,  229 

DNA-dyes,  782,  874.  See  also,  DAPI; 

Dyes  GFP  excitation,  798,  873 
high-content  screening,  811 
ion-imaging,  346,  383,  529,  738,  742 
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Ultraviolet  (UV)  confocal  microscopy 
(cont.) 

multi-photon  excitation,  535,  538,  544, 
559,  646,  706,  905 
photoactivation,  759 
safety,  117-118,  839,  900,  903-904 
simultaneous  with  DIC  imaging,  846, 

850 

as  source  of  stray  signal  in  PMT 
envelopes,  257 

Ultraviolet  performance  of  objective  lenses, 
154,  159-161,  706. 

Ultraviolet  widefield  light  sources,  132,  136, 
139,  143,  226,  542. 
table,  226 

Ultraviolet  transmission  of  optical  fibers, 

88. 

Ultraviolet  (UV)  light,  effects  produced  by 
multiphoton  intrapulse  interference, 
88. 

Ultraviolet  scanning  light  microscope,  6-7. 

Uncaging,  multi-photon  microscopy,  383, 
385,  545,  693,  760-764.  See  also. 
Photoactivation. 

Unconjugated  bodipy/ceramide  dyes,  760. 

Under-sampling,  79,  635,  640,  652,  662, 

831,  833,  836,  839,  841. 
example,  640 
uses,  68 

Uniformity,  of  light  source,  127-129. 

Unit  image  body,  3D  Airy  figure,  147. 

Upright  vi.  inverted  microscope,  140,  157, 
217,  230,  413,  722,  727,  870-872. 

Unmixing.  See  Spectral  unmixing; 
structured  illumination. 

Up-conversion,  fiber  lasers.  110. 
doped  ZB  LAN,  110 
dual-ion  doped,  110 

UV.  See  Ultraviolet. 

V 

Vacuum  avalanche  photodiode  (VAPD),  31, 
254,  255. 
definition,  254 
schematic,  31,  255 

VAPD.  See  Vacuum  avalanche  photodiode. 

Vertical-cavity  semiconductor  diode  laser 
(VCSEL),  108. 

Vibration. 

compensation,  732 
from  cooling  water,  84,  102,  499 
of  disk  scanner,  753 
causing  distortion,  16,  39-41,  166,  201 
of  galvanometer  mirrors,  40,  201 
high-frequency,  of  acousto-optic  devices, 
55,  84 

isolation,  85,  201,  219,  541 
measurement,  30-41,  652 
of  mechanical  shutters,  929 
of  objective  lens  motion,  754 
optical  fiber  isolation,  505,  507 
of  optical  fiber  scrambler,  8,  84,  131 

Vibronic  laser,  Ti:  Sa  four-level,  109. 


Video,  2,  4,  5-7,  11-14,  17,  37,  52-53, 

61-62,  88,  219,  237,  261,  263,  346, 
372,  430,  451,  505,  539,  554,  556, 
589-590,  593,  604,  860,  885. 
confocal,  25,  237,  914 
impact  on  light  microscopy,  5-7,  14 
results,  14 
signal,  258-259 

Video-enhanced  contrast  microscopy, 
imaging  small  features,  14,  68. 

Vignetting,  210-211,  229,  245-247,  492, 
541. 

objective,  off-axis  performance,  245-247 

Visibility,  and  signal-to-noise  ratio,  37-38, 
68.  See  also.  Rose  Criterion. 

Visilog/Kheops,  software,  282,  301-302, 
312. 

Visitech,  confocal  manufacturer, 

descriptions,  88,  119-120,  226,  237, 
908. 

VT  eye,  119-120,  908,  914 
VT  Infinity,  119-120,  908,  914 

Visual  cortex,  identification  of  primary,  724. 

Visual  observation,  magnification  for,  146. 
non-linearity,  72-73 

Visualization,  280,  282-283.  See  also. 
Multidimensional  microscopy 
images;  Rendering, 
definition,  280,  292 
software  packages  for,  table,  282-283 

Vitrea2/Voxel  View,  software,  282,  335. 

Volocity  (software),  281,  236,  282,  295, 

299,  312,  757,  762-764. 

VolumeJ,  software,  282,  304,  764. 

VolVis,  281-282. 

VoxBlast,  283,  301-302,  309,  312. 

Voxel,  defined,  20. 

Voxel  rendering,  speed,  290. 

Voxx,  software,  283,  377,  764. 

W 

WAD.  See  Whole-area;  Non-descanned 
detection. 

Water,  as  immersion  medium,  409,  410. 
refractive  index  mismatch,  table,  409,  410 
two-edge  response  curves,  410 

Water-coverslip  interface,  spherical 
aberration  generated  at,  147. 

Water-immersion  objectives,  15,  23,  36, 

141,  148-149,  154,  190,  235, 
241-242,  247,  261,  377,  386-387, 
389,  395,  411-412,  513,  542,  552, 
556,  562,  567-568,  584,  654-656, 
708,  727-728,  737,  747,  772.  See 
also.  Spherical  aberration. 

4Pi,  562,  567-568 
advantages,  149 
biofilms,  870,  872 
brain  slices,  727-728,  730,  737 
chapter,  404-413 

correction-color/flatness/transmission, 

154 

deep  imaging,  395 


dipping  objectives,  161,  209,  411,  429, 
568,  613,  727,  737,  870,  872 
in  fluorescence  ion  measurement,  737 
ion  measurement,  737 
living  cells,  386-387,  389,  395,  398 
performance  measured,  47,  655-656 
plant  cells,  429,  433,  772 
STED,  576 

transmission  curves,  159-161 
use  and  limitations,  15 
Watershed  algorithm,  322-325,  777,  822. 

for  segmentation,  plant  cell  images,  777 
Wave  optics,  4,  10. 

for  calculating  axial  resolution.  4,  146, 
154 

Wavefront  error,  217. 

lower,  with  hard  coatings  on  filters,  45 
Wavelength,  24,  28,  43-51,  62,  88,  107, 
114-115,  118,  129-130,  135-139, 
165-166. 

calculation  of  Forster  radius,  FRET,  793 
and  CCD  coupling  tube  magnification,  62 
filters  for  selecting,  43,  44,  88 
in  multi-photon  lasers,  165-166.  415,  750 
multiple,  dynamic  embryo  analysis,  756 
of  non-laser  light  sources,  129-130, 
135-136 

and  optimal  zoom  setting,  24 
vs.  pinhole  size,  28 
selecting,  with  interference  filter,  88, 
165-166 

stability,  in  non-laser  light  sources, 
137-139 

tunability,  of  lasers,  107,  109. 

Wavelength  expansion,  non-linear,  114-115. 
Wavelength  ratioing,  346.  See  also,  FRET; 
FLIM. 

Wavelength  response,  chromatic  aberration, 
663. 

Wavelength-selective  filters,  43-51,  88. 
Wavelength-tunable  lasers,  summary,  107, 
113,  116,  118,  550. 

Wavelet  compression,  581-584. 

Wavelet  de-noising  protocol,  733-734, 
819-820. 

Waxes,  plant,  420,  428,  434-435,  714-715. 
Website  references,  123. 

2  photon  excitation  spectra,  546,  727, 

729,  782 
brain  slices,  727 
CCDs,  76,  234,  927,  931 
components,  58 
confocal  Listserve,  390,  901 
deconvolution,  495 
dyes,  221,  343-344,  782 
fluorescent  beads,  653 
FRET  technique,  185,  803 
high-content  screening  systems,  811 
image  management,  865 
lasers,  104,  115,  120,  123-125 
live-cell  chambers,  388-389,  870 
movies  related  to  book,  235,  392 
muscles,  237 
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non-laser  light  sources,  138,  143 
plants,  769 

safety,  117-118,  839,  900,  903-904 
software,  282,  376,  594,  734,  762,  764, 
776,  777,  820,  824,  827,  831-833, 
844,  845,  864-862,  865-867,  869 
SPIM,  672 

Wedge,  compensator,  566-567. 

Wedge,  rotating,  for  light  scrambling,  84, 
131. 

Wedge  error,  in  interference  filters,  45-46, 
151,  211-212,  630. 
in  traditional  filters,  45 
Wedged  fiber-optics,  reduce  reflections,  85. 
Well-by-well  data,  817. 

WF.  See  Widefield. 

WFF.  See  Widefield  fluorescence 
microscopy. 

White  light  continuum  lasers,  88,  109,  113 
continuum,  88,  109 
He :  Cd,  113. 

Whole-area  and  external  detection,  541-542. 

See  also ,  Non-descanned  detectors. 
Whole-cell  patch  pipet  delivery,  360, 
726-727. 

Widefield  deconvolution,  751-753,  785.  See 
also.  Deconvolution, 
botanical  specimens,  785 
for  living  imaging,  751-753 
Widefield  (WF)  fluorescence  microscopy,  3, 
22-23,  26,  172-173,  219,  453-467, 
518.  See  also,  Epifluorescence 
microscopy,  Deconvolution, 
compared  to  confocal,  453-467,  644-647 
CCD/confocal  comparison,  458-459, 

465 

same  specimen,  465,  482 
compared  to  structured  illumination,  274 
deconvolution,  imaging  living  cells,  23, 
392 

deconvolving  confocal  data,  461-464, 

466 

fluorescence  detection,  459-460 
fluorescence  excitation,  459 
fluorescence  lifetime  imaging,  518 
gain-register  CCDs,  460-461 
images  utilizing  out-of-focus  light,  26 
imaging  as  convolution,  453^157 
imaging  thin  specimens,  172-173 
integration  of  fluorescence  intensity, 

459 

interaction  of  photons  with  specimen, 
22-23 

light-emitting  diode  sources,  136 
limits,  linearity/shift-in  variance,  457,  490, 
564 

model  specimens,  461 
noise,  459-463 

optical  sectioning  schematic,  469 
optical  tweezers/cutters,  219,  89,  383,  385 
out-of-focus  light,  461 
point-spread  function,  453-457,  459-463 
resolution,  3 


sensitivity,  459-463 
single  point  images,  454 
pros/cons,  644-648 
table,  459 

temporal  resolution,  458 

Wiener  filtering,  494,  496.  See  also, 
Gaussian  filtering, 
image  enhancement,  496 
image  restoration  by,  image,  494 

Windows  software,  for  automated  confocal, 
810. 

WinZip,  580. 

Wollaston  prisms,  DIC,  156,  468,  473,  475. 
See  also,  Nomarski;  DIC  contrast. 

Working  distance  (WD)  of  objective  lenses, 
5,  9,  129,  145,  154,  157,  198,  249, 
511,  568,  598,  634,  673,  678, 
727-728,  747,  774,  779,  781,  872. 
table,  157-158 

WORM  disks  (write  once,  read  many),  586. 

X 

Xenon  arc  lamps,  44,  132,  137-138,  144. 
iso-intensity  plot  of  discharge,  132 
pulsed-operation,  137-138 
shapes  of  electrodes,  132 
spectral  distribution,  144 
super-pressure,  spectrum,  44,  136 
explosion  hazard,  136 
wavelengths  available  for  detection,  44 

Xenon/iodine  fill  arc,  radiance,  137-138. 

Xenopus  laevis,  13,  610,  746,  748-753. 
blastomere,  757 

confocal/multi-photon  comparison,  750 
embryo 

viewed  with  confocal,  748-753 
viewed  with  OCT,  610,  749 
embryo  viewed  with  MRM,  623-264 
in  situ  imaging,  746,  748 
oocyte  wound  closure,  749 

X-Y  resolution,  confocal/widefield 
compared,  36. 

Y 

Yellow  fluorescent  protein  (YFP),  221-222, 
429. 

FRET  pair  with  CFP,  791-803 

YFP,  221-222,  429 

Yokogawa  disk-scanning  confocal  system,  6, 
12-13.  16,  216,  224-226,  231, 
234-237,  458,  754. 

CSU-10/22  model,  223,  231,  236,  915 
with  EM-CCD,  234,  237,  755 
high  speed  acquisition,  11,  220,  222-226, 
229,  231.  458,  667,  754,  784 
results,  236-237,  755,  783 
vibration,  16 

Ytterbium  tungstate  (Yb:KGW)  laser,  108. 

Z 

ZB  LAN  up-conversion  glass  fiber,  110. 

Z-buffering,  304-305. 

Z-contrast,  in  confocal  microscopy,  180. 


Zea  mays.  See  Maize. 

Zebrafish,  174,  176,  761. 

GFP  image,  176,  176 
autofluorescence,  174 
pancreas  expressing  DsRed,  176 
scatter  labeling/lineage  tracers,  761 

Zeiss,  confocal  manufacturer,  212,  214, 

217,  226,  231-232,  655,  771, 
916-917. 

510  META  confocal  microscope,  655, 
908,  916 

Achrogate  beam-splitter/LSM  5-Live,  50, 
119-120,  212,  231-232,  916 
Axioimager  system,  217 
fluorescence  correlation  spectrometer 

(FCS),  383,  385,  602,  801,  803,  805, 
917 

HBO-100  source,  self-aligning,  134-135 
high-content  screening,  811 
LSM  5-Live  line-scanning  confocal 
microscope,  50,  51,  231-232,  237, 
784,  908,  916 

META  confocal  spectral  detector,  51, 
119-120,  161,  202,  660,  663,  796, 
916. 

mini-PMT  arrays,  51,  667FRET,  706 
tests,  663 

objectives,  advantages  of,  155-156 
Infinity  Color-corrected  System,  155, 
217 

plan  objectives,  table,  152 
transmission  specifications,  161 
tube  length  conventions,  157,  239 
working  distance  of  objectives,  table, 

158 

Zernike  moments,  247-249,  818-820. 

Zernike  polynomial  fit,  245-247. 
table,  247 

wavefront  aberration  function,  247 

Zinc  selenide  (ZnSe)  diode  lasers,  106. 

Zirconium  arc  lamps,  136,  141. 
spectrum,  136 

Zone  System  (  Ansel  Adams),  71-72. 

Zoom  magnification,  11,  24,  37,  63-64,  66, 
70.  See  also  Magnification 
optimal,  24 

optical  vs.  electronic  bandwidths,  70 
relationship  to  area  scanned,  63 

Z-position  and  pinhole/slit  size,  227. 

Z-resolution,  3-4,  22,  36,  149-150,  224, 
225-228,  563,  752.  See  also.  Axial 
resolution. 

4Pi  microscopy,  563 

in  confocal  fluorescence  microscopy,  36 

effect,  of  fluorescence  saturation,  22 

improvement,  752 

of  pinhole  disks,  224 

in  STED,  576 

Z-scanners,  evaluating,  215. 

Z-sectioning,  imaging  brain  slices,  729. 

Z-stack,  23,  754. 

of  images  of  cheek-cell  specimen,  23 
speed  acquisition  constraint,  754 


